Investigating generative AI in computer education
By Steve Nelson
There is no shortage of buzz around the use of generative AI in education and how to best use this powerful new tool. But when it comes to learning effectiveness and outcomes, educators don’t yet know how or if generative AI is working to close the learning gap — or only making that gap wider. A team of researchers from MIT CSAIL and MIT RAISE, Georgia State University, and Quinsigamond Community College investigated how this powerful tool can be used to advance equitable educational pathways in computing education at minority-serving institutions.
In the recent whitepaper “Opportunities, Issues, and Challenges for Generative AI in Fostering Equitable Pathways in Computing Education,” the team focused heavily on the challenges that impede underrepresented student success, as well as the impact of technology on equitable learner success — two aspects that form a strong determining factor for how well technology will help or hurt in the classroom. In essence, the paper argues these challenges can be overcome by providing adequate support to students as well as flexible learning options. The researchers also note that one key is early recognition of when students might be struggling and provide them with more personalized learning experiences. Researchers also highlight “constructionist approaches to learning,” and “learning-through-making in student-driven projects — computational thinking and skills, like coding” can help enhance student learning.
The authors suggest that maintaining learning as a social construct — a learning process that shouldn’t lean heavily on technology tools — is more supportive of learner growth. They also point out that while technology can be helpful in the classroom, access to certain technology is not always available to those students who need it the most. It is this unfortunate scenario that often leads to the widening gap between students of different socioeconomic backgrounds.
The researchers warn that several factors limit the effectiveness of using generative AI tools in the classroom including privacy, safety, cost, biased outputs, over reliance on tech tools, and academic integrity. These and other factors should give pause to educators as they continue to find safe and reliable ways to use generative AI to augment educational goals.
The team worked with a focus group of 60 students from Georgia State University to learn more about their own experiences with learning technologies. A majority of the participants in the study (65 percent) were first-generation college students. The students were separated into four groups and assigned to use a large language model (LLM) to do one of the following:
- concept comprehension, specifically through comprehending the concept of an LLM dictionary;
- debugging, specifically detecting and resolving errors in loops;
- quiz preparation, specifically how to apply loops to new problem settings; and
- program development, specifically developing programs with loops.
The student results varied by which tool students preferred based on their learning objectives. For example, students looking to deepen conceptual understanding preferred the use of Khanmigo’s LLM, while students looking for quick feedback preferred using the CS50.ai tool. Also of note, while CS50.ai seemed to express less empathy, the matter-of-fact responses were valued for precision and clarity.
A second set of research participants focused on employers and how generative AI was being used in their industry and what impact it had on helping develop computing skills. One takeaway from this group was that beyond learning how to interact with generative AI software development tools, entry-level coders will continue to need the same skills that talented coders currently need. The group also warned that generative AI still couldn’t account for some of the soft skills that workers will need for the future of work, including problem solving, innovation, collaboration, and general business skills like networking and salesmanship.
Both studies suggest that investing in human-centered skills — teamwork, curiosity, critical thinking, lifelong learning, creativity, problem solving, and systems thinking — remain important for learners and workers to cultivate. And while generative AI has the potential to make learning more effective and workers more productive, finding the right balance between soft skills and technology seems to be the most important lesson learned.
Authors of “Opportunities, Issues, and Challenges for Generative AI in Fostering Equitable Pathways in Computing Education” include: Cynthia Breazeal, dean for digital learning at MIT Open Learning and director of MIT RAISE; Arun Rai; Balasubramaniam Ramesh; Liwei Chen; Yuan Long; Andrea Aria; Hao Loi; Antonio Torralba; Jeremy Bernstein; Justin Reich; Eric Klopfer; Hal Abelson; George Westerman; and Christina Bosch.
The MIT RAISE Initiative is an Institute-wide initiative headquartered in the MIT Media Lab and in collaboration with the MIT Schwarzman College of Computing and MIT Open Learning.
Investigating generative AI in computer education was originally published in MIT Open Learning on Medium, where people are continuing the conversation by highlighting and responding to this story.