Experimenting with AI and Personalized Learning
A recent article from Tech & Learning (Syracuse University Gave AI Access To 30,000+ Students and Faculty) explored Syracuse University giving AI access to their faculty and students and included an experiment with personalized learning done by Chief Digital Officer Jeff Rubin. The T&L article talks about a few different topics, but I want to focus on using AI to improve student outcomes through study tools.
Rubin, who also teaches an introduction to information technology course for several hundred students, fed his recorded lectures to Claude (from Anthropic) and had Claude generate practice multiple-choice questions for students noting how hard it was to scale up to generating a large set of questions. “It is actually really hard for my brain to come up with thousands of different ways of asking questions,” Rubin said.
His hope was that this would meet the requests of students for a larger set of questions and improve student outcomes. To his surprise, the results didn’t show the expected improvements. The students practicing with the Claude generated questions weren’t doing any better than the rest of his students.
Rubin met with members of Syracuse University’s education department to discuss this and realized the problem might be that the multiple-choice questions Claude had created weren’t requiring his students to think before answering.
So, he worked with Claude to change the style of question from multiple-choice to short answers and additionally had Claude generate a response to that answer, as immediate feedback.
“It'll throw out a term that we've talked about in class in the form of a question and ask, basically, what do you know about this?" Rubin says. "Instead of just picking an A, B, C, or D, the student would type out their response. And then what Claude does is it grades the response and basically says, ‘Hey, this is a B type of response. Here’s the things you got right,’ and ‘These are the things you should know about.’”
Rubin saw immediate results with a 12-point average increase in exam scores.
This, I believe, is the true power and possibility of AI-based tools — acting as the "professor-in-a-box" that Jon Bergmann describes in his Mastery Flip framework (discussed in Flipped Classroom Pioneer Launches New Instructional Framework) and helping students work through misunderstandings between classes.
Rubin goes on to state that while he just started using the system, he believes he’ll continue to see positive trends.
The T&L article goes on to discuss the struggles some professors have with AI use and whether to condemn it or allow it with strict guidelines — but Rubin asks a better question:
“How do we think about AI from a pedagogy standpoint?”
Rubin recommends more active assignments, such as asking students to think about what they’d do in a specific situation as opposed to essays.
In the original article from T&L, Rubin also discussed the importance of transparency for any faculty use of AI.
“We have a responsibility to let students know if we're using AI. Are we using AI to generate our lectures? Are we using AI to generate our exams?” he says. “I don't think there's something wrong with it. I just think there's a level of transparency that is needed to show, 'Hey, we're still owning this, but I'm using AI in certain aspects of it.’”
If you would like to learn more about active assignments, and active learning in general, there’s still time to register for the Rutgers Active Learning Symposium (RALS), open to Rutgers faculty and staff, that is being held on Thursday, May 21st at the Richard Weeks Hall of Engineering on Busch Campus. Learn more and register here.