A New York University professor used AI to host an exam, choosing to ‘fight fire with fire’ after noticing a trend of ‘suspicious’ work from students.
With AI only growing more prevalent, it’s challenging the foundations of education. For many current students, AI programs have become a crutch to rely on, falling back on the likes of ChatGPT to assist, or in some cases, outright handle entire assignments.
This trend is what NYU business school professor Panos Ipeirotis looked to combat at the tail end of 2025. The data scientist began to notice that a few assignments were “suspiciously good.” Not just good for a ‘strong student,’ as he described, but far superior.
So to turn the tables and truly test his students’ knowledge, he himself turned to AI for help. Ipeirotis conducted an exam with the aid of AI, having his pupils converse directly with AI chatbots to deduce their expertise.
Students have been relying more on technology in the classroom than ever before.
Professor uses AI voice bots for exam
After first growing suspicious of AI use by students, the professor began “cold calling students randomly during class.” Although many had submitted “thoughtful, well-structured work” in their assessments, some struggled to explain “basic choices” under scrutiny.
“Students now have immediate access to LLMs that can handle most exam questions we traditionally use for assessment,” Ipeirotis wrote in a blog. “Suddenly, a student could deliver a polished, sophisticated presentation about a project they barely touched.”
This led the professor to oral examinations, wherein students speak without the aid of textbooks, or in today’s age, AI programs to help. “They force real-time reasoning, application to novel prompts, and defense of actual decisions,” as Ipeirotis explained.
Using ElevenLabs Conversational AI, the professor built an ‘examiner’ from the ground up to quiz his students. This AI bot would use student names and project details to personalize the assessment for all 36 individuals who took part.
Behind the enemy linesThe AI ‘graders’ even exposed “weaknesses” of the professor and their teaching methods.
It took, on average, 25 minutes as students spoke with the AI model, responding to questions about various case studies.
From there, three different AI programs were then used to check over transcripts of the conversations and even grade the students. The shortest exam took just nine minutes, yet provided the highest result with a 19/20 grade.
Ultimately, the feedback given by these AI programs “was better than any human would produce,” according to Ipeirotis. However, students found it more stressful, with 57% wanting to return to traditional written exams.
The vast majority agreed that the conversation with an AI program thoroughly tested their understanding of the course material. In concluding, the professor stated: “The concept works. The execution needs iteration.”
