OAN Roy Francis
UPDATED 9:18 AM PT – Wednesday, January 25, 2023
ChatGPT, the viral chatbot, has recently taken the Wharton MBA final exam, the United States Medical Licensing Exam, and parts of the Bar exam.
Professor Christian Terwiesch from the Wharton School of the University of Pennsylvania proctored a final exam from a Master of Business Administration course to the chatbot, and the program had passed and scored a grade of “B” on the exam.
Terswiesch claims that the chatbot had scored high on open-ended questions that were based on specific case studies, but had apparently struggled with 6th grade arithmetic. The one area the professor had found that he chatbot could not handle was “process analysis.”
When provided with help from humans, the chatbot had been able to improve its answers, similarly to how students can improve when helped by a teacher.
“This has important implications for business school education, including the need for exam policies, curriculum design focusing on collaboration between human and AI, opportunities to simulate real world decision making processes, the need to teach creative problem solving, improved teaching productivity, and more,” Terwiesch wrote in a report that he had published about the chatbot.
Researchers also tested the chatbot on the United States Medical Licensing Exam (USMLE), a three-part exam that is required to be a licensed medical professional in the United States. ChatGPT had passed or nearly passed all three exams and showed high levels of insight into its explanations.
ChatGPT has also taken two of the seven components of the Bar exam, a test that law school graduates must pass to be able to practice law. The first part of the test that it took was the multiple-choice portion, on which it scored a 50 percent, which is higher than the baseline guessing rate of 25 percent. The second component was the Torts and Evidence where it had scored within the passing range.
Looking at these results, researchers have confirmed that the ChatGPT, or a similar program, will be able to pass the Multistate Bar Examination component in the near future.
A Professor of Economics at George Mason University tested a competitor of ChatGPT, the Claude AI on a law and economics exam. The professor, Alex Tabarrok, announced that the Claude AI had marginally passed the exam.
Microsoft has recently announced a multiyear, multibillion-dollar investment in OpenAI, the maker of ChatGPT. This is the third phase of the partnership between the two companies which started in 2019.