LOS ANGELES (AP) — The State Bar of California revealed that certain multiple-choice questions in an examination marred by issues were created using assistance from artificial intelligence.
The regulatory licensing authority stated
news release
On Monday, it will inquire about the
California Supreme Court
To modify the test scores for individuals who sat for the February bar exam.
“The fiasco surrounding the February 2025 bar examination has turned out to be even more severe than we initially thought,” stated Mary Basick, who serves as the assistant dean for academic skills at the University of California, Irvine, Law School.
told
The Los Angeles Times reported, “I’m nearly lost for words. It’s astonishing to have questions prepared by individuals who aren’t lawyers and utilize artificial intelligence.”
In February, the introduction of the new exam resulted in numerous complaints as many participants were unable to successfully complete their bar exams. Online testing platforms frequently malfunctioned before several candidates had even begun. Other examinees faced difficulties finishing and saving their essays, encountered sluggish screens and error messages, and found they couldn’t copy and paste text, according to reports from the Times.
reported
earlier.
According to a recent
presentation
By the State Bar, out of the 171 scored multiple-choice questions, 100 were created by Kaplan and 48 came from an examination for first-year law students. Additionally, a smaller group of 23 scored questions was produced by ACS Ventures, which acts as the State Bar’s psychometrician, and these were developed using artificial intelligence.
“We believe in the accuracy and fairness of the (multiple-choice questions) in effectively evaluating the legal competency of those taking the exam,” stated Leah Wilson, the Executive Director of the State Bar, to the newspaper.
Katie Moran, an associate professor at the University of San Francisco School of Law with expertise in bar exam prep, told the newspaper, “This is a shocking revelation.”
The State Bar acknowledged employing a firm to have someone who isn’t an attorney utilize artificial intelligence for crafting questions used during the official bar examination,” she stated. “Subsequently, this same company was compensated to evaluate and finally sanction the questions featured on the test, encompassing those which they had composed.
Andrew Perlman
, serves as the dean of Suffolk University Law School and acts as an advisor
council
A member of the American Bar Association Task Force on the Law and Artificial Intelligence mentioned that they hadn’t come across AI being utilized for crafting bar examination questions or establishing regulations overseeing these applications.
However, he noted that he wasn’t shocked considering the swift advancement of AI technology. Perlman mentioned that AI could prove helpful in crafting evaluation questions, yet a crucial safeguard is ensuring all outputs from an AI tool are thoroughly reviewed by specialists in the relevant field.
He anticipates its usage will keep expanding.
Even though there may currently be public doubt regarding new technologies in law, “in the future we’ll worry about the effectiveness of attorneys who do not utilize these tools,” Perlman forecasted.
