California Bar discloses AI was used to develop questions in Exam
By AP Staff
A photo taken on November 27, 2024 shows the logo of the ChatGPT application developed by US artificial intelligence research organization OpenAI on a smartphone screen (L) and the letters AI on a laptop screen in Frankfurt am Main, western Germany. (Photo by Kirill KUDRYAVTSEV / AFP) (Photo by KIRILL KUDRYAVTSEV/AFP via Getty Images)
LOS ANGELES (AP) — The State Bar of California has disclosed that some multiple-choice questions in a problem-plagued bar exam were developed with the aid of artificial intelligence.
The legal licensing body said in a news release Monday that it will ask the California Supreme Court to adjust test scores for those who took its February bar exam.
“The debacle that was the February 2025 bar exam is worse than we imagined,” Mary Basick, assistant dean of academic skills at the University of California, Irvine, Law School, told the Los Angeles Times. “I’m almost speechless. Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable.”
In February, the new exam led to complaints after many test-takers were unable to complete their bar exams. The online testing platforms repeatedly crashed before some applicants even started. Others struggled to finish and save essays, experienced screen lags and error messages and could not copy and paste text, the Times reported earlier.
According to a recent presentation by the State Bar, 100 of the 171 scored multiple-choice questions were made by Kaplan and 48 were drawn from a first-year law students exam. A smaller subset of 23 scored questions were made by ACS Ventures, the State Bar’s psychometrician, and developed with artificial intelligence.
“We have confidence in the validity of the (multiple-choice questions) to accurately and fairly assess the legal competence of test-takers,” Leah Wilson, the State Bar’s executive director, told the newspaper in a statement.
Katie Moran, an associate professor at the University of San Francisco School of Law who specializes in bar exam preparation, told the newspaper, “It’s a staggering admission.”
“The State Bar has admitted they employed a company to have a non-lawyer use AI to draft questions that were given on the actual bar exam,” she said. “They then paid that same company to assess and ultimately approve of the questions on the exam, including the questions the company authored.”
Andrew Perlman, dean of Suffolk University Law School and an advisory council member of the American Bar Association Task Force on the Law and Artificial Intelligence, said he had not heard of AI being used to develop bar exam questions or standards being put in place governing such uses.
But he said he was not surprised, given the rapid growth of AI technology. Perlman said AI can be useful for developing questions for assessment, but a critical guard rail is making sure that everything that comes from an AI tool is vetted carefully by experts in the subject matter.
He expects its use to continue to grow.
Although there might be public skepticism of the emerging technology in the legal profession at this time, “we will be worried in the future about the competence of lawyers who don’t use these tools,” Perlman predicted.
Universal Music Group and AI platform Udio have settled a copyright lawsuit and will collaborate on a new music creation and streaming platform. The companies announced on Wednesday that they reached a compensatory legal settlement and new licensing agreements. These agreements aim to provide more revenue opportunities for Universal's artists and songwriters. The rise of AI song generation tools like Udio has disrupted the music streaming industry, leading to accusations from record labels. This deal marks the first since Universal and others sued Udio and Suno last year. Financial terms of the settlement weren't disclosed.
Microsoft says users of its Azure cloud portal may be not be able to access Office 365, Minecraft or other services due to issues with its global content delivery network services. The tech company posted a note to its Azure status page that its teams are currently deploying a fix to address the outage.
Nvidia on Wednesday became the first public company to reach a market capitalization of $5 trillion. The ravenous appetite for the Silicon Valley company’s chips is the main reason that the company’s stock price has increased so rapidly since early 2023.
A new poll finds that as the United States rapidly builds massive data centers for the development of artificial intelligence, many Americans are concerned about the environmental impact.
Brain.fm merges music and neuroscience to enhance focus, creativity, and mental health—Dr. Kevin Woods reveals how sound is transforming cognitive performance.
An internet outage on Monday morning highlights the reliance on Amazon's cloud services. This incident reveals vulnerabilities in the concentrated system. Cloud computing allows companies to rent Amazon's infrastructure instead of building their own. Amazon leads the market, followed by Google and Microsoft. The outage originated in Northern Virginia, the biggest and oldest cloud hub in the U.S. This region handles significantly more data than other hubs. Despite the idea of spreading workloads, many rely on this single hub. The demand for computing power, especially for AI, is driving a construction boom for data centers.
Ashley Fieglein Johnson, CFO & President at Planet, joins us to share the story behind the Owl launch—and how strategy, tech, and vision are fueling liftoff.
OpenAI has announced that ChatGPT will soon engage in "erotica for verified adults." CEO Sam Altman says the company aims to allow more user freedom for adults while setting limits for teens. OpenAI isn't the first to explore sexualized AI, but previous attempts have faced legal and societal challenges. Altman believes OpenAI isn't the "moral police" and wants to differentiate content similar to how Hollywood differentiates R-rated movies. This move could help OpenAI, which is losing money, turn a profit. However, experts express concerns about the impact on real-world relationships and the potential for misuse.
