Will New CA Law Regulating Lawyer & Arbitrator AI Use Solve GenAI Problems?

By Leslie King O’Neal

Hallucinations, Deep Fakes, Synthetic Evidence, Oh My![i]  

New CA Law First to Regulate Attorneys’ & Arbitrators’ GenAI Use

California is the first state to enact a law regulating lawyers’ and arbitrators’ generative AI use. But will it solve the generative AI problems in law practice and arbitration or create new ones? On September 30, 2026, California Governor Gavin Newsom signed SB 574[ii], the first U.S. law regulating lawyers’ and arbitrators’ generative AI use. The new law prohibits attorneys from “delegating the practice of law to generative AI.[iii]

“Practice of Law” Not Defined

 However, the bill does not define what constitutes “the practice of law,” creating an obvious issue for future litigation. What is the line between using GenAI to assist a lawyer in reviewing a contract, drafting a memorandum, developing a deposition question outline and delegating the practice of law to GenAI? What level of human lawyer input and involvement is needed to comply with the statute?

 Attorneys Cannot Enter Confidential Information into Open AI Systems

The new California law also prohibits attorneys from entering confidential, personal identifying information[iv] and other nonpublic information into generative AI systems which do not restrict access to the information to the attorney or authorized persons obligated to protect the information’s confidentiality. The law specifies “attorneys” only, creating another source of potential disputes. What if paralegals or interns input confidential information into non-secure GenAI platforms? Presumably the supervising attorney would be responsible for any breach of client confidentiality.

Attorneys Must Verify Gen AI Outputs’ Accuracy

Under the new law, attorneys must take reasonable steps to: (1) verify the accuracy of GenAI outputs, including, but not limited to, all case and statutory citations and (2) correct any erroneous or hallucinated output in any material used. Finally, attorneys must disclose GenAI use to the court for all documents submitted and should consider disclosing GenAI use for content provided to the public.

Attorneys Must Vet All Citations in Court Filings

The CA Code of Civil Procedure is amended to provide: “A brief, pleading, motion or any other paper filed in any court shall not contain any citations that an attorney responsible for submitting the pleading has not personally vetted, including any citation provided by generative artificial intelligence.”[v] The statute does not include expert reports put into evidence, which could also contain factual or other hallucinations.[vi] Should attorneys be responsible for those also? Should this obligation extend to any documents submitted into evidence or by affidavit to support a claim or a motion? In Kohls v. Ellison[vi] an expert (in AI, ironically) filed a report containing hallucinated citations. The court did not sanction the attorneys but noted their non-delegable duty to validate the truth of papers filed under Rule 11. [vii]

Court May Impose Sanctions for Violations

Under CA Code of Civil Procedure §128.7(c), the court may sanction the court may sanction the attorneys, law firms, or parties who violated or who are responsible for violating the vetting requirement. In considering what sanctions, if any, to impose, the court shall consider whether the party moving for sanctions acted with due diligence.

Arbitrators May Not Delegate Decision-Making to GenAI

The law also provides that “an arbitrator shall not delegate any part of their decision-making process to any generative AI tool.” Further, it prohibits arbitrators from relying on GenAI information outside the record without making appropriate disclosures to the parties beforehand and allowing the parties to comment on its use.[ix] Recently a Canadian court annulled an arbitration award because the arbitrator used GenAI in drafting (without advising the parties) and the award relied on hallucinated cases and authorities. See prior post:

Canadian Court Annuls Arbitration Award for Arbitrator’s Improper AI Use

What is an Arbitrator’s “Decision-Making Process?”

The statute does not define the arbitrator’s “decision-making process,” creating another area for disputes. Some arbitrators may say that everything about reviewing the evidence and legal arguments presented is part of the “decision-making” process and, therefore, GenAI use is prohibited. Others may assert that only the amount of the dollar award or non-monetary relief given is “decision-making.”  Without some guidance, arbitrators are left adrift in determining what, if any, GenAI use is allowed.

Takeaways

• California’s new law attempts to regulate attorneys’ and arbitrators’ GenAI use, but its lack of definitions may create additional disputes and frustrate its intent.
• As the law surrounding GenAI develops, attorneys and arbitrators in California and elsewhere must be cautious in how they and their staffs (and experts) use GenAI tools.

---

[i] Reminiscent of the famous “Lions and Tigers and Bears, Oh My!” clip from “The Wizard of Oz” starring Judy Garland, Ray Bolger, Jack Haley and Bert Lahr, (MGM Studios, 1939) https://youtu.be/31j4YJzvwFI?si=L0SovkAH7LVB3kFo

[ii] Titled “Attorneys, arbitrators, judicial officers, and alternative resolution providers,” the Act amends Section 6173 of, and adds Section 6068.1 to, the Business & Professions Code. It also amends Section 128.7 of and adds Sections 180 and 1282.1 to, the CA Code of Civil Procedure. https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB574

[iii] “Generative artificial intelligence” (GenAI) is defined as an artificial intelligence system that can generated derived synthetic content, including text, images, video, and audio that emulates the structure and characteristics of the system’s training data.

[iv]“Personal identifying information” includes: driver’s license numbers; dates of birth; social security numbers; national crime information and criminal identification and information numbers; addresses and phone numbers of parties, victims, witnesses and court personnel; medical or psychiatric information; financial information; account numbers; any other content sealed by court order or deemed confidential by court rule or statute.

[v] New Section 1287(2)(A), CA Code of Civil Procedure.

[vii]  Kohls v. Ellison, (Order granting in part and denying in part  Plaintiffs’ Motion to Exclude Expert Testimony, (Case No: 0:2024cv03754, D. Minn., January 10, 2025); later appeal, affirming judgment on merits, 166 F.4th 728 (8th Cir. 2026) https://law.justia.com/cases/federal/district-courts/minnesota/mndce/0:2024cv03754/220348/46

[viii]Tyler J. Brewster, Egregious Errors in Expert Evidence: Ethical Oversight for Experts Who Use Generative AI, Louisiana L. Rev. (2025) https://lawreview.law.lsu.edu/archives/3332

[ix] New Section 1282.1, CA Code of Civil Procedure