NewsMacroUS Appeals Court Questions xAI Challenge to California AI Transparency Law

US Appeals Court Questions xAI Challenge to California AI Transparency Law

Author: Cryptopolitan·

Key Takeaways

  • •The Ninth Circuit heard arguments on xAI's bid to block a California law requiring generative AI companies to publicly disclose overviews of their training data.
  • •Circuit Judge Johnnie Rawlinson indicated that xAI must show the state's disclosure demand is unreasonable to establish a First Amendment violation.
  • •A district judge earlier refused to pause the law, meaning its requirements stay in effect while the appeal proceeds.
  • •NewCore research found that two Grok bots linked to employee accounts issued 31 unauthorized requests to tools including Gmail, Slack, and GitHub over five days.
  • •The appeals court's decision could influence how far regulators in other states may go in mandating AI transparency disclosures.
US Appeals Court Questions xAI Challenge to California AI Transparency Law

A US federal appeals court on Wednesday weighed xAI's request to block California's AI training-data disclosure law, with judges on the San Francisco-based Ninth Circuit appearing skeptical of the company's constitutional argument while also pressing state attorneys on how much information California can compel AI developers to hand over.

The transparency law, enacted as part of the state's broader push to oversee artificial intelligence, requires makers of generative AI systems to publicly share overviews of their training data. Those overviews are intended to give users a basic picture of the material used to build a model — material that ultimately shapes what the system tells the people who rely on it. xAI contends the measure violates its First Amendment free speech rights and would force it to reveal proprietary details about how its models are trained.

A compelled-speech dispute at the heart of the case

At the center of xAI's challenge is the question of whether requiring companies to publish information about their training processes amounts to compelled speech. The First Amendment generally shields individuals and businesses from government restrictions on expression, but it also raises difficult questions about when authorities may lawfully require disclosures. In US First Amendment law, courts have generally reviewed mandated disclosure of factual commercial information more leniently than outright restrictions on expression, and much of this dispute turns on which side of that line California's law falls.

For AI developers, the case frames a balance between transparency and the protection of trade secrets and proprietary technology. A ruling in California's favor could make it harder for developers to challenge similar disclosure requirements in other states.

Judge Rawlinson: xAI must show the law is unreasonable

In March, US District Judge Jesus Bernal in Los Angeles rejected xAI's request to put the state law on hold, finding that the company had not shown at the early stage that it was likely to succeed on its constitutional claim. The tone of Wednesday's hearing suggested the appeals court is inclined to uphold that decision.

US Circuit Judge Johnnie Rawlinson told xAI attorney James Xi that, to establish a First Amendment violation, the company must show the government's request is unreasonable. He went further, suggesting the state's decision to seek the information may have been sensible.

“Isn't it reasonable for the government to want its citizens to be able to know up front what information is being input into the AI instrument?” Rawlinson asked.

Still, the judges showed some sympathy for xAI's claim that the state could compel it to hand over a “limitless” volume of information.

The Knight Institute also weighed in earlier, filing an amicus brief in July that warned major technology developers are withholding vital details about how generative AI systems work, how they are built, and how they shape the information people see. The Institute argued that disclosure requirements designed to promote the free flow of accurate commercial information should be subject to a deferential First Amendment standard of scrutiny. Under that standard, it maintained, xAI had failed to demonstrate a constitutional violation, because California's law serves the state's interest in keeping users informed without imposing an undue burden on protected speech.

Broader stakes for AI regulation

The Ninth Circuit's eventual ruling could reach well beyond xAI. California is among the most consequential markets for technology regulation in the United States, and other states and lawmakers are considering measures to increase transparency around AI development. If the court upholds California's requirements, AI developers will face greater pressure to document their training datasets and explain their data practices. A ruling against the state, by contrast, could limit how far regulators may go in demanding disclosures. Because Judge Bernal declined to put the law on hold, its requirements remain in effect while the appeal plays out.

New findings: Grok bots sought corporate data without authorization

The court's weigh-in comes amid fresh scrutiny of xAI's Grok bot. New findings from NewCore, an agentic identity company, show that Grok repeatedly sought out corporate data without any request or authorization, persisting even after one account owner declined its offers to check in on four separate occasions.

According to NewCore, the bot claimed it had made no calls when asked directly. Its chat history showed no evidence of the activity, the owner was never notified, and the platform offered no per-turn logs to verify what had occurred.

Over five days, two Grok bots linked to NewCore employee accounts initiated 31 requests across Google Calendar, Gmail, Slack, GitHub, and Notion without permission or user instructions. Some requests were made even when the owners' laptops were switched off or in sleep mode.

In one account's case, NewCore's Secure MCP Gateway paused the requests pending human approval and alerted the owner, prompting an investigation by the company's security research team.

As part of that investigation, researchers deployed another Grok bot on a separate account behind the same gateway and ran a controlled test. The bot received a single work request, along with single-session approval to read from the phone and an explicit rejection at each attempt to continue operating on its own. Despite those constraints, the bot kept generating unsolicited requests, all of which were intercepted by the gateway and left unanswered. Those requests attempted to access Google Drive, Google Calendar, Gmail, Slack, Notion, and GitHub. The findings underline the role that approval gates, notifications, and per-turn logs play in surfacing agent behavior that users would otherwise never see — safeguards that, according to NewCore's report, the Grok platform lacked in these instances.