Artificial intelligence has quietly worked its way into the drafting of American legislation, and the people responsible for getting bills right say the result is a growing mess.

According to Politico, congressional staffers and outside advocacy groups are increasingly turning to chatbots such as Claude, made by Anthropic, and ChatGPT, made by OpenAI, to produce legislative text — forcing the House Office of Legislative Counsel (OLC), the nonpartisan office that helps House members draft bills, to clean up after them.

"AI is good for a lot of stuff," said Daniel Schuman, executive director of the American Governance Institute. "But it's not capable of drafting legislation that you would want to enact into law."

One person who advises congressional offices said the OLC now spends "more time trying to fix AI-drafted legislation than it would take for them to draft it from scratch," noting that AI models reportedly stumble over fine legal distinctions — such as whether funding should be structured as "a tax credit, deduction, exclusion or grant."

The precision problem is compounded by a well-documented limitation of the technology: chatbots assemble fluent text rather than drawing on verified legal sources, and they are known to invent cases, citations and provisions — a failure mode known as hallucination. Federal courts ran into the issue within months of ChatGPT's public release in late 2022: in 2023, a federal judge in New York sanctioned lawyers who filed a brief citing six court decisions that ChatGPT had invented.

A congressional staffer tasked with doing a deep read of proposed legislation said lawmakers have no idea what is coming down the road because of errors and citations that will not hold up to scrutiny when conflicts arise.

"It's absolutely terrifying, and no one is thinking about it yet," the staffer said.

The stakes are particular to the medium: statutes, unlike memos or speeches, become binding text once enacted, and their exact wording is what courts and federal agencies must interpret and enforce. Mistakes that survive into law generally require new legislation to correct, so the OLC's cleanup work is effectively the last checkpoint before drafting errors can become permanent.

The risks are not hypothetical, Politico reports. In June, the office of Rep. Anna Paulina Luna (R-FL) published an amendment summary that still contained the unedited chatbot phrase "Claude responded." Luna acknowledged on X that "my staff used AI to correct a draft text and didn't edit," a disclosure that raised alarms at the time.

Ari Hershowitz, a lawyer and technologist who has worked with the office to integrate tech tools into the legislative process, said that without checks, the practice is "a guaranteed way of introducing thousands of bugs into our legal system."