NewsMacroNew York City Council unveils AI regulation bills requiring outside validation and kill switches while vowing to remain the 'AI capital of the world'

New York City Council unveils AI regulation bills requiring outside validation and kill switches while vowing to remain the 'AI capital of the world'

Author: Fortune Crypto·

Key Takeaways

  • •The broadest bill would bar businesses from selling or deploying AI systems in New York City without third-party validation and a kill switch, with violations carrying a $25,000 fine for each instance.
  • •A separate measure would grant whistleblowers a portion of fines recovered from AI lawbreakers, which the council describes as the first such provision in the United States.
  • •The October 5 hearing will be the city's first Committee of the Whole since 2022, and sources indicate none of the five invited tech CEOs are likely to testify, though the council retains subpoena power.
  • •Major AI companies are deeply established in New York, with Google employing more than 14,000 people there, Meta leasing 1.2 million square feet at 50 Hudson Yards, and Anthropic taking an entire 16-story building.
  • •Federal developments, including a bipartisan Senate bill that could override state AI laws and a Justice Department task force suing states, will shape how much of the city's package can ultimately take effect.
New York City Council unveils AI regulation bills requiring outside validation and kill switches while vowing to remain the 'AI capital of the world'

Congress has spent nearly a decade holding hearings on artificial intelligence while passing almost nothing to make it safer. Now the New York City Council wants to find out whether a city can accomplish what Washington has not.

On Friday, City Council Speaker Julie Menin unveiled a package of bills that would require AI systems sold in the city to pass outside validation and carry a kill switch, pay whistleblowers who report violations, and allow New Yorkers to sue AI companies when jailbroken tools cause them harm. The legislation will be heard on October 5 at a Committee of the Whole hearing of all 51 council members. Unlike the stalled federal effort, the council is claiming direct jurisdiction: the companies behind these systems sell their products in New York and maintain some of their largest offices there.

"We obviously are not in any way looking to stifle innovation. We want to ensure that New York stays the AI capital of the world, but with that comes responsibility," Menin told Fortune. "I don't believe these two ideas are mutually exclusive at all. In fact, I think they go hand in hand."

"You want to be the capital of innovation, and you want to be the capital of smart, sensible regulation and guardrails around it, because you want people to have faith in the product itself," she said.

A high-profile hearing

The council has invited Dario Amodei, Sam Altman, Sundar Pichai, Elon Musk and Mark Zuckerberg—the CEOs of Anthropic, OpenAI, Google, SpaceXAI and Meta, respectively—to testify. The October 5 hearing will be a Committee of the Whole, a format the City Council has not convened since 2022.

Sources with knowledge of the hearing told Fortune that none of the five requested executives are likely to appear, though the City Council reserves the right to subpoena them. The companies did not immediately respond to Fortune's requests for comment. Whether any of the five attend—or whether the council moves to compel their testimony—will be the first visible test of how much resistance the package meets.

Still, that does not rule out other leaders representing the companies, and Menin said the aim is for the public to gain trust in these institutions. "These kinds of regulations actually improve people's faith in these AI technologies," she said.

Cities taking the lead

For Menin, a regulatory attorney turned politician, the legislation is not meant to stay within the city limits. "I feel strongly as a regulatory attorney, and as someone who served as commissioner of the Department of Consumer and Worker Protection, that there absolutely is a leading role for New York City to play," she said.

She described the bills as a case study from a course she taught at Columbia University, "When Cities Take the Lead," which examined what cities should do "when the federal government is deregulating or failing to act."

"This is perhaps one of the greatest cases of that, where you have a product in AI that has very little regulation," Menin said. "In New York City we regulate all sorts of products and services, everything from barber shops to nail salons, and yet here we have a situation with AI where there is very little regulation."

A sparse federal record

The federal track record Menin is pushing against is long and mostly empty. The first federal AI bill, the FUTURE of AI Act, was introduced in 2017 and went nowhere. In 2023, OpenAI CEO Sam Altman asked the Senate to regulate his industry; then-Majority Leader Chuck Schumer responded with nine closed-door forums that produced a roadmap but no bill. By 2025, Altman was telling senators that requiring government approval to release AI would be a disaster.

When Washington finally did act, it moved to stop others from acting The Senate stripped Sen. Ted Cruz's proposed 10-year ban on state AI laws by a 99-1 vote in July 2025, and President Trump then created a Justice Department task force to sue states over their AI laws. Its first target, Colorado, gutted its own law five weeks after the suit was filed.

Even after Anthropic researcher Jacob Coxon resigned this month with a warning that AI companies were gambling with human lives, the federal response has consisted of letters and investigations. The most recent effort comes from a bipartisan safety bill from Cruz, Sen. Amy Klobuchar and Majority Leader John Thune, which would likely override most state laws, including New York's current RAISE Act. How that preemption fight is ultimately resolved will shape how much of the council's package can take effect.

In a July post, OpenAI argued that states could fill the gap by passing laws that mirror one another in place of a federal one. Yet it is in New York City, rather than in most of the states now writing AI laws, that these companies have put down roots. Google has more than 14,000 employees in the city, and Meta leases 1.2 million square feet at 50 Hudson Yards. Anthropic leased an entire 16-story building at 330 Hudson Street this summer and expects to have more than 1,000 city employees by year's end, while OpenAI took 90,000 square feet at the Puck Building in 2024.

"These companies have offices in New York. The product is being sold in New York, and we believe this falls into our domain to be able to regulate," Menin said.

What the bills would do

The broadest of the bills would bar any business from selling or deploying an AI system in the city unless an outside validator had checked it for data quality, bias, privacy and security under standards set by the city's Office of Cyber Command. Every system would also need a kill switch—a human override capable of shutting it down. Because that requirement applies to any business selling or deploying AI in the city, it would reach well beyond the major labs.

"This is not an industry that should self-regulate," Menin said. "Right now, the problem is that's basically what the standard is." A $25,000 fine would apply to each instance, which Menin said means that "if there's a swarm of agents, the penalty would apply per agent."

A separate bill would give whistleblowers a share of the fines recovered from AI companies that break the law, which the council believes would be the first measure of its kind in the nation. A companion measure would protect city employees and contractors who report AI threats to public safety.

A third bill would let people sue AI companies for harm caused by third parties who get around safety controls, provided the harm was foreseeable and the company lacked reasonable safeguards. Courts have not settled whether Section 230, the federal law that shields platforms from liability for users' content, covers AI-generated output, so the bill may also test federal law.

Other bills would require city contractors to report AI safety incidents within 24 hours, bar false or misleading safety claims, set privacy rules for chatbots based on the Electronic Privacy Information Center's People-First Chatbot Bill, and require the city to plan for AI-driven attacks on its systems. The council has said more bills are coming, including a ban on deceptive deepfakes and a study of how algorithmic tools change job duties. The October 5 hearing will be the first public checkpoint for how far that effort extends.

"Our bills deal with all sorts of different ways that we can address the safety issue," Menin said. "We want to continue that as a hub of innovation. The idea of regulation in any way hampering that, I don't believe in."

This story was originally featured on Fortune.