Sarah Yager, OpenAI's first head of human rights and responsible deployment, said the questions raised by military adoption of artificial intelligence weigh on her constantly—and that the technology's potential use in targeting and autonomous weapons is what worries her most.

“Everyone is very worried and rightly so,” Yager said on stage Tuesday at the ScaleUp:AI conference in New York City. “This keeps me at night: ‘What is the military going to do with AI when it comes to targeting? And what about autonomous weapons? And where is all of this going?’”

Yager took on the newly created role in June, bringing roughly 30 years of human rights experience to a company whose partnership with the Department of War (DoW)—the rebranded Department of Defense—has placed it at the center of the U.S. military's push into AI. Her brief focuses squarely on military applications.

Militaries around the world are moving quickly to fold AI into their operations as they develop new weapons and autonomous systems such as drones and robots. From inside one of the most powerful AI companies in the world, Yager now grapples with the difficult questions this shift poses.

“I have always wanted to go into places and persuade people where human rights was not necessarily the dominant idea,” she said. “With apologies to my OpenAI colleagues, I am now in the belly of the beast, and it's actually a very comfortable place for me to be because what I love to do is persuade and discuss and debate and hopefully move the needle.”

An unlikely hire, after a contested contract

OpenAI approached Yager for the job shortly after the company signed a contract with the DoW in late February to deploy advanced AI systems in classified environments. The high-profile deal followed a public falling out between the DoW and Anthropic. Anthropic had sought guardrails on military use of its technology—specifically bans on mass public surveillance and fully autonomous weapons—that the DoW would not accept. The department then labeled Anthropic a supply-chain risk, and OpenAI stepped in to take the contract, claiming it had persuaded the DoW to agree to similar guardrails on surveillance and autonomous weapons. OpenAI described the agreement in a company blog post.

For Yager, the idea of working at OpenAI was not “an immediate yes.” She feared, as the on-stage interviewer put it, that she might be hired as a “branding exercise”—a figurehead who could make the company look responsible without actual influence over the technology. Her due diligence involved talking with employees, particularly “the younger generation who are very worried about what they are creating and who's at the wheel.” That convinced her to take the position and see whether she could make a difference.

Real risks and real influence

Since joining, Yager has had some success influencing the direction of the technology, alongside continuing concerns about where it is headed. The stakes were underscored last month, when CNN reported that the U.S. military nearly targeted a Chinese military ship based on an inaccurate, AI-generated intelligence report—a near-miss that landed squarely on the targeting questions she said worry her most.

Yager has worked directly with researchers and engineers to improve the models, including making sure they give appropriate responses to foundational international agreements such as the laws of war established in 1949 at the post–World War II Geneva Conventions. She brought in international experts “to literally sit at keyboards and put in prompts for really high stakes engagements,” then had engineers tweak the models based on the responses.

“That had never been done before, so that was extremely gratifying,” Yager said.

In her view, AI companies are trying to “reinvent the wheel” on questions such as defining humanity and morality that societies settled long ago. “Societies have already figured out a lot of those things, and it's called human rights. That should be the baseline,” she said, referring to the Geneva Conventions agreements. “I do not want an AI company telling me what is good. If we allow the industry to determine what is good, then it's a free-for-all.”

'Get back to the human of human rights'

Yager also expressed unease with a running Silicon Valley debate over whether AI systems—and particularly the new agent technologies—should have rights. OpenAI posed the question during her interview process. She answered “no” at the time and reiterated that her focus was human rights, though she has since thought more about it. She now feels more open to the “extremely important” conversations about “ agents fit into the legal structures, the labor structures, the tax structures,” while noting that these questions are not a matter of rights.

“This is not at all comparable to human rights, and I would really love if we could just get back to the human of human rights,” she said. The crowd—mostly suited businessmen—began to clap. “Big round of applause for that,” the on-stage interviewer said.

Who builds AI matters

Another of Yager's concerns is the relatively insular demographic makeup of the groups building AI. Last week, President Trump gathered dozens of tech industry leaders in Washington to discuss AI safety. All the signatories of the voluntary safety accord were men, and photos of the broader group present at a lunch event showed that all of the attendees were men as well, save for AMD CEO Lisa Su.

“Men are awesome, but it was a very male photograph of who is going to be making the choices about AI,” Yager said. “One problem with that is that AI is fundamentally shifting the power balance between people all over the world. You did not see affected communities up there.”

Yager noted that the government itself is rapidly adopting AI. She is helping oversight bodies, such as the Government Accountability Office and the Office of Inspector General, build tools to keep tabs on what it is doing—an accountability mechanism she said “feels important for democracy.” In doing so, her role stretches beyond shaping OpenAI's own models to helping federal watchdogs audit the government's own adoption of AI.

This story was originally featured on Fortune.com.