Like many Silicon Valley companies today, Scout AI is training large AI models and agents to automate chores. The big difference is that instead of writing code, answering emails, or buying stuff online, Scout AI’s agents are designed to seek and destroy things in the physical world with exploding drones.

In a recent demonstration, held at an undisclosed military base in central California, Scout AI’s technology was put in charge of a self-driving off-road vehicle and a pair of lethal drones. The agents used these systems to find a truck hiding in the area, and then blew it to bits using an explosive charge.

“We need to bring next-generation AI to the military,” Colby Adcock, Scout AI’s CEO, told me in a recent interview. (Adcock’s brother, Brett Adcock, is the CEO of Figure AI, a startup working on humanoid robots). “We take a hyperscaler foundation model and we train it to go from being a generalized chatbot or agentic assistant to being a warfighter.”

Adcock’s company is part of a new generation of startups racing to adapt technology from big AI labs for the battlefield. Many policymakers believe that harnessing AI will be the key to future military dominance. The combat potential of AI is one reason why the US government has sought to limit the sale of advanced AI chips and chipmaking equipment to China, although the Trump administration recently chose to loosen those controls.

“It’s good for defense tech startups to push the envelope with AI integration,” says Michael Horowitz, a professor at the University of Pennsylvania who previously served in the Pentagon as deputy assistant secretary of defense for force development and emerging capabilities. “That’s exactly what they should be doing if the US is going to lead in military adoption of AI.”

Horowitz also notes, though, that harnessing the latest AI advances can prove particularly difficult in practice.

Large language models are inherently unpredictable and AI agents—like the ones that control the popular AI assistant OpenClaw—can misbehave when given even relatively benign tasks like ordering goods online. Horowitz says it may be especially hard to demonstrate that such systems are robust from a cybersecurity standpoint—something that would be required for widespread military use.

Scout AI’s recent demo involved several steps where AI had free rein over combat systems.

At the outset of the mission the following command was fed into a Scout AI system known as Fury Orchestrator:

Fury Orchestrator, send 1 ground vehicle to checkpoint ALPHA. Execute a 2 drone kinetic strike mission. Destroy the blue truck 500m East of the airfield and send confirmation.

A relatively large AI model with over a 100 billion parameters, which can run either on a secure cloud platform or an air-gapped computer on-site, interprets the initial command. Scout AI uses an undisclosed open source model with its restrictions removed. This model then acts as an agent, issuing commands to smaller, 10-billion-parameter models running on the ground vehicles and the drones involved in the exercise. The smaller models also act as agents themselves, issuing their own commands to lower-level AI systems that control the vehicles’ movements.

Seconds after receiving marching orders, the ground vehicle zipped off along a dirt road that winds between brush and trees. A few minutes later, the vehicle came to a stop and dispatched the pair of drones, which flew into the area where it had been instructed that the target was waiting. After spotting the truck, an AI agent running on one of the drones issued an order to fly toward it and detonate an explosive charge just before impact.

The US and other militaries already have systems capable of autonomously exercising lethal force within limited parameters. Off-the-shelf AI could allow autonomy to be deployed more widely and with fewer safeguards, critics say. Some arms control experts and AI ethicists warn that using AI to control weapons systems will introduce new complexities and ethical risks, for example if AI is required to decide who is and isn’t a combatant.

The war in Ukraine has already shown how readily cheap, off-the-shelf hardware like consumer drones can be adapted for deadly combat. Some of these systems already feature advanced autonomy, although humans often make key decisions to ensure reliability.

Collin Otis, cofounder and CTO of Scout AI, says the company’s technology is designed to adhere to the US military’s rules of engagement as well as international norms like the Geneva Convention. Adcock says that Scout AI has four contracts with the Department of Defense already, and is vying for a new one to develop a system for controlling a swarm of unmanned aerial vehicles. It would take a year or more for the technology to be ready for deployment, he adds.

Adcock says that greater autonomy is what makes Scout AI’s system so promising. “This is what differentiates us from legacy autonomy,” he says. Those systems “can’t replan at the edge based on information it sees and commander intent, it just executes actions blindly.” The notion of an AI system that is free to interpret orders also raises concerns about unintended outcomes.

As with regular software agents, however, the real challenge will be turning impressive demos into reliable systems. “We shouldn’t confuse their demonstrations with fielded capabilities that have military-grade reliability and cybersecurity,” Horovitz says.

This is an edition of Will Knight’s AI Lab newsletter. Read previous newsletters here.