We've spent the last quarter putting reasoning models on hardware that didn't exist as 'AI hardware' six months ago. Tactical vehicles, ground stations, drone control nodes: all carry compute, all carry GPU silicon for ISR pipelines, and none of them were designed for transformer inference. The constraint isn't the model anymore. It's the integration.
Quantization is the unlock. INT4 and dynamic FP8 bring a 7B-parameter reasoning model (Llama, Qwen, Granite, take your pick) into 4–6 GB of memory. On a current-gen tactical GPU we measure 35–55 tokens per second on a reasoning-style workload, which is fast enough for operator-pace question answering and slow enough that you don't burn the battery in a five-minute window.
Distillation works for the narrow tasks. We train smaller models on the specific decision surfaces an operator actually needs (recognize this radar signature, classify this comms anomaly, suggest the next standard operating procedure), and the distilled models often match the parent on the in-domain workload at 10× the speed. The bet is that you compose narrow distilled agents at the edge, not one wide generalist.
Where it falls apart: anything that needs the network. Reasoning models at the edge are useful precisely because they don't assume connectivity. The moment you architect a workflow that calls back to a centralized model for hard cases, you've reintroduced the failure mode the tactical edge was supposed to escape. Plan for fully disconnected behavior or don't deploy.
The operational pattern we're settling on: model artifact ships as part of the build, gets cryptographically pinned to a known-good version, and runs inside the existing accreditation boundary of the host system. No external model registry, no over-the-air weight updates without a new ATO. That's a step backward from how commercial AI ships, and it's the right step backward for the mission.