Coordinating Adaptive Inference: From Executable Model States to Agent Trajectories
Foundation-model inference is undergoing two major shifts: it is expanding beyond cloud-centric deployment into heterogeneous edge, embodied, and personal settings, while also evolving from a single model call into iterative generation and agentic trajectories involving models, tools, memory, verification, and recovery. Together, these shifts increase both the diversity of resource constraints and the decision horizon of inference, making isolated optimizations increasingly insufficient. This talk presents a unified view of adaptive inference across three scopes: executable model states, generation schedules, and full agent trajectories. Drawing on our recent work, I will discuss how models can expose controllable execution states, how generation can allocate computation across tokens and steps, how agents can reshape future workloads through routing, decomposition, context management, and memory, and how evaluation must audit the evidence used to guide adaptation. The central thesis is that efficiency should be optimized over the complete, stateful path to a successful and verified outcome—not over a single model call in isolation.