My research is in machine learning systems. I design systems that adapt how compute and memory are allocated at runtime, so that AI workloads meet their performance goals at lower cost. My work so far has focused on LLM serving, and I am now extending it to compound AI systems, where many model and tool calls must be optimized end to end.
In long-context serving, the KV cache grows with every generated token and the batch keeps changing, so offloading plans fixed ahead of time cannot keep up with GPU memory demand. ORBITFLOW re-plans KV cache placement per request and per layer with a lightweight ILP solver and runtime feedback, raising SLO attainment for TPOT and TBT by up to 66% and 48% and throughput by up to 3.3× over existing offloading methods.
Modern data augmentation is expensive enough that CPU-side preprocessing, not the GPU, becomes the training bottleneck. FusionFlow offloads part of each batch's augmentation onto idle GPU cycles without disturbing training, speeding up single-machine training by 16–285% over CPU-only preprocessing with 50–60% fewer CPUs than scaling out.
