Posts by Nidhi Bhatia
Developer Tools & Techniques
Nov 17, 2026
Co-Designing Sparse Attention for Efficient Long-Context Inference
Agentic workloads, long documents, and large codebases are pushing context lengths from tens of thousands of tokens to hundreds of thousands. At those lengths,...
16 MIN READ
Developer Tools & Techniques
Sep 02, 2026
Co-Designing AI Models Using Speculative Decoding for Faster LLM Inference
This post is the third in a series on AI model co-design. It explores how to accelerate LLM inference while maintaining accuracy using speculative decoding and...
15 MIN READ
Developer Tools & Techniques
Jul 31, 2026
Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference
As agentic and long-context workloads become common, the context lengths increase and attention consumes a larger share of inference time (Figure 1). Because...
14 MIN READ
Agentic AI / Generative AI
Jul 10, 2026
AI Model Co-Design: Hardware-Friendly LLM Design
AI performance comes down to three dimensions: Accuracy: How well the model reasons and produces outputs Throughput: How many tokens per second a...
17 MIN READ
Developer Tools & Techniques
Jul 07, 2025
Think Smart and Ask an Encyclopedia-Sized Question: Multi-Million Token Real-Time Inference for 32X More Users
Modern AI applications increasingly rely on models that combine huge parameter counts with multi-million-token context windows. Whether it is AI agents...
8 MIN READ