Dr Matthew Adiletta, & Profs David Brooks & Gu-Yeon Wei
Pistil: Chiplet-Based Architecture for Edge Language Model Inference

Project Overview
Personal AI is moving onto phones, laptops and embedded devices, but today's edge hardware isn't built for fast, conversational reasoning. The bottleneck is scalable memory bandwidth rather than raw compute, and general-purpose platforms waste power and latency on the most important part of the workload. Pistil is a chiplet-based accelerator specialized for the decode stage of small language model (SLM) inference, and a fabricated prototype delivers substantially higher token throughput and energy efficiency than conventional edge platforms. The team is now developing a next-generation prototype, demonstrating multi-package scale-out for larger SLMs, and co-designing a higher-efficiency memory solution, with the aim of spinning out a startup.
Interested in this technology? Send a note to grid@harvard.edu.




Comments