LLM
Speculative Decoding: How LLMs Guess Ahead to Run Faster
Speculative decoding lets large language models generate text faster by using a smaller draft model to predict tokens ahead, then verifying them in parallel. Here's how this inference optimization technique works under the hood.