How DeepSeek’s DSpark Makes LLMs 85% Faster Without Retraining

DeepSeek has introduced DSpark, an open-source speculative decoding method that accelerates LLM inference by up to 85% ...

DeepSeek open sources DSpark, a new framework to speed up LLM inference by up to 85%

DSpark can make decoding faster, but acceptance quality still determines how much speed the system actually realizes.
Computing

DeepSeek claims new technique boosts LLM serving efficiency by up to 85%

DeepSeek has documented a new inference acceleration framework that it claims increases the efficiency of how LLMs are run. The company, which this month become China’s most valuable AI startup, says ...
Tech Times

NVIDIA’s New LLM Decodes 6x More Tokens Without an Auxiliary Draft Model

NVIDIA Nemotron-Labs-Diffusion is a new tri-mode language model that eliminates the separate draft model in speculative ...
Hosted on MSN

Speculative decoding made my local LLM actually usable

Local LLMs have this annoying middle ground problem. They're good enough that you can see the potential, but just slow enough to get in the way. You really feel the ...
EurekAlert!

SPECTRA: Towards a new framework that accelerates large language model inference

This figure shows an overview of SPECTRA and compares its functionality with other training-free state-of-the-art approaches across a range of applications. SPECTRA comprises two main modules, namely ...
Geeky Gadgets

Google’s Secret Weapon to Make AI Faster and Smarter Revealed

Have you ever been frustrated by how long it takes for AI systems to generate responses, especially when you’re relying on them for real-time tasks? As large language models (LLMs) become integral to ...
Semiconductor Engineering

Microarchitecture Tailored to 3D-Stacked Near-Memory Processing LLM Decoding (U. of Edinburgh, Peking U., Cambridge et al.)

A new technical paper, “Rethinking Compute Substrates for 3D-Stacked Near-Memory LLM Decoding: Microarchitecture-Scheduling Co-Design,” was published by researchers at University of Edinburgh, Peking ...
Tech Times

Diffusion LLM Learns to Be Its Own Draft Model: NVIDIA Releases Tri-Mode Open Weights

Nemotron-Labs-Diffusion, NVIDIA’s new tri-mode language model, eliminates the separate draft model in speculative decoding: ...
Semiconductor Engineering

Arithmetic Intensity In Decoding: A Hardware-Efficient Perspective (Princeton University)

“LLM decoding is bottlenecked for large batches and long contexts by loading the key-value (KV) cache from high-bandwidth memory, which inflates per-token latency, while the sequential nature of ...