NVIDIA Reports 42.5% Throughput Gain for Ling-3.0-flash-fp4 on DGX Spark
Photo: TokenPost
NVIDIA said three software changes lifted decode throughput for Inclusion AI's Ling-3.0-flash-fp4 by 42.5% on DGX Spark in a test covering all 1,319 GSM8K questions, while mean latency fell as well. Throughput increased to 53.89 tokens per second from 37.81, and mean latency dropped to 11.49 seconds from 15.21. The end-to-end test ran at batch size one. The model has 124 billion total parameters, with 5.1 billion active for each token. NVIDIA used a Humming MXFP4 mixture-of-experts backend, online FP8 quantization of the language-model head and NEXTN multi-token-prediction speculative decoding.
Source: TokenPost
