NVIDIA Details TensorRT Tuner’s 1.14x to 1.63x Performance Across 10 Workloads

NVIDIA detailed Friday, Oct. 23, how its TensorRT Global Performance Tuner searches alternative engine build routes and selects the fastest candidate that stays within an application’s accuracy limit. Across 10 workloads, the fastest qualifying route delivered between 1.14x and 1.63x the default route’s end-to-end performance. Each workload’s default route was normalized to 1.0x, making the figures relative measurements rather than absolute latency or throughput. A build route combines values for compiler knobs, settings that influence how TensorRT builds neural-network inference engines for NVIDIA GPUs. TensorRT 11.1 offers more than 200 tunable compiler knobs.

Source: TokenPost