Lila Sciences and Anyscale Train GLM-5.3-Flash on 1 Million Tokens

Lila Sciences and Anyscale trained GLM-5.3-Flash on sequences of up to 1 million tokens using 64 NVIDIA B200 GPUs, extending its reach for scientific datasets such as human genomes. The run used SkyRL, an open-source training stack, and marked a 64-fold increase over the initial training capacity. At 1 million tokens, throughput was 13,700 tokens per second and peak memory use reached 144.5 GiB per GPU. Throughput peaked at 24,900 tokens per second at 256,000 tokens. A separate fine-tuning test used human-genome sequences of 512,000 tokens.

Source: TokenPost