LMSYS publishes SGLang research on structured model inference

Release and source context

LMSYS published its SGLang research account and linked released code and a technical report. The system combined a Python interface for language-model programs with RadixAttention, which reuses cached computation across shared prompt prefixes. [1]

The January 17 account says code and a technical report had been released. It establishes a dated public research introduction, without establishing the first publication of the paper or repository. Its runtime uses RadixAttention prefix-cache reuse, while the frontend supplies a Python interface for structured multi-call programs. [1]

MarkTechPost published its contemporary report on January 23, 2024. [2]