sgl-project/sglang 发布 v0.5.20

Highlights

713 PRs from 237 contributors.

New models in this release (see the cookbook for all supported models):

Model	Type	PRs	Cookbook
GLM-5.3-Flash	Autoregressive	#36507, #38621	link
Hy4-Preview	Autoregressive	#36805	link
Qwen3.8-Flash-Next	Autoregressive	#37500	link
K2 Horizon	Autoregressive	#37654, #38033	link
Nanbeige4.2	Autoregressive	#32151
SenseNova-U1.5-8B-MoT	Diffusion	#36606	link
FastH3 (4-step MiniMax-H3 distill)	Diffusion	#37480	link
VDN-H3 (hybrid-attention MiniMax-H3 distill)	Diffusion	#37903	link

Sampling masks for RL rollouts. With return_sampling_mask, each decode step returns the exact token support the sampler drew from and the log-probability of the sampled token under it, so a trainer can replay the rollout without reconstructing top-k or top-p (#36630). Masks now run under overlap scheduling: on Qwen3-8B, decode throughput is 17% higher at batch 1 and 52% higher at batch 64 than the previous implementation. Capacity is set by --sampling-mask-max-tokens (default 4096) (#36631). DisallowedTokensLogitsProcessor is supported alongside masks (#38279).

Unified radix tree. Branching-point caching for the SWA component keeps the sliding-window state at the point where requests fork from a shared prefix, so branches reuse it instead of recomputing. On DeepSeek-V4-Flash with a shared system prompt, token hit rate rises from 43.8% to 60.8% and mean TTFT falls from 1.57 s to 1.07 s (#34565). An opt-in external linker lets the tree address a shared global memory pool through Mooncake or UMBP (#37381).

DSpark under PD with decode context parallelism. A DCP1 prefill can now transfer its DSpark draft KV to a DCP-N decode, so hybrid models such as Kimi-Linear run DSpark in disaggregated, context-parallel serving. Verified on 8x B300 over NIXL and Mooncake up to 256K input (#37709).

Responses API storage is opt-in. /v1/responses no longer retains results in memory unless the server starts with --enable-response-store. Without it, retrieval, previous_response_id chaining, and background requests return 400; PD deployments cannot enable it (#39122).

SGLang Simulator. A CPU-only simulator runs the real scheduler, radix cache, and hierarchical cache with a latency predictor in place of the model forward. Against measured serving traces it predicts TTFT within about 6% on most traces (up to 10% on the longest 32K to 128K ones) and prefix reuse within 0.05 percentage points, for cache and scheduling studies without GPUs (#33824).

Prefill context parallelism v1 removed. The strategy-based implementation is now the only prefill CP path; the v1 runtime and its CLI options are gone. Prefill CP on HIP, NPU, and MUSA is rejected until those platforms are ported (#36228).

Faster model loading on ROCm. Large pageable host-to-device copies are staged rather than pinned in place, which stops the driver from suspending GPU queues on every eviction. GLM-5.2 at TP4 on 4x MI355X loads in 40.4 s instead of 505.7 s (#37720).

Intel XPU joins the release images. Every tagged release now builds and publishes lmsysorg/sglang:vX.Y.Z-xpu from the XPU Dockerfile, so Intel GPU users get a versioned image instead of relying on nightly builds (#37340).

DeepSeek-V4 on Blackwell. TRT-LLM attention kernels now cover DeepSeek-V4's CSA and HCA layers on SM100 and SM103: about 1.2x faster prefill and 1.45x faster decode than FlashMLA at the kernel level on B200 (#30805). FlashInfer MegaMoE is available as --moe-runner-backend flashinfer_megamoe; on DeepSeek-V4-Flash NVFP4 at TP4/DP4 it adds up to 11.9% prefill throughput at 8192 tokens per rank, with decode within 2% of the trtllm runner at saturation (#31470).

DeepSeek-V4 on RTX PRO 6000. On SM120 the sparse-MLA indexer now runs on DeepGEMM's paged-MQA kernel and the DeepGEMM FP4 MoE backend is enabled, replacing the torch fallback that was the only working path. On 4x RTX PRO 6000, DeepSeek-V4-Flash decode TPOT drops from 36.1 to 10.5 ms at batch 1 and TTFT falls 20% from 8K to 128K input. Opt-in through the environment flags in the PR (#29927).

Dependencies and images. The CUDA 12 lane is retired; v0.5.19 was the last release with -cu12x wheels and images (#38404). sglang-kernel moves to 0.4.7 (#39346) and sgl-deep-gemm to 0.2.0 (#39371). New images: ROCm 10 for MI30x and MI35x with a matching kernel wheel (#38763), gfx1151 for Strix Halo / Ryzen AI MAX+ (#33939), and Moore Threads MUSA (#36709). ROCm 7.0 CI, images, and kernel wheel are retired (#38632, #38767).

Full release notes by category below; breaking changes are at the end.

Speculative Decoding

• [Speculative Decoding] Add native UNO serving support: #37667
• feat: add optimized Domino rollout to DFlash V2: #36899
• feat: support TP>1 Domino rollout for DFlash V2: #37069
• [Spec] Stage Inkling MTP draft metadata before verify: #38169
• [Spec] Allow speculative workers to stage prefill shared reads: #38554
• [Spec] Support large MTP batches in short-convolution metadata: #38558
• [EAGLE] Prune draft-extend logits to selected rows (+2.5% output tokens/s/GPU at 102K context per rank): #35546
• [KDA] Support ReplaySSM ring-write in the fused chain-verify kernel: #36821
• [GDN] Amortize ReplaySSM checkpoint materialization: #35544
• perf(gdn): select ReplaySSM verify loop unrolling by shape: #36970
• Allow custom policy for adaptive speculative decoding: #37274
• Support speculative decoding with unified SWA memory: #36403
• Improve CUDA graph and speculative execution output handling: #37329
• Fix DSpark CUDA graph replay with MegaMoE TP attention: #34919
• [Fix][Mamba] Clear deferred init metadata before speculative decode: #37165
• [Fix] Track DFlash Mamba state at checkpoint boundaries: #37818
• [Fix] Load Qwen3.5 MTP embedding under PP: #37471
• fix: stop shadowing the DSpark shared-experts fusion guard: #39366
• [Profiler] Label draft-runner steps DRAFT and target verify VERIFY in step spans: #38630

Piecewise & Breakable CUDA Graph

• [Memory] Size the CUDA graph pool from warmup measurements and fix graph-pool borrowing: #36911
• [Memory] Retire graph borrow pool before updating static runs: #37966
• [Memory] Reuse output storage across full prefill CUDA graphs: #38038
• [Unified Memory] Enable prefill cuda-graph capture: #37418
• Decouple ragged CUDA graph request and token capacities: #37300
• Reuse live CUDA graph executables during dedup registration: #39176
• Scope graph-pool borrowing to the runtime and reduce fragmentation: #39177
• Expose a capacity check for graph-pool borrows: #39178
• Keep graph-pool borrows on their allocation stream: #39180
• Add a provider hook for prefill-buffer ceilings: #39182
• [Refactor] Generalize attention graph variants in the decode runner: #38993
• [Fix] Coordinate FullCG prefix variants across DP ranks: #37888
• [Fix] Keep a shared MAX_LEN prefill CUDA graph bucket when the graph captures a DP gather (MegaMoE sparse-DP hang): #37933
• [Fix] Key CUDA graph dedup signatures on kernel function identity: #37657
• [Bug] Include DSA variant in exact-bucket graph admission: #38992
• Fix KV-canary workspace accounting after graph capture: #38596

Attention Backends

• [DSv4] Integrate TRT-LLM DSv4 Attention for SM100/103 (kernel-level prefill about 1.2x and decode about 1.45x faster than FlashMLA on B200): #30805
• [DSV4] Support raw-index output in TopK v2: #33672
• [DSV4] hc-prenorm: fuse the combine step into a Triton kernel: #35118
• [FlashInfer] Avoid D2H sync for sliding-window lengths (TPOT 5.4 to 5.1 ms on 8x A800): #32218
• [Attention] Size FlashInfer MLA indptr buffers to the padded max batch: #38590
• [Perf] trtllm_mla: reuse the fused fp8 KV/Q prepare on target verify: #39232
• [Performance] Optimize Qwen3.5 GDN prefill projection layouts: #36267
• Fix Qwen3.5 GDN multi-item scoring (+18.8% throughput, P50 latency -16%): #33922
• Support NoPE layers in the tokenspeed_mla FP8 prefill hook: #38152
• fix(qsa): make the paged sparse-decode gather memory-safe (zero-fill scratch, int64 offsets, dequant FP8 on gather): #38851
• fix(qsa): dequantize FP8 cached prefixes in the sparse prefill kernels: #38855
• Reland fix(qsa): clamp the compress gather to the rows (#38346): #39446
• [Qwen 3.8 Next] reuse old cuda stream instead of endlessly creating streams: #39474
• [Fix][DSA] Bound prefill Triton specializations for page-table stride: #37093
• [Fix] Wait for PDL before reading DeepSeek V4 K cache locations: #38409
• [Fix] Fix Llama 4 FA3 local attention with paged KV cache: #32902

MoE & Expert Parallelism

• [NVIDIA] Support flashinfer Mega Moe (prefill input throughput up to +11.9% vs the trtllm runner on DeepSeek-V4-Flash NVFP4 at 8192 tokens per rank): #31470
• [MoE Refactor] Migrate SM100 trtllm-gen mxfp4 MoE onto MoeRunner: #32405
• [Deepseek V4] Keep fp32 routing weights in the mxfp4 trtllm MoE: #33608
• Keep fp32 routing weights in the fp8 block-scale and bf16 trtllm MoE: #33631
• Drop the routing bias casts in flashinfer trtllm MoE: #33591
• Fix DeepSeek-V4 routing: sqrtsoftplus underflow and unfloored renorm: #34459
• keeping router GEMM in fp32 for deterministic inference (DeepSeek V3/V4): #38176
• Fuse Nemotron latent MoE projection and shared add: #30430
• [Kernel] Replace dsv3_router_gemm with the unified tiny GEMM: #34693
• [Kernel] Avoid zero-bias allocation in fused softmax routing: #36811
• [Kimi-K3] Accept fp32 routing weights in the fused MoE finalize: #38612
• Cast fp32 routing weights to bf16 in the Kimi-K3 fused finalize: #38588
• Use fp32 in TRTLLM all reduce buffers: #36143
• [Comm] Drop the in-tree MNNVL CuTe DSL port in favor of FlashInfer 0.6.18: #37206
• [Moe] Fix flashinfer_trtllm silently dropping swiglu_limit clamped SwiGLU activation (GLM-5.3-Flash FP8 GSM8K 0.928 to 0.936, matching the Triton runner): #39920
• Fix MegaMoE buffer allocation and caching for effective SM budgets: #39223
• Fix native MoE handling of noncontiguous top-k IDs: #36407
• [Fix] Accept int64 top-k IDs in FlashInfer routed MoE packer: #32882
• fix(moe): cast filtered-activation expert_ids to int32 for torch.compile: #38085
• fix(gpt-oss): avoid duplicate MoE reduction with DP attention: #37199
• Fix mooncake scale joiner groups: #38503

Quantization

• [Quant][ue8m0 fix] group requant_weight_ue8m0 reduce reserved gpu memory: #31755
• [1/N] Quantization Refactor: remove dead code and dedup the FP4 marlin helpers: #37552
• Delete cutlass_mla, non-Marlin GPTQ, AWQ AOT kernel, and Dual Chunk Flash Attention: #32114
• fix(modelopt_fp4): skip NVFP4 swiglu-fusion interleave for shared experts with swiglu_limit: #37378
• Disable Hopper GLM shared-expert fusion for modelopt_fp4 Marlin: #37325
• [Fix] Preserve FP32 in SM107 MXFP8 fallback: #37489
• Fix block-scale swizzling device placement: #37849
• Keep NVFP4 blockscale swizzle padding on the input device: #39141
• Fix dummy initialization of inverse weight scales: #35491
• [Chore] harden checkpoint quantization metadata parsing: #36922

Parallelism & Disaggregation

• [PD] Transfer the DCP-replicated DSPARK draft KV in DCP1->DCP-N relayouts: #37709
• [Qwen3.8-Next] Add PD state transfer for Flash Next: #36651
• [Intra-node PD][DSV4] Pack all layers into one batch for INTRA_NODE_NVLINK path (KV transfer latency 178 to 119 ms, E2E latency -20% on a 2P1D B200 node): #38984
• [PD] Share the prefill->decode failure notification across backends: #36612
• [PD] Gate deferred decode KV release on backend capability: #37454
• [PD] Bound transfer engine init with SGLANG_DISAGGREGATION_ENGINE_INIT_TIMEOUT: #37874
• [PD] Preserve the prefill rank during rebootstrap: #39357
• [PD] Optimize paged allocator free-list release: #37146
• [PD][LoRA] Gate decode admission on adapter slots: #39332
• [Unified Memory] PD disaggregation for every unified pool shape: #37506
• fix(disagg): poll receivers during decode preallocation: #37483
• Fix DSA compression tail capacity for PD decode request slots: #38417
• fix(glm-5.2-nvfp4): bound Mooncake synchronous transfer batches: #32758
• [DCP] Allow fi_a2a on single-node systems Blackwell without MNNVL fabric ( ex B200 B300) (median TPOT 1.5 to 5.5% lower than the NCCL all-to-all on 8x B200): #37767
• [DCP] Resolve --dcp-comm-backend to fi_a2a/a2a by default for every model: #39165
• [Feature] Unified memory: support decode context parallelism for Kimi-Linear: #36890
• [Feature] Unified memory: support decode context parallelism for the trtllm_mla family: #37693
• [Perf] Unified memory: close the DCP decode gap on Blackwell: #37926
• [Fix] Merge adjacent KV-row frees so a mid-page split under DCP cannot double-free: #38941
• [Fix][DCP] Localize widened KV ids in MLA retraction CPU backup/restore: #39487
• [CP V1 Deprecation 2/5] Make strategy prefill CP canonical: #36223
• [CP V1 Deprecation 3/5] Remove generic prefill CP v1 runtime: #36228
• [CP V1 Deprecation 3.5/5] Deprecate HIP/NPU/MUSA prefill CP and remove legacy implementation: #38293
• [CP V1 Deprecation 4/5] Canonicalize prefill CP API names: #36229
• [CP V1 Deprecation 5/5] Update prefill CP documentation: #36230
• fix: gather CP-sharded tokens before TP-sharded dense MLP under prefill CP: #38078
• Fix inflated row pitch when a CP round-robin shard has a single row: #34142
• [SP] Make attention-TP sequence sharding a per-forward batch property: #37546
• [Fix] Stamp sequence-parallel state on dummy forward batches: #38564
• [DP Attn] Fix crash for no token all-gather case: #39899
• [Fix] Disable NCCL graph buffer registration for the TP LM-head all-to-all (pure-DP decode hang under request bursts): #38936
• [Misc] Extract PP dynamic chunk sizing into a DynamicChunkSizer scheduler component: #37674
• [Fix] Lock PP dynamic-chunk profiling requests before releasing through the tree cache: #37509
• [Fix] Apply the attention-CP broadcast result in PP dynamic-chunk profiling: #37669
• [Fix] Broadcast PP dynamic-chunk profiling failures so every rank disables together: #37675

Scheduler & Runtime

• [Scheduler] Add HRRN schedule policy to significantly reduce TTFT (--schedule-policy hrrn; mean TTFT -69% and p99 -8% vs FCFS on a GLM-5.2 production trace): #32911
• [Scheduler] Make request-timeout aborts rank-consistent to fix TP collective hangs: #37143
• [Scheduler] Count the parked chunked-prefill request in the busy mem check: #37502
• [Scheduler] Unify per-iteration request intake into ingest_requests(): #38389
• Fail fast on undersized swa pool: #37610
• Reduce tokenizer overhead and offload CUDA VMM publication: #37330
• Keep VMM capability votes on CPU: #38722
• state_capturer: pin the exact host-cache size via mmap + cudaHostRegister: #37285
• Generalize auxiliary outputs: #39164
• [Model Loader] Split weight loading from postprocessing: #34981
• Fix gpt-oss RunAI streamer weight ownership: #38908
• Fix RunAI object-storage checkpoint index filtering: #38988
• Clean logging under --weight-loader-prefetch-checkpoints: #33930
• [Fix] Release NCCL on scheduler exit and let the ASGI server own shutdown: #39560
• [Fix] Aggregate all IPC weight update responses: #39534
• [Fix] Avoid scanning crash-dump token buffers during GC: #37297
• add reindex_device_id to device OOT plugin: #36220
• [Metrics] Add rolling scheduler utilization counters: #37461
• [Metrics] Export scheduler stage wall time: #37636
• [Metrics] Report logical prefill token counts: #38566
• [Observability] Fix missing e2e/decode/inference latency span attributes: #37789
• [Ray] Support Ray metric backend for engine metrics: #31415
• [Profiler] Add SGLANG_PROFILE_BY_STAGE_DECODE_MIN_BS to defer the decode-stage capture: #38067
• Add num_prealloc_ready_tokens to decode load snapshot: #38065
• [Bench] Support real-traffic replay with early-stop-aware steady-state metrics in bench_one_batch_server: #37469
• [Benchmark] Add an opt-out for the token-capacity check: #39284
• Fix reasoning metrics and add TPOT to bench_multiturn: #35443

Sampling

• [Sampling] Capture masks from sampler support: #36630
• [Sampling] Support sampling masks with overlap scheduling (Qwen3-8B decode throughput +17% at batch 1 and +52% at batch 64 vs the previous non-overlap path): #36631
• [Sampling] Allow sampling-mask replay with DisallowedTokensLogitsProcessor: #38279
• perf: use Gumbel-max trick in the main sampler to cut decode CPU dispatch (Qwen3.5-2B decode +75% at batch 1 and +23% at batch 32): #38117
• Fix custom logit processor params when num_tokens_in_batch is used: #38730

HiCache & Radix Cache

• [Unified Tree] Support Branching-Point Caching for the SWA Component (shared-prefix token hit rate 43.8% to 60.8%, mean TTFT 1.57 to 1.07 s on DeepSeek-V4-Flash): #34565
• [Unified Tree] Port SWA Branching-Point Caching to the Rust TreeCore: #37584
• [Unified Tree] fix: make dfs weight ordering iterative: #38313
• [Unified Tree] Preserve aux LRU recency when splitting nodes: #38482
• [Unified Cache][5/N]: Integrate external linker mode end to end: #37381
• [Unified Cache][6/N]: Add UMBP external linker: #37578
• [Unified Cache][7/N] Support MTP, EAGLE, and DSpark draft KV caches in the external linker: #37914
• [Unified Cache][AMD] Support DeepSeek-V4 unified KV in direct external linkers: #38269
• [Rust TreeCore] Harden runtime and CI parity: #37303
• [Rust TreeCore] Support external cache linker: #37306
• [Rust] Rename mem-cache to sglang-radix-tree: #37290
• [KV Shard 1/4] Logical-page placement with UnifiedRadixCache: #38356
• [Cache] Forward fast prefix matching capability: #37844
• [Refactor] Let eviction policies take construction parameters: #37795
• [Perf] Walk the radix tree by offset instead of re-slicing token storage (ported from #36507): #37324
• [Perf] Vectorize alloc_extend_naive to remove the per-request Python loop: #37938
• [Radix Cache] Fix PureSWA tail release without insertion: #38349
• fix: preserve SWA host lock on node split: #38138
• fix: collect prefix hash values iteratively: #38204
• fix: align write-through pending across tree cores: #37278
• [HiCache] Add MLA host-dedup primitives: #36800
• [HiCache] Replace skip_lock_node_ids with a segment lock protocol: #36848
• [HiCache] Buffer mode support sidecar pool: #37424
• [HiCache] buffer mode: anchor-lock staged prefetches by default: #37464
• [HiCache] L3 storage prefetch lifecycle metrics and cross-tier attribution fixes: #37503
• [HiCache] Count hit allocations and in-flight backups in the buffer pipeline idle check: #37883
• [HiCache] Fix side pools to use resolved host allocator: #38350
• [HiCache] Account for newly pinned ancestors in load-back quota: #38481
• [HiCache] Release buffer prefetch anchor locks during storage cleanup: #38483
• [HiCache] Publish a host store event for storage-prefetch refills: #38486
• [HiCache][LoRA] Isolate storage pages by extra key: #38577
• [HiCache][LoRA] Simplify decode offload hash inputs: #39162
• [HiCache] fix: preserve SWA host lock boundaries across splits: #38835
• [HiCache] Keep the file backend temp file name within NAME_MAX: #38925
• [HiCache] Rework the buffer-mode storage prefetch pipeline and retry bookkeeping: #39283
• [HiCache] Optimize buffer-mode storage existence bookkeeping: #39480
• HiCache: Add @rank_consensus to various functions: #37425
• HiCache: Reduce the number of all_reduce in check_hicache_events for PP (TTFT -7% on DeepSeek-V4-Flash with HiCache L3, PP4 TP2): #37562
• Fix buffer-mode idle tracking and VLM memory sizing: #37567
• fix: keep queued Mooncake linker loads after abort: #38195
• fix(nixl): make FILE path-mode devId globally unique: #34362
• Scope prefetch cache state to the request attempt: #39318
• [Fix] HiCache startup ImportError on the pinned kernel wheel: #39516
• [Mem Cache] Make free_swa sync-free on page_size == 1: #36723
• [Mem Cache] Drop the torch.unique sync from the SWA page expansion: #37463
• [Mem Cache] Split duplicate insert frees at the SWA eviction floor: #37481
• [Mem Cache] Require page-aligned starts in free_segment and drop the boundary trim: #37729
• [Mem Cache] Route hybrid SWA full-side kv-row frees through free_segment: #37876
• [Mem Cache] Move the unified-memory allocators into allocator/ and split the composites out: #38072
• [Mem Cache] Free hybrid SWA pages by one representative per page on page_size > 1: #38159
• [Mem Cache] skip duplicates host evict via environ: #38462
• [Misc] Resolve SWA ownership at enqueue time for grouped free(): #36646
• Fix unified SWA: size a non-owner's v2p by the id space it must address: #37560
• Size the unified read-table grid from bs, and fuse the allocator's tombstone scatters: #37511
• Build the unified read stream directly, without the page-table rectangle (decode CUDA graph memory 2.89 to 1.89 GB on Kimi-Linear-48B TP2): #37512
• fix(unified-memory): evict Full KV for Mamba byte shortfalls: #36713
• Store mamba prefix-cache checkpoints at the configured SSM state dtype: #34820
• Fix mamba radix cache ssm state indexing: #37836
• [Fix] Vacuous marker writes in the cache tests, and an undebited Mamba admission slot: #36415

LoRA

• [LoRA] Support MoE in full and breakable prefill CUDA graphs (prefill throughput +39.5 to 40.9% with Triton LoRA on Qwen breakable graphs, GB300): #38578
• [Intel][XPU][LoRA] Enable LoRA on Intel XPU: #30345

Multimodal

• [Perf] Optimize Qwen3-VL unique-image serving on H100: #36411
• fix(multimodal): handle tensor images in exact-token preprocessing: #30368
• Fix multimodal embedding cache retaining full batches through views: #39120
• [Fix][Qwen-VL] Normalize <image> sentinel on artifact fast path: #39278
• fix(glm4v): disambiguate mixed image video offsets: #37971
• [Fix] Alpha-channel images and tool-result media ordering (port of #36507): #37320
• [MM] Add flag to force Kimi image preprocessing onto CPU: #39148
• [VLM] Retire aborted disaggregated prefill results: #36988
• fix(vlm): contain EPD request lifecycle failures: #36944
• fix(vlm): harden EPD receiver validation and liveness: #36945
• fix(vlm): make EPD cache publication transactional: #36949

Model Support & Optimizations

• [Cosmos3] Add cosmos3 Reasoner to llm only inference: #33572
• Support NemotronH_Omni_Reasoning_V3 in SGLang: #35599
• Inference Support Mamba 2 and 1 (Mamba-Codestral-7B, Falcon-Mamba-7B, and the state-spaces Mamba checkpoints): #34556
• [GLM-5.3 Flash] Restore and enable KPool metadata fusion: #38845
• MiniMax-M3: share the sparse index top-k across layers and reuse the decode top-k buffer (output throughput +6.4 to +20.6% at 80K input on 8x MI350X): #36527
• MiniMax-M3: Triton split-K router GEMV with in-kernel fixup: #36557
• perf(lfm2): fuse gating and short convolution on SM90: #37622
• [GraniteMoE] Load split per-expert quantized MoE weights: #37679
• Fix Muse Glimmer ModelOpt mixed weight mapping: #37510
• [Fix] Fix Spark2.5 hybrid SWA config: #37335
• fix(mamba): unify causal_conv1d col* dtype to x (MiniCPM-V-4.6 GDN prefill bf16/fp16 mismatch): #38039
• [Refactor] Clarify DeepSeek V4 metadata names for V4.1: #38947
• [Refactor] Generalize DeepSeek V4 compressed pool management: #38954

Kernel Library

• [Kernel] GLM 5.3 Flash related kernels (ported from #36507): #37477
• [Kernel] Register merged diffusion agent kernels with KDA backend: #37385
• [Kernel] Add fused MoE Triton configs for Qwen3.8-Flash-Next FP8 on NVIDIA H200 NVL (TP2+EP2) (1.14 to 1.23x at decode-sized batches): #38116
• [Kernel] Add tuned LFM2.5 Triton MoE configs on B300: #37158
• [Kernel] Drop the vendored dense BF16 GEMM port in favor of FlashInfer 0.6.18: #38124
• [Kernel] Add OOT dispatch for clamp position: #38687
• [JIT] Port the expert-pack MXFP4 kernels to load_jit and fix their launch limits: #38830
• feat(kernels): generalize persistent CuTe JIT cache: #33911
• Add SGLANG_CRASH_ON_JIT_COMPILE to forbid on-the-fly JIT compilation: #36615

Configuration System

• [Config] Round 6.1: "unset" gets its own spelling, and the declaration says what it means: #38046
• [Config] Round 6.2: the field declarations move to their namespaces, and the record is assembled from them: #38047
• [Config] Round 6.3: the record remembers how it was asked for, and is sealed while resolution runs: #38048
• [Config] Round 6.4: the runtime reads the bags, not the record: #38049
• [Config] Round 6.5: a namespace declares what it derives, next to what it derives it from: #38113
• [Config] Retire get_global_server_args, and clear the deprecated flags that have a replacement: #38375
• [Config] One writer for the declaration stash; no exception to the write seal: #38752
• [Config] msgspec.Struct for the config tier: #38753
• config: an out-of-tree replacement point for every resolution-pipeline step: #39134
• Fix dataclasses.asdict on the msgspec ServerArgs: #38958

OpenAI-Compatible API

• [Feat][Responses API] Support custom tools, encrypted reasoning replay, developer tier and model validation: #38690
• [PD][OpenAI] Gate /v1/responses persistence behind --enable-response-store, default off: #39122
• [OpenAI] Propagate PD routing metadata through /v1/responses: #35503
• [Responses] Fix empty-prompt routing for token-first chat encoders (kimi_k3, inkling): #35486
• [Feature] Add response-level input/output token ids to chat completions via SglExt: #34488
• Support non-strict GLM47 tool calls with EBNF constraints: #38890
• [Fix] Preserve GLM tool argument types across JSON Schema unions: #39136
• Auto-detect GLM-5.3 chat templates as glm45/glm47 parsers: #38297
• Force reasoning mode for GLM-5.3 chat templates: #39227
• Add granite_thinking_parser reasoning parser for Granite 4.2: #38693
• [Kimi K3] Rework skipped-think fix as opt-in force_nonempty_content with streaming coverage: #34187
• [Kimi-K3] Recover the reply when the model skips the think channel: #37743
• [Session] Fix session idle timeout after rejected requests: #39035
• [Session] Work with PD and Fix empty continuations: #39038
• [Session] Fix image append positions and parent metadata: #39145
• [Session + MM] Fix text positions in session continuations: #39144
• feat(agent sessions): attribute stored KV cache blocks to sessions: #37482
• Fix /model_info serialization when a config value is a class: #39237
• Fix: abort handling for dispatched requests after client disconnect: #35255

Rust Server

• [Router] Add composable scoring and eligibility policies: #37731
• [Router] Add load-aware prefill admission and bounded policy proposals: #37843
• [Router] Add bucket-aware policy domains and native cache indexing: #38108
• [Router] Publish cache-aware load state: #38139
• [Router] Preserve global cache affinity with bucket routing (TTFT p50 -18.5% vs power-of-two bucket routing): #38814
• [Router] Honor KV-event storage tiers in the cache-aware tree (1/4): #39108
• [Router] Expose the KV storage-tier stream and tree occupancy on /metrics (2/4): #39109
• [Router] Stream outcome observability for 2xx SSE streams: #38737
• [PD] Add /v1/responses support to the HTTP PD router: #36141
• [Rust] Gate health on startup warmup completion: #37994
• [Rust] Use Dynamo native renderers when chat templates are missing: #38939
• [Rust Server] Use node-local HTTP ports for DP attention: #34430
• [Rust Server] fix p/d bootstrap across dp listeners: #36234
• Rust server: align launcher and request validation behavior: #37327
• Allow CUDA VMM feature transport with the Rust frontend: #39347
• [gRPC] Stream engine state changes: #39915
• [gRPC] Expose native pause status: #37488
• [Fix] Seed raw tokenizer_path for smg-grpc-servicer in gRPC mode: #39105
• [SGL Router] Prepare dynamo-render dependencies: #39457

Simulator

A CPU-only inference simulator under tools/sglang-simulator that reuses the real scheduler and cache stack with a latency predictor in place of the model forward.

• [Simulator] Add high-fidelity CPU-based inference simulator: #33824
• [Simulator] Give the OFFLINE/BLOCKING comparison tolerances real headroom: #38732

SGLang-Diffusion