All pull requests

Pull requests

• fix(server): reject empty completions prompt and keep error code

  #850·techsaurabh-394 opened Nov 2, 2026·

  ·
  1

• fix(scheduler): clip max_tokens on a copy of the shared SamplingParams

  #849·techsaurabh-394 opened Nov 2, 2026·

  ·
  1

• fix(sampler): bound top_k to the vocabulary before the int32 cast

  #848·techsaurabh-394 opened Nov 2, 2026·

  ·
  1

• Feat/completions n

  #843·A5bhinav opened Oct 28, 2026Contributor·

  ·

• [PoC, DO NOT MERGE] feat(spec): add speculative decoding with MTP drafts

  spec decode
  #832·jason-fxz opened Oct 27, 2026Collaborator·

  ·

• feat(windows): expand Qwen and Gemma checkpoint support for OOB Huggingface uncensored models

  #831·trcwebdesign opened Oct 27, 2026·

  ·

• feat(models): add nvidia/Qwen3.8-Flash-Next-FP4-Local checkpoint support

  #820·chrisqianz opened Oct 24, 2026Contributor·

  ·
  1

• feat(server): report per-request prefill and decode timings in api responses

  #814·void-mckenzie opened Oct 23, 2026Contributor·

  ·
  1

  4

• fix(gguf): rename the ggml shim's type_info lookup, ambiguous under MSVC

  gguf
  #809·YevheniiKotyrlo opened Oct 23, 2026Contributor·

  ·

• perf(moe): fuse selection-biased sigmoid routing

  #802·rainhjrui opened Oct 22, 2026·

  ·
  1

• perf(ple): add an opt-in bounded disk row cache

  #792·wenlive opened Oct 22, 2026·

  ·
  1

• fix(sampling): bound top-k before int32 conversion

  #790·wenlive opened Oct 22, 2026·

  ·
  1

• fix(scheduler): preserve shared sampling parameters when clipping

  #788·wenlive opened Oct 22, 2026·

  ·
  1

• perf(attention): fuse short bf16 decode output gates on gfx1201

  amd
  #781·zihaomu opened Oct 21, 2026Contributor·

  ·
  1

• perf(kernel): add opt-in hip fp8 decode kernels for r9700

  amd
  #779·zihaomu opened Oct 21, 2026Contributor·

  ·

• test(models): cover the Ornith-1.5-122B NVFP4 export

  #777·chrisqianz opened Oct 21, 2026Contributor·

  ·
  1

• feat(mimo): add text-only MiMo v2.6 Flash support

  #761·jcmh0782 opened Oct 20, 2026·

  ·

• fix(server): opt-in backend recovery after runtime CUDA OOM

  #688·oscarlius opened Oct 14, 2026·

  ·

• fix(rocm): route moe_align around Triton 3.8 miscompile

  amd
  #687·thedandano opened Oct 14, 2026·

  ·

• perf(qwen4): add opt-in layer-major long prefill

  #686·oscarlius opened Oct 14, 2026·

  ·

• fix(moe): admit a wide offloaded decode step's experts 128 at a time

  #680·YevheniiKotyrlo opened Oct 13, 2026Contributor·

  ·

• feat(moe): load a short prefill chunk's experts on demand through the slot cache

  #676·YevheniiKotyrlo opened Oct 13, 2026Contributor·

  ·
  1

  2

• perf(moe): promote recurring hybrid misses into empty cache slots

  #674·taking-lying-flat opened Oct 13, 2026Contributor·

  ·

• feat(kvcache): --kv-placement host keeps the QSA K/V in pinned host memory

  #672·YevheniiKotyrlo opened Oct 13, 2026Contributor·

  ·

• test(moe): grant one bf16 ulp only at rounding ties in the ds_fp4 transcription check

  #668·void-mckenzie opened Oct 13, 2026Contributor·

  ·
  1