New models

Aleph-Alpha/Kolibri-1 – 78.1B total / 3.46B active, up to 1M context, Apache 2.0 A new open-weight English-German Mixture-of-Experts reasoning model from the German lab Aleph Alpha, open-sourced Oct 3, 2026 (Germany’s Reunification Day). 78.1B total params, ~3.46B active per token, native context up to 1,048,576 tokens, weights on Hugging Face under Apache 2.0. Trained on ~20T tokens (mostly English ~62.5%, German ~23.9%, code ~13.6%). Notable as one of the largest sovereign open-weight releases from Europe. Why it matters to you: the 3.46B-active MoE makes it a strong candidate to replace GPT-OSS at roughly two-thirds the size for German/English workloads, and the 1M context is a big draw. Commenters note the model is “basiclly worse than Qwen3.6-35B at twice the size” (u/Training_Visual6159) and that it is “comparable to Qwen 3.5 35B” (u/SnooPaintings8639), so treat it as a strong first attempt rather than a clear winner. At ~78B it likely needs a 96 GB VRAM build to run comfortably. As of this digest there were no community quant releases (no GGUF/INT4/NVFP4 quants posted), and a commenter asked whether llama.cpp supports the architecture yet – so do not plan a deployment until a serving path and quants exist. https://llm.r.homelab.internal/r/LocalLLaMA/comments/1wwl7y6/alephalphakolibri1_hugging_face_78b_parameters/ Tech report: https://aleph-alpha.com/downloads/tech-report.pdf Model: https://huggingface.co/aleph-alpha/Kolibri-1

Qwen3.8 Flash-Next (~176B total, 6B active) – the recurring constrained-hardware test subject Not new today, but it is the model two of the five featured posts are benchmarking on 16 GB-class hardware. The MoE with 512 experts and a large n-gram/PLE table makes it unusually suited to tiered SSD offloading. See the “Most interesting posts” section for the two 16 GB data points.

bilibili “Index-Translate” – multilingual translation family on Qwen3.5 (lower signal) A small (15?) post announcing a multilingual translation model family based on Qwen3.5. Translation-focused, so limited relevance to your coding/agentic backend; flagging it only because it is a new Qwen3.5-based release. No quants or serving configs posted. https://llm.r.homelab.internal/r/LocalLLaMA/comments/1wxa1wr/bilibili_released_indextranslatea_a_multilingual/

Most interesting posts

1. https://llm.r.homelab.internal/r/LocalLLaMA/comments/1wwocik/two_300b_moe_models_each_on_one_128_gb_mini_pc/ u/Yaniss916 reports two ~300B MoE models, each on ONE 128 GB AMD Strix Halo (Ryzen AI Max+ 395, gfx1151) running a new ExLlamaV3/ROCm engine called Kyojin. Concrete numbers from the post: GLM-5.3-Flash (99.7 GB) hits ~580 tok/s prefill (at 3.5K), 546 at 64K, 26-30 tok/s decode (MTP); MiMo-V2.6-Flash-MOPD (105 GB) hits ~650 tok/s prefill at 4K and up to 44 tok/s decode (speculative, code). Quality: KLD vs official FP8 of 0.151 / 0.0713 and top-1 agreement 89.3% / 92.0% respectively. The GLM pack mixes turboderp’s public 2.05 and 3.05 bpw EXL3 tensors with a custom layer mix. Includes a quickstart command (clone, ./build.sh, hf download, python tools/glm/serve.py --model ./glm-pack -c 131072 --num-draft 2, OpenAI-style API). Reproducible on Strix Halo hardware only – the author explicitly lists as “not measured yet” any GPU other than gfx1151, and the conversion pipeline stays private. Highly relevant as a technique reference (EXL3 layer-mix quant + ROCm) even though the target ha rdware is AMD, not your NVIDIA stack. Resources: Engine https://github.com/Yamz-Labs/kyojin . Weights https://huggingface.co/yamz-labs . Base fork https://github.com/vcruz305/exllamav3-amd

2. https://llm.r.homelab.internal/r/LocalLLaMA/comments/1wwwmy1/running_qwen38_flash_next_176b_on_a_16gb_rtx_3080/ u/fuzhongkai runs Qwen3.8 Flash-Next 176B on an RTX 3080 Laptop (16 GB VRAM) + 32 GB RAM + SSD using his open-source engine TensorSharp, which does MoE-aware scheduling across cache, VRAM, system RAM, and SSD. Reported results: ~11.09 tok/s decode vs Strata’s 10.24, but end-to-end 16.54 s vs Strata’s 62.15 s on his test (peak GPU 14.8 GB, OS working set 19.74 GiB). Reproducibility caveat (flagged in comments): the post does not name the quantization, and top comments ask for exactly that (u/DigitalguyCH: “very vague without specifying the quantization”; u/wizard_of_menlo_park: “Quantization?”). u/leonbollerup also says he gets ~80 tok/s decode and ~2200 tok/s prefill on the same card with Strata, which contradicts the post’s headline – so treat the end-to-end speedup as single-source and unverified. This is directly relevant to whether a 24 GB 3090 can serve the 176B Flash-Next via SSD tiering, but verify the quant and benchmark method before trusting it. Resources: https://github.com/zhongkaifu/TensorSharp . Model docs https://github.com/zhongkaifu/TensorSharp/blob/main/docs/models/qwen38-flash-next.md

3. https://llm.r.homelab.internal/r/LocalLLaMA/comments/1wwv0fj/i_built_ninfer_4080_for_16gb_class_gpus/ u/roofkid built NInfer 4080 to run ISTA-DASLab Qwen3.8-27B-GSQ at 100k context on an RTX 4080 16 GB, claiming up to 2720 tok/s prefill and 262 tok/s generation. Part of the NInfer 5090/4090/3090 family of from-scratch engines. Reproducibility caveat: the post is a project announcement; the headline numbers (2720 pp / 262 gen) are the author’s max-observed, and a top comment (u/Pyrolistical) notes these custom engines often compromise with a quantized KV cache and that his own roofline analysis of llama.cpp on an R9700 found <5% decode improvement left on the table – so the gap between custom and general engines is hardware/kernel-specific, not universal. Comments are mostly “how do I get this on a 3080/5080/4070 Ti” requests, so portability to other cards is unconfirmed. Relevant to you primarily as another data point in the overfit-engine trend and for the ISTA-DASLab 3-bit GSQ quant on Qwen3.8-27B. Resources: https://github.com/roofkid/ninfer-4080 . Quant https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-3Bit-GSQ . ByteShape Qwen3.8-27B reference https://byteshape.com/blogs/Qwen3.8-27B/

4. https://llm.r.homelab.internal/r/LocalLLaMA/comments/1wwu6zj/the_rise_of_overfit_inference_engines/ u/carteakey’s discussion post on the overfit-engine trend (see topics above). 315?, 199 comments. No single reproducible config, but the most substantive community engineering observations of the day: SGLang’s current gaps (u/buttplugs4life4me), the Xeon-Max NUMA speed gap (u/TokenRingAI), and a practitioner who “vibe-coded” his own engine because upstream PRs were ignored (u/wishstudio). Worth reading for the state of general vs. narrow runtimes rather than as a runnable config.

5. https://llm.r.homelab.internal/r/LocalLLaMA/comments/1wwl7y6/alephalphakolibri1_hugging_face_78b_parameters/ The Kolibri-1 announcement (see New models above). 510?, 153 comments. Not a reproducible-serving post (no quants yet), but the most-discussed new model of the day and directly relevant to a European/sovereign model pipeline and as a GPT-OSS-sized MoE candidate.