@magnitudedev/cli@0.2.172026-11-08
Patch Changes
- a68074a Thanks @thrgreenwald! - Faster CPU inference on ARM. On ARM CPUs with the dot-product instructions, such as AWS Graviton and Ampere, Q8_0 models use new kernels for prompt processing and generation, and Linux ARM64 builds now detect these instructions at run time.
- Prompt processing is 4–13× faster and generation 1.4–5× faster than 0.2.16. For example, Qwen3.5 4B Q4 on Graviton4 goes from 3.8 to 45 tokens/s for prompts and from 3.0 to 14.7 tokens/s for generation.
- a68074a Thanks @thrgreenwald! - - Removing one version of a catalog model no longer deletes the image file shared by another installed version, which left that version unable to read images.
- Concurrent requests to a model that drafts with its own MTP heads no longer fail with "a causal head in a drafting batch has no selection".
- A dropped connection while a download is starting is now retried instead of failing the download, and a failed download logs the reason.
- When a newer release comes out after an update was downloaded, Magnitude offers the newer one and replaces the older download.
- The macOS app icon has its tighter rounded corners back.
- a68074a Thanks @thrgreenwald! - See what a model's memory is made of: weights, context and overhead.
- In the app, the info icon next to a model's memory on its card, and next to Fit on the Discover radar, shows the breakdown.
- magnitude catalog show, magnitude models status and their --json output include the breakdown. magnitude hardware shows a loaded model's weights, context and overhead separately, counted the same way as in Settings.
- a68074a Thanks @thrgreenwald! - More accurate memory estimates on NVIDIA and AMD GPUs, so models fit when they should and don't run out of memory when loaded.
- On CUDA, the estimate counts only memory the allocator can't reuse instead of a fixed 1 GB allowance. Small GPUs fit more: models on an 8 GB card that were limited to 32K now get up to their full 128K context.
- Mixture-of-experts models no longer use gigabytes more GPU memory than their estimate on CUDA and Vulkan.
- On CUDA, memory the driver reserves for running kernels is now counted, so it can't push a loaded model past what was estimated.
- a68074a Thanks @thrgreenwald! - Run models your computer can't hold at full context. Each model is now served at the largest context that fits in memory, from 32K up to its full length, instead of being refused when its full context doesn't fit.
- Discover recommends the highest-quality version that reaches 64K context, falling back to 32K.
- Chat, the API (/v1/models) and connected agents such as Claude Code, Codex and OpenCode use the served context. Claude Code compacts before reaching it instead of assuming 200K. A prompt longer than the served context is refused with contextlengthexceeded.
- A model's memory breakdown shows the context it's served at when the full context doesn't fit, and models status explains why a model doesn't fit.
- A model waits for its memory check before loading, so it never loads at a context the computer can't hold.
@magnitudedev/cli@0.2.162026-11-03
0.2.16 修复了共享提示前缀的并发对话触发 "target state capacity changed after availability check"、导致模型被卸载的问题,现在共享同一前缀的请求会合并推进,不再互相拖垮。性能提升明显:8 路并发共用 15K token 提示,A10G 上从每路 226 秒降到 3.5 秒;前缀缓存批次结束后仍保留,重复请求也更快。同时新增官方 Docker 镜像 magnitudedev/magnitude,模型与密钥存在 /data 卷,支持 NVIDIA、AMD、Intel GPU 及 Compose、Kubernetes、Podman。模型管理方面,删除已下载模型不再中断正在使用的模型,Linux 下软件 Vulkan 设备也不会再被误判为 GPU。量化支持扩容:Q10、IQ1S、IQ1M、IQ2S、TQ10 可加载运行,Q3K 体积从 6.56 降到 3.44 bit/权重,CUDA 解码提速 13–23%,IQ3_XXS 快 2.5 倍;另补上 IPv6 连接支持。
@magnitudedev/cli@0.2.152026-10-30
推理引擎的 Anthropic 兼容消息接口与 token 计数接口放宽了版本校验:anthropic-version 头可为任意值,不传也行,不再强制要求 2023-06-01,此前因此被拒的客户端现在可以直接接入。
另外修复了 Metal 上模型加载的一个内存问题(#231):流式读取权重表时会超出预留的导入内存。现在导入批次共用同一份内存预算计划,并把映射文件区间与并发暂存上传计算在内,权重导入不再撑爆预留空间。
@magnitudedev/cli@0.2.142026-10-29
0.2.14:模型目录新增两种排序——「最快」依据各模型在本机 50K 上下文下的预估速度,「最聪明」依据智能评分,缺少估算或评分的模型排在末尾。CPU 上量化 GGUF 模型解码提速,Q4 版 Qwen3.5 4B 在 M4 Pro 上单 token 耗时减少约 13%。推理引擎下载容错大幅加强:分段重试直至 10 分钟无数据才放弃,重启后从断点续传,失败时服务持续重试,magnitude status 可查看错误与下次重试时间,启动也不再因下载超过五分钟而中止。修复 OpenCode 2 无法携带 Magnitude 模型启动的问题——它已移除 --model 并共享后台服务,现改为检测已装版本、单独启动服务并在配置中写入模型,OpenCode 1 照旧。另修复 Qwen3.8 Flash Next 在 Apple 芯片上加载报错 "prepared 105 bytes but the ordered program plan charged 102"(#227):内核内存改为推导计算,两套计数不再冲突。
@magnitudedev/cli@0.2.132026-10-28
0.2.13:修复了 Discover 页面与模型卡片启动后每隔几秒就闪回「评估中」状态的问题,状态显示恢复稳定。模型目录新增 LFM3-1.7B 与 LFM3-VL-2B,提供 Q4 QAT、Q6、Q8 三种量化版本,其中 LFM3-VL-2B 支持图像输入。
@magnitudedev/cli@0.2.122026-10-27
应用端几处体验优化:侧边栏在「设置」上方新增了文档入口;聊天窗口只在你位于底部时才跟随新输出,上翻阅读即暂停,回到底部自动恢复;模型重新评估期间,上一次的评估结果会保留显示,不再退回评估中状态;硬件列表补上了 Intel GPU。
Qwen3.8 Flash Next NVFP4 不再作为独立模型,而是 Qwen3.8 Flash Next 的一个变体,ID 改为 qwen3.8-flash-next:gguf:nvfp4(原为 qwen3.8-flash-next-nvfp4:gguf:nvfp4)。
安装脚本大幅重写,macOS、Linux、Windows 全平台一条命令完成,README 与安装文档同步改用脚本方式。安装器会显示每个下载的速度与剩余时间,不再出现空白帧;慢速连接从「10 分钟即失败」改为「60 秒无数据才判定失败」并给出提示。Windows 恢复了静默安装,且会在下载前要求先退出正在运行的 Magnitude,而非到最后一步才报错;macOS 15 以下的系统会直接说明版本要求,避免装上打不开的应用;Ctrl-C 或关掉终端会立即中止下载并清理残留;无 systemd 的 Linux 环境(容器、WSL)不再提供服务器模式。
Linux 服务器的搭建与卸载也更可靠:首次启动即开启网络访问,中途打断 magnitude server setup 后服务器仍可被其他机器访问,重跑即可显示访问密钥;magnitude server remove 现在会关闭 setup 开放的 firewalld 端口;安装器被中断时会提示用 magnitude server setup 继续完成。
@magnitudedev/cli@0.2.112026-10-26
Patch Changes
- 2e321c6 Thanks @thrgreenwald, @anerli! - Run custom models: any GGUF of a model family Magnitude supports, not just catalog models, including other quantizations, finetunes, merges and uncensored versions.
- Search Hugging Face and add models from the desktop app, the web app or the CLI (magnitude catalog search, inspect and pull). Every version shows its size and whether it fits your computer, can't run (with the reason) or is gated, before you download it.
- Sign in to Hugging Face once to download gated models; Magnitude says when a model needs sign-in, its terms accepted, or the author's approval.
- Models already on your computer are found and used in place, without copying: the Hugging Face cache, LM Studio, the llama.cpp cache, Ollama, and folders you add. Found models can be hidden without touching their files, and a catalog model already on your computer is used instead of downloaded again.
- Each model gets the best drafter known to work with it, when there is one: from its own repository or folder, the matching catalog model, or its built-in MTP heads. You can choose a different drafter or none.
- Models that can't run are listed with the reason, such as an architecture without kernels yet or an unsupported weight type.
- 2e321c6 Thanks @thrgreenwald! - Discover and the model catalog are now one page.
- Model cards show one clear status and an estimated download time, and catalog and custom models share the same details view.
- Loading and optimizing show live progress.
- Service status and model memory moved into Settings.
- 2e321c6 Thanks @thrgreenwald, @anerli! - Faster generation.
- On x86 CPUs with AVX2 or VNNI, k-quant, Q8 and 4-bit codebook weights run faster.
- On NVIDIA GPUs, IQ2_XXS weights read faster, and models that stream a token table decode faster: Nemotron 3.5 Bolt on a GB10 went from 36.0 to 41.9 tok/s with drafting off.
- Drafting decides from the time recent steps actually took, so one slow step no longer turns it off, and it resumes once it pays off again.
- AMD GPUs on Linux (Vulkan, RADV) prepare a model's programs in a fraction of the time and memory.
- Chat templates that scan the whole conversation for every message no longer take seconds to render, and short follow-up turns resume from the cached conversation.
- A catalog model's one-time optimization waits until the GPU is free, and loading a model you want takes priority over it.
- 2e321c6 Thanks @thrgreenwald! - Fix installation and platform issues.
- On Windows 10, upgrades no longer fail to replace the installed app.
- The desktop app no longer takes over a Magnitude server running as a Windows service, and says why.
- Hermes connections on Windows are written where Hermes reads them.
- Fix installer readiness checks, macOS onboarding, and verification error messages.
- CLI commands piped into a reader that stops early end quietly instead of crashing.
- 2e321c6 Thanks @thrgreenwald, @anerli! - More accurate memory planning and more reliable loading.
- Fit checks count each backend's own memory costs, so a model shown as fitting loads, and a long conversation's cached prompt no longer grows past what was planned.
- Models too large for a small GPU show as not fitting instead of failing to load.
- Magnitude serves on the discrete GPU when your computer also has an integrated one, and shows which device it's using.
- Requested downloads, drafters and projectors included, resume after a restart, and a cancelled download stays cancelled.
- A large image no longer holds a model running on the CPU for minutes.
- 2e321c6 Thanks @thrgreenwald, @anerli! - Fix incorrect output and load failures for several models.
- Reasoning works with LFM2.5 Instruct and Thinking models, Gemma 4 thinks unless thinking is turned off, and a response that ends inside its reasoning keeps that reasoning instead of failing.
- Default sampling is now temperature 1.0, topp 0.95 and topk 20 on the OpenAI and Anthropic APIs alike. The OpenAI APIs used 0.8 and 0.95, and the Anthropic API 1.0 and 1.0, with no topk on either. The Anthropic API now accepts topk.
- Finetunes run with their base model's vision projector, and Qwen 3.5 models without built-in prediction heads run.
- Fix Vulkan sampling probabilities, Vulkan weight formats failing to compile, and a Metal weight-packing bug.
- ac67c47 Thanks @thrgreenwald! - Add a native Chat tab with local model streaming, performance metrics, reasoning controls, generation settings, and SQLite conversation history.
Render code blocks with language labels, syntax highlighting, and individual copy actions, and improve Markdown headings and nested spacing.
- 2e321c6 Thanks @anerli! - Add Nemotron 3.5 Bolt 74B-A9B, Nemotron 3.5 Super VL 120B-A12B with vision, and Qwen 3.8 Flash Next NVFP4 to the catalog.
- Nemotron 3.5 models draft with LiLiCorr, a new drafting method ranked above DFlash2. On a GB10, Bolt decodes at 55.7 tok/s against llama.cpp's 51.6.
- NVFP4 models that carry NVIDIA's activation scales, such as Qwen 3.8 Flash Next NVFP4 and Super VL's NVFP4 variant, run on FP4 tensor cores on GPUs that have them. On a GB10, Super VL NVFP4 prefills at 826 tok/s against llama.cpp's 552.
- Nemotron 3 Super and Ultra can now run as custom models.
- 2e321c6 Thanks @anerli, @thrgreenwald! - Speed estimates in Discover are now based on measurements of your own hardware instead of a generic memory-bandwidth figure, so they track real performance more closely. The measurement runs once, in the background, and never while a model is in use.
@magnitudedev/cli@0.2.102026-10-20
0.2.10 是一个以修复加载失败为主的补丁版本。统一内存与集成显卡上的内存规划得到修正:NVIDIA GB10 和 Apple Silicon 会把已加载模型占用的内存计入预算,AMD Strix Halo 这类核显也能同时使用共享内存与专用分区,不再因分区占满而拒绝分配;GPU 拒绝分配时会自动重试。各后端现已统一支持 IQ2XXS 权重,并修复了 IQ2XS 模型在 Mac 上无法加载的问题。Linux 与虚拟机上的加载失败也一并修好——文件页缓存占用的内存现在视为可用,真正的内存不足会提示可重试,CPU 指令不匹配则在加载前直接说明。此外修复了 Q8_0 输出头模型加载失败(涉及 MiniCPM5 1B Q8、Qwen3.6 35B-A3B 等),RPM 包不再与其他应用冲突,网页端的模型状态与更新按钮文案也补充了说明。
@magnitudedev/cli@0.2.92026-10-15
Claude Code 现在可以不登录 Claude 直接接入,包括它内置子代理在内的所有请求都会走本地已加载的模型;日后想用自己的 Claude 登录,重新连接即可。
修复集中在安装与更新:Linux 包支持 glibc 2.34(含 RHEL 9),Debian/Ubuntu 包不再往无头机器拉入 KDE 桌面;桌面端重启应用以应用设置时不再崩溃;Windows 下「重启以更新」后窗口能正常回来,防火墙提示、任务管理器与文件属性中也正确显示 Magnitude 的名称、发布者和版本;macOS 上其他账号遗留的安装锁不再挡住启动。装完却起不来的更新会被标记为失败并停止自动重装,magnitude update status、magnitude update check 与设置页都能看到可读的错误信息。取消模型下载会清掉残留文件,磁盘检查把模型与其草稿一并计入,端口 10100 被占时会说明占用者是谁。
性能方面,一次性内核优化把预算集中到耗时最多的算子,生成速度提升,Apple Silicon 上的 Gemma 4 尤其明显;token packing 新增 8 位权重与 DFlash 草稿模型支持,提示词处理也更快。模型目录新增 Liquid LFM2.5 24B-A2B,以及支持图像理解与 DSpark 投机解码的 MiniCPM-V 4.7(1B、35B-A3B)。
@magnitudedev/cli@0.2.82026-10-12
0.2.8 修复了一批稳定性与资源占用问题。zstd 压缩的 Codex 请求一旦超过 128MB 路由上限就立即停止解码,避免几 KB 的压缩输入膨胀成数 GB 内存才被拒掉。模型侧:Gemma 4 系列在目录中配对了草稿模型以加速生成;嵌入表改为流式加载,大模型内存占用明显下降(Qwen3.8 27B 少约 600 MB);首载优化后 Gemma 解码更快,且升级后不再出现部分未优化加载的情况。硬件适配上修了不少坑:M1 Max/Ultra 上 Qwen 3.5、3.6、3.8 报 "sampling distribution is nonfinite" 的问题已解决;8GB 的 NVIDIA 和 AMD 显卡在长提示加并发时不再误卸载模型,长上下文能力大幅提升(MiniCPM5 2B 从约 26K 提升到 78K tokens);显卡驱动预留的显存不再计入可用,推荐与 "Fits" 标签现在与实际能加载的模型一致;Windows 下 AMD 显卡加载必崩、长提示在 8GB AMD 卡上卡死的问题也已修复。核显不达标的机器(如 Intel UHD 770)会自动回落到 CPU 运行,而不是直接加载失败。另外还修了并发时偶发的 "target rows differ" 报错。