cuTile Rust 0.4.02026-09-25
0.4.0 基于 CUDA 13.4 Tile IR 构建，以下是相对 0.3.0 的变化。调试方面：开发构建在 O0 下编译并保留完整设备调试信息，CUDARUSTDEBUG 可在 none/line/full 间切换；line 模式兼顾优化与行号表，Nsight Compute 和 Systems 能把 SASS 归因到产生它的 .rs 行，cuda-gdb 也能沿跨模块调用链回溯到指定行。每次启动的 CompileOptions（devicedebug、lineinfo、sanitizememcheck、optlevel）现均参与内核缓存键。项目已支持 DGX Spark 的 GB10（sm121）：16 位 Qwen3-4B 解码 29.5 tokens/s，32B 达 3.7 tokens/s，均接近 LPDDR5X 屋顶线九成。新增 programmaticdependentlaunch()，让后继内核在前一内核收尾前进入序言，要求 Tile IR 13.4 与 sm_90 以上 GPU，不支持时直接报错而非静默串行。13.4 另带来 insert、fpowi、fpowf、`f8e5m3
cuTile Rust 0.3.12026-09-06
0.3.1 makes getting started one line — cargo add cutile is all a kernel crate needs — and brings profiling support, enhanced autotuning, and a faster launch path.
Highlights
- One-crate install: generated launchers resolve compiler support through the cutile crate root, so cargo add cutile alone builds and runs kernels (#248).
- Profiling support: CompileOptions::{devicedebug, lineinfo, sanitizememcheck, opt_level} (#236) and source locations in the bytecode debug section (#238), so Nsight Compute/Systems attribute SASS to your kernel's .rs lines. See Debugging and Profiling.
- Enhanced autotuning (experimental): resumable trial logs on the engine-objective path, required incumbent configs, kernel-cache management for sweeps, and a dedicated book chapter: Autotuning (#239).
- Faster launches: cache-hit launches cost ~2.4x less host time (3.9 → 1.6 µs measured) via per-site resolution caching and allocation-free marshalling (#265).
- Kernel API: programid(axis)/numprograms(axis) (Triton/cuTile Python vocabulary), const T raw pointers with Rust's safety model and Tensor::asptr, x.load_like(z), and shape![..] (#240, #259, #261).
- Host-side crates: cuda-oxide's host surface (simt modules, #[derive(DeviceCopy)]), CUDA VMM wrappers, NVLink switch multicast (#202, #233, #251).
Breaking changes in a patch release
0.3.1 contains breaking API changes that semantically belonged in 0.4.0 — several soundness fixes (unsafe fn atomics and pointer stores, unsafe trait DType) and one silent semantics fix: integer / in kernels now rounds toward zero as in Rust, and u8/u16 arithmetic is properly unsigned, so programs that relied on the old lowering produce different results. The full list is in the CHANGELOG.
To keep this from recurring: the crate family is now version-pinned in lockstep so cargo can never mix releases (#267), CI runs cargo-semver-checks against the latest published release so an API break without the matching version bump fails the PR, and the next breaking release will bump the minor version.
Requires CUDA 13.2+.
v0.3.02026-08-21
This release introduces compiler optimizations that improve safe kernel performance, async launch performance improvements, persistent kernel tuning, and zero-copy interop with externally owned device memory. The front-end compiler now proves a broader class of symbolic facts about loads and stores during JIT compilation. Checks which the compiler cannot discharge outright are progressively hoisted out of control flow constructs (e.g. hot loops) and, when possible, out of the kernel entirely, re-materializing as single host-side comparisons at launch. This frees kernel code from bounds checks and bounds-related register pressure.
Highlights
- Bounds-check placement: Every partition access walks one proof ladder (axis provenance, static folding, declared preconditions, host-side launch checks) and only then pays an in-kernel assert, hoisted to the outermost provable loop preheader. Kernels indexed by loop iterands, mapped schedule components, or tile-block ids typically compile with zero in-kernel checks and no annotations, and denyinkernelchecks = true turns any residual in-kernel check into a compile error. PartitionMut::store joins the checked-by-default surface (storeunchecked is the explicit escape) at identical register counts. The placement optimizer is differentially validated: Every kernel in the test corpus runs both with optimized placement and with all checks forced to their access sites, and the optimized build must never accept an execution the reference build rejects. The new book chapter, Bounds-Check Placement, documents the placement rules and the verification workflow.
- Partial partition mapping: A partitioned output normally maps every block to a tile program; partition_prefix allows a partial, still-disjoint mapping — the leading blocks along each axis get tile programs, the rest are left untouched. Mapping beyond the partition remains a launch error, and partition still requires the mapping to be total.
- Zero-copy interop: Tensor::fromforeign borrows device memory owned by an external framework (cudarc, torch, VMM ranges), holding the owner alive by refcount and verifying the addressable extent at construction; Tensor::borrowrawparts, Device::borrowwithowner, and Stream::borrowwith_owner cover bare handles.
- Autotuner core: declared configuration spaces, pluggable searchers, resumable grid search, and provenance-checked tuning records.
- cutile::bench for device-event kernel benchmarking, an opt-in persistent on-disk cubin cache, a process-global single-flight kernel cache with a .compile() warmup API, and async launch latency at parity with sync.
Performance results
- Fully checked kernels match their zero-check unsafe twins: Paired A/B runs on B200 (Qwen3-32B, 16K and 32K prefill) agree within 0.5% in both directions with byte-identical output, and on RTX 5090 the fully checked flash-attention prefill kernel runs within 2.5% of its checks-disabled floor. Checked and unchecked stores compile to identical register counts.
- Async launch latency is at parity with sync.
Changed
- with_bounds and Dim::new are deprecated: bounds inference and declared preconditions prove everything they proved, with the checks placed at launch instead of in the kernel. Migration notes are on the deprecations.
- Tile IR bytecode version selection reads the CUDA toolkit when discoverable and otherwise probes tileiras with a representative module (an entry with a for region and view chain), so an accepted version is one real kernels compile at.
- The persistentgemm example is rewritten in the zero-annotation form and compiles under denyinkernelchecks = true.
Fixed
- An entry declared as bare #[cutile::entry], without an argument list, was silently treated as a plain function; both attribute forms are now recognized.
- Interval range facts no longer survive arithmetic that can wrap i32; a wrapped intermediate could previously discharge a check the machine value violated.
- Bounds analysis guards division and remainder against zero divisors, saturates interval arithmetic, and uses overlap analysis for eq/ne.
- Device-op Send bounds, CUDA driver flag types, and aarch64 flake support.
- Security bumps for crossbeam-epoch (RUSTSEC-2026-0204) and anyhow (RUSTSEC-2026-0190).
v0.2.02026-06-16
cuTile Rust 0.2.0 adds low-precision inference support and accompanies our paper, Fearless Concurrency on the GPU.
Paper: https://arxiv.org/abs/2606.15991
Artifacts: cutile-benchmarks/paper/
Announcement: https://github.com/NVlabs/cutile-rs/discussions/164
Highlights
- CUDA 13.3-oriented low-precision support, including NVFP4 packing and unpacking and block-scaled matrix multiply.
- Runnable NVFP4 and MXFP8 examples.
- New cutile-kernels crate with reusable inference-oriented kernels written in cuTile Rust.
- Paper reproducibility artifacts under cutile-benchmarks/paper/, including benchmark harnesses, CSV results, hardware details, and clock settings.
- Expanded compile-only regression coverage and example/test cleanup for release validation.
Performance context
On NVIDIA B200, cuTile Rust reaches 7 TB/s for element-wise operations and 2 PFlop/s for GEMM, about 91% of peak memory bandwidth and 92% of dense f16 peak, respectively. Safe Rust persistent GEMM reaches 2.07 PFlop/s at M=N=K=8192, within 0.3% of the corresponding low-level Tile IR variant.
We also evaluated Grout, a Qwen3 inference engine built with cuTile Rust in collaboration with Hugging Face. In batch-1 Qwen3 decode, Grout reaches 171 tokens/s for Qwen3-4B on NVIDIA GeForce RTX 5090 and 82 tokens/s for Qwen3-32B on B200, showing strong performance on memory-bound inference tasks, consistent with our HBM roofline analysis.
Notes
cuTile Rust remains early-stage research software. CUDA 13.3 is required for the new low-precision features, and hardware-specific features such as native FP4 require architectures that support them.
v0.1.12026-06-15
0.1.1 是一个小版本，主要补齐了 NVFP4 的支持面。Tile IR 及其编译器降级路径新增了 NVFP4 类型与算子覆盖，配套的示例、测试和书籍材料也一并更新，方便开发者直接参考使用。书籍构建与发布流程改为带版本号，之后每个版本的文档都能独立留存。工作区各 crate 版本号升至 0.1.1，CI 与文档同步做了刷新。
v0.1.02026-05-16
cuTile Rust v0.1.0 已上线 crates.io。主机端与设备端 API 正式定稿，内核编写模型、张量启动 API、DeviceOp 执行模型与核心设备操作接口今后不再有破坏性变更；日常写内核只需 cargo add cutile@0.1.0，无需再依赖内部 workspace 各 crate。
本版定稿了张量内核、DeviceOp、异步执行、CUDA graphs 与 CUDA 互操作的接口；设备端操作紧贴 Tile IR，覆盖张量视图、分区视图、原子操作、内存序、形状运算与 tile 数学；JIT 编译器类型推断更强，新增静态分派降级、类型别名、全局常量、Global、else if，诊断信息也更清晰；映射分区支持安全的持久化调度，附持久化 GEMM 示例；动态形状与只读类 tile 加载在常见情形下生成更快的代码；运行时新增动态 CUDA 绑定、tileiras 覆盖、自定义内存池与内存计量。README 与 book 已同步更新为精简的快速上手示例与当前 API 文档。
