SkillEvaluator 0.9.02026-11-13
SkillEvaluator 0.9.0 重点升级了评估数据集的构建与审查:可为脚本执行、工作区文件和最终回复声明确定性检查,支持从技能说明、作者意图或运行轨迹生成数据集,导入生产轨迹时自动脱敏,且预期项须经人工审核后才能启用。版本追踪方面新增 --compare-to、compare-versions 与 --reuse-baseline,可复用兼容的无技能基线。
预算控制是本版亮点:可设置 token、评审调用、尝试次数与成本上限,中断后报告仍保留部分结果,并按用例和智能体展示消耗。报告证据更完整,区分满足、未满足、未检查与不适用的预期,Skill Lift 作为核心对比指标,pass@k 仅作辅助,HTML 图表支持离线查看。审计能力同步增强:tier3 validate-grader 校验评分器,tier3 feedback 支持对已完成运行提问,可导入外部自定义指标及其来源。CI 方面可选接入 GitHub/GitLab,在必需的 Tier 1 检查旁增加 Tier 2、Tier 3 建议性评估。
修复涵盖各报告格式间指标对齐、失败基线标注、安全与 PII 误报降低、依赖公告结构化输出,以及 Tier 3 后端升级至 Harbor 0.25.0。
升级需注意:数据集生成默认改为 auto 基准,纯技能流程需显式加 --grounding skill;部分评分口径有变,跨版本比较前建议重新评估;基线超时容忍改为可选参数。
SkillEvaluator v0.8.02026-11-06
SkillEvaluator 0.8.0:Tier 3 评分逻辑重构,所有分数改由 judge 裁决经 verifier 计算得出,准确率只看结果本身(FORMATFOLLOWED 取代 SKILLIDENTIFIED),HTML 报告会解释低分原因。新增 tier3 feedback 命令,可在终端里针对已完成的运行结果提问;judge 的回退模型改为通过 LLMJUDGEFALLBACKPOLICY 等环境变量按需开启。报告中每条 finding 都有了稳定的 findingid 与 occurrence_id,JSON、Markdown、HTML 三处统计口径统一。Tier 1 支持导入经审计的 SkillSpector 抑制基线和绑定证据的 PII 豁免。
升级前注意:共享验证策略改名为 default(旧名仍兼容);同一批运行的 Tier 3 分数会变化;Claude Code 拒答现在会直接导致该 arm 失败、tier3 evaluate 退出码为 1,需重跑或换模型;Tier 1 对非 agent 可达文件的威胁匹配不再判失败,但许可证检查更严格。
SkillEvaluator v0.7.22026-11-04
SkillEvaluator 0.7.2 重点补齐了 GitHub 工作流的托管报告能力:在保留原有共享组件、目录结构、阈值与触发条件的前提下,新增托管 HTML 报告和自动更新的 PR 评论,直接调用与共享组件调用两种方式都支持。集中维护的组件(如 @v3)现在可以自行升级锁定的评估器版本,不必再逐个改动消费方仓库;每份报告仍会保留签名的评估器身份与解析出的工作流修订。设置前会先校验共享组件的身份权限,已有的 Actions 限制、无关的 Pages 站点和部署审批不受影响。企业版组织仓库可复用管理员配置的私有 Pages 站点,若私有托管不可用,检查与可下载报告照常可用。另外还改进了重复配置、自定义扫描任务名、Windows 下的工作流写入与 CRLF 审批哈希。需注意私有 Pages 部署未做端到端测试。
Python 包方面:新增 Tier 3 可复现的本地目录诱饵选择(tier3 select-decoys);metadata.domain 接受列表,上限 16 条、每条 128 字符;修复 Tier 2 自匹配与分块预算,以及 Tier 3 评分、源错误处理、Codex 凭据目录清理和基准显示等问题。扫描镜像未重建,报告发布器修订为 8096c6966a498f35f6f7dc1a596d1cc4da85cf11。
SkillEvaluator v0.7.12026-11-03
SkillEvaluator 0.7.1 packages guided GitHub onboarding, improves CI report feedback, and includes scanner and evaluation runtime fixes.
Highlights
- Set up repository checks with skillevaluator ci init; use ci status to inspect setup and ci demo to open an optional test PR.
- Check changed skills with Tier 1 quality checks and SkillSpector security scanning. Optional GitHub feedback includes hosted HTML reports and one updated PR comment.
- Improve scanner setup, local agent installation, Tier 3 progress and results, and provider readiness checks. See the changes below.
Python package
SkillEvaluator 0.7.1 on PyPI · NVIDIA Python package index
Requires Python 3.12, 3.13 or 3.14. Install or upgrade:
``bash
pip install --upgrade skillevaluator
`
Both the wheel and source distribution are published on PyPI and NVIDIA's index. Package files and SHA-256 checksums are attached below.
CI integration
From your cloned GitHub repository, run:
`bash
skillevaluator ci init
``
The wizard prepares a setup PR for review. Automatic setup requires an existing GitHub CLI login and repository admin access. Checks-only supports public and private repositories; automatic hosted reports and PR comments support public repositories.
For manual GitHub or GitLab setup, download the ready-to-copy CI templates. The bundle includes immutable workflow/image pins, scanner versions and checksums. Open the template folder to copy individual files. The ZIP and checksums are also stored there.
CLI setup guide · GitHub Actions · GitLab CI
What's Changed
* fix(tier2): report a chat-model refusal as an advisory dedup finding by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/281
* fix(security): make the SkillSpector timeout configurable by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/271
* feat(doctor): add Tier 1 scanner and Tier 2 provider readiness checks by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/293
* fix: decode Git roots as file names and keep PowerShell prefix findings (#312 review) by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/320
* feat(tier3): add --skip-with-skill for baseline-only runs by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/284
* fix(security): accept verified referenced images in partial SkillSpector scans by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/299
* test(security): cover SkillSpector watchdog duration boundaries by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/319
* fix(tier3): write plain progress heartbeats once a minute by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/274
* fix(security): handle read-only passwd mounts with SkillSpector 2.13.3 by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/289
* fix(docs): render README images in PyPI descriptions by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/323
* Private runtime: prepared packages, native loading and evidence over the existing engine by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/264
* Offer to install a missing agent CLI in local mode by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/317
* Show per-case Tier 3 results and a "what to fix" list by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/316
* chore(release): reconcile published 0.7.0 metadata by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/314
* Set up Tier 1 scanners on first use and add --skip by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/318
* fix(ci): require Semgrep in scanner subprocesses by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/324
* fix(ci): show concise PR feedback with fresh HTML report links by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/326
* feat(ci): ship guided GitHub onboarding in SkillEvaluator 0.7.1 by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/327
Full Changelog: https://github.com/NVIDIA/SkillEvaluator/compare/0.7.0...v0.7.1
SkillEvaluator v0.7.02026-11-02
SkillEvaluator 0.7.0
SkillEvaluator 0.7.0 adds Python 3.14 support alongside Python 3.12 and 3.13, reusable Tier 1 CI integration, configurable quality policies, and clearer evaluation reports.
Highlights
- Python 3.14 support, with full-suite CI and built-package installation checks across Python 3.12, 3.13, and 3.14.
- Reusable GitHub Actions and GitLab CI checks, parallel skill scanning, and configurable quality headings and thresholds.
- Verdict-led HTML reports, case-first Tier 3 summaries, matching-judge comparisons, and actionable recommendations in whattofix.json.
- More precise Tier 2 completeness reporting, Tier 3 timeout/recovery and local-mode fixes, and an optional --max-cases limit for exploratory runs.
- SkillSpector 2.13.3 installation and CI pins, preserved NVIDIA Build model-registry handling, and validated LLM provenance in JSON/HTML reports.
- Refreshed locked runtime and dashboard dependencies. The Python package's dependency ranges are unchanged.
Python package
PyPI publication: published and verified. Both the wheel and source distribution are available on PyPI and NVIDIA’s Python package index. Their downloaded SHA-256 hashes match the release files below.
Versioned package page: skillevaluator 0.7.0 on PyPI.
Create a virtual environment using Python 3.12, 3.13, or 3.14 and install the package:
``bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install skillevaluator==0.7.0
`
For the optional Python evaluation and security dependencies:
`bash
python -m pip install 'skillevaluator[all]==0.7.0'
`
A standalone uv installation is also supported:
`bash
uv tool install --python 3.14 'skillevaluator==0.7.0'
`
The all extra includes the Tier 2, Tier 3, and Python security dependencies. Full static security scanning additionally requires separately installed SkillSpector, Gitleaks, and Semgrep executables. Upgrade existing SkillSpector installations explicitly to v2.13.3; Claude 5.5 requires a supported explicit SKILLSPECTOR_MODEL with this scanner release. Live evaluations require provider credentials and the selected agent/sandbox runtime. The optional dashboard is installed with the separate dashboard extra.
See the installation guide for scanner and runtime setup.
Release files
This release uses the GitHub tag 0.7.0 at commit ad2815fe78c045db91d230681d32863698da08cf. The earlier v0.7.0 tag name was consumed by an immutable release and cannot be reused. The Python package version is 0.7.0.
The Assets section contains the installable Python wheel (skillevaluator-0.7.0-py3-none-any.whl), Python source distribution (skillevaluator-0.7.0.tar.gz), and their SHA-256 checksums. GitHub also supplies repository source archives and a release attestation. The wheel supports all three Python versions listed above.
Installation verification
Fresh PyPI installations were verified with pip using both wheel and source distributions on Python 3.12, 3.13, and 3.14, plus uv tool installation and the all extra on Python 3.14. Checks included dependency consistency, CLI execution, and real offline validation reports in JSON, HTML, Markdown, and SARIF formats. New provider-backed LLM or Tier 3 evaluations were not run for this package publication.
Package checksums
| File | SHA-256 |
| --- | --- |
| skillevaluator-0.7.0-py3-none-any.whl | 886a9713ab33f0e904d848f649977d02214c2412901b4eecd9c01fe51b79340e |
| skillevaluator-0.7.0.tar.gz | c8634c39ce7bbbf5138bf6a7a99c841637fc40a8407caf88ae2c49cdc90f0a31` |
What's Changed
* Fix Windows subprocess decoding, reasoning-model token budget, and harbor bin resolution by @Minervaowl7 in https://github.com/NVIDIA/SkillEvaluator/pull/194
* fix(cli): include nested skills in catalog validation by @YMuskrat in https://github.com/NVIDIA/SkillEvaluator/pull/210
* fix(quality): keep minimum score failures after policy changes by @YMuskrat in https://github.com/NVIDIA/SkillEvaluator/pull/211
* fix(tier3): keep skill lift consistent when rebuilding reports by @YMuskrat in https://github.com/NVIDIA/SkillEvaluator/pull/212
* feat(tier1): score tier 1 quality checks via a configurable YAML file by @westonplatter in https://github.com/NVIDIA/SkillEvaluator/pull/213
* feat(tier3): let installed packages add custom execution backends by @rmalani-nv in https://github.com/NVIDIA/SkillEvaluator/pull/215
* fix(tier3): score unrecovered failures and align recovery verdicts by @mavericksea-ai in https://github.com/NVIDIA/SkillEvaluator/pull/217
* docs: simplify first runs with keyless checks and NVIDIA Build by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/218
* fix(judge): treat an "unavailable" schema option as a schema rejection by @MaxFreedomPollard in https://github.com/NVIDIA/SkillEvaluator/pull/219
* Pass Windows program dirs and Python UTF-8 mode to Harbor by @GodfredArhinful in https://github.com/NVIDIA/SkillEvaluator/pull/220
* fix(tier3): retain scores after verified timeout recovery by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/222
* fix(reporting): retain completed-agent results in Tier 3 reports by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/223
* docs: refresh evaluation guidance and expose dashboard guide by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/224
* fix(quality): score non-mapping metadata instead of crashing by @MaxFreedomPollard in https://github.com/NVIDIA/SkillEvaluator/pull/226
* fix(tier2): bound embedding chunks and preserve source evidence by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/227
* fix(tier3): avoid destructive scoring for plain cache cleanup by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/228
* fix(tier3): count explicit root searches as wasted tool calls by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/229
* fix(tier3): flag deletions that escape temporary paths by @YMuskrat in https://github.com/NVIDIA/SkillEvaluator/pull/230
* fix(tier3): flag destructive rm targets supplied by xargs by @YMuskrat in https://github.com/NVIDIA/SkillEvaluator/pull/231
* fix(tier3): gate nested cleanup findings on symlink evidence by @YMuskrat in https://github.com/NVIDIA/SkillEvaluator/pull/232
* docs(cli): compare direct tier workflows with validate --tiers by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/237
* feat(ci): add reusable Tier 1 scans and GitHub onboarding by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/238
* fix(tier3): preserve local verifier uploads and shared step files by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/239
* Add opt-in Unicode character facts for Tier 3 judges by @mavericksea-ai in https://github.com/NVIDIA/SkillEvaluator/pull/242
* fix(pii): recognize reserved placeholder email domains by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/246
* test: wait for descendant exit after streamed exec cleanup by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/247
* Add reusable GitLab skill evaluation and repository onboarding by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/248
* Share deterministic grading and add bounded input readers by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/249
* Add private manifest discovery and configuration checks by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/250
* fix(reporting): preserve unavailable Tier 3 evaluator scores by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/251
* fix(tier3): preserve local Claude Code instruction variables by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/252
* fix(rubric): preserve source and complete linked guidance by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/254
* feat(tier3): case-first CLI summary and c
PyPI packages for v0.6.02026-11-01
Wheel and source distribution for the existing SkillEvaluator v0.6.0 release, built from commit 5320d9329c36ecc72bc94c4f58cc813d6b54224b.
These files provide stable download URLs for managed PyPI publication. Their presence here does not indicate that publication to the package index has completed. Package version and product behavior remain v0.6.0.
| File | SHA-256 |
| --- | --- |
| skillevaluator-0.6.0-py3-none-any.whl | 92b4be7b0023b77ce219ed57de8a189365b94502bf209037333292e96eed2ada |
| skillevaluator-0.6.0.tar.gz | f1e75f88c85df5989f744e5aba485c149aeb20776e910e98f49888a2b9529bb4 |
SkillEvaluator v0.6.02026-10-20
What's Changed
* docs: bring the published docs in line with v0.5.0 by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/186
* fix(embedding): preserve multiline descriptions in catalogs by @rioyu123 in https://github.com/NVIDIA/SkillEvaluator/pull/191
* test: clear two test-only CodeQL alerts by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/184
* feat(report): show the SkillSpector version in the Tier 1 HTML card by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/187
* fix: avoid redundant Node/npm during Docker agent setup by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/193
* feat: default to GPT-6.1 Sol and Claude Opus 5.5 across Tiers 1-3 by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/188
* fix(tier3): honor per-step tokens when detecting runtime failures by @YMuskrat in https://github.com/NVIDIA/SkillEvaluator/pull/199
* fix(tier3): include ATIF content parts in judge answer evidence by @YMuskrat in https://github.com/NVIDIA/SkillEvaluator/pull/203
* fix: target dependency coverage and Tier 3 result integrity by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/196
* feat(tier3): run hardened Docker trials under Kata Containers by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/182
* fix(tier3): derive accuracy scores from criterion verdicts by @mavericksea-ai in https://github.com/NVIDIA/SkillEvaluator/pull/190
* fix(tier3): exclude failed skill reads from execution credit by @YMuskrat in https://github.com/NVIDIA/SkillEvaluator/pull/201
* fix(reporting): require complete tier evidence for publication by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/77
* fix(tier3): use one overall-score formula by @Saibernard in https://github.com/NVIDIA/SkillEvaluator/pull/124
* feat(tier3): expose Harbor's Runta and Mosaic backends by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/183
* feat(tier3): support Claude Code on Agent Platform and add active preflight probe by @kweinmeister in https://github.com/NVIDIA/SkillEvaluator/pull/140
* feat(reporting): add quantitative skill performance dashboard by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/179
* fix(tier3): mark never-activated skill evaluations inconclusive by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/209
* ci: cover every workflow with the hardening guards, and fix the gaps they surface by @dberzan in https://github.com/NVIDIA/SkillEvaluator/pull/126
* chore(release): prepare v0.6.0 by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/216
New Contributors
* @YMuskrat made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/199
* @Saibernard made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/124
* @dberzan made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/126
Full Changelog: https://github.com/NVIDIA/SkillEvaluator/compare/v0.5.0...v0.6.0
SkillEvaluator v0.5.02026-10-06
What's Changed
* feat(tier3): make evidence bundle budgets and final response limit configurable by @kweinmeister in https://github.com/NVIDIA/SkillEvaluator/pull/160
* feat(tier3): enforce structured JSON schemas and rate-limit retries in LLM judges by @kweinmeister in https://github.com/NVIDIA/SkillEvaluator/pull/145
* feat(tier3): rebuild ATIF from OpenCode and Codex agent logs by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/119
* fix(tier3): propagate Harbor dual-arm task suffixes and normalize canonical case ID resolution by @kweinmeister in https://github.com/NVIDIA/SkillEvaluator/pull/161
* fix(validation): report non-string YAML keys by @rioyu123 in https://github.com/NVIDIA/SkillEvaluator/pull/178
* feat(tier3): upgrade Harbor to 0.24.0 by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/83
* chore(release): prepare v0.5.0 by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/185
Full Changelog: https://github.com/NVIDIA/SkillEvaluator/compare/v0.4.0...v0.5.0
SkillEvaluator v0.4.02026-09-30
SkillEvaluator v0.4.0 simplifies running evaluations and strengthens reporting, security checks, and large-catalog analysis.
Highlights
- Run tiers directly with skillevaluator tier1 PATH, tier2 PATH, or tier3 PATH, with clearer provider setup, gateway model defaults, and evaluation progress.
- Scan larger Tier 2 collections with explicit entry and comparison budgets, faster vector comparisons, and actionable service-failure diagnostics.
- Improve Tier 3 script-invocation evidence, protected-path write detection, token masking, and incomplete-run feedback.
- Harden CLI rendering and secret redaction, run the Docker image as an unprivileged user, and pin the HTML report's Chart.js dependency with integrity verification.
Upgrade notes
- validate PATH now runs all three tiers for skills by default. Use --tiers 1 for keyless static CI; --tiers, --no-tier3, and --no-autopilot provide explicit scope controls.
- When --agents is omitted, Tier 3 selects a provider-native agent. Extra agent runtime preflight is opt-in through --agent-runtime-preflight.
- The NVIDIA Build defaults are now nvidia/nemotron-3-super-120b-a12b for chat and nvidia/nemotron-3-embed-1b for embeddings. Rebuild existing catalogs and caches when switching embedding models.
See the v0.4.0 changelog for the full release notes.
What's Changed
* test: stabilize FIFO subprocess checks by @deepujain in https://github.com/NVIDIA/SkillEvaluator/pull/148
* docs: link SkillEvaluator research paper by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/146
* docs: add docs/livestream/blog links to README by @mosheabr in https://github.com/NVIDIA/SkillEvaluator/pull/159
* feat(cli): simplify tier workflows and improve evaluation feedback by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/152
* fix(reporting): verify the pinned Chart.js dependency by @deepujain in https://github.com/NVIDIA/SkillEvaluator/pull/162
* fix(container): use an unprivileged runtime account by @deepujain in https://github.com/NVIDIA/SkillEvaluator/pull/163
* Keep fenced code within its Markdown section during chunking by @sylvesterkaczmarek in https://github.com/NVIDIA/SkillEvaluator/pull/165
* fix: keep --no-llm negative cases off-skill by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/100
* fix: harden all CI workflow guards by @efegokdemir in https://github.com/NVIDIA/SkillEvaluator/pull/158
* fix(tier2): support larger bounded collection scans by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/166
* fix(tier3): require invocation evidence for script execution credit by @mavericksea-ai in https://github.com/NVIDIA/SkillEvaluator/pull/154
* fix(security): stop reporting User-Agent versions as IP addresses by @JayOfTheKeyboard in https://github.com/NVIDIA/SkillEvaluator/pull/168
* fix(reporting): escape untrusted text in Rich CLI output by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/173
fix(tier3): detect applypatch writes to protected paths and mask GitHub tokens by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/174
* fix(redaction): make sensitive-key, JWT, and private-key matching linear-time by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/172
* fix(reporting): strip terminal controls from catalog and view output by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/175
* fix(tier3): make log JWT redaction linear-time by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/176
* chore(release): prepare v0.4.0 by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/177
New Contributors
* @sylvesterkaczmarek made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/165
* @efegokdemir made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/158
* @mavericksea-ai made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/154
* @JayOfTheKeyboard made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/168
Full Changelog: https://github.com/NVIDIA/SkillEvaluator/compare/v0.3.0...v0.4.0
SkillEvaluator v0.3.02026-09-17
What's Changed
* docs: link the NVIDIA Verified Skills docs from SkillEvaluator by @mosheabr in https://github.com/NVIDIA/SkillEvaluator/pull/46
* Fix Tier 2 scans for contained compatibility alias by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/36
* fix: replace OSS boundary line allowlist with stable anchors by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/40
* ci: skip heavy jobs for docs-only pull requests by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/48
* fix(tier3): isolate per-task inputs and evaluator fixtures by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/44
* feat(validate): replay explicit tier gating onto main by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/49
* Ignore short env values in secure Docker redaction by @Hashim1999164 in https://github.com/NVIDIA/SkillEvaluator/pull/51
* fix(quality): avoid substring false positives by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/45
* chore(release): prepare v0.2.0 by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/52
* chore: centralize frontier OpenAI/Anthropic chat defaults by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/38
* feat(reporting): roll out unified benchmark publication card by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/47
* fix(tier3): propagate judge model overrides to Harbor verifiers by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/54
* fix(reporting): use commit SHA for pull request links by @danielgaskins in https://github.com/NVIDIA/SkillEvaluator/pull/60
* fix(tier3): align Anthropic endpoints and fail closed on judge errors by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/56
* Fix quality scoring with CommonMark structural parsing by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/53
* fix(tier3): handle negative controls in insights by @rookepoole in https://github.com/NVIDIA/SkillEvaluator/pull/58
* fix(tier3): parse SKILL.md frontmatter as YAML by @genkey6 in https://github.com/NVIDIA/SkillEvaluator/pull/68
* chore(release): prepare v0.2.1 by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/76
* fix(tier3): score Harbor single-step null results by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/74
* fix(tier3): fail fast on rejected credentials by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/75
* feat(tier3): report pass@k uncertainty and paired outcomes by @AdemVessell in https://github.com/NVIDIA/SkillEvaluator/pull/63
* fix(ci): scope Gitleaks to checked-out history by @rioyu123 in https://github.com/NVIDIA/SkillEvaluator/pull/106
* fix(tier3): avoid transcript runtime false positives by @rioyu123 in https://github.com/NVIDIA/SkillEvaluator/pull/84
* fix: scan Markdown headings for PII instead of skipping them by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/96
* fix(tier3): retry malformed structured judge responses by @chrisknvidia in https://github.com/NVIDIA/SkillEvaluator/pull/78
* fix(tier3): detect a results directory the Docker daemon cannot see by @genkey6 in https://github.com/NVIDIA/SkillEvaluator/pull/80
* fix: skip gitleaks test dirs, not substring names by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/102
* fix(hygiene): use CommonMark for dead-link checks by @rioyu123 in https://github.com/NVIDIA/SkillEvaluator/pull/109
* fix: keep --llm-verify file reads inside the skill by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/101
* feat: add SARIF reporter for GitHub Code Scanning by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/117
* fix: do not let frontmatter MIT hide a GPL LICENSE file by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/97
* fix: treat tools/ as an executable directory by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/103
* fix: flag Windows PII usernames that start with s by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/95
* fix: evaluate full SPDX license expressions by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/98
* fix(tier3): decode native Codex exec calls by @Tomauskasz in https://github.com/NVIDIA/SkillEvaluator/pull/111
* fix(hygiene): detect unpinned dependencies with environment markers by @rioyu123 in https://github.com/NVIDIA/SkillEvaluator/pull/107
* fix: accept UTF-8 BOM in SKILL.md frontmatter by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/99
* Honor SkillSpector analysis completeness by @Tomauskasz in https://github.com/NVIDIA/SkillEvaluator/pull/112
* feat(model-catalog): allow plain HTTP for named hosts by @Tomauskasz in https://github.com/NVIDIA/SkillEvaluator/pull/135
* Fix empty process-control environment resets by @RyaliNvidia in https://github.com/NVIDIA/SkillEvaluator/pull/133
* fix: map Harbor trial folders to eval case ids by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/104
* fix(cli): report malformed policy files cleanly by @rioyu123 in https://github.com/NVIDIA/SkillEvaluator/pull/129
* feat(cli): write catalog-summary.json after catalog validate by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/121
* feat(cli): parallel catalog validation and fleet rollup by @mimran-khan in https://github.com/NVIDIA/SkillEvaluator/pull/123
* fix(security): accept fully covered documentation scans by @deepujain in https://github.com/NVIDIA/SkillEvaluator/pull/143
* fix(tier3): prevent false positives in network exfiltration checks by @kweinmeister in https://github.com/NVIDIA/SkillEvaluator/pull/141
* fix(embedding): reject non-progressing chunk windows by @deepujain in https://github.com/NVIDIA/SkillEvaluator/pull/142
* feat(reporting): record the evaluated source identity on benchmark cards by @MaxFreedomPollard in https://github.com/NVIDIA/SkillEvaluator/pull/131
* chore(release): prepare v0.3.0 by @rng1995 in https://github.com/NVIDIA/SkillEvaluator/pull/144
New Contributors
* @mosheabr made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/46
* @Hashim1999164 made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/51
* @danielgaskins made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/60
* @rookepoole made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/58
* @genkey6 made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/68
* @AdemVessell made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/63
* @rioyu123 made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/106
* @mimran-khan made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/96
* @Tomauskasz made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/111
* @RyaliNvidia made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/133
* @deepujain made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/143
* @kweinmeister made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/141
* @MaxFreedomPollard made their first contribution in https://github.com/NVIDIA/SkillEvaluator/pull/131
Full Changelog: https://github.com/NVIDIA/SkillEvaluator/compare/v0.1.0...v0.3.0