Rank #03 · Daily

Rank #03 · Daily · DevTools

SkillEvaluator: NVIDIA's Multi-Tier Framework for Evaluating Agent Skills

· Daily Top10 · License: Apache-2.0

Rank #03 Daily — NVIDIA/SkillEvaluator

Check whether an agent skill is well-formed and safe, then measure whether adding it actually changes how the agent performs a task.

  • Board: Rank #03 · Daily ·
  • Repo: NVIDIA/SkillSpector
  • Stars / lang / license: 2,696 stars · Python · Apache-2.0
  • License label: License: Apache-2.0
  • Data verified: 2026-11-17 08:55 Asia/Shanghai (official Daily Top10 gate; language/license confirmed via GitHub API get_repository + README + LICENSE; docs, PyPI, and arXiv links verified from the README)
  • Trending note: +873 stars today (captured 08:55 Asia/Shanghai)
  • Tags: DevTools · agent-evaluation · skill-evaluation · Python · Apache-2.0 · CLI · CI · NVIDIA

What it does

SkillEvaluator is a multi-tier evaluation framework from NVIDIA for agent skills and plugins. It is an evaluation tool, not a skills or prompt collection. GitHub description: "Multi-tier framework for evaluating AI agent skills and plugins with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior." Docs: docs.nvidia.com/skills/skillevaluator. PyPI: skillevaluator. Paper: arXiv:2608.20614. NVIDIA Developer blog: Evaluating AI Agent Skill Performance with NVIDIA SkillEvaluator.

The repository's root skills/ directory holds the project's own skill (skills/skillevaluator) so a coding agent can run the evaluator. It is not a dump of skills to install as content.

README claims (as verified):

License: Apache-2.0 (LICENSE file — Copyright 2026 NVIDIA CORPORATION & AFFILIATES — and GitHub SPDX agree). The README notes that the project may download additional third-party open-source software; review those licenses before use.

Who it's for

Less ideal if you are looking for a library of skills to install (this repository does not ship one for that purpose), need NVIDIA enterprise support or an SLA, or only want a one-off linter and cannot set up scanners, a model provider, or a live-evaluation sandbox.

Stars / language / license

2,696 stars · Python · Apache-2.0

Data verified: 2026-11-17 08:55 Asia/Shanghai (official Daily Top10 gate; language/license confirmed via GitHub API get_repository + README + LICENSE; docs, PyPI, and arXiv links verified from the README)

Trending note: +873 stars today (captured 08:55 Asia/Shanghai)

About the license: Apache-2.0 applies to this repository as published. Third-party scanners and extras may carry their own terms. This is a summary, not legal advice.

Category tags

DevTools · agent-evaluation · skill-evaluation · Python · Apache-2.0 · CLI · CI · NVIDIA

How it differs from similar projects

Compared with skills or prompt packs (including filtered trending repos). Those distribute instructions for an agent to follow. SkillEvaluator evaluates a skill you already have: quality, overlap, and live impact. Do not treat the root skills/ folder as a content pack.

Compared with generic agent eval harnesses (including Harbor on its own). Harbor is documented here as the sandbox runner for Tier 3. SkillEvaluator adds skill-specific quality gates, semantic overlap detection, optional synthetic eval-dataset generation, and a combined CLI/CI path around that live run.

Compared with static scanners alone (Semgrep, Gitleaks, and similar). Those can be part of Tier 1. The framework's further claim is rubric scoring and a with-versus-without-skill live evaluation — capabilities that need a model and, for Tier 3, an agent sandbox. TidyHow has not audited detection rates or score quality.

Caveats. Support is Experimental. A scanner that cannot run is marked INCOMPLETE and can fail the command. Model endpoints and managed sandboxes can incur charges. The README's example scores apply to bundled reference skills, not to your skill and not to a TidyHow benchmark. No independent accuracy, coverage, or cost figures are claimed here.

Repo link

https://github.com/NVIDIA/SkillEvaluator