skill-tester

---
name: skill-tester
description: Use when authoring skills, auditing for tier promotion, setting up quality gates, or integrating skill QA into CI; validates structure, Python scripts, and scores quality with letter grades.
category: Engineering
version: 1.0.0
tools: []
---

# Skill Tester

Meta-skill for validating, testing, and scoring skills. Four tools: structure validation, Python script testing, quality scoring (letter grade + tier recommendation), security scoring.

## Quick Start

All commands run from skill root:

```
# 1. Validate structure
python3 scripts/skill_validator.py path/to/skill --json

# 2. Test Python scripts
python3 scripts/script_tester.py path/to/skill --json

# 3. Score quality
python3 scripts/quality_scorer.py path/to/skill --json --minimum-score 75
```

Consume the JSON: validator emits `overall_score`, `compliance_level`, per-check results; scorer emits `overall_score`, `letter_grade`, `tier_recommendation`, and an `improvement_roadmap`.

## What Each Tool Checks

### skill_validator.py
- SKILL.md frontmatter, required sections, minimum line counts per tier
- Required structure: SKILL.md, README.md, scripts/, references/, assets/, expected_outputs/
- Python scripts: argparse present, stdlib-only imports

### script_tester.py
- AST-based syntax validation; import analysis (flags external dependencies)
- Controlled execution with timeout protection (default 30s)
- `--help` functionality and sample-data runs compared against expected_outputs/

### quality_scorer.py
Four dimensions (25% each): **Documentation** (depth, examples, references), **Code Quality** (complexity, error handling, output consistency), **Completeness** (required dirs, sample data, expected outputs), **Usability** (help text, example clarity).

Outputs: 0-100 score + A-F grade + tier recommendation.

## Tier Classification

| Tier | SKILL.md lines | Scripts | CLI Surface |
|---|---|---|---|
| BASIC | >=100 | 1 (100–300 LOC) | Basic argparse |
| STANDARD | >=200 | 1–2 (300–500 LOC) | Subcommands, JSON + text output |
| POWERFUL | >=300 | 2–3 (500–800 LOC) | Multiple modes, CI integration |

## Verification Loop

A skill "passes" when, in one run:
1. `skill_validator.py <skill> --json` exits 0
2. `script_tester.py <skill>` reports all scripts passing
3. `quality_scorer.py <skill> --minimum-score <target>` exits 0

If any step fails, apply the top improvement_roadmap item and re-run all three.

## CI Integration

```yaml
- name: "validate-changed-skills"
  run: |
    for skill in $changed_skills; do
      python3 skill_validator.py "$skill" --json
      python3 script_tester.py "$skill"
      python3 quality_scorer.py "$skill" --minimum-score 75
    done
```

Pre-commit hook: run validator on staged skill directory, block commit on non-zero exit.

## Troubleshooting

- **Timeout errors** → raise `--timeout` or optimize the script
- **Import failures** → external deps detected; stdlib-only is required
- **Tier misclassification** → check line counts against tier table