Benchmarking Claude Opus 5.5: Why Agent Skills and CLI Updates Still Matter
Driftproof benchmarked Claude Opus 5.5 within 15 hours of release across three core agent skills from addyosmani/agent-skills. Without explicit SKILL.md guidance, Opus 5.5 regressed on conventional commit prefixes and review severity labels compared to Opus 5. Calling the new model in Claude Code also strictly requires upgrading to version 2.1.280 or newer.
Impact: Medium
Why it matters
You should retain your SKILL.md configuration files rather than assuming the upgraded frontier model makes explicit system skills redundant.
TL;DR
- 01Do not remove SKILL.md steering files; unassisted Opus 5.5 scored 0.300 on conventional commit formatting compared to 0.765 on Opus 5.
- 02Update Claude Code to version 2.1.280 or later before attempting to run prompts against Claude Opus 5.5.
- 03Verify benchmark receipts locally using the open-source Driftproof CLI validator tool.
Key facts
- Required Claude Code CLI Version
- >= 2.1.280
- Total Model Invocations Evaluated
- 204 calls
- Unassisted Git Conventional Commits Score (Opus 5.5 vs Opus 5)
- 0.300 vs 0.765
- Unassisted Code Review Severity Tagging Score (Opus 5.5 vs Opus 5)
- 0.680 vs 0.862
- Metered Benchmark Cost
- $0.00 subscription (~$10 API equivalent)
Model Upgrades Require Runtime Upgrades
When testing Claude Opus 5.5 immediately after release, Driftproof identified that Claude Code CLI version 2.1.272 stopped at its first API call to Opus 5.5 with an explicit error requiring version 2.1.280 or newer. Transitioning automated agent harnesses to Opus 5.5 is not just a configuration change; local environments and automated pipelines must update the Claude Code package before changing target models.
Skills Still Prevent Format and Lint Regressions
Across three public benchmarked skills (code-review-and-quality, git-workflow-and-versioning, documentation-and-adrs), the presence of a SKILL.md file neutralized output regressions:
- Conventional Commits: Without instructions, Opus 5.5 dropped conventional
fix:prefixes, scoring0.300against Opus 5's0.765. - Code Review Severity: In unassisted code reviews, Opus 5.5 omitted severity labels in two out of four runs under category headers, scoring
0.680compared to0.862for unassisted Opus 5. - Assisted Equality: When guided by
SKILL.md, score deltas between Opus 5 and Opus 5.5 remained narrow (+0.009,-0.023,-0.143on a 0 to 1 scale).
Verifiable Evaluation Receipts
The 204 model calls consumed 0.00 dollars on a flat subscription (approximately $10 at metered API pricing). Every execution is pinned by byte hashes, allowing developers to inspect and validate outputs locally with npx driftproof validate <file>.
Try it in 2 minutes
# Update Claude Code to support Opus 5.5
npm install -g @anthropic-ai/claude-code@latest
# Validate benchmark receipts from Driftproof Report 011
npx driftproof validate report-011.receipt.jsonbash
✓ When to use
- Use when transitioning developer agent harnesses and automated code review pipelines to Claude Opus 5.5.
- Use when validating whether agentic skill files provide measurable value across model iterations.
✕ When NOT to use
- Do not run Claude Code on version 2.1.272 when configured to invoke Opus 5.5.
- Do not strip out SKILL.md prompt files assuming Opus 5.5 natively enforces specific linting formats without explicit rules.
What to do today
- Run npm install -g @anthropic-ai/claude-code@latest to verify Claude Code is at least 2.1.280.
- Keep SKILL.md active in your repositories for git commit generation and code review tasks.
Sources