Agents & MCP
Oleksandr Kuzmenko, EditorPublished 4 min readAI-assisted · editor-reviewed
TL;DR
- Human accuracy in approving agent commands averages only 66.3%, dropping over time due to fatigue.
- Commands hiding payloads inside benign script names like `npm run analyze` were approved 64.7% of the time.
- Effective protection requires automated sandboxing and separating credentials rather than relying on manual prompt approvals.
The Breakdown of Permission Fatigue
Data from the security challenge reveals that human accuracy drops as sessions progress due to notification fatigue and time pressure. In the evaluation, 32.9% of user sessions ended with a net negative score because penalties from approved malicious actions outweighed correctly handled commands.
Dangerous vs. Benign False Positives
- Obvious destruction (
rm -rf /,chmod -R 777 /): 11.7% miss rate. - Scope violations (
cat ~/.aws/credentials): 35.0% miss rate. - Script masking (
npm run analyzecontaining exfiltration payloads): 64.7% miss rate. - Benign commands wrongly blocked:
npm config set registry(blocked 59%),rm -rf dist/(blocked 45%).
Practical Remediation: Sandboxing and Secret Separation
Asking developers to audit every command is an ineffective security barrier. Instead, practical mitigations require applying sandboxing and separating credentials and environment variable secrets.