Oleksandr Kuzmenko, EditorPublished 6 min readAI-assisted · editor-reviewed
TL;DR
- DeepSeek V4.1 Flash achieved 73% task completion on network intrusion benchmarks versus 81% for OpenAI GPT-6 Sol.
- Agent testing costs drop from roughly $22 to under $3 per simulation run when leveraging open-weight alternatives.
- Public weights enable fully local and air-gapped execution for red-teaming infrastructure without cloud gating.
Key facts
- DeepSeek V4.1 Flash Success Rate
- 73% of objectives
- OpenAI GPT-6 Sol Success Rate
- 81% of objectives
- DeepSeek Run Cost
- <$3 per test
- OpenAI GPT-6 Sol Run Cost
- ~$22 per test
- Model Weights Distribution
- Public / Openly downloadable
Source: Key facts — DeepSeek V4.1 Flash Success Rate, OpenAI GPT-6 Sol Success Rate
Source: Key facts. DeepSeek V4.1 Flash Success Rate: 73% of objectives. OpenAI GPT-6 Sol Success Rate: 81% of objectives. The chart compares only values that share a unit. Each figure is copied from the story.
Metric chart
- 73% of objectivesDeepSeek V4.1 Flash Success Rate
- 81% of objectivesOpenAI GPT-6 Sol Success Rate
- DeepSeek V4.1 Flash Success Rate73% of objectives
- OpenAI GPT-6 Sol Success Rate81% of objectives
Autonomous Penetration Testing Benchmarks
Independent evaluations conducted by Vals AI measured the capability of autonomous agent systems executing corporate cyber intrusions. The benchmark placed models in sandboxed enterprise environments with objectives including perimeter compromise, internal credential harvesting, and lateral movement across distinct subnets.
DeepSeek V4.1 Flash completed 73% of total assigned objectives across the evaluation suites. In comparison, OpenAI's specialized cyber checkpoint, GPT-6 Sol, completed 81% of objectives. Notably, DeepSeek's open architecture secured higher objective completion on one of the two isolated target networks, demonstrating that general agentic weights can navigate multi-step exploitation topologies effectively.
Inference Economics and Deployment Trade-Offs
Cost efficiency remains the defining divergence between the evaluated architectures. A full evaluation execution using DeepSeek V4.1 Flash registered under $3 in compute spend, compared to approximately $22 per execution run with GPT-6 Sol. For continuous security verification harnesses, this 86% cost reduction alters the feasibility of running automated nightly penetration tests against staging infrastructure.
Unlike GPT-6 Sol, which OpenAI reserves behind a vetted cybersecurity access program, DeepSeek weights remain publicly downloadable. While this unlocks unmonitored local execution and offline air-gapped deployments, it places the full burden of access control, sandboxing, and execution monitoring on the hosting development team.
Try it in 2 minutes
ollama run deepseek-v4.1-flashWhen to use
- Automated continuous red-teaming pipelines where API costs make frontier proprietary models prohibitive.
- Air-gapped and local security simulations requiring full telemetry control without external network calls.
When NOT to use
- Environments without isolated network sandboxing where unaligned models could impact production systems.
- Compliance workflows where model usage requires strict provider-guaranteed provenance and vendor safety attestation.
What to do today
- Evaluate open-weight DeepSeek checkpoints for internal security unit-tests to reduce monthly agent orchestration costs.
- Enforce hypervisor-level sandboxing when running multi-step security agent experiments using unguarded public models.
- Compare multi-subnet credential discovery rates across local models against proprietary API checkpoints in staging.
Step names are taken from What to do today.
Step names are taken from What to do today. Evaluate open-weight DeepSeek checkpoints for internal security unit-tests to reduce monthly agent orchestration costs.. Enforce hypervisor-level sandboxing when running multi-step security agent experiments using unguarded public models.. Compare multi-subnet credential discovery rates across local models against proprietary API checkpoints in staging.
- Evaluate open-weight DeepSeek checkpoints for internal security unit-tests to reduce monthly agent orchestration costs.
- Enforce hypervisor-level sandboxing when running multi-step security agent experiments using unguarded public models.
- Compare multi-subnet credential discovery rates across local models against proprietary API checkpoints in staging.