OpenLake Storage Engine Beats MLPerf Storage v3.0 for LLM Checkpointing
OpenLake launched its asynchronous storage engine for LLM training and KV cache offloading, taking first place in MLPerf Storage v3.0. It reached 6.72 GiB/s write and 11.55 GiB/s read bandwidth on Llama 3.1 8B checkpointing.
Impact: Medium
Why it matters
Cut GPU idle time during large distributed training runs and accelerate host-to-storage KV cache offload.
TL;DR
- 01OpenLake achieved 6.72 GiB/s write and 11.55 GiB/s read bandwidth on MLPerf Storage v3.0.
- 02Combines Rust compio, io_uring, and GPUDirect Storage to eliminate kernel I/O scheduling bottlenecks.
- 03Reduces Llama 3.1 8B checkpoint pauses to 29.42 seconds over standard S3-compatible endpoints.
Key facts
- Checkpoint write bandwidth
- 6.72 GiB/s (MLPerf Storage v3.0)
- Checkpoint read bandwidth
- 11.55 GiB/s
- Mean write duration (Llama 3.1 8B)
- 29.42 s
- Mean read duration (Llama 3.1 8B)
- 9.37 s
- Network fabric
- 400 Gb/s InfiniBand to S3 API
MLPerf Storage v3.0 Benchmark Results
OpenLake's Infinity Core I/O Engine ranked first among Closed division S3 results in the MLCommons MLPerf Storage v3.0 benchmark. Evaluating the Llama 3.1 8B checkpointing workload (8 processes writing and reading 16 files across 10 iterations), OpenLake delivered 6.72 GiB/s write bandwidth—1.98× faster than the next best submission (NVIDIA AIStore 6-node at 3.40 GiB/s) and Nebius Object Storage (2.81 GiB/s).
Thread-Per-Core Architecture and KV Offloading
OpenLake implements an asynchronous I/O engine built with Rust compio using a thread-per-core design, asynchronous io_uring kernel operations, GPUDirect Storage, and on-GPU compression. The system connects via an S3 API interface over a 400 Gb/s InfiniBand fabric to NVMe-backed storage gateways. In production, OpenLake achieved over 1M IOPS with sub-millisecond latency on a 96-node cluster.
Slashing Distributed Training Idle Time
In synchronous distributed training, all accelerators freeze execution until state checkpoints commit durably. OpenLake reduced the mean checkpoint write duration for Llama 3.1 8B to 29.42 seconds (compared to 37.24s for Nebius) and recovery read duration to 9.37 seconds (11.55 GiB/s bandwidth). This throughput minimizes GPU idle time during routine checkpointing and accelerates post-failure cluster recovery.
Try it in 2 minutes
git clone https://github.com/openlake-project/openlake.git
cd openlake && cargo build --releasebash
✓ When to use
- Use when scaling multi-node pre-training or RL runs where synchronous checkpoint pauses stall GPU compute.
- Use to offload large context-window KV cache data from GPU memory to NVMe over S3-compatible interfaces.
✕ When NOT to use
- Do not deploy for simple single-GPU inference where KV cache comfortably fits in VRAM.
- Avoid if your infrastructure lacks NVMe storage or high-speed fabric (InfiniBand/RoCE), which limits io_uring gains.
What to do today
- Inspect the openlake repository to benchmark your cluster's checkpointing write speeds against 6.72 GiB/s.
- Evaluate io_uring-based object storage gateways if GPU clusters spend over 10% of runtime paused for state writes.
What the community says
“Today we are launching OpenLake, a KV offloading solution for modern LLM workloads. OpenLake achieved 1M+ iops in 1ms on a 96 node cluster... through rust compio with on GPU compression.”
Sources