← Back to Blog

Shelby.xyz: The Decentralized Hot Storage Protocol Built for AI & Web3

Published2026-04-26
AuthorDevansh
Tags
Web3Decentralized StorageAptosAI InfrastructureBlockchainNodeTestnetNode Operation

⚡ TL;DR

Shelby.xyz is a decentralized, cloud-grade "hot storage" network co-built by Aptos Labs and Jump Crypto. It combines erasure coding, a private fiber network, and Aptos smart contracts to deliver sub-second global data access with cryptographic provenance — built for AI, streaming, and Web3 workloads.


What Is Shelby.xyz?

Shelby.xyz is not your typical blockchain project. It's a decentralized hot storage protocol — think AWS S3, but trustless, provenance-aware, and built natively for AI and Web3 workloads.

The project is co-developed by Aptos Labs (the team behind the Aptos L1 blockchain) and Jump Crypto (one of the most technically deep firms in crypto infrastructure). Together, they've built a system designed to solve one of the most overlooked problems in Web3: fast, reliable, verifiable data storage at scale.

While blockchains like Filecoin or Arweave focus on cold/archival storage, Shelby targets the opposite end of the spectrum — high-frequency reads and writes where milliseconds matter. This makes it uniquely positioned for:

  • AI model inference and training pipelines
  • Real-time data streaming
  • Data-intensive decentralized applications (dApps)
  • Web3-native media and content platforms

Network Status

Shelby is currently live on testnet (Early Access) as of early 2026, with full production expected later in 2026.


How Shelby.xyz Works: The Core Architecture

Shelby's architecture is elegant in how it separates concerns. Rather than reinventing everything from scratch, it leverages existing battle-tested infrastructure where it matters most.

The Three Pillars

1. Aptos Smart Contracts — The On-Chain Source of Truth

Shelby uses the Aptos L1 blockchain for coordination and verification, not for storing data itself. Smart contracts manage:

  • System state and blob metadata — what data exists, where it lives
  • Placement groups — which storage nodes hold which chunks
  • Cryptographic commitments — Merkle roots for data integrity
  • Auditing logic — on-chain verification of storage proofs
  • Slashing conditions — penalties for misbehaving nodes

This design keeps "correctness-critical" operations on-chain while pushing high-throughput data operations off-chain — a smart architectural tradeoff.

2. Storage Provider (SP) Nodes — Where Data Actually Lives

Storage Providers are the decentralized nodes that store user data. Each SP holds erasure-coded chunks of data and participates in the network's auditing system.

Key Technical Details

  • Use Clay Codes for erasure coding — 10 data chunks + 6 parity chunks per 10 MB chunkset
  • Data can be reconstructed even if multiple nodes go offline
  • SPs validate incoming chunks on write, serve them on reads, and sign cryptographic acknowledgments
  • They respond to audit challenges, proving they actually possess the data they claim to store
  • They can trigger slashing for peers caught cheating or dropping data

🔧 Reference Implementation: Cavalier

The reference implementation for Storage Providers is Cavalier — an open-source, high-performance node client written in C by Jump Crypto, inspired by their Firedancer Solana validator client.

3. Shelby RPC Servers — The Client Gateway

RPC nodes are the interface between end users/developers and the storage network. They:

  • Accept client reads and writes via S3-compatible APIs and SDKs
  • Handle erasure encoding of data before distribution
  • Query smart contracts to discover SP locations (placement groups)
  • Manage micropayment sessions for read access
  • Validate chunks against on-chain metadata
  • Reassemble and serve data with cryptographic provenance receipts

The Private Fiber Network: Why Shelby is "Hot" Storage

One of Shelby's most distinctive features is its dedicated private fiber mesh between RPC nodes and Storage Providers. This is not the public internet — it's a purpose-built, high-bandwidth network designed for:

  • Sub-second global latency on reads and writes
  • Predictable, low-jitter performance — critical for AI and streaming
  • Security — data in transit stays off the public internet

This fiber network is what earns Shelby the "hot storage" label. Cold storage protocols like Filecoin are cheap and durable, but retrieval can take minutes or hours. Shelby aims for the performance profile of a premium cloud object store — with the verifiability of a blockchain.


Data Flow: What Happens When You Store or Retrieve Data

Writing Data

1

Client → RPC Server: You send data via the S3-compatible SDK or CLI

2

RPC Server: Erasure-encodes your data into 16 chunks (10 data + 6 parity)

3

RPC → 16 Storage Providers: Chunks are distributed across a placement group over the fiber network

4

SPs → On-Chain: Storage Providers sign acknowledgments; confirmation is recorded on Aptos

Client receives: A cryptographic receipt with provenance proof

Reading Data

1

Client → RPC Server: Request data, initiate a micropayment session

2

RPC Server: Queries Aptos smart contract to locate the placement group

3

RPC → SPs: Fetches chunks from the relevant Storage Providers

4

RPC Server: Validates chunks against on-chain Merkle commitments, reassembles data

Client receives: Data + a cryptographic provenance receipt


Cavalier: The Reference Node Implementation

Cavalier is Jump Crypto's open-source reference implementation for Shelby Storage Provider nodes. It's written in C and takes inspiration from Firedancer — Jump's high-performance Solana validator client — making it one of the most performance-optimized node implementations in the Web3 storage space.

Cavalier's Tile-Based Architecture

Cavalier uses an isolated tile architecture where each functional component runs as a separate process on a dedicated CPU core, communicating via shared memory. This design eliminates cache conflicts and delivers predictable, low-latency performance.

Tile Function
Server Tile Handles RPC/protobuf communication with other nodes
Engine Tile Manages disk I/O using io_uring and direct I/O
Client Tile Streams Aptos state via gRPC indexer; runs audit and rebuild state machines
Rebuild Tile Recovers lost chunks using Clay erasure codes
Sign Tile Generates cryptographic signatures for acknowledgments and Aptos transactions

💡 Why This Matters

This architecture ensures that a slow disk I/O operation in the Engine Tile never stalls the Server Tile handling network requests — each tile is isolated and independently schedulable.


The Incentive Model: Why Nodes Stay Honest

Shelby uses a hybrid auditing system to ensure Storage Providers actually store the data they claim to hold. The system has two layers:

1. Internal Peer Auditing

Storage Providers audit each other. Nodes within a placement group regularly challenge their peers to prove data possession using cryptographic proofs. Honest behavior is provably a Nash equilibrium — meaning no individual SP has an incentive to cheat, given that their peers are also auditing them.

The system is designed to be ε-coalition resistant, meaning even coordinated groups of malicious nodes cannot profitably cheat without getting caught and slashed.

2. On-Chain "Audit-the-Auditor"

To prevent auditors from lying about audit results, Shelby also runs on-chain verification of the auditing process itself. Smart contracts can verify that audits are being performed correctly and slash nodes that falsify results.

Read-Based Incentives

Uniquely, Shelby's economic model includes read-based incentives — Storage Providers earn rewards not just for storing data, but for serving it. This aligns node operators' incentives with actual network utility, not just passive storage.


Node Types: Is There a "Validator Node" in Shelby?

Shelby is not a blockchain and does not have traditional Proof-of-Stake validator nodes. However, the network has nodes that perform validation in different senses:

Node Type Validation Role
Aptos Smart Contracts On-chain source of truth; validates data placements, audits, and commitments
Storage Provider (SP) Nodes Validate data integrity via audit challenges and proofs
Shelby RPC Nodes Validate chunks against on-chain metadata during reads/writes
Aptos Validators (shelbynet) Run separately for the testnet; standard Aptos validators, not Shelby-specific

Note

When people ask about "Shelby validator nodes," they're most often referring to Storage Provider nodes — the nodes that actively prove and verify data possession.


Shelby Testnet: Current Status and Networks

Shelby is currently in Early Access on testnet (as of early 2026). Two network environments exist:

Network Description
Main Testnet api.testnet.shelby.xyz — the primary testnet for developers
Shelbynet A more frequently wiped development network with its own Aptos full nodes, indexer, and validators

📊 Real-World Scale Demonstrated

Despite being in testnet, Shelby has already demonstrated significant scale — including storing 400TB+ of archival data in demos. This signals that the architecture is production-ready, even if access is still gated.


How to Run a Shelby Storage Provider Node

⚠️ Important

As of early 2026, Shelby node setup is not fully self-serve. Infrastructure setup is managed by the Shelby team.

To get involved as a node operator:

1

Join the Shelby Discord — find the link at shelby.xyz

2

Reach out via email — contact details are available on the docs

3

Review the Cavalier implementationdocs.shelby.xyz is the reference for storage operators

Expected Hardware Requirements (When Public)

  • High-bandwidth network connectivity (fiber preferred)
  • NVMe or high-performance SSD storage
  • Multi-core CPUs for tile-based parallel processing
  • Staking/collateral on Aptos for slashing conditions

Shelby vs. Other Decentralized Storage Protocols

Protocol Storage Type Speed Verifiability AI/Streaming Focus
Shelby.xyz Hot (object storage) Sub-second Cryptographic proofs + on-chain ✅ Primary focus
Filecoin Cold/warm Minutes to hours Proof of Spacetime
Arweave Permanent/archival Slow retrieval Blockweave
Storj Hot Fast Audit-based Partial
AWS S3 Hot Sub-second Centralized ✅ (but centralized)

Key Insight

Shelby's unique position is at the intersection of S3-grade performance and blockchain-grade verifiability — a gap that no other protocol has convincingly filled.


Who Should Build on Shelby?

Shelby is purpose-built for builders who need fast, verifiable, decentralized data storage. Ideal use cases include:

🤖 AI Companies

Running inference workloads that need low-latency model artifact storage

🎮 Web3 Gaming Studios

Storing game assets with provenance guarantees

💰 DeFi Protocols

Storing off-chain data with on-chain verifiability

📺 Media Platforms

Streaming content with cryptographic ownership receipts

🗳️ Data DAOs

Managing large datasets with transparent access controls

📦 Archival Services

Needing 400TB+ scale with redundancy guarantees

The S3-compatible API means that migrating from AWS S3 requires minimal code changes — a major developer experience advantage.


FAQ

What is Shelby.xyz?

Shelby.xyz is a decentralized, cloud-grade hot storage protocol designed for fast reads and writes. It is co-developed by Aptos Labs and Jump Crypto and uses the Aptos blockchain for coordination and verification. It targets AI, streaming, and Web3 workloads.

Is Shelby.xyz a blockchain?

No. Shelby is not a blockchain. It is a storage network that uses the Aptos L1 blockchain for on-chain coordination, metadata management, and audit verification — but data itself is stored on decentralized Storage Provider nodes, not on-chain.

What are Storage Provider nodes in Shelby?

Storage Provider (SP) nodes are the decentralized nodes that store erasure-coded data chunks and participate in the network's mutual auditing system. They use the Cavalier reference implementation, developed by Jump Crypto. SPs prove data possession via cryptographic challenges and can slash misbehaving peers.

How does Shelby ensure data integrity?

Shelby uses a hybrid auditing system: (1) internal peer auditing where Storage Providers challenge each other to prove data possession, and (2) on-chain "audit-the-auditor" verification via Aptos smart contracts. The system uses Clay Codes (erasure coding) so data can be recovered even if multiple nodes fail.

Can I run a Shelby node?

As of early 2026, Shelby node setup is managed and not fully self-serve. You need to contact the Shelby team via Discord or email (details at shelby.xyz or docs.shelby.xyz). The Cavalier reference implementation is publicly available for review.

What is Cavalier in Shelby?

Cavalier is the open-source reference implementation for Shelby Storage Provider nodes, written in C by Jump Crypto. It uses a tile-based architecture with isolated processes on dedicated CPU cores for high-performance, predictable I/O.

What makes Shelby different from Filecoin or Arweave?

Filecoin and Arweave focus on cold or archival storage where retrieval speed is secondary to cost and permanence. Shelby focuses on "hot storage" — sub-second reads and writes via a private fiber network — making it competitive with AWS S3 in performance, while adding cryptographic provenance and decentralization.


Get Started with Shelby.xyz

👉 Need help setting up Web3 infrastructure?

Contact DevOpsby.me — we help teams set up and manage high-performance Web3 node infrastructure.

DevOpsBy Assistant

How can we help?

Select a topic to get started: