From Filecoin, Arweave to Walrus, Shelby: The Development History and Future Prospects of Decentralization Storage

Storage has been one of the popular tracks in the blockchain industry. Filecoin, as the leading project of the last bull market, once had a market capitalization exceeding 10 billion USD. In contrast, Arweave, which focuses on permanent storage, reached a peak market capitalization of 3.5 billion USD. However, as the availability of cold data storage comes into question, the development prospects of decentralized storage have also been cast into doubt. Recently, the emergence of Walrus has brought new attention to the long-silent storage track, and the Shelby project, jointly launched by Aptos and Jump Crypto, attempts to achieve new breakthroughs in the field of hot data storage. This article will analyze the evolution path of decentralized storage based on the development history of these representative projects and explore its future development prospects.

Filecoin: The Essence of Mining Coins Beneath the Storage Cloak

Filecoin is one of the early rising blockchain projects, with its development direction centered around Decentralization. Filecoin aims to shift centralized storage to decentralized storage, but the trade-offs made to achieve decentralization during this process have also become pain points that subsequent projects attempt to address.

IPFS: Architecture Decentralization, but limited by transmission bottlenecks

IPFS(, the InterPlanetary File System, was launched in 2015 with the aim of disrupting the traditional HTTP protocol through content addressing. However, the biggest drawback of IPFS is its extremely slow retrieval speed. In an era where traditional data service providers can achieve millisecond-level responses, retrieving a file from IPFS still takes several seconds, making it difficult to promote in practical applications.

IPFS is mainly suitable for "cold data," which refers to static content that does not change frequently. However, when it comes to handling hot data, such as dynamic web pages, online games, or AI applications, P2P protocols do not show significant advantages compared to traditional CDNs.

Although IPFS itself is not a blockchain, its design concept of a directed acyclic graph )DAG( is highly compatible with many public chains and Web3 protocols, making it suitable as a foundational building framework for blockchains.

) The logic of mining coins under the storage cloak

In the token economic model of Filecoin, there are mainly three roles: users pay fees to store data; storage miners receive token incentives for storing user data; retrieval miners provide data when users need it and receive incentives.

This model has potential malicious space. Storage miners may fill garbage data after providing storage space to obtain rewards. Since this garbage data will not be retrieved, even if lost, it will not trigger the penalty mechanism. This allows storage miners to delete garbage data and repeat this process. Filecoin's proof-of-replication consensus can only ensure that user data has not been privately deleted, but it cannot prevent miners from filling garbage data.

The operation of Filecoin largely depends on the continuous investment of miners in the token economy, rather than the actual demand from end users for distributed storage. Although the project is still iterating, at this stage, the ecological construction of Filecoin is more in line with the "mining coin logic" rather than the definition of "application-driven" storage projects.

Arweave: Founded on Long-Termism, Defeated by Long-Termism

If the design goal of Filecoin is to build an incentivized, verifiable Decentralization "data cloud" shell, then Arweave takes another extreme direction in storage: providing the capability for permanent storage of data. Arweave does not attempt to build a distributed computing platform; its entire system is based on a core assumption—that important data should be stored once and retained forever on the network. This extreme long-termism makes Arweave fundamentally different from Filecoin in terms of mechanisms, incentive models, hardware requirements, and narrative perspectives.

Arweave uses Bitcoin as a learning object, attempting to continuously optimize its permanent storage network over long periods measured in years. Arweave does not care about marketing, nor does it care about competitors and market trends. It is simply moving forward on the path of iterating its network architecture, indifferent even if no one pays attention, because this is the essence of the Arweave development team: long-termism. Thanks to long-termism, Arweave was warmly embraced during the last bull market; and because of long-termism, even if it falls to the bottom, Arweave may still survive several rounds of bull and bear markets. The only question is whether there will be a place for Arweave in the future of decentralized storage. The value of permanent storage can only be proven over time.

Despite losing market discussion heat, the Arweave mainnet has been working hard to enable a wider range of miners to participate in the network at minimal cost, and to incentivize miners to maximize data storage, thereby continuously enhancing the robustness of the entire network. Arweave has taken a conservative approach, fully aware that it does not align with market preferences, avoiding embracing the miner community, resulting in a completely stagnant ecosystem, upgrading the mainnet at minimal cost while continuously lowering hardware thresholds without compromising network security.

Review of the upgrade path from 1.5 to 2.9

The Arweave version 1.5 exposed a vulnerability that allowed miners to rely on GPU stacking instead of actual storage to optimize block creation chances. To curb this trend, version 1.7 introduced the RandomX algorithm, which restricts the use of specialized computing power and instead requires general-purpose CPUs to participate in mining, thereby weakening computing power centralization.

In version 2.0, Arweave adopts SPoA, transforming data proofs into a concise path of the Merkle tree structure, and introduces format 2 transactions to reduce synchronization burdens. This architecture alleviates network bandwidth pressure, significantly enhancing the collaborative capability of nodes. However, some miners can still evade the responsibility of holding real data through centralized high-speed storage pool strategies.

To correct this bias, version 2.4 introduced the SPoRA mechanism, which incorporates global indexing and slow hash random access, requiring miners to genuinely hold data blocks to participate in effective block generation, thereby weakening the effects of hash power stacking from a mechanism perspective. As a result, miners began to pay attention to storage access speeds, driving the application of SSDs and high-speed read-write devices. Version 2.6 introduced a hash chain to control the rhythm of block generation, balancing the marginal benefits of high-performance devices and providing a fair participation space for small and medium miners.

Subsequent versions further strengthen network collaboration capabilities and storage diversity: 2.7 adds collaborative mining and pool mechanisms to enhance the competitiveness of small miners; 2.8 introduces a composite packaging mechanism that allows large-capacity low-speed devices to participate flexibly; 2.9 introduces a new packaging process in the replica_2_9 format, significantly improving efficiency and reducing computational dependence, completing the closed loop of data-driven mining models.

Overall, Arweave's upgrade path clearly presents its long-term strategy oriented towards storage: while continuously resisting the trend of computing power centralization, it consistently lowers the participation threshold to ensure the long-term viability of the protocol.

![From FIL, Arweave to Walrus, Shelby: How far is the popularization of Decentralization storage?]###https://img-cdn.gateio.im/webp-social/moments-1ebd281e65dedbe6216b5e1496a2963e.webp(

Walrus: Is Embracing Hot Data Hype or a Hidden Opportunity?

The design philosophy of Walrus is completely different from that of Filecoin and Arweave. Filecoin starts with the goal of creating a decentralized and verifiable storage system, at the cost of cold data storage; Arweave aims to create an on-chain library of Alexandria for permanent data storage, at the cost of having too few scenarios; Walrus, on the other hand, aims to optimize storage costs for hot data storage protocols.

) Magic Modified Error-Correcting Code: Cost Innovation or New Wine in an Old Bottle?

In terms of storage cost design, Walrus believes that the storage overhead of Filecoin and Arweave is unreasonable. Both latter systems adopt a fully replicated architecture, whose main advantage is that each node holds a complete copy, providing strong fault tolerance and independence between nodes. This type of architecture ensures that even if some nodes are offline, the network still possesses data availability. However, this also means that the system requires multiple copies of redundancy to maintain robustness, thereby increasing storage costs. Especially in the design of Arweave, the consensus mechanism itself encourages node redundant storage to enhance data security. In contrast, Filecoin is more flexible in cost control, but at the expense of potentially higher data loss risk for some low-cost storage. Walrus attempts to find a balance between the two, whose mechanism enhances availability while controlling replication costs, thus establishing a new trade-off path between data accessibility and cost efficiency.

The Redstuff created by Walrus is a key technology for reducing node redundancy, which originates from Reed-Solomon ### RS ( coding. RS coding is a very traditional erasure code algorithm, and erasure codes are a technology that allows data sets to be doubled by adding redundant fragments ) erasure code (, which can be used to reconstruct the original data. From CD-ROMs to satellite communications to QR codes, it is frequently used in daily life.

Erasure coding allows users to take a block, for example, 1MB in size, and then "amplify" it to 2MB in size, where the additional 1MB is special data known as erasure coding. If any byte in the block is lost, users can easily recover those bytes through the code. Even if up to 1MB of the block is lost, the entire block can still be recovered. The same technology allows computers to read all the data on a CD-ROM, even if it has been damaged.

The most commonly used is RS coding. The implementation method starts with k information blocks, constructs the related polynomial, and evaluates it at different x coordinates to obtain the encoded blocks. Using RS erasure codes, the probability of randomly sampling large chunks of data loss is very small.

What is the main feature of the RedStuff encoding algorithm? By improving the erasure coding algorithm, Walrus can quickly and robustly encode unstructured data blocks into smaller shards, which are distributed across a storage node network. Even if up to two-thirds of the shards are lost, the original data block can be quickly reconstructed using partial shards. This is made possible while maintaining a replication factor of only 4 to 5 times.

Therefore, it is reasonable to define Walrus as a lightweight redundancy and recovery protocol redesigned around the Decentralization scenario. Compared to traditional erasure codes ) such as Reed-Solomon (, RedStuff no longer pursues strict mathematical consistency, but instead makes realistic trade-offs regarding data distribution, storage verification, and computational costs. This model abandons the instantaneous decoding mechanism required for centralized scheduling, opting instead to verify whether nodes hold specific data copies through on-chain Proof, thus adapting to a more dynamic and marginalized network structure.

The core design of RedStuff is to split data into two categories: primary slices and secondary slices. Primary slices are used to restore the original data, and their generation and distribution are subject to strict constraints, with a recovery threshold of f+1, requiring 2f+1 signatures as availability endorsement. Secondary slices are generated through simple operations like XOR combinations, serving the purpose of providing elastic fault tolerance and enhancing the overall robustness of the system. This structure essentially reduces the requirements for data consistency—allowing different nodes to temporarily store different versions of data and emphasizing the practical path of "eventual consistency." Although it is similar to the lenient requirements for backtracking blocks in systems like Arweave, achieving some effectiveness in reducing network burden, it also weakens the guarantees of data immediacy and integrity.

It is important to note that although RedStuff has achieved effective storage in low computing power and low bandwidth environments, it essentially remains a "variant" of erasure coding systems. It sacrifices some data read determinism in exchange for cost control and scalability in a decentralized environment. However, at the application level, whether this architecture can support large-scale, high-frequency interactive data scenarios remains to be seen. Furthermore, RedStuff has not truly broken through the long-standing computational bottleneck of erasure coding, but rather avoided the high coupling points of traditional architectures through structural strategies. Its innovation is more reflected in the combinatorial optimization on the engineering side rather than a disruption at the foundational algorithm level.

Therefore, RedStuff is more like a "reasonable modification" aimed at the current reality of decentralized storage. It indeed brings improvements in redundancy costs and operational load, allowing edge devices and non-high-performance nodes to participate in data storage tasks. However, in large-scale applications, general computing adaptation, and business scenarios with higher consistency requirements, its capability boundaries are still quite apparent. This makes Walrus's innovation more like an adaptive transformation of the existing technological framework, rather than a decisive breakthrough in promoting the migration of the decentralized storage paradigm.

) Sui and Walrus: High-performance public chains can drive storage practicality