Accelerating Content Routing with Bitswap: A multi-path file transfer protocol in IPFS and Filecoin

Alfonso de la Rocha
Protocol Labs
alfonso@protocol.ai

David Dias
Protocol Labs
david@protocol.ai

Yiannis Psaras
Protocol Labs

Abstract

Bitswap is a Block Exchange protocol designed for P2P Content Addressable Networks. It leverages merkle-linked graphs to parallelize retrieval and verify content integrity. Bitswap is being used in the InterPlanetary File System architecture as the main content exchange protocol, as well as in the Filecoin network, as part of the block synchronization protocol. In this work, we present Bitswap’s baseline design and then apply several new extensions with the goal of improving Bitswap’s efficiency, efficacy and minimizing its bandwidth fingerprint. Most importantly, our extensions result in a substantial increase to the protocol’s content discovery rate. This is achieved by using the wealth of information that the protocol acquires from the content routing subsystem to make smarter decisions on where to fetch content from.

Index Terms

P2P, Permissionless, merkle-link, IPFS, Filecoin, DHT, Kademlia, multi-path, Content Addressing

I. INTRODUCTION

Peer-to-Peer (P2P) overlay networks have been researched extensively as an alternative content distribution model promising more efficient use of resources and faster access to content. P2P technology leverages user resources to offload servers, reducing the need for the content distributor to invest in additional server and network hardware equipment as the demand for content grows. P2P architectures are used today by P2P Content Distribution Networks (CDNs), hybrid P2P-centralized CDNs, and rapidly emerging Distributed Ledger Technology (DLT) platforms. In fact, P2P research and development has seen a renaissance as a core tenet of DLTs and blockchains with dozens of projects and initiatives pushing towards a decentralized version of the Internet and its services. There is significant momentum already being in place by hundreds of high-profile projects with large user bases, such as: Ethereum, Filecoin, Dfinity, Oasis, among others.

Content-addressable networks have also received significant attention during the last decade due to the promising features they offer. Location-independent content retrieval and arbitrary in-network caching can increase delivery performance significantly and reduce network resource requirements.

What has been mostly missing is protocol designs to optimize content routing as the primary mechanism. However, content routing systems often disregard a wealth of information that they acquire through their interactions: a DHT peer A that receives a request for content x from peer B and forwards it further along the DHT ring now knows that peer B caches content x. Subsequent requests received from A for x do not need to "walk" the DHT again – instead, A can redirect the request to node B directly. The utility of this information is not limited to networks using a DHT, but can apply to any content routing system where the content – rather than its original host – is explicitly identified.

In this paper, we introduce several novel extensions to Bitswap, the IPFS block exchange protocol, in order to enhance content resolution for content-addressable networks. Baseline Bitswap leverages the node’s connection pool and the architecture’s content resolution system to gather information and optimise content discovery. It can therefore run ahead of the main content resolution component in an attempt to discover content faster.

Bitswap operates on explicit content naming. It assumes an underlying P2P network where every peer is connected to a number of other peers forming its swarm. The baseline version of the protocol makes use of a Content Identifier (CID) to request content from the peers in its own swarm. Our novel design extensions add functionality optimising both discovery and delivery performance to the baseline Bitswap specification.

II. RELATED WORK

Additionally, we have built and make publicly available our benchmarking and testbed for IPFS and Bitswap. Our testbed is built on Testground and deployed on AWS, a platform for testing and evaluation of P2P networks, which allows for extension and reproducibility.

Bitswap can work in other environments too but for the purposes of this study we focus on P2P Networks. The promise has always been that by dealing with data transfers in a P2P manner, both central network servers and backhaul links will be offloaded. The architectural paradigm was not only considered promising for general file transfers but also for CDN architectures, VoD platforms, as well as real-time VoIP communication. Skype was among the first VoIP applications that built on P2P, while initial versions of Spotify were taking advantage of user-assisted storage.

To the best of our knowledge, not many content exchange protocols for P2P networks have been proposed in the literature. The vast majority of P2P file sharing protocols are usually focused on content discovery. One of the most widespread solutions for piece exchange is Bittorrent’s "tit-for-tat" algorithm. Generally speaking, Bittorrent peers leverage one of the available content routing systems in the network (mainly the DHT or a centralized tracker) to find seeders storing the pieces of the content.

Proposals to improve file-sharing in P2P networks have mainly focused around the use of network coding and rateless coding to leverage multiple path streams and make all content pieces equally valuable.

III. THE BITSWAP PROTOCOL

Bitswap is a message-oriented exchange protocol used to request and send content blocks between peers in a P2P content-addressable network. Bitswap’s key value proposition is the ability to traverse a hash-linked graph that represents a file structure, fetching the multiple parts of the graph from different peers, optimizing for throughput and bandwidth usage.

A. Content Objects in Bitswap

Blocks are the minimum content unit exchanged in the Bitswap protocol. When a peer wants to provide a file to the network, it chunks the file into several blocks of 256KiB (default) and up to 1MiB. Each of these blocks has its own unique identifier, the CID (Content IDentifier). The CID of a block is generated from the hash digest of the content included in the block. Any change to the content itself results in a totally different hash – and therefore, a different identifier – making CIDs immutable, permanent, and a very convenient way to uniquely identify blocks of content in the network. Blocks relate to each other through merkle-links. Blocks belonging to the same file are linked to each other following a Merkle DAG structure. In DAGs, several branches can re-converge, so nodes of the structure can have several parents. Merkle DAGs are self-verified structures.