HBS2 P2P Storage and Platform

What is HBS2?

An open-source, decentralized content-addressable storage (CAS) system with peer-to-peer replication. No central servers. Local-first. End-to-end encryption.

HBS2 implements and provides functions required for P2P and local-first applications development.

Why HBS2?

Have you ever thought that all your team members have a full copy of the source code of projects you work on because of Git's nature, and that you actually do not need GitHub or any other central service?

Have you ever thought that something like Git + BitTorrent may bring a totally decentralized way of working, without relying on domain systems, service providers, particular IP addresses, hosts, specific physical servers, desktops, phones, phone numbers, emails, or accounts in social networks?

Anything that could be broken, stolen, leaked, banned, confiscated, cut off, censored, compromised, and so on.

That any data of any size may be described as a 44-symbol (32-byte) hash code, and that’s basically all you need to retrieve any of your data back at any time.

For example, this entire website may be identified as 4X65y4YvUjRL2gtA9Ec3YDDP4bnxjTGhfjpoah96t3z1

Have you ever thought that all you need to address someone, send them a message, share data, authenticate yourself, and verify counterparty authorization is just a couple of encryption keys (32 bytes / 44 symbols each).

Have you ever thought that all your data may be dynamically replicated across multiple servers (and you can know exactly how many, if you want)?

If so, HBS2 might be useful for you.

HBS2 Status

It works.

HBS2 has been under development since December 2022, with earlier attempts dating back to 2018-2019 when I was searching for decentralized collaboration tools but found nothing that suited my needs.

Since March 2023, HBS2 has been fully self-hosted. All work happens within HBS2 itself — GitHub serves only as a mirror and a way for non-HBS2 users to access the source code.

HBS2 can now host websites in a P2P fashion, which is how this page appeared.

Supported platforms

NixOS 22.11+ Tested/Works
Ubuntu 22.04+ Tested/Works
Linux Mint Launched/Works
Debian 12 Tested/Works
Raspberry Pi OS Tested/Works
Mac OS X/aarch64 Tested/Works
Mac OS X/x86 Launched/Works
Nix-on-Droid Launched/?

Features

Bootstrap / Peer Discovery

Enables decentralized discovery of new peers without relying on central servers, using DNS bootstrap, pre-set seed peers and multicasting

(PEX) Peer Exchange

Peers share information about other active peers

NAT Traversal

HBS2 effectively handles NAT traversal using a combination of peer-assisted routing, UDP, gossip-based discovery and message relaying, and duplex channels over client-initiated connections, allowing seamless bidirectional communication.

Pull Model

HBS2 follows a pull-based model, where peers proactively request the data they need. This reduces load and ensures that most of the work is done by the requesting side.

P2P Gossip

Gossip-based message forwarding enables peer communication even when there is no direct connection between them.

P2P Authentication / Authorization

P2P Messaging / PubSub

Peers only listen to and relay data for topics they are subscribed to.

This allows multiple independent groups and channels to coexist, enabling different applications to operate without interference.

Transport Layer Encryption

Uses elliptic curve cryptography (libsodium) for message encryption. Ensures uniform encryption across all transport protocols, including TCP and UDP.

Mutable References with Cryptographic Access Control

HBS2 provides multiple types of CRDT-based mutable references with built-in cryptographic proofs and access control, ensuring secure and decentralized data updates.

Storage Layer Encryption

Blocks and Merkle trees can be stored with end-to-end encryption for multiple recipients using group encryption keys.

Any peer can seed encrypted data without being able to decrypt it, ensuring secure distribution while maintaining privacy.

Synchronization and Replication

Network-enabled CRDT data structures and API for data replication with cryptographic authentication and authorization

Merkle Trees Tooling

Efficient data verification and replication mechanisms using Merkle Trees.

Cryptographic Primitives for P2P

Built-in cryptographic tools for secure peer-to-peer communication.

Consensus

HBS2 primarily avoids global online consensus, relying instead on CRDT entities for data replication.

However, certain channels use a PBFT-like quorum consensus mechanism. Additionally, reference implementations of other PBFT-like protocols and Raft exist, built on top of HBS2 networking primitives.

RPC and Libraries

HBS2 provides an RPC mechanism for interacting with protocol nodes, allowing external applications to integrate without requiring direct use of the HBS2 codebase or re-implementing peer (node) protocols.

HBS2 also includes a set of libraries for networking, data structures, and cryptography, which can be used independently in projects unrelated to HBS2 itself.

Applications

HBS2 includes a number of built-in applications.

Distributed P2P Git

While Git is known as a DVCS (Distributed Version Control System), it is not truly decentralized. It lacks a distributed transport layer, leading to centralization, with GitHub being a prime example.

"hbs2-git" offers a fully decentralized solution for collaborative development, extending Git's capabilities.

Built on top of Git’s existing mechanisms, "hbs2-git" enables content distribution over HBS2 protocols. Developers can work with distributed storage just like a regular Git repository, with automatic replication across all participants—no central servers required.

Supports end-to-end group encryption, making it suitable for both private projects and open-source collaboration (if not encrypted).

Distributed Issue Tracker

fixme A simple yet powerful distributed CLI issue tracker, fully integrated with Git repositories.

Demonstrates the use of CRDTs and RefChans for SQL database synchronization, with built-in access control and encryption.

Decentralized Content Sharing and Publishing

Published data is automatically replicated across all peers subscribed to a specific channel or reference.

Data can be end-to-end encrypted using group encryption keys, ensuring secure distribution.

P2P / Decentralized Directory Synchronization

The hbs2-sync app enables directory synchronization across teams in a fully decentralized P2P environment. It supports end-to-end encryption with group keys and fine-grained read/write access control.

Decentralized Messaging

Encrypted, peer-to-peer messaging without relying on central servers.

Encrypted Data Replication

Synchronize and distribute encrypted datasets across multiple peers securely.

Related Projects