The Next Generation of Renting Sia Storage

We are excited to announce version 1.0 of renterd, the successor of siad for renting storage on the Sia network.

Almost a decade has passed since siad‘s launch. We are very proud of what it has become. We have gathered invaluable experience along the way we have received amazing feedback from our community and third-party developers. However, we realised that the monolithic architecture ofsiad has reached its limit. It was not quite user-friendly enough for endusers and not flexible and scalable enough for enterprise. So we decided to start over from scratch using everything we’ve learned along the way to build renterd.

Horizontal Scalability

renterd can be broken apart into 3 modules with their own API:

• Bus: The core of renterd. Manages persistence via SQL and contains all consensus — related code.
• Worker: Communicates with hosts for tasks such as uploading and downloading. Stateless by itself.
• Autopilot: (optional) Talks to the Bus and Workers to automate tasks such as contract maintenance and file repairs.

By default all of the modules are enabled and run within a single process. However, horizontal scalability can be achieved by running them separately in a cluster. A cluster can share 1 Bus, 1 Autopilot and any number of Worker — nodes.

This eliminates one of the biggest downsides of siad which was the need for running multiples instances with different contract sets which all duplicates the consensus data (Blockchain) to achieve scalability. So all of the metadata would be fragmented across multiple servers and files could only be retrieved on the instance that uploaded it. With renterd any Worker of the cluster can download any file.

No Downtime

The persistence of siad was exclusivey done via custom data formats. They were very performant but also not interoperable at all. Backing them up required shutting down siad to prevent it from writing to disk which results in downtime.

With renterd and its support of different SQL databases (SQLite/MySQL), users can pick from a vast ecosystem of tooling to monitor and duplicate the metadata without downtime such as database replication. Making it more viable in an enterprise environment while keeping it simple for individuals via SQLite.

Overhauled UI

With renterd we have made the UI significantly more useful. It’s a website hosted by the Bus module instead of a desktop app to be easily accessible from any device on the network. Due to built-in authentication it can also be set up to be accessed remotely.

Metrics

Thanks to built-in collection of metrics, the UI can also provide useful breakdowns of historical data. Such as contract spending, contract formations and the wallet balance.

Spending breakdown of a contract

Alerts

In siad users had to regularly check in on their logs. That is quite cumbersome and often times quite overwhelming. With renterd we have taken a UI-first approach. Users are able to monitor the health of their node directly from the UI via renterd‘s alert system. It provides useful info on what’s happening at the moment and will alert users of issues with additional information.

The alerts can also be consumed via renterd‘s webhook API. Making it easy for third-party developers to build useful tools like chat bots to notify users of alerts wherever they may be at the time.

S3 Compatible

One of the most anticipated features in siad for a long time has been compatibility with S3. The de-facto standard for cloud storage APIs. With renterd this is finally happening. You can now use your favourite S3 compatible tools to back up your files to the Sia network or mount your object bucket locally as a folder. You could even use renterd as a datastore for IPFS to start pinning files without requiring the physical space on disk.

No More Wasted Storage

renterd also introduces two more new features over siad which people have been asking us about for a long time.

• Built-In upload-packing: siad pads small files to 40MB at the default redundancy settings. So uploading a 1KB file is a huge waste of space. With upload-packing, renterd will wait for 40MB to accumulate before actually uploading the data to the Sia network reducing the waste to effectively 0. Of course this feature can be disabled by advanced users who wish for more control over how files are packed.
• Contract pruning: Data uploaded to siad never gets deleted. Once it’s part of a contract, the user just keeps paying for it even if they delete the file in the UI. In renterd we introduced contract pruning which removes deleted files from a contract periodically.

With these two features combined, users will be able to save even more money when using Sia.

This release marks a new milestone in the history of Sia, but it is only the beginning. renterd is already a huge step up in scalability and usability and with the protocol changes we have planned for 2024 it will be able to reach even greater heights. We will keep raising the bar for decentralised storage and hope you will be part of the journey with us.

You can find the latest software at https://sia.tech/software/renterd