Sia S3 Integration: Duplicati

What is Duplicati?

Duplicati is an open-source backup software that allows users to store compressed, encrypted, and incremental backups on a wide variety of storage destinations, including traditional cloud providers, local storage, and other remote servers. With built-in deduplication functionality, it ensures efficient use of storage by saving only the unique chunks of data. Duplicati offers both a user-friendly graphical interface and command-line utility, making it versatile for different types of users. In essence, Duplicati provides a secure and efficient solution for safeguarding your data against loss.

What is Sia?

Sia is a decentralized data storage network where anyone can rent cloud storage and pay with Sia tokens. This storage is rented from people, called “hosts”, who want to earn these tokens. The Sia network consists of hosts all around the world that can be selected based on price, speed, location, or a combination of the three. Your data will be encrypted and erasure-coded before being uploaded to the host network, keeping you, the user, in control of said data.

renterd is the renting software developed by the Sia Foundation. This is the successor of siad, which is being deprecated and should no longer be used. It is important to note that renterd has to continuously run in the background to ensure the integrity of your data. This is necessary because hosts on the Sia network can come and go as they please, so the data has to be constantly monitored and migrated to preferable hosts.

Step 1: Setup renterd

Start by setting up renterd on your system. Follow the setup guide for your operating system.

Setting up renterd

S3 Setup

To use Sia with Duplicati, we must rely on its integration with S3. This API is not enabled by default, but configuring it is straightforward and can be done by adding the following to your renterd.yml file.

s3:
  enabled: true
  address: "localhost:8080"
  keypairsV4:
    your_access_key: your_private_key

The S3 API is authenticated through key pairs that you will need to specify under the keypairsV4 section of your renterd.yml. A key pair consists of a 20-character access key followed by a 40-character private key, separated by a colon. So, for example, we will be replacing your_access_key: your_private_key with abcde4-bozTah-gugxev: ABCDhvKW959kR0Gs2IPeHkMY1fhwHC1Ww7u9ucT0. Keep these values on hand for Step 3.

After setting up renterd and enabling S3, it’s a good idea to test it out quickly. This can be done using either the AWS CLI Client or Minio Client. If you use the minio client, setting an alias is very useful. Let’s try and list the buckets to see if it works. It should return the default bucket that was created automatically when you started renting for the first time.

# alias our s3 endpoint for ease of use
mc alias set s3 <http://localhost:8080> abcde4-bozTah-gugxev ABCDhvKW959kR0Gs2IPeHkMY1fhwHC1Ww7u9ucT0

# list all buckets
mc ls s3
[2023-09-18 19:09:37 CEST]     0B default/

Step 2: Install Duplicati

Download and install the latest version of Duplicati from their website.

Duplicati

After installing it, the application should open in your browser. If not, navigate to http://localhost:8200, where the application is being served by default.

Step 3: Configure Duplicati

Add backup

Select Add backup from the side bar and configure a new backup.

General

Fill in the general backup settings and make sure to save your passphrase somewhere safe.

Destination

Select “S3 Compatible” from the Storage Type drop-down. We have to set a custom server URL to point it to the endpoint you configured renterd’s S3 API address with, which is[localhost:8080](<http://localhost:8080>) by default.

⚠️ Duplicati offers a storage provider option called "Sia Decentralized Cloud" that uses the soon to be deprecated siad software. renterd is superior in many ways, and it is also S3 compatible, making it easy to integrate with third-party services like Duplicati.

It’s important not to leave the folder path blank but instead give it a logical name. We advise using the backup name you used when configuring the general backup settings. Leaving this blank causes issues if you configure multiple backup jobs.

You may get a warning that says the bucket name is usually prefixed with the access ID. You could do that, but then you will have to switch to that bucket in the renterd UI for your Duplicati files to appear. For the purpose of this tutorial, we advise you to use the default bucket name.

Source Data

Select the files you want to back up to Sia.

Schedule

Create a backup schedule.

Options

Duplicati provides some backup options that might be interesting for you. We recommend never setting the remote volume size below 40MByte unless upload packing is enabled in the renterd UI. The defaults work fine.

Conclusion

The integration of renterd with Duplicati marks a significant step towards harnessing the power of decentralized technologies for everyday applications. As data breaches and privacy concerns continue to rise, such collaborations offer a promising alternative for those who prioritize data security and sovereignty. As both platforms evolve, users can look forward to even more streamlined experiences and enhanced features.

If you are looking for a storage solution that is truly decentralized, secure, private, and resilient, consider Sia. You can learn more about Sia from our website and documentation. You can also join our community of data storage enthusiasts and developers.

Sia is the future of decentralized storage, and you can have a part in shaping it!