Sia Antfarm

Sia’s Decentralized Storage Mini Testnet and Test Lab

Image credit: Designed by brgfx / Freepik

Sia

Sia is a decentralized data storage platform and marketplace backed by its own blockchain and cryptocurrency. It is a base layer upon which other platforms and end-user applications can be built with Skynet being the most prominent.

Sia Antfarm

The purpose of Sia Antfarm is for testing the Sia network. It can be used to test predefined scenarios like version upgrades or to test an application that is being built on top of Sia.

The Sia Antfarm creates a local Sia testnet by spinning up Sia nodes and connecting them. These Sia nodes are referred to as ants and the quantity and makeup is configurable based on the user's needs.

Ants

Each ant can perform one or more jobs such as being a renter, host, or miner on the Antfarm network. These jobs can run perpetually in the background. An example would be an autoRenter job, where the ant connects with other ants (hosts, miners) on the Antfarm, sets its own renter storage allowance, forms storage contracts with hosts, and starts periodically uploading, downloading, and deleting files to and from the local Sia testnet.

While the Antfarm is running, users can interact with each of the ants through the Sia API (e.g.: curl -A "Sia-Agent" "127.0.0.1:10980/renter") or siac commands (e.g. siac --addr 127.0.0.1:10980 renter).

Image credit: Designed by brgfx / Freepik

Configuration

The Sia Antfarm can be configured to meet the needs of different test cases by using the config.jsonfile. This config.jsonfile allows for managing the type of ants within the Antfarm as well as specific settings for each ant. Each setting option is described in the README file and a sample configuration we use can be also found in our GitLab repository.

Each ant is assigned port numbers for its services. To interact with the ant you need its API address, which you can set in the ant configuration with APIAddr setting, e.g. 'APIAddr': '127.0.0.1:10980'. If you omit these settings, API ports (and also other service ports) will be assigned random free port numbers which you can find in the Antfarm log after Antfarm starts the ants.

By default, the Sia Antfarm builds Sia binaries with the dev tag enabled. This allows users to monitor the network and see changes as they are happening. As an example, with the dev tag the block time is 12s as oppose to 10min in production and 1s in testing.

The Antfarm itself can be installed in two versions: sia-antfarm and sia-antfarm-debug, where the later version logs debug information which is helpful while the app still matures.

Sia Upgrade Tests

One area that the Antfarm can really provide value to developers is with the new Version Tests. These Version Test (or Upgrade Tests) allow for easy testing of a new siad version without having to upgrade production Sia nodes.

The Version Tests build 5 latest released Sia versions (i.e. siad-dev binaries), then they continue with 2 paths: Renter Upgrade Tests and Hosts Upgrade Tests.

Renter Upgrade Tests spin up a testnet with the renter ant having the oldest siad release (currently v1.4.7 as of this writing) and the rest of ants having the latest released version (or the latest Sia master on GitLab). The renter then uploads and downloads a file to the network to check it is working correctly, then it upgrades one version up, uploads a file and downloads all uploaded files (uploaded with the current version and with all previous versions), then continues iteratively up to the latest released version (or the latest master).

Similarly, the Hosts Version Test iteratively upgrades all host ants from the oldest version up to the latest released version (or to the latest master).

There are several configuration options for Version Tests. The first being binariesDirwhich defines a directory where siad-dev binaries built for each tested released version or latest master are stored. minVersion option defines what is the oldest Sia version to be used in the Version Tests. The test finds all released Sia versions in Sia GitLab repository from this version on. If the test has a postfix BaseLatestMaster it will include the latest Sia master commit. The generated list of Sia versions is referred to as the upgrade path. The Version Test then buildssiad-dev binaries for each version in the upgrade path and tests them. If there are some released Sia versions, like v1.4.10, which should be excluded from the upgrade path, they can be specified in excludeReleasedVersions. When repeating a Version Test it is possible to skip rebuilding binaries with rebuildReleaseBinaries and rebuildMaster settings. Last setting allowLocalIPs enables performing tests on a machine that is not accessible on public internet either directly or via port forwarding.

For the Sia versions prior to v1.5.0 we had to create a “mini-patches” to the officially released versions to allow them to be used on local network, so e.g. instead of testing Sia release v1.4.8 we use a version patched for antfarm tagged v1.4.8-antfarm.

Image credit: Designed by brgfx / Freepik

Docker Image

For those looking for simple renter testing we have prepared a Sia Antfarm Docker image. The source is located in the GitHub repository and the image itself is published on Docker Hub.

Docker image contains `sia-antfarm` and `siad-dev` binaries and a sample json configuration file to start with.

Running

docker run \
    --publish 127.0.0.1:9980:9980 \
    nebulouslabs/siaantfarm

starts the Antfarm using the default configuration with the renter ant accessible at `127.0.0.1:9980` API address.

If the port `9980` on your machine is already taken you can change the renter API port to any other free port using the publish setting, e.g.:

docker run \
    --publish 127.0.0.1:39980:9980 \
    nebulouslabs/siaantfarm

To use a custom Antfarm configuration, you can create your own configuration file, save it to the local config directory and start the Antfarm Docker container using:

docker run \
    --publish 127.0.0.1:9980:9980 \
    --volume $(pwd)/config:/sia-antfarm/config \
    --env CONFIG=config/custom-config.json \
    nebulouslabs/siaantfarm

You can access more than one ant from outside of Docker via HTTP API. First you have to set ant’s APIAddr option in configuration file, e.g.: "APIAddr": "127.0.0.1:10980" and you have to publish it when starting a Docker container:

docker run \
    --publish 127.0.0.1:9980:9980 \
    --publish 127.0.0.1:10980:10980 \
    --volume $(pwd)/config:/sia-antfarm/config \
    --env CONFIG=config/custom-config.json \
    nebulouslabs/siaantfarm

When you want to persist Antfarm’s and ants’ logs even after the Antfarm’s Docker container is stopped or removed, you can start the container with the current folder mounted:

docker run \
    --publish 127.0.0.1:9980:9980 \
    --volume $(pwd):/sia-antfarm/data \
    nebulouslabs/siaantfarm

Then you can find the logs at your ./antfarm-data directory.

Image credit: Designed by brgfx / Freepik

Looking Forward

Having a testnet in the Sia ecosystem is important to enabling developers building on top of Sia. We hope that these upgrades to the Sia Antfarm encourage more development on top of Sia.

If you are a developer with Sia experience we invite you to give us your feedback on the Sia Antfarm testnet. If you haven’t started yet, we encourage you to enter web3 space to design, develop, and deploy your decentralized app of Sia or Skynet.

Read more about Sia or Skynet, discover the latest news from our blog or Twitter, join our community on Discord.

Sia Antfarm is an open-source application hosted on Gitlab under the MIT license.