The State of Sia, September 2023

We’re back with the State of Sia, a monthly installment looking at development and team updates for The Sia Foundation.

With summer fully drawing to a close, we can focus on taking the next important steps for the Sia ecosystem to round out the year. We’re moving towards full releases of our app suite and looking beyond. Oliver, our new Developer Relations expert, is getting revved up and ready to go. We’re really excited to see his impact on Sia and the community.

We’ve got a team get-together coming up where we’ll be able to have some much needed face time, and sync up on a number of initiatives. In this world of remote work, sometimes it’s nice to work together.

Enough chatter, let’s get to the good stuff quick.

Development

All public repos can be found here.

renterd

renterd will be our new stand-alone renter. The renterd repo is currently public and can be found here, and the app is available in beta.

• Added YAML config support (Link)
• Added ‘before’ parameter to /bus/transactions endpoint (Link)
• Added an alert system to renterd (Link)
• Object health is now cached in the database (Link)
• New CLI flag to configure the number of slabs the autopilot reparis in parallel (Link)
• Added a setting to bypass API authentication for downloads only (Link)
• The MaxUploadPrice setting doesn’t account for storage cost and redundancy anymore, only for upload bandwidth since people found that to be confusing (Link)
• Support for renaming objects in the API (Link)
• Optional upload packing support to support uploading small files without wasting space (Link)
• The seed generated by renterd is now printed to stdout instead of stderr for easier scripting (Link)

• New flag to change the location of the log path in the CLI (Link)
• Slabs returned through the API now have a ‘health’ field (Link)
• The id of an object is now guaranteed to be case sensitive (Link)
• The /autopilot/hosts endpoint now returns all hosts instead of just the ones that are cached (Link)
• Hosts will be scanned before the blockchain is fully synced (Link)
• New /bus/wallet endpoint that contains more info regarding the wallet’s balance (Link)
• /bus/wallet now returns unconfirmed siacoins (Link)
• The autopilot now periodically submits revisions to the blockchain (Link)
• The /autpilot/status endpoint now returns whether the autopilot is configured (Link)
• New endpoint to fetch all objects for a slab (Link)
• Added webhook support to renterd. Initially all alerts can be subscribed for using webhooks (Link)

related to sector deletion (Link), (Link), (Link), (Link), (Link), and (finally)

related to database query optimisations and fixes (Link), (Link), (Link), (Link), (Link)

miscellaneous others of significance:

• added low-balance alert (Link)
• added state route (Link)
• drastically improved build times (Link)

bug fixes:

• multiple write (Link)
• broken jape (Link)
• broken build (Link)
• underflow panic (Link)
• NDF fix (Link)

renterd ui

This is a small collection of monthly changes. A full detailed changelog can be found here.

• File upload and directory creation are now disabled until enough contracts are formed. (Link)
• The failed to migrate slab alert now lists the associated objects/files. (Link)
• The host explorer now has an interactive map. Filtered hosts can be selected via the list or map, and hosts on the map are colored based on whether renterd is actively contracting with the host. For hosts with active contracts the size of the hosts on the map is based on renterd’s used storage, for hosts without active contracts the size is based on the hosts remaining storage. (Link)

• New users are now more clearly instructed to configure autopilot and to wait for enough contracts before files can be uploaded. (Link)
• Node profile information now includes the build version. (Link)
• The connectivity and login check no longer depends on consensus APIs which in some rare cases can be unresponsive. (Link)
• Alerts now show contract additions and removals, formatted address, balance, and more. (Link)
• Health percentage values are now clamped to a range of 0–100%. (Link)
• File health tooltip now includes redundancy info and supports partial slabs. (Link)
• The host explorer now works before autopilot is configured, the table shows the autopilot columns with empty values and tooltips explaining that autopilot is not configured. (Link)
• Upload packing can now be enabled from the configuration tab. (Link)
• The file stats do not show redundancy factor or health until there is at least 1 byte of data. (Link)
• Fixed a crash issue when loading the hosts explorer. (Link)
• Wallet balance is now more accurately calculated as spendable + unconfirmed. (Link)

hostd

hostd is the app you’ll be using to rent out your excess storage space to the Sia network. The hostd repo is public, with a beta out now.

• Fixed underflow in collateral metrics when broadcasting storage proof for expired contracts (Link)
• Added additional contract lifecycle tests to ensure contracts are completing correctly (Link)
• Optimized storage location selection increasing performance around 2x for large nodes (Link)

• Added caching to improve RPC performance when revising frequently used contracts. This is primarily noticeable when revising contracts storing more than 2TB of sectors (Link)
• Fixed an issue with the No-IP DDNS provider when the IP address hasn’t changed (Link)
• Added a 5m interval to the host’s metrics (Link)
• Improved contract sector pruning to be more performant (Link)
• Fixed an issue with sectors not being distributed between all available storage locations (Link)
• Fixed an issue with sector shrink when the volume is not a multiple of 64 (Link)
• Fixed an issue with the wallet balance metric not being properly reset if the seed changes (Link)
• Made logging more consistent with renterd (Link)
• Added additional context and coloring to the CLI prompts to make them more user friendly (Link, Link)

hostd ui

This is a small collection of monthly changes. A full detailed changelog can be found here.

• Fixed an issue where alert error messages were being cut off. Full error messages are now displayed above all other fields. (Link)
• The connectivity and login check no longer depends on consensus APIs which in some rare cases can be unresponsive. (Link)
• The contracts table now has an egress usage column. (Link)

walletd

walletd is our brand new wallet-only software. Find the public repo here.

• Finished benchmarking walletd. There wasn't much (at least in the way of major performance improvements) to change.

walletd ui

This is a small collection of monthly changes. A full detailed changelog can be found here.

• The connectivity and login check no longer depends on consensus APIs which in some rare cases can be unresponsive. (Link)
• The app now warns the user if it is running on the testnet with a prominent banner. (Link)
• Table headers now freeze in view when the table scrolls. (Link)
• Fixed an issue where walletd would indefinitely report as “syncing” — the synced state is now based on the most recent block’s timestamp being within the last 2 hours. (Link)
• The contrast was improved on the syncing progress text in the daemon profile. (Link)

core

The core repo is public and can be found here.

• We’ve made big strides on v2 in the its-happening branch (Link)
• The consensus architecture has been refactored such that even v1 transactions do not require any I/O to process, which improves performance and allows for better isolation of consensus code (Link)
• Added support for v2 transactions in the core transaction pool (Link)

• Attestations (i.e. host announcements) will now be present in the v2 state accumulator, making it possible for a new node to efficiently retrieve a list of announced hosts without having to scan the entire blockchain (Link)
• Multisig v2 addresses now support hiding the non-signing parties, improving privacy (Link)
• Added some much-needed testing for the v2 validation functions (Link)
• Reduced memory allocations in core to speed things up a little (Link)

Looking ahead

• Publish siad to hostd migration utility
• Stability on large nodes
• S3 compatibility
• Designing and implementing the gateway protocol for v2 nodes. This is already underway and we hope to have an MVP working soon. The protocol will probably go through a number of redesigns before launch.

That’s all folks

Thanks for your continued support and dedication as we build the foundation of the decentralized future.

Take care, and see you next month.

Discord, Email

Steve

@steveisonline on Discord