Wrapper Script Update

Keith · January 10, 2025, 12:16am

Next week, January 13-17, 2025 - exact date TBD, the Technical Committee will push an update to the Wrapper Script. The update disables the CloudWatch function of the wrapper script by default. We are opting to disable the function rather than removing it entirely so there is the ability to enable it in the future if warranted.

Why disable CloudWatch?

In the past, the CloudWatch function was very useful to the development teams, specifically during BetaNet and in the early days of MainNet. However, it is now considered a legacy function, and if you’re not familiar with AWS S3 costs, it’s surprisingly expensive to maintain. In Q4 of 2024 alone, the xx Foundation was able to reduce the annual operational costs of the xx network project by tens of thousands of dollars by removing, reconfiguring and/or reorganizing servers and services related to its websites, MTP, etc. To get 2025 started in the right direction, we’ve made the decision to disable CloudWatch which will reduce the annual operational costs by over 15,000 USD.

What will Node Runners need to do?

Not much. Assuming Node Runners configured their node and gateway computers per the xx Network Node Handbook one will only need to restart the xx network cMixx related services after the update has been pushed. The update function of the Wrapper Script will handle the rest.

I will be posting additional information in the coming days and once the update has gone live.

Thanks for running a node!

HBW · January 10, 2025, 7:51pm

I welcome the reduction of fixed costs, which enhances financial stability

Keith · January 17, 2025, 5:14am

Hey everyone!

Just wanted to update you all on the ETA of the Wrapper Script Update. I had planned for it to be done by 2025-01-17 UTC but there were a couple of things that had to happen first. I had to be added to the Technical Committee, which I have been. And the Technical Committee has to be re-authorized to be able to update the cMixx binaries/hashes.

Democracy Referendum #29, to re-authorize the Technical Committee to be able to update the cMixx binaries/hashes, looks to be on course to be approved and will complete in about 12 hours.

Once Referendum #29 completes, members of the Technical Committee will vote on Proposal #59. You may notice there is a Proposal #57 and #58. Those proposals were submitted in error so Technical Committee members are expected to vote “nay” on #57 and #58, and “aye” on #59.

The exact ETA of the update will be determined by the closure of Proposal #59. Since members of the Technical Committee are located in different time zones I can’t give a specific time. However, we have all spoken and will vote ASAP.

With all of that said, I expect the update to happen in the next 18-24 hours.

I will make an official announcement, on Discord and Telegram once it is completed with a back-link to a post about what Node Runners should expect to see in their logs and when/how to restart the cMixx related services.

Talk to you all again soon and thanks for running a node!

rothbardian · January 17, 2025, 7:13am

What will Node Runners need to do?
Not much.

Why can’t we just deploy the change ourselves now and start saving today?
Speaking of decentralization, it used to be possible to disable push style updates. The handbook should have that information added.

Keith · January 17, 2025, 5:46pm

The Wrapper Script Update has been pushed. Node and Gateways should have received the update. If not, to trigger the update, all one should need to do is restart the xxnetwork-cmix and xxnetwork-gateway services.

On the NODE computer run …

sudo systemctl restart xxnetwork-cmix.service

Once restarted the cmix-wrapper.log will contain a message that indicates the update is complete.
You can search the log with …

grep 'wrapper update' /opt/xxnetwork/log/cmix-wrapper.log

… or review the log with something like …

tail -n 50 /opt/xxnetwork/log/cmix-wrapper.log

You should see a message similar to …

[INFO] DATE TIME: wrapper update required: efbc1be7b6922fbc70da224bcf00bdc3599326c35fcdb0747b549f05ad8b7ccd -> 8d8480cc96d76556a9f63dd1bd6f42254b0c1dfdb64b18d1e3d34528312b3bc8

Another way to check is to verify the checksum of the wrapper script files. The scripts on the node and gateway are actually the same file so the checksums will be the same. The checksum of the updated wrapper script is …
07363a6a9b3b0fb34dfc9f0995d704e4cb182f6e16bf376fabbdb23419307700

If you run the following, it will display the checksum.

sha256sum /opt/xxnetwork/cmix-wrapper.py

On the GATEWAY computer run …

sudo systemctl restart xxnetwork-gateway.service

The gateway-wrapper.log will contain a message that indicates the update is complete.
You can search the log with …

grep 'wrapper update' /opt/xxnetwork/log/gateway-wrapper.log

or review the log with something like …

tail -n 50 /opt/xxnetwork/log/gateway-wrapper.log

You should see a message to the affect of …

[INFO] DATE TIME: wrapper update required: efbc1be7b6922fbc70da224bcf00bdc3599326c35fcdb0747b549f05ad8b7ccd -> 8d8480cc96d76556a9f63dd1bd6f42254b0c1dfdb64b18d1e3d34528312b3bc8

Another way to check is to verify the checksum of the wrapper script files. The scripts on the node and gateway are actually the same file so the checksums will be the same. The checksum of the updated wrapper script is …
07363a6a9b3b0fb34dfc9f0995d704e4cb182f6e16bf376fabbdb23419307700

If you run the following, it will display the checksum.

sha256sum /opt/xxnetwork/gateway-wrapper.py

Thanks for running a node!

Keith · January 18, 2025, 2:30am

One was always able to disable CloudWatch with --disable-cloudwatch but kind of moot at this point since CloudWatch is now disabled and the back-end services will be decommissioned.

Disable push style updates has not been an option of the wrapper script for many years if I’m not mistaken. The decision to remove it would have been one made by the previous devs. I don’t recall exactly when and why it was removed. I think the reasoning was, MainNet is a production network so overall, it’s better everyone have the latest binaries. However, if someone had the development skills and ability to contribute code to the binaries and chose to test/run them, they likely have the development skills and ability to modify the wrapper script so their binaries are not updated.

rothbardian · January 24, 2025, 8:21am

It’d be good if in the future these announcements (or entirely separate announcements) were created when the node & gateway archives get updated.

There was one time (2022?) when the archives lagged because reasons, and the chain binary was outdated for months.

This this time they were updated quickly, but not immediately, so some of us (I did) assumed it will go the same way because the wrapper script auto-updates anyway.

But I had a failed checksum happen in my script

After checking I noticed both node & gateway are new compared to 2 days ago.

I know there’s no specific “repo” for these, so maybe no special “announcements” are needed.

But that may be precisely why they are needed, because we have no other recipe to get these but by downloading, so a lot of manual inspection is needed to figure out what happened, and how can we even be sure?

I need to unpack the archives, then trace where the new files (or maybe all files, since dates can be manipulated) came from and compare their individual checksums.

Not to mention that the dates on the page don’t quite match. It says Jan 18, but my screenshot shows that gateway.tar.gz from Jan 22 still didn’t have the new wrapper script.

Then the other problem is the checksums are old checksums (that match node.tar.gz and gateway.tar.gz I got on Jan 22) while they don’t match what is available right now.

It took me 30 minutes to figure out where these problems are and I still can’t tell what I’m supposed to have.
How do we know what’s posted now and what was posted before was posted by xx Network?

I think every time these are updated, there should be an announcement, and archives should be signed (using packager’s wallet for example).

Alternatively, we should have a build-from-source doco in the Handbook or scripts in a repo. I’m not asking for that because that’s more time-consuming, but that would be the right way - it’s easier to build from the source which was signed in commits, than come up with another signing thing based on the archaic PGP sigs that nobody likes to use.
And with that we could simply get CI builds knowing the latest one is right, but if we lock onto a specific release (e.g. node-v24.06.tar.gz - when the chain binary was last updated) that should still work fine and wouldn’t break users’ scripts.

Keith · January 24, 2025, 9:45am

All valid points.

NOTE: Date: on the website is the date the tarballs were uploaded.

FWIW, the tarballs were updated on the 18th but with an error …

So were updated again on the 24th.

Additionally so was the handbook …

Maybe you’re seeing a cached copy.

rothbardian · January 24, 2025, 10:01am

Well, the archive files were downloaded automatically with wget and the Web site screenshot I opened anew when I was about to get that screenshot.
Maybe that Web page cache was stale when I took the screenshot, who knows, but I don’t think it was in my browser. I didn’t think to check, but now the content is correct.

I was looking for a “docs” repo but couldn’t find it. I didn’t expect that the docs.xx.network archives are created in the repo normally available at handbook.xx.network…

At least now we know where to go to check.

If the CI workflow could be adjusted to symlink (or republish) latest archive version as -latest in addition to -version (which could be a timestamp), that would be helpful.