August 2020 Uptime Policy

On August 6th we sent an email highlighting the new uptime policy for xx network nodes coming online in August. Please read below and let us know if you have any questions.

August 2020 Uptime Policy

The xx network is tentatively targeting a 50% uptime policy for August 2020. This means that, with caveats, a node needs to be online for at least 50% of the month of August to receive compensation for the month.

A node will be online as long as it is pinging the permissioning server regularly. If at the dashboard and your node is listed as either “Online” or “Error” it is still online for the purpose of compensation for the month of August.

On the individual node pages on the dashboard there is now a monthly uptime metric to inform you of your progress.

Node uptime is not tracked while xx networks servers are down and such periods will not be counted. In the event the network is down due to a software update or a bug caused by the xx network team, those periods will also not count towards uptime.

Per the handbook, grace periods for software updates will not be considered part of uptime calculations for nodes who do not have automatic updates enabled. We are still working on ensuring the dashboard reflects this.

Grace Period

Nodes who are joining the network for the first time this month will have the entire month of August as a grace period. This only includes nodes who registered initially to start in August. If this month is a grace period for your node, you will receive an email notifying you.

We are sympathetic to node operators who are struggling due to the difficult times. We will strive to work with nodes on a one on one basis to ensure they can participate. If your situation warrants a grace period outside of the previous policy, please email us at [email protected]

Technical details of uptime tracking

Whenever a node polls the permissioning server for network state updates, as long as the network is up, the node is properly authenticated, the node is not banned, it will increment a poll counter for that node. Every 5 minutes the poll counters are stored as “node metrics” in a database and cleared. As long as that counter is greater than 0, the node is considered online for that period. That is what is included in the uptime metric on the dashboard.

3 Likes

Why are you targeting a so low uptime?

Yes 50% seems very low. :thinking:

Does a node that is online at least 50% receive the same as a node that has been up for 100%? Sorry, haven’t read about compensation yet. :slight_smile:

At least the Beta net can be checked thoroughly if you set the bar low for uptime performance. As well the speed of the connection-requirement from the nodes is also lowered.

For mainnet, at least that’s my opinion, the bar can be set higher on both issues.

1 Like

Thanks guys.

That uptime won’t be an issue at all, but I just wanted to say how I appreciate the grace and flexibility you’ve offered these initial two months while people get set-up.

Like everyone else, I agree that the uptime can be increased but there shouldn’t be a high uptime requirement until there’s at least some form of notification service that can send you a text/email when your node has gone offline. A few days ago I was surprised that mine had gone off and only knew because I check the dashboard page multiple times a day.

That’s how I do service monitoring from another host:

$ cat xxnetwork_monitor_running.sh 
#!/bin/bash

set -e
set -u

if ! systemctl --quiet is-active xxnetwork-node && systemctl --quiet is-active xxnetwork-gateway; then
        echo "xxnetwork not running"
fi
$ crontab -l
[email protected]

# monitor xx network node
15 22 * * *  ssh [email protected] "bash -s" -- < ./xxnetwork_monitor_running.sh

Just make sure that the host where this cronjob is set up can send mails to the given address.

PS: with this solution, the host running the crontab of course needs to be well secured, given that it has ssh access to the node.

@d10r please leave this technical comments in discord. There are multiple channels there where you can post this kind of messages. For the rest… All the best