Netris Upgrade and Rollback Procedures¶
Upgrade Procedure¶
Due to potential database structural changes between Netris versions, it’s highly recommended to take a backup of the database before upgrading. The backup will be used in the unlikely event of the need to perform a rollback.
To create a database backup, run the following command by first SSHing the Controller:
kubectl -n netris-controller exec -it netris-controller-mariadb-0 -- bash -c 'mysqldump -u $MARIADB_USER -p${MARIADB_PASSWORD} $MARIADB_DATABASE' > db-snapshot.sql
Ensure that SQL file db-snapshot.sql is generated and present in the current directory.
Stop all Netris agents on devices managed by the controller (switch & SoftGate).
For the switch agent, first SSH to the switch and run the following command:
sudo systemctl stop netris-sw
For the SoftGate agent, first SSH to the SoftGate and run the following command:
sudo systemctl stop netris-sg
Ensure that all devices in the Net → Inventory section are “red” with the “check_agent” status being “Agent is unavailable”.
Note
A stopped Netris agent has no impact on production traffic through the device.
Before upgrading the Netris Controller, take a note of the “Netris Version” by navigating to Setting → General in the Controller web interface. This version number will be used in case of the need to perform a rollback procedure.
Start the upgrade of the Netris Controller using the one-liner after SSHing to the Controller.
curl -sfL https://get.netris.ai | sh -
Note
This process can take up to 5 minutes
Afterwards, make sure that all pods have either “Running” or “Completed” status by executing the following command:
kubectl -n netris-controller get pods
The output is similar to this:
NAME READY STATUS RESTARTS AGE
svclb-netris-controller-haproxy-6tkgj 4/4 Running 0 38d
netris-controller-haproxy-bcb944b7c-qcbf8 1/1 Running 0 13d
netris-controller-squid-7f6fdc6cf9-7fdx8 1/1 Running 0 38d
svclb-netris-controller-squid-58rnp 1/1 Running 0 38d
netris-controller-graphite-0 1/1 Running 0 38d
netris-controller-mongodb-0 1/1 Running 0 38d
netris-controller-redis-master-0 1/1 Running 0 38d
netris-controller-smtp-76778cf85f-lw5v5 1/1 Running 0 10d
netris-controller-mariadb-0 1/1 Running 0 10d
netris-controller-web-session-generator-8b9dbbcd8-8snhd 1/1 Running 0 10d
netris-controller-telescope-notifier-647975848f-fs5dn 1/1 Running 0 10d
netris-controller-app-b9b8d8f8d-4ssqb 1/1 Running 0 10d
netris-controller-grpc-987669fb9-jjskp 1/1 Running 0 10d
netris-controller-telescope-777c98c5d9-mqwl6 1/1 Running 0 10d
helm-install-netris-controller-lqmq7 0/1 Completed 0 20h
Warning
If, after 5 minutes, you see pods with a status other than “Running” or “Completed”, please reach out to us via Slack.
Then verify that the “Netris Version” reflects the version change by navigating to Setting → General in the Controller web interface.
Once you have verified that the Netris controller is up-to-date, it is time to update the switch and SoftGate agents.
Upgrade the switch & SoftGate agents by copying the one-liner from the “Install Agent” option of the device’s 3-dot menu found under the Net → Inventory section and pasting it after SSHing to the corresponding device.
After all the agents have finished the upgrade process, make sure all devices in the Net → Inventory section have a “green” status and the Netris version for each device reflects the version change.
In the event the “check_agent” status is “Agent is unavailable” after the agent upgrade has finished, perform agent restart on the affected device(s).
For the switch agent, first SSH to the switch and run the following command:
sudo systemctl restart netris-sw
For the SoftGate agent, first SSH to the SoftGate and run the following command:
sudo systemctl restart netris-sg
Rollback Procedure¶
A rollback procedure can be executed in the event the upgrade introduced any adverse impact on the production traffic.
Stop all Netris agents on the devices managed by the controller (switch & SoftGate).
For the switch agent, first SSH to the switch and run the following command:
sudo systemctl stop netris-sw
For the SoftGate agent, first SSH to the SoftGate and run the following command:
sudo systemctl stop netris-sg
Restore the database from the previously taken snapshot.
Copy the backup file from the controller host system to the MariaDB container by running the following command after SSHing to the Controller:
kubectl -n netris-controller cp db-snapshot.sql netris-controller-mariadb-0:/opt/db-snapshot.sql
While still connected to the Controller, restore the database:
kubectl -n netris-controller exec -it netris-controller-mariadb-0 -- bash -c 'mysql -u root -p${MARIADB_ROOT_PASSWORD} $MARIADB_DATABASE < /opt/db-snapshot.sql'
Downgrade Netris Controller application with the following command.
Note
For the version number, use the number collected from step #3 during the upgrade procedure.
Example:
curl -sfL https://get.netris.ai | sh -s -- --ctl-version 3.0.10-031
Afterwards, verify that the version of the “Netris Version” reflects the downgraded version by navigating to Setting → General in the Netris Controller.
Once you have verified that the Netris controller has been downgraded to the correct version, it is time to downgrade the switch and SoftGate agents.
Install the correct and appropriate versions of the switch & SoftGate agents simply by copying the one-liner from the “Install Agent” option of the device’s 3-dot menu found under the Net → Inventory section and pasting it after SSHing to the corresponding device.
After all the switches and SoftGates have been successfully downgraded, make sure all the devices in the Net → Inventory section have a “green” status and the Netris version for each device reflects the version downgrade.
In case the “check_agent” status is “Agent is unavailable” after agent downgrade, perform agent restart.
For the switch agent, first SSH to the switch and run the following command:
sudo systemctl restart netris-sw
For the SoftGate agent, first SSH to the SoftGate and run the following command:
sudo systemctl restart netris-sg