This page describes steps to take before starting a Sentinel IP-SM-GW upgrade.
Note that there is no specific need to take an export prior to starting the upgrade. Taking an export is the first step
orca will perform when issued the upgrade command, and the export will be available at the end of the upgrade
Create validation test plans
Devise a validation test plan to ensure that, after the upgrade:
traffic is routed as expected and calls work (including audio and, where appropriate, video)
any features in the downlevel version that you rely on still work as expected
any new features you are expecting to use in the uplevel version work as expected.
Your network configuration may support setting up test numbers where calls to these numbers are routed to specific nodes. This is a particularly useful tool for verifying the upgrade early, as after upgrading the first node you can make calls using test numbers configured to route specifically to the first node.
During or after the upgrade, if you find the validation tests are failing, contact your Metaswitch Customer Care Representative to discuss next steps.
Ensure the nodes are configured with the standardized path structure
orca requires the nodes to have a standardized path structure. Within the
HOME directory (normally
/home/sentinel) there must be:
one or more Rhino installation directories with a name of the following format:
a symbolic link named
rhinopointing at the currently live Rhino installation
a directory named
a directory named
See Standardized Path Structure for more information.
If your installation does not conform to this setup, consult your Metaswitch Customer Care Representative who can advise on how to convert it.
Record the old cluster ID
In case you need to rollback the upgrade, you will need the current cluster ID.
status command on all the hosts:
orca --hosts <host1,host2,…> status
The first part of the output for each host will include the cluster status. One of the clusters should be marked as "LIVE".
Status of host host1 Clusters: - ipsmgw-220.127.116.11-cluster-41 - ipsmgw-18.104.22.168-cluster-42 - LIVE
On the "LIVE" cluster, the last part of the name given is the cluster ID. For example, in the above output the cluster
Ensure all clusters have the same live cluster ID. Make a note of this value.
Verify license validity
In the status output from the previous section, check the
License information section for each host to ensure
the license is valid and will remain valid until after the planned finish time of the upgrade.
If your upgrade includes a new major Rhino version, e.g 2.6.0 → 2.7.0 then you may require a new license,
as all products deployed on Rhino are licensed against a Rhino version.
Your Metaswitch Customer Care Representative should be
able to provide you with one, in which case you should copy it to the directory where you extracted the upgrade bundle,
and pass it to
orca by appending the parameter
--license <license file> when
running the first upgrade command. Alternatively, they may advise that
the license is already present in the upgrade bundle, or that no new license is needed (in which case
automatically transfer the existing license to the uplevel cluster).
Verify disk space
Log into each node in turn and verify the disk space using
df -h. Ensure there is at least 3GB of free disk space.
Refer to this section if you need to clean up old clusters and/or exports in
order to free disk space.
Check for unexpected alarms
Log into REM and access the connection to the Rhino cluster. Check there are no unexpected alarms. Fix any issues you find. If required, make a note of expected active alarms for correlation with the post-upgrade list of alarms.
Upgrade Rhino Element Manager (REM) nodes
With REM upgraded, you will be able to monitor the nodes as they join the new cluster during the upgrade process.
Check that other network elements are configured to redirect traffic on failure
During the upgrade, at any given point one node of the cluster will be unavailable to handle traffic. As such, calls directed to this node will fail. Network elements such as loadbalancers and S-CSCFs generally utilize a blacklist mechanism, so that INVITEs that get a timeout failure cause both a retry for the call on a different node, and also block that node from being tried for further calls for a while afterwards.
Enable symmetric activation state mode if required
orca only supports upgrading a cluster with symmetric activation state mode enabled. If your cluster normally
operates with it disabled, you will need to enable it here and restore the state after the upgrade.
To check if the cluster has symmetric activation state mode enabled, examine the
orca status output.
At the top under "Global info" it will say
Symmetric activation state mode is currently enabled or
Nodes with per-node activation state: <node list>. If you see the former output then symmetric activation state mode
is enabled, while the latter means it is disabled.
To enable symmetric activation state mode, follow the instructions in the Rhino documentation here.
orca status command
orca --hosts <host1,host2,…> status and verify that the status reports
Symmetric activation state mode is currently enabled.