This page describes steps to take before starting a Sentinel IP-SM-GW upgrade.

Note that there is no specific need to take an export prior to starting the upgrade. Taking an export is the first step that orca will perform when issued the upgrade command, and the export will be available at the end of the upgrade process.

Create validation test plans

Devise a validation test plan to ensure that, after the upgrade:

  • traffic is routed as expected and calls work (including audio and, where appropriate, video)

  • any features in the downlevel version that you rely on still work as expected

  • any new features you are expecting to use in the uplevel version work as expected.

Your network configuration may support setting up test numbers where calls to these numbers are routed to specific nodes. This is a particularly useful tool for verifying the upgrade early, as after upgrading the first node you can make calls using test numbers configured to route specifically to the first node.

During or after the upgrade, if you find the validation tests are failing, contact your Metaswitch Customer Care Representative to discuss next steps.

Ensure the nodes are configured with the standardized path structure

orca requires the nodes to have a standardized path structure. Within the HOME directory (normally /home/sentinel) there must be:

  • one or more Rhino installation directories with a name of the following format: ipsmgw-<version>-cluster-<id>, e.g. ipsmgw-2.7.1.2-cluster-41

  • a symbolic link named rhino pointing at the currently live Rhino installation

  • a directory named export

  • a directory named install.

See Standardized Path Structure for more information.

If your installation does not conform to this setup, consult your Metaswitch Customer Care Representative who can advise on how to convert it.

Record the old cluster ID

In case you need to rollback the upgrade, you will need the current cluster ID.

Execute the status command on all the hosts:

orca --hosts <host1,host2,…​> status

The first part of the output for each host will include the cluster status. One of the clusters should be marked as "LIVE".

Status of host host1

Clusters:
 - ipsmgw-2.7.1.2-cluster-41
 - ipsmgw-2.7.1.4-cluster-42 - LIVE

On the "LIVE" cluster, the last part of the name given is the cluster ID. For example, in the above output the cluster ID is 42.

Ensure all clusters have the same live cluster ID. Make a note of this value.

Verify license validity

In the status output from the previous section, check the License information section for each host to ensure the license is valid and will remain valid until after the planned finish time of the upgrade.

If your upgrade includes a new major Rhino version, e.g 2.6.0 → 2.7.0 then you may require a new license, as all products deployed on Rhino are licensed against a Rhino version. Your Metaswitch Customer Care Representative should be able to provide you with one, in which case you should copy it to the directory where you extracted the upgrade bundle, and pass it to orca by appending the parameter --license <license file> when running the first upgrade command. Alternatively, they may advise that the license is already present in the upgrade bundle, or that no new license is needed (in which case orca will automatically transfer the existing license to the uplevel cluster).

Verify disk space

Log into each node in turn and verify the disk space using df -h. Ensure there is at least 3GB of free disk space. Refer to this section if you need to clean up old clusters and/or exports in order to free disk space.

Check for unexpected alarms

Log into REM and access the connection to the Rhino cluster. Check there are no unexpected alarms. Fix any issues you find. If required, make a note of expected active alarms for correlation with the post-upgrade list of alarms.

Upgrade Rhino Element Manager (REM) nodes

See Upgrade Rhino Element Manager.

With REM upgraded, you will be able to monitor the nodes as they join the new cluster during the upgrade process.

Check that other network elements are configured to redirect traffic on failure

During the upgrade, at any given point one node of the cluster will be unavailable to handle traffic. As such, calls directed to this node will fail. Network elements such as loadbalancers and S-CSCFs generally utilize a blacklist mechanism, so that INVITEs that get a timeout failure cause both a retry for the call on a different node, and also block that node from being tried for further calls for a while afterwards.

Enable symmetric activation state mode if required

orca only supports upgrading a cluster with symmetric activation state mode enabled. If your cluster normally operates with it disabled, you will need to enable it here and restore the state after the upgrade.

To check if the cluster has symmetric activation state mode enabled, examine the orca status output. At the top under "Global info" it will say Symmetric activation state mode is currently enabled or Nodes with per-node activation state: <node list>. If you see the former output then symmetric activation state mode is enabled, while the latter means it is disabled.

To enable symmetric activation state mode, follow the instructions in the Rhino documentation here.

Rerun the orca status command orca --hosts <host1,host2,…​> status and verify that the status reports Symmetric activation state mode is currently enabled.

Previous page Next page