RVT VM Install Guide (CDMA)

The current version of the VMs being upgraded is known as the downlevel version, and the version that the VMs are being upgraded to is known as the uplevel version.

A rolling upgrade is a procedure where each VM is replaced, one at a time, with a new VM running the uplevel version of software. The Rhino VoLTE TAS nodes are designed to allow rolling upgrades with little or no service outage time.

As with installation, upgrades and rollbacks use the SIMPL VM. The user starts the upgrade process by running csar update on the SIMPL VM. SIMPL VM destroys, in turn, each downlevel node and replaces it with an uplevel node. This is repeated until all nodes have been upgraded.

Configuration for the uplevel nodes is uploaded in advance. As nodes are recreated, they immediately pick up the uplevel configuration and resume service.

If an upgrade goes wrong, rollback to the previous version is also supported.

See the Rolling upgrades and patches page for detailed instructions on how to perform an upgrade.

A TAS Storage Node (TSN) is a VM that runs two Cassandra databases and provides these databases' services to the other node types in a Rhino VoLTE TAS deployment. TSNs run in a cluster with between 3 and 30 nodes per cluster depending on deployment size; load-balancing is performed automatically.

A Management and Authentication Gateway (MAG) node is a node that runs the XCAP server and Sentinel AGW, Metaswitch’s implementation of the 3GPP Generic Authentication Architecture (GAA) framework, consisting of the NAF Authentication Filter and BSF components. These components all run in Rhino. It also runs the Rhino Element Manager management and monitoring software.

An Sh Cache Microservice node provides HTTP access to the HSS via Diameter Sh, as well as caching some of that data to reduce round trips to the HSS.

An MMTel (MMT) node is a VM that runs the Sentinel VoLTE application on Rhino. It provides both SCC and MMTel functionality. It is available in both a GSM and CDMA variant.

A Short Message Gateway and OCSS7 (SMO) node is a VM that runs the Sentinel IP-SM-GW application on Rhino, which provides IP Short Message Gateway functionality. It also runs the OCSS7 application, which provides the SS7 protocol stack for the MMT and SMO nodes.

The SIMPL Virtual Appliance provides orchestration software to create, verify, configure, destroy and upgrade RVT instances. Following the initial deployment, you will only need the SIMPL VM to perform configuration changes, patching or upgrades - it is not required for normal operation of the RVT deployment.

Rhino VoLTE TAS deployments use Metaswitch Deployment Manager (MDM) to co-ordinate installation, upgrades, scale and healing (replacement of failed instances). MDM is a virtual appliance that provides state monitoring, DNS and NTP services to the deployment. It is deployed as a pool of at least three virtual machines, and can also manage other Metaswitch products that might be present in your deployment such as Service Assurance Server (SAS) and Clearwater. A single pool of VMs can manage all instances of compatible Metaswitch products you are using.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

Anyone can perform these MOP steps.

This page references an external document: the SIMPL VM Documentation. Ensure you have a copy available before proceeding.

This can either be done on your local Linux machine or on a SIMPL VM.

The Solution Definition File (SDF) contains all the information required to set up your cluster. It is therefore crucial to ensure all information in the SDF is correct before beginning the deployment. One SDF should be written per deployment.

It is recommended that the SDF is written before starting the deployment. The SDF must be named sdf-rvt.yaml.

In addition, you will need to write a secrets file and upload its contents to QSG. For security, the SDF no longer contains plaintext values of secrets (such as the password to access the VM host). Instead, the SDF contains secret IDs which refer to secrets stored in QSG.

See the various pages in the Writing an SDF section for more detailed information.

Example SDFs are included in every CSAR and can also be found at Example SDFs. We recommend that you start from a template SDF and edit it as desired instead of writing an SDF from scratch.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have access to a local computer (referred to in this procedure as the local computer) with a network connection and access to the vSphere client.

This page references an external document: the SIMPL VM Documentation. Ensure you have a copy available before proceeding.

Follow the SIMPL VM Documentation on how to deploy the SIMPL VM and set up the configuration.

Create configuration YAML files relevant for your node type on the SIMPL VM. Store these files in the same directory as your prepared SDF.

See Example configuration YAML files for example configuration files.

Generate a template secrets.yaml file by running csar secrets create-input-file --sdf <path to SDF>.

Replace the value of any secrets in your SDF with a secret ID. The secret ID and corresponding secret value should be written in secrets.yaml.

Run the command csar secrets add <path to secrets.yaml template> to add the secrets to the secret store.

Refer to the Refer to the SIMPL VM documentation for more information.

MDM requires the following certificates and keys. Refer to the MDM documentation for more details.

If the CA used is an in-house CA, keep the CA private key safe so that you can generate a new static certificate and private key from the same CA in the future. Add the other credentials to QSG as described in MDM service group.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have access to the SIMPL VM, and the SIMPL VM must have the right permissions on the VMware vSphere deployment.

In the below steps, replace parameters marked with angle brackets (such as <deployment ID>) with values as follows. (Replace the angle brackets as well, so that they are not included in the final command to be run.)

Run csar deploy --vnf tsn --sdf <path to SDF> --sites <site name>.

This will validate the SDF, and generate the terraform template. After successful validation, this will upload the image, and deploy the number of TSN nodes specified in the SDF.

Validate the configuration for the TSN nodes to ensure that each TSN node can properly self-configure.

To validate the configuration after creating the YAML files, run

rvtconfig validate -t tsn -i <yaml-config-file-directory>

on the SIMPL VM from the resources subdirectory of the TSN CSAR.

Upload the configuration for the TSN nodes to the CDS. This will enable each TSN node to self-configure.

To upload configuration after creating the YAML files and validating them as described above, run

rvtconfig upload-config -c <CDS address> <CDS auth args> -t tsn -i <yaml-config-file-directory> (--vm-version-source this-rvtconfig | --vm-version <vm version>)

on the SIMPL VM from the resources subdirectory of the TSN CSAR.

See Example configuration YAML files for example configuration files.

An in-depth description of RVT YAML configuration can be found in the Rhino VoLTE TAS Configuration and Management Guide.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have access to the SIMPL VM, and the SIMPL VM must have the right permissions on the VMware vSphere deployment.

In the below steps, replace parameters marked with angle brackets (such as <deployment ID>) with values as follows. (Replace the angle brackets as well, so that they are not included in the final command to be run.)

Validate the configuration for the MAG nodes to ensure that each MAG node can properly self-configure.

To validate the configuration after creating the YAML files, run

rvtconfig validate -t mag -i <yaml-config-file-directory>

on the SIMPL VM from the resources subdirectory of the MAG CSAR.

Upload the configuration for the MAG nodes to the CDS. This will enable each MAG node to self-configure when they are deployed in the next step.

To upload configuration after creating the YAML files and validating them as described above, run

rvtconfig upload-config -c <CDS address> <CDS auth args> -t mag -i <yaml-config-file-directory> (--vm-version-source this-rvtconfig | --vm-version <vm version>)

on the SIMPL VM from the resources subdirectory of the MAG CSAR.

See Example configuration YAML files for example configuration files.

An in-depth description of RVT YAML configuration can be found in the Rhino VoLTE TAS Configuration and Management Guide.

Run csar deploy --vnf mag --sdf <path to SDF> --sites <site name>.

This will validate the SDF, and generate the terraform template. After successful validation, this will upload the image, and deploy the number of MAG nodes specified in the SDF.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have access to the SIMPL VM, and the SIMPL VM must have the right permissions on the VMware vSphere deployment.

In the below steps, replace parameters marked with angle brackets (such as <deployment ID>) with values as follows. (Replace the angle brackets as well, so that they are not included in the final command to be run.)

Validate the configuration for the ShCM nodes to ensure that each ShCM node can properly self-configure.

To validate the configuration after creating the YAML files, run

rvtconfig validate -t shcm -i <yaml-config-file-directory>

on the SIMPL VM from the resources subdirectory of the ShCM CSAR.

Upload the configuration for the ShCM nodes to the CDS. This will enable each ShCM node to self-configure when they are deployed in the next step.

To upload configuration after creating the YAML files and validating them as described above, run

rvtconfig upload-config -c <CDS address> <CDS auth args> -t shcm -i <yaml-config-file-directory> (--vm-version-source this-rvtconfig | --vm-version <vm version>)

on the SIMPL VM from the resources subdirectory of the ShCM CSAR.

See Example configuration YAML files for example configuration files.

An in-depth description of RVT YAML configuration can be found in the Rhino VoLTE TAS Configuration and Management Guide.

Run csar deploy --vnf shcm --sdf <path to SDF> --sites <site name>.

This will validate the SDF, and generate the terraform template. After successful validation, this will upload the image, and deploy the number of ShCM nodes specified in the SDF.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have access to the SIMPL VM, and the SIMPL VM must have the right permissions on the VMware vSphere deployment.

In the below steps, replace parameters marked with angle brackets (such as <deployment ID>) with values as follows. (Replace the angle brackets as well, so that they are not included in the final command to be run.)

Validate the configuration for the MMT CDMA nodes to ensure that each MMT CDMA node can properly self-configure.

To validate the configuration after creating the YAML files, run

rvtconfig validate -t mmt-cdma -i <yaml-config-file-directory>

on the SIMPL VM from the resources subdirectory of the MMT CDMA CSAR.

Upload the configuration for the MMT CDMA nodes to the CDS. This will enable each MMT CDMA node to self-configure when they are deployed in the next step.

To upload configuration after creating the YAML files and validating them as described above, run

rvtconfig upload-config -c <CDS address> <CDS auth args> -t mmt-cdma -i <yaml-config-file-directory> (--vm-version-source this-rvtconfig | --vm-version <vm version>)

on the SIMPL VM from the resources subdirectory of the MMT CDMA CSAR.

See Example configuration YAML files for example configuration files.

An in-depth description of RVT YAML configuration can be found in the Rhino VoLTE TAS Configuration and Management Guide.

Run csar deploy --vnf mmt-cdma --sdf <path to SDF> --sites <site name>.

This will validate the SDF, and generate the terraform template. After successful validation, this will upload the image, and deploy the number of MMT CDMA nodes specified in the SDF.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have access to the SIMPL VM, and the SIMPL VM must have the right permissions on the VMware vSphere deployment.

In the below steps, replace parameters marked with angle brackets (such as <deployment ID>) with values as follows. (Replace the angle brackets as well, so that they are not included in the final command to be run.)

Validate the configuration for the SMO nodes to ensure that each SMO node can properly self-configure.

To validate the configuration after creating the YAML files, run

rvtconfig validate -t smo -i <yaml-config-file-directory>

on the SIMPL VM from the resources subdirectory of the SMO CSAR.

Upload the configuration for the SMO nodes to the CDS. This will enable each SMO node to self-configure when they are deployed in the next step.

To upload configuration after creating the YAML files and validating them as described above, run

rvtconfig upload-config -c <CDS address> <CDS auth args> -t smo -i <yaml-config-file-directory> (--vm-version-source this-rvtconfig | --vm-version <vm version>)

on the SIMPL VM from the resources subdirectory of the SMO CSAR.

See Example configuration YAML files for example configuration files.

An in-depth description of RVT YAML configuration can be found in the Rhino VoLTE TAS Configuration and Management Guide.

Run csar deploy --vnf smo --sdf <path to SDF> --sites <site name>.

This will validate the SDF, and generate the terraform template. After successful validation, this will upload the image, and deploy the number of SMO nodes specified in the SDF.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

Anyone can perform these MOP steps.

This page references an external document: the SIMPL VM Documentation. Ensure you have a copy available before proceeding.

This can either be done on your local Linux machine or on a SIMPL VM.

The Solution Definition File (SDF) contains all the information required to set up your cluster. It is therefore crucial to ensure all information in the SDF is correct before beginning the deployment. One SDF should be written per deployment.

It is recommended that the SDF is written before starting the deployment. The SDF must be named sdf-rvt.yaml.

In addition, you will need to write a secrets file and upload its contents to QSG. For security, the SDF no longer contains plaintext values of secrets (such as the password to access the VM host). Instead, the SDF contains secret IDs which refer to secrets stored in QSG.

See the various pages in the Writing an SDF section for more detailed information.

Example SDFs are included in every CSAR and can also be found at Example SDFs. We recommend that you start from a template SDF and edit it as desired instead of writing an SDF from scratch.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have:

This page references an external document: the SIMPL VM Documentation. Ensure you have a copy available before proceeding.

Follow the SIMPL VM Documentation on how to deploy the SIMPL VM and set up the configuration.

Create configuration YAML files relevant for your node type on the SIMPL VM. Store these files in the same directory as your prepared SDF.

See Example configuration YAML files for example configuration files.

Generate a template secrets.yaml file by running csar secrets create-input-file --sdf <path to SDF>.

Replace the value of any secrets in your SDF with a secret ID. The secret ID and corresponding secret value should be written in secrets.yaml.

Run the command csar secrets add <path to secrets.yaml template> to add the secrets to the secret store.

Refer to the Refer to the SIMPL VM documentation for more information.

You can find the possible flavors in the Flavors section, and it is recommended to use the same flavor name as described there.

Some node types share flavors. If the same flavor is to be used for multiple node types, only create it once.

You have now created the OpenStack flavor you will need when following the procedure to install the nodes on OpenStack virtual machines.

MDM requires the following certificates and keys. Refer to the MDM documentation for more details.

If the CA used is an in-house CA, keep the CA private key safe so that you can generate a new static certificate and private key from the same CA in the future. Add the other credentials to QSG as described in MDM service group.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have access to the SIMPL VM, and the SIMPL VM must have the right permissions on the OpenStack deployment.

In the below steps, replace parameters marked with angle brackets (such as <deployment ID>) with values as follows. (Replace the angle brackets as well, so that they are not included in the final command to be run.)

The SIMPL VM creates one server group per VM, and one security group per interface on each VM. OpenStack sets limits on the number of server groups and security groups through quotas.

View the quota by running openstack quota show <project id> on OpenStack Controller node. This shows the maximum number of various resources.

You can view the existing server groups by running openstack server group list. Similarly, you can find the security groups by running openstack security group list

If the quota is too small to accommodate the new VMs that will be deployed, increase it by running
openstack quota set --<quota field to increase> <new quota value> <project ID>. For example:
openstack quota set --server-groups 100 125610b8bf424e61ad2aa5be27ad73bb

Run csar deploy --vnf tsn --sdf <path to SDF> --sites <site name>.

This will validate the SDF, and generate the heat template. After successful validation, this will upload the image, and deploy the number of TSN nodes specified in the SDF.

Validate the configuration for the TSN nodes to ensure that each TSN node can properly self-configure.

To validate the configuration after creating the YAML files, run

rvtconfig validate -t tsn -i <yaml-config-file-directory>

on the SIMPL VM from the resources subdirectory of the TSN CSAR.

Upload the configuration for the TSN nodes to the CDS. This will enable each TSN node to self-configure.

To upload configuration after creating the YAML files and validating them as described above, run

rvtconfig upload-config -c <CDS address> <CDS auth args> -t tsn -i <yaml-config-file-directory> (--vm-version-source this-rvtconfig | --vm-version <vm version>)

on the SIMPL VM from the resources subdirectory of the TSN CSAR.

See Example configuration YAML files for example configuration files.

An in-depth description of RVT YAML configuration can be found in the Rhino VoLTE TAS Configuration and Management Guide.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have access to the SIMPL VM, and the SIMPL VM must have the right permissions on the OpenStack deployment.

In the below steps, replace parameters marked with angle brackets (such as <deployment ID>) with values as follows. (Replace the angle brackets as well, so that they are not included in the final command to be run.)

The SIMPL VM creates one server group per VM, and one security group per interface on each VM. OpenStack sets limits on the number of server groups and security groups through quotas.

View the quota by running openstack quota show <project id> on OpenStack Controller node. This shows the maximum number of various resources.

You can view the existing server groups by running openstack server group list. Similarly, you can find the security groups by running openstack security group list

If the quota is too small to accommodate the new VMs that will be deployed, increase it by running
openstack quota set --<quota field to increase> <new quota value> <project ID>. For example:
openstack quota set --server-groups 100 125610b8bf424e61ad2aa5be27ad73bb

Validate the configuration for the MAG nodes to ensure that each MAG node can properly self-configure.

To validate the configuration after creating the YAML files, run

rvtconfig validate -t mag -i <yaml-config-file-directory>

on the SIMPL VM from the resources subdirectory of the MAG CSAR.

Upload the configuration for the MAG nodes to the CDS. This will enable each MAG node to self-configure when they are deployed in the next step.

To upload configuration after creating the YAML files and validating them as described above, run

rvtconfig upload-config -c <CDS address> <CDS auth args> -t mag -i <yaml-config-file-directory> (--vm-version-source this-rvtconfig | --vm-version <vm version>)

on the SIMPL VM from the resources subdirectory of the MAG CSAR.

See Example configuration YAML files for example configuration files.

An in-depth description of RVT YAML configuration can be found in the Rhino VoLTE TAS Configuration and Management Guide.

Run csar deploy --vnf mag --sdf <path to SDF> --sites <site name>.

This will validate the SDF, and generate the heat template. After successful validation, this will upload the image, and deploy the number of MAG nodes specified in the SDF.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have access to the SIMPL VM, and the SIMPL VM must have the right permissions on the OpenStack deployment.

In the below steps, replace parameters marked with angle brackets (such as <deployment ID>) with values as follows. (Replace the angle brackets as well, so that they are not included in the final command to be run.)

The SIMPL VM creates one server group per VM, and one security group per interface on each VM. OpenStack sets limits on the number of server groups and security groups through quotas.

View the quota by running openstack quota show <project id> on OpenStack Controller node. This shows the maximum number of various resources.

You can view the existing server groups by running openstack server group list. Similarly, you can find the security groups by running openstack security group list

If the quota is too small to accommodate the new VMs that will be deployed, increase it by running
openstack quota set --<quota field to increase> <new quota value> <project ID>. For example:
openstack quota set --server-groups 100 125610b8bf424e61ad2aa5be27ad73bb

Validate the configuration for the ShCM nodes to ensure that each ShCM node can properly self-configure.

To validate the configuration after creating the YAML files, run

rvtconfig validate -t shcm -i <yaml-config-file-directory>

on the SIMPL VM from the resources subdirectory of the ShCM CSAR.

Upload the configuration for the ShCM nodes to the CDS. This will enable each ShCM node to self-configure when they are deployed in the next step.

To upload configuration after creating the YAML files and validating them as described above, run

rvtconfig upload-config -c <CDS address> <CDS auth args> -t shcm -i <yaml-config-file-directory> (--vm-version-source this-rvtconfig | --vm-version <vm version>)

on the SIMPL VM from the resources subdirectory of the ShCM CSAR.

See Example configuration YAML files for example configuration files.

An in-depth description of RVT YAML configuration can be found in the Rhino VoLTE TAS Configuration and Management Guide.

Run csar deploy --vnf shcm --sdf <path to SDF> --sites <site name>.

This will validate the SDF, and generate the heat template. After successful validation, this will upload the image, and deploy the number of ShCM nodes specified in the SDF.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have access to the SIMPL VM, and the SIMPL VM must have the right permissions on the OpenStack deployment.

In the below steps, replace parameters marked with angle brackets (such as <deployment ID>) with values as follows. (Replace the angle brackets as well, so that they are not included in the final command to be run.)

The SIMPL VM creates one server group per VM, and one security group per interface on each VM. OpenStack sets limits on the number of server groups and security groups through quotas.

View the quota by running openstack quota show <project id> on OpenStack Controller node. This shows the maximum number of various resources.

You can view the existing server groups by running openstack server group list. Similarly, you can find the security groups by running openstack security group list

If the quota is too small to accommodate the new VMs that will be deployed, increase it by running
openstack quota set --<quota field to increase> <new quota value> <project ID>. For example:
openstack quota set --server-groups 100 125610b8bf424e61ad2aa5be27ad73bb

Validate the configuration for the MMT CDMA nodes to ensure that each MMT CDMA node can properly self-configure.

To validate the configuration after creating the YAML files, run

rvtconfig validate -t mmt-cdma -i <yaml-config-file-directory>

on the SIMPL VM from the resources subdirectory of the MMT CDMA CSAR.

Upload the configuration for the MMT CDMA nodes to the CDS. This will enable each MMT CDMA node to self-configure when they are deployed in the next step.

To upload configuration after creating the YAML files and validating them as described above, run

rvtconfig upload-config -c <CDS address> <CDS auth args> -t mmt-cdma -i <yaml-config-file-directory> (--vm-version-source this-rvtconfig | --vm-version <vm version>)

on the SIMPL VM from the resources subdirectory of the MMT CDMA CSAR.

See Example configuration YAML files for example configuration files.

An in-depth description of RVT YAML configuration can be found in the Rhino VoLTE TAS Configuration and Management Guide.

Run csar deploy --vnf mmt-cdma --sdf <path to SDF> --sites <site name>.

This will validate the SDF, and generate the heat template. After successful validation, this will upload the image, and deploy the number of MMT CDMA nodes specified in the SDF.

This procedure assumes that:

This procedure does not require a maintenance period. However, if you are integrating into a live network, we recommend that you implement measures to mitigate any unforeseen events.

This procedure does not impact service.

You must be a system operator to perform the MOP steps.

You must have access to the SIMPL VM, and the SIMPL VM must have the right permissions on the OpenStack deployment.

In the below steps, replace parameters marked with angle brackets (such as <deployment ID>) with values as follows. (Replace the angle brackets as well, so that they are not included in the final command to be run.)

The SIMPL VM creates one server group per VM, and one security group per interface on each VM. OpenStack sets limits on the number of server groups and security groups through quotas.

View the quota by running openstack quota show <project id> on OpenStack Controller node. This shows the maximum number of various resources.

You can view the existing server groups by running openstack server group list. Similarly, you can find the security groups by running openstack security group list

If the quota is too small to accommodate the new VMs that will be deployed, increase it by running
openstack quota set --<quota field to increase> <new quota value> <project ID>. For example:
openstack quota set --server-groups 100 125610b8bf424e61ad2aa5be27ad73bb

Validate the configuration for the SMO nodes to ensure that each SMO node can properly self-configure.

To validate the configuration after creating the YAML files, run

rvtconfig validate -t smo -i <yaml-config-file-directory>

on the SIMPL VM from the resources subdirectory of the SMO CSAR.

Upload the configuration for the SMO nodes to the CDS. This will enable each SMO node to self-configure when they are deployed in the next step.

To upload configuration after creating the YAML files and validating them as described above, run

rvtconfig upload-config -c <CDS address> <CDS auth args> -t smo -i <yaml-config-file-directory> (--vm-version-source this-rvtconfig | --vm-version <vm version>)

on the SIMPL VM from the resources subdirectory of the SMO CSAR.

See Example configuration YAML files for example configuration files.

An in-depth description of RVT YAML configuration can be found in the Rhino VoLTE TAS Configuration and Management Guide.

Run csar deploy --vnf smo --sdf <path to SDF> --sites <site name>.

This will validate the SDF, and generate the heat template. After successful validation, this will upload the image, and deploy the number of SMO nodes specified in the SDF.

In the below steps, replace parameters marked with angle brackets (such as <deployment ID>) with values as follows. (Replace the angle brackets as well, so that they are not included in the final command to be run.)

You must have the SSH keys required to access the SIMPL VM and the TSN VMs that are to be upgraded.

The SIMPL VM must have the right permissions on the VNFI. Refer to the SIMPL VM documentation for more information:

rvtconfig is a command-line tool for configuring and managing Rhino VoLTE TAS VMs. All TSN CSARs include this tool; once the CSAR is unpacked, you can find rvtconfig in the resources directory, for example:

The rest of this page assumes that you are running rvtconfig from the directory in which it resides, so that it can be invoked as ./rvtconfig. It assumes you use the uplevel version of rvtconfig, unless instructed otherwise. If it is explicitly specified you must use the downlevel version, you can find it here:

Log into the SIMPL VM and run the command simpl-version. The SIMPL VM version is displayed at the top of the output:

Ensure this is at least 6.15.3. If not, contact your Customer Care Representative to organise upgrading the SIMPL VM before proceeding with the upgrade of the TSN VMs.

Output shown on this page is correct for version 6.15.3 of the SIMPL VM; it may differ slightly on later versions.

Your Customer Care Representative will have provided you with the uplevel TSN CSAR. Use scp to copy this to /csar-volume/csar/ on the SIMPL VM.

Once the copy is complete, run csar unpack /csar-volume/csar/<filename> on the SIMPL VM (replacing <filename> with the filename of the CSAR, which will end with .zip).

The csar unpack command may fail if there is insufficient disk space available. If this occurs, SIMPL VM will report this with instructions to remove some CSARs to free up disk space. You can list all unpacked CSARs with csar list and remove a CSAR with csar remove <node type>/<version>.

On the SIMPL VM, run csar list.

Ensure that there is a TSN CSAR listed there with the current downlevel version.

If you are upgrading to an image that doesn’t require patching, or have already applied the patch, skip this step.

To patch a set of VMs, rather than modify the code directly on the VMs, the procedure is instead to patch the CSAR on SIMPL VM and then upgrade to the patched CSAR.

If you have a patch to apply, it will be provided to you in the form of a .tar.gz file. Use scp to transfer this file to /csar-volume/csar/ on the SIMPL VM. Apply it to the uplevel CSAR by running csar efix tsn/<uplevel version> <patch file>, for example, csar efix tsn/4.2-8-1.0.0/csar-volume/csar/mypatch.tar.gz. This takes about five minutes to complete.

Check the output of the patching process states that SIMPL VM successfully created a patch. Example output for a patch named mypatch on version 4.2-8-1.0.0 and a vSphere deployment is:

You can verify that a patched CSAR now exists by running csar list again - you should see a CSAR named tsn/<uplevel version>-<patch name> (for the above example that would be tsn/4.2-8-1.0.0-mypatch).

For all future steps on this page, wherever you type the <uplevel version>, be sure to include the suffix with the patch name, for example 4.2-8-1.0.0-mypatch.

If the csar efix command fails, be sure to delete any partially-created patched CSAR before retrying the patch process. Run csar list as above, and if you see the patched CSAR, delete it with csar remove <CSAR>.

If you keep the configuration hosted on the SIMPL VM, find it and rename it to /home/admin/current-config. Verify the contents by running ls /home/admin/current-config and checking that at least the SDF (sdf-rvt.yaml) is present there. If it isn’t, or you prefer to keep your configuration outside of the SIMPL VM, then create this directory on the SIMPL VM:

Use scp to upload the SDF (sdf-rvt.yaml) to this directory.

On the SIMPL VM, run mkdir /home/admin/uplevel-config. This directory is for holding the uplevel configuration files.

Use scp (or cp if the files are already on the SIMPL VM, for example in /home/admin/current-config as detailed in the previous section) to copy the following files to this directory. Include configuration for the entire deployment, not just the TSN nodes.

Open the /home/admin/uplevel-config/sdf-rvt.yaml file using vi. Find the vnfcs section, and within that the TSN VNFC. Within the VNFC, locate the version field and change its value to the uplevel version, for example 4.2-8-1.0.0. Save and close the file.

You can verify the change you made by using diff -u2 /home/admin/current-config/sdf-rvt.yaml /home/admin/uplevel-config/sdf-rvt.yaml. The diff should look like this (context lines and line numbers may vary), with only a change to the version for the relevant node type:

The upgrade procedure requires a maintenance period. For upgrading nodes in a live network, implement measures to mitigate any unforeseen events.

Ensure you reserve enough time for the maintenance period, which must include the time for a potential rollback.

To calculate the time required for the actual upgrade or roll back of the VMs, run rvtconfig calculate-maintenance-window -i /home/admin/uplevel-config -t tsn --site-id <site ID>. The output will be similar to the following, stating how long it will take to do an upgrade or rollback of the TSN VMs.

You must also reserve time for:

The csar update dry run command carries out more extensive validation of the SDF and VM states than rvtconfig validate does.

Carrying out this step now, before the upgrade is due to take place, ensures problems with the SDF files are identified early and can be rectified beforehand.

Please run the following command to execute the dry run.

Confirm the output does not flag any problems or errors. The end of the command output should look similar to this.

Please confirm the set of nodes you are upgrading looks correct, and that the software version against the service group correctly indicates the software version you are planning to upgrade to.

If you see any errors, please address them, then re-run the dry run command until it indicates success.

Before starting the upgrade procedure, run VNF validation tests from the SIMPL VM against the downlevel nodes: csar validate --vnf tsn --sdf /home/admin/current-config/sdf-rvt.yaml

This command performs various checks on the health of the VMs' networking and services:

If all is well, then you should see the message All tests passed for CSAR 'tsn/{example-downlevel-version}'!.

If the VM validation fails, you can find details in the log file. The log file can be found in /var/log/csar/ansible_output-<timestamp>.log. The msg field under each ansible task explains why the script failed.

If there are failures, the upgrade cannot take place. Investigate them with the help of your Customer Care Representative and the Troubleshooting pages.

Once the VNF validation tests pass, you can proceed with the next step.

Only perform this step if this is the first, or only, node type being upgraded.

Run ./rvtconfig enter-maintenance-window -c <CDS address> <CDS auth args> -d <deployment ID> --site-id <site ID> --hours <MW duration in hours>. The output will look similar to:

This will prevent scheduled tasks running on the VMs until the time given in the output.

If at any point in the upgrade process you wish to confirm the end time of the maintenance window, you can run ./rvtconfig maintenance-window-status -c <CDS address> <CDS auth args> -d <deployment ID> --site-id <site ID>.

Run rm -rf /home/admin/config-output on the SIMPL VM to remove that directory if it already exists. Then use the command ./rvtconfig compare-config -c <CDS address> <CDS auth args> -d <deployment ID> --input /home/admin/uplevel-config
--vm-version <downlevel version> --output-dir /home/admin/config-output -t tsn to compare the live configuration to the configuration in the /home/admin/uplevel-config directory.

Example output is listed below:

You can then view the differences using commands such as cat /home/admin/config-output/sdf-rvt.yaml.diff (there will be one .diff file for every file that has differences). Aside from the version parameter in the SDF, there should normally be no other changes. If there are other unexpected changes, pause the procedure here and correct the configuration by editing the files in /home/admin/uplevel-config.

When performing a rolling upgrade, some elements of the uplevel configuration must remain identical to those in the downlevel configuration. The affected elements of the TSN configuration are described in the following list:

The rvtconfig compare-config command reports any unsupported changes as errors, and may also emit warnings about other changes. For example:

Ensure you address the reported errors, if any, before proceeding. rvtconfig will not upload a set of configuration files that contains unsupported changes.

First, establish an SSH session to the management IP of the first TSN node. To check that the primary Cassandra cluster is healthy, run nodetool status on the TSN node:

Check that all TSN nodes are present and listed as UN (Up and Normal). The output in the Owns colomn may differ and is irrelevant.

Next, check that the ramdisk-based Cassandra cluster is healthy. Run nodetool status -p 17199 on the TSN node. Again, check that all TSN nodes are present and listed as UN.

If either the primary or ramdisk-based Cassandra cluster is not healthy (i.e. not all TSN nodes show up as UN in the output from nodetool status and nodetool status -p 17199), stop the upgrade process here and troubleshoot the node. Only continue after both the Cassandra clusters are healthy.

Run the command ./rvtconfig validate -t tsn -i /home/admin/uplevel-config to check that the configuration files are correctly formatted, contain valid values, and are self-consistent. A successful validation with no errors or warnings produces the following output.

If the output contains validation errors, fix the configuration in the /home/admin/uplevel-config directory

If the output contains validation warnings, consider whether you wish to address them before performing the upgrade. The VMs will accept configuration that has validation warnings, but certain functions may not work.

Upload the configuration to CDS:

Check that the output confirms that configuration exists in CDS for both the current (downlevel) version and the uplevel version:

We recommend gathering diagnostic archives for all TSN VMs in the deployment.

On the SIMPL VM, run the command

If <diags-bundle> does not exist, the command will create the directory for you.

Each diagnostic archive can be up to 200 MB per VM. Ensure you have enough disk space on the SIMPL VM to collect all diagnostics. The command will be aborted if the SIMPL VM does not have enough disk space to collect all diagnostic archives from all the VMs in your deployment specified in the provided SDF.

Set the running state of initconf processes in non-TSN VMs to a paused state.

./rvtconfig set-desired-running-state --sdf /home/admin/uplevel-config/sdf-rvt.yaml --site-id <site ID> --state Stopped.

You should see an output similar to this, indicating that the initconf process of non-TSN nodes are in state Stopped.

Take a backup of the CDS database by issuing the command below.

./rvtconfig backup-cds --sdf /home/admin/uplevel-config/sdf-rvt.yaml --site-id <site ID> --output-dir <backup-cds-bundle> --ssh-key-secret-id <SSH key secret ID> -c <CDS address> <CDS auth args>

The output should look like this: