Planning for the procedure
Background knowledge
This procedure assumes that:
-
you are installing into an existing OpenStack deployment
-
The OpenStack deployment must be set up with support for Heat templates.
-
-
you are using an OpenStack version from Icehouse through to Train inclusive
-
you are thoroughly familiar with working with OpenStack machines and know how to set up tenants, users, roles, client environment scripts, and so on.
(For more information, refer to the appropriate OpenStack installation guide for the version that you are using here.) -
you are upgrading an exisiting downlevel deployment for REM.
-
you have deployed a SIMPL VM, unpacked the uplevel CSAR, and prepared an uplevel SDF.
Method of procedure
Refer to the SIMPL VM Documentation for details on the commands mentioned in the procedure. |
Step 1 - Generate the uplevel heat template
Run csar generate --vnf rem --sdf <path to SDF>
.
This will validate the uplevel SDF, and generate the uplevel Terraform template. If any errors occur, check the documentation on preparing the SDF and fix the uplevel SDF.
Step 2 - Upgrade the downlevel REM nodes using the uplevel heat template
Run csar update --vnf rem
.
This will upload the uplevel image.
The following will occur one REM node at a time:
-
The downlevel node will be quiesced.
-
The uplevel node will be created and boot up.
-
The VM will automatically start applying configuration from the files you uploaded to CDS in the above steps. During this phase, the status of the VM in MDM will be Orange.
-
Once configuration is complete, the status will change to Green, and the node will be ready for service. At this point the
csar update
command will move on to the next REM VM, or report that the upgrade of the REM was successful if all nodes have now been upgraded.
To perform a canary upgrade, run
csar update --vnf rem --sites <site> --service-group <service_group> --index-range <range> . The
range accepts a comma delimited index of nodes starting from 0. Only the nodes specified
in the index will be upgraded.
|
Backout procedure
If the upgrade has brought up uplevel VMs to replace the downlevel VMs, then the uplevel VMs can be rolled back to the downlevel VMs.
To rollback, repeat the steps 1 and 2 with the downlevel REM CSAR and downlevel SDF,
appending the --skip-pre-update-checks
flag to the csar update
command in step 2.
The --skip-pre-update-checks
flag allows rollbacks when a node is unhealthy.
If the upgrade has failed to bring up the uplevel VMs or the rollback has failed to bring up the downlevel VMs,
then you must redeploy the downlevel VMs.
Repeat step 1 with the downlevel CSAR and SDF,
then run csar deploy --redeploy --vnf rem --sites <site>
.
Diagnostics during the quiesce stage
When the downlevel VMs are quiesced, they upload some diagnostics to CDS. These may be useful if the upgrade or rollback fails. To get these diagnostics, follow instructions from Retrieving deployment TAS audit logs with export-audit-history and Retrieving Initconf and Rhino logs with export-log-history.
Next Step
Verify your REM upgrade here: Verify the state of the nodes and processes.