Bootstrap
Bootstrap is the process whereby, after a VM is started for the first time, it is configured with key system-level configuration such as IP addresses, DNS and NTP server addresses, a hostname, and so on. This process runs automatically on the first boot of the VM. For bootstrap to succeed it is crucial that all entries in the SDF (or in the case of a manual deployment, all the bootstrap parameters) are correct.
Successful bootstrap
Once the VM has booted into multi-user mode, bootstrap normally takes about one minute.
SSH access to the VM is not possible until bootstrap has completed. If you want to monitor
bootstrap from the console, log in as the sentinel user with the password you set in the SDF
and examine the log file bootstrap/bootstrap.log.
Successful completion is indicated by the line Bootstrap complete.
Troubleshooting bootstrap
If bootstrap fails, an exception will be written to the log file. If the network-related portion
of bootstrap succeeded but a failure occurred afterwards, the VM will be accessible over SSH and
logging in will display a warning Automatic bootstrap failed.
Examine the log file bootstrap/bootstrap.log to see why bootstrap failed. In the majority of cases
it will be down to an incorrect SDF or a missing or invalid bootstrap parameter. Destroy the VM and recreate it with
the correct SDF or bootstrap parameters (it is not possible to run bootstrap more than once).
If you are sure you have the SDF or bootstrap parameters correct, or it is not obvious what is wrong, contact your Customer Care Representative.
Configuration
Configuration occurs after bootstrap. It sets up product-level configuration such as:
-
configuring Rhino and the relevant products (on systems that run Rhino)
-
SNMP-based monitoring
-
SSH key exchange to allow access from other VMs in the cluster to this VM
-
authentication settings for the Cassandra clusters on the TSN VNFCs
To perform this configuration, the process retrieves its configuration in the form
of YAML files from the CDS. The CDS to contact is
determined using the cds-addresses parameter from the SDF or bootstrap parameters.
The configuration process constantly looks for new configuration, and reconfigures the system if new configuration has been uploaded to the CDS.
Currently CDS services are provided by the TSN nodes.
The YAML files describing the configuration should be prepared in advance.
rvtconfig
After spinning up the VMs, configuration YAML files can be validated and uploaded to CDS using the rvtconfig tool.
The rvtconfig tool can be run either on the SIMPL VM or any Rhino VoLTE TAS VM.
|
The TSN nodes should be ready for service before any other nodes are booted. Since TSN nodes provide the CDS services, boot all TSN nodes and wait for them to complete initconf before booting a node of any other type. |
Configuration files
The configuration process reads settings from YAML files. Each YAML file refers to a particular set of configuration options, for example, SNMP settings. The YAML files are validated against a YANG schema. The YANG schema is human-readable and lists all the possible options, together with a description. It is therefore recommended to reference the Configuration YANG schema while preparing the YAML files.
Some YAML files are shared between different node types. If a file with the same file name is required for two different node types, the same file must be used in both cases.
An in-depth description of RVT YAML configuration including a downloadable archive of the high-level YAML files and detailed instructions on how to create them for your deployment can be found in the Rhino VoLTE TAS Configuration and Management Guide.
|
|
When uploading configuration files, you must also include a Solution Definition File containing all nodes in the deployment (see below). Furthermore, for any VM which runs Rhino, you must also include a valid Rhino license. You may also want to upload an HTTPS certificate and private key. |
Solution Definition File
You will already have written a Solution Definition File (SDF) as part of the creation of the VMs. As the configuration process discovers other RVT nodes using the SDF, this SDF needs to be uploaded as part of the configuration.
|
|
The SDF must be named |
Successful configuration
The configuration process on the VMs starts after bootstrap completes. It is constantly
listening for configuration to be written to CDS (via rvtconfig upload-config). Once it detects configuration has been
uploaded, it will automatically download and validate it. Assuming everything passes validation,
the configuration will then be applied automatically. This can take up to 20 minutes depending on
node type.
The configuration process can be monitored using the report-initconf status tool. The tool can be run via an VM SSH session.
Success is indicated by status=vm_converged.
Troubleshooting configuration
Like bootstrap, errors are reported to the log file, located at initconf/initconf.log in the default
user’s home directory.
initconf initialization failed due to an error: This indicates that initconf initialization has irrecoverably failed.
Contact a Customer Care Representative for next steps.
Task <name> marked as permanently failed: This indicates that configuration has irrecoverably failed.
Contact a Customer Care Representative for next steps.
<file> failed to validate against YANG schemas: This indicates something in one of the YAML files
was invalid. Refer to the output to check which field was invalid, and fix the problem. For
configuration validation issues, the VM doesn’t need to be destroyed and recreated.
The fixed configuration can be uploaded using rvtconfig upload-config.
The configuration process will automatically try again once it detects the uploaded
configuration has been updated.
|
|
If there is a configuration validation error on the VM, initconf will NOT run tasks until new configuration has been validated and uploaded to the CDS. |
Other errors: If these relate to invalid field values or a missing license, it is normally safe to fix the configuration and try again. Otherwise, contact a Customer Care Representative.
Configuration alarms
The configuration process can raise the following SNMP alarms,
which are sent to the configured notification targets (all with OID prefix 1.3.6.1.4.1.19808.2):
| OID | Description | Details |
|---|---|---|
12355 |
Initconf warning |
This alarm is raised if a task has failed to converge after 5 minutes. Refer to Troubleshooting configuration to troubleshoot the issue. |
12356 |
Initconf failed |
This alarm is raised if the configuration process irrecoverably failed, or if the VM failed to quiesce (shut down prior to an upgrade) cleanly. Refer to Troubleshooting configuration to troubleshoot the issue. |
12361 |
Initconf unexpected exception |
This alarm is raised if the configuration process encountered an unexpected exception, or if initconf received invalid configuration. Examine the initconf logs to determine the cause of the exception.
If it is due to a validation error, correct any errors in the configuration and try again.
(This won’t normally be the case, as If initconf hit an unexpected error when applying the configuration, initconf attempts to retry the failed task up to five times. Even if it eventually succeeds on a subsequent attempt, the eventual configuration of the node might not match the desired configuration exactly, or a component may be left in a partly-failed state. We therefore recommend that you investigate further. This alarm must be administratively cleared as it indicates an issue that requires manual intervention. |
12363 |
Configuration validation warning |
This alarm is raised if the VM’s configuration contains items that require attention, such as expired or expiring XCAP, BSF, or REM certificates. The configuration will be applied, but some services may not be fully operational. Further information regarding the configuration warning may be found in the initconf log. |
12364 |
OCSS7 reconfiguration attempt blocked |
This alarm is raised if the VM configuration has changed, and the change would result in the OCSS7 SGC being reconfigured. It is not currently possible to reconfigure OCSS7 through changing the YAML configuration alone. Components other than the OCSS7 SGC will be updated to the new configuration, but the OCSS7 SGC component will retain its existing configuration. Review the configuration changes and revert the SS7-related changes if they are not required. To apply the changes to the OCSS7 SGC, follow the procedure documented in Reconfiguring the SGC. |
12365 |
The MDM certificates are about to expire. |
This alarm is raised when the expiration date will be reached less than 30 days. The MDM certificates need to be up to date so that the node can communicate with MDM. The certificates are about to expire. They should be replaced as soon as possible, so that the node can continue to communicate with MDM. Failing to do so will result in config updates and upgrades failing. Replace the MDM certificates using the instructions in the documentation, or reach out to Support for assistance. |
12366 |
The XCAP certificates are about to expire. |
This alarm is raised when the expiration date will be reached less than 30 days. The XCAP certificates are expiring soon. They need to be renewed before the expiry date. These certificates are used to secure the communication between the XCAP server and the clients. If these certificates expire, the communication will not be secure. Replace the certificates with new ones using the documentation provided by Metaswitch. Or contact Metaswitch Customer Care for assistance. |
12367 |
The BSF certificates are about to expire. |
This alarm is raised when the expiration date will be reached less than 30 days. The BSF certificates are expiring soon. They need to be renewed before the expiry date. These certificates are used to secure the communication between the BSF server and the clients. If these certificates expire, the communication will not be secure. Replace the certificates with new ones using the documentation provided by Metaswitch. Or contact Metaswitch Customer Care for assistance. |
|12368 |Detected Read-Only Filesystem. |This alarm is raised when an ext3 or ext4 partition on the filesystem has been detected as read-only.
This can cause multiple services to fail.
Find the detected RO partition in initconf logs or using the 'mount' command and
remount the filesystem as read-write using the following command: mount -o remount,rw <partition>.
If the issue persists, restart the VM or contact Metaswitch Customer Care for assistance.
:is-rvt!: :has-tsn!: :cds-name-lowercase!: :cds-name-uppercase!: :solution-type!: :all-node-types!: :all-node-type-commands!: :username!: :platform-choice!: :platform-choice-with-indefinite-article!: :supports-sas!: :has-certificates!: :products-with-certificates!: :generic-simpl-url-suffix!: :platform-simpl-url-suffix!: |
