Sentinel IP-SM-GW or OCSS7 not running after installation

Note

Sentinel IP-SM-GW can be disabled in smo-vmpool-config.yaml. If Sentinel IP-SM-GW has been disabled, Rhino will not be running.

Check that bootstrap and configuration were successful:

[sentinel@smo1 ~]$ grep 'Bootstrap complete' ~/bootstrap/bootstrap.log
2019-10-28 13:53:54,226 INFO bootstrap.main Bootstrap complete
[sentinel@smo1 ~]$

If the bootstrap.log does not contain that string, examine the log for any exceptions or errors.

[sentinel@smo1 ~]$ report-initconf status
status=vm_converged
[sentinel@smo1 ~]$

If the status is different, examine the output from report-initconf for any problems. If that is not sufficient, examine the ~/initconf/initconf.log file for any exceptions or errors. If bootstrap and configuration were successful, check the Rhino and OCSS7 journalctl logs.

[sentinel@smo1 ~]$ journalctl -u rhino -l
[sentinel@smo1 ~]$ journalctl -u ocss7 -l

Further information can be found from the SMO logs in /var/log/tas. In particular, the Rhino logs are found in a subdirectory of /var/log/tas with the same name as the Rhino directory has in the home directory, e.g. gaa-4.0.0.0-cluster-110. Similarly, the OCSS7 logs are found in /var/log/tas/ocss7.

Rhino Alarms

Rhino alarms indicate issues that should be reserved promptly. Rhino alarms can be monitored using MetaView Server or REM on the MAG node. Some common Rhino alarms are described below.

Not Connected to Cassandra

Node: 101
Level: Critical
Type: CassandraCQLRA.ConnectToCluster
Message: Not connected to Cassandra. Attempting to connect each 10s

  • Check that the Cassandra server is active on the TSN nodes.

  • Check the network connectivity to the TSN nodes.

  • As TSN nodes are discovered automatically, no further configuration should be necessary. Ensure this node has been provided (as part of its configuration bundle) with the correct SDF for the TSN nodes, as the IP addresses to connect to are derived from this SDF.

Lost connection to SGC

Node: 101
Level: Major
Type: noconnection
Message: Lost connection to SGC localhost:11002

  • Check that SGC on this node is active.

Connection to Diameter Ro peer is down

Node: 101
Level: Warning
Type: diameter.peer.connectiondown
Message: Connection to [host]:[port] is down

  • Check the Diameter Ro peers are configured correctly.

  • Check the network connectivity to the Diameter Ro peer host and port.

Connection to SAS server is down

Node: 101
Level: Major
Type: rhino.sas.connection.lost
Message: Connection to SAS server at [host]:[port] is down

  • Check that SAS is active.

  • Check the network connectivity to the SAS server host and port.

Not connected to any instances of the configured Sh Cache Microservice host.

Node: 101
Level: Critical
Type: ShCMRA.ShCMConnectFailed
Message: Not connected to any instances of the configured Sh Cache Microservice host.

  • Check that ShCM is active on the ShCM nodes.

  • Check the network connectivity to the ShCM nodes.

  • Check that the DNS SRV records for ShCM are set up correctly. The ShCM domain that is configured in common-config.yaml should have DNS SRV records set up for every ShCM node.

OCSS7 SGC

The OCSS7 SGC is not running

  • Use systemctl status ocss7 to determine if the ocss7 service is enabled and running.

  • Check using jps to see if an SGC process is running.

  • Check the most recent startup.log and ss7.log in /var/log/tas/ocss7/ for information relating to any failed startup.

OCSS7 SGC Alarms

The OCSS7 SGC CLI may be used to query the SGC for its active alarms. The SGC CLI executable is located at ~/ocss7/<deployment_id>/<node_id>/current/cli/bin/sgc-cli.sh.

Use the display-active-alarm command in the SGC CLI to show the active alarms.

See the OCSS7 Installation and Administration Guide for a full description of the alarms that can be raised by the OCSS7 SGC.

Previous page Next page
Rhino VoLTE TAS VMs Version 4.0.0