Besides the information on the console, orca provides detailed output of the actions taken in the log file. The log file by default is located on the host that executed the command under the path logs.

orca can’t connect to the remote hosts

Check if the trusted connection via ssh is working. The command ssh <the host to connect to> should work without asking for a password.

You can add a trusted connection by executing the steps below

  • Create SSH key by using default locations and empty passphrase. Just hit enter until you’re done

ssh-keygen -t rsa

  • Copy the SSH key onto all VMs that need to be accessed including the node you are on. You will have to enter the password for the user

ssh-copy-id -i $HOME/.ssh/id_rsa.pub sentinel@<VM_ADDRESS>

where VM_ADDRESS is the host name you want the key to be copied to.

To check run:

ssh VM_ADDRESS

It should return a shell of the remote host (VM_ADDRESS). If it prompts for a password ensure that group permission is correct.

chmod -R go= ~/.ssh

orca failed to create the management database

orca uses the psql client to connect to and do operations on the PostgreSQL database. Check the first host in the --hosts list has psql installed.

Get the database hostname from the file /home/sentinel/rhino/node-xxx/config/config_variables Search for the properties MANAGEMENT_DATABASE_NAME, MANAGEMENT_DATABASE_HOST , MANAGEMENT_DATABASE_USER, MANAGEMENT_DATABASE_PASSWORD

example

MANAGEMENT_DATABASE_NAME=rhino_50
MANAGEMENT_DATABASE_HOST=postgresql
MANAGEMENT_DATABASE_PORT=5432
MANAGEMENT_DATABASE_USER=rhino_user
MANAGEMENT_DATABASE_PASSWORD=rhino_password

Test the connection:

psql -h postgresql -U rhino_user -p 5432 rhino_50

Enter the password and you expect to see

Password for user rhino:
psql (9.5.14)
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)
Type "help" for help.

rhino_50=>

If the connection fails try with the database name from the previous cluster. If it still fails the procedure below is a guide on how to add permission to the database to accept remote connections.

  • Log in to the database host.

  • Open up the pg_hba.conf file, using sudo.

sudo vim /var/lib/pgsql/data/pg_hba.conf

  • Replace the line that looks like this…​ host rhino-xxx sentinel xxx.xxx.xxx.xxx/xx password with a line that looks like this…​

host  all  sentinel  xxx.xxx.xxx.xxx/xx  password

Where 'rhino-xxx' is the name of cluster xxx’s postgres database, and xxx.xxx.xxx.xxx/xx covers the signalling addresses of the nodes.

  • Reload the config. /usr/bin/pg_ctl reload

  • Or failing that, try this command. sudo systemctl restart postgresql

Installer runs but does not install

The recommendation is to rollback the node to the previous cluster, cleanup the prepared cluster and start again.

./orca --hosts host1 rollback
./orca --hosts host1 cleanup --cluster <cluster id>
./orca --hosts host1 minor-upgrade <path to the product sdk zip> <path to the install.properties> --pause

Rhino nodes shutdown or restart during migration

orca requires the node being migrated to allow management commands via rhino-console. If the node is not active orca will fail.

If while executing orca --hosts <list of hosts> minor-upgrade packages install.properties --continue the node being currently migrated shuts down or restart, there are 3 options:

1- Skip the node and proceed to the other nodes and migrate the failed node later

Except for the first host name, remove the hosts that were already migrated from the orca command, including the host that failed and execute the command orca --hosts <list of hosts> minor-upgrade packages install.properties --continue again to continue with the upgrade.

2- Restart the node manually and continue with the upgrade

If the node shutdown and hadn’t restart, execute the rhino.sh start script. If the node restarted, check if rhino-console command works:

<path to the old cluster >/client/bin/rhino-console

Except for the first host name, remove the hosts that were already migrated from the orca command and execute the command orca --hosts <list of hosts> minor-upgrade packages install.properties --continue again to continue with the upgrade.

3- Do the migration manually

This is an extreme case, but the procedure is simple.

For each node that was not migrated:

Identify the new cluster

cd $HOME
ls -ls rhino

lrwxrwxrwx 1 sentinel sentinel 24 Nov 21 21:48 rhino -> shcm-1.0.0.0-cluster-50

will show you the current cluster name in the format <product>-<version>-cluster-<id>.

The new cluster will have a similar name <product>-<new version>-cluster-<id + 1>, but the cluster id will be one number higher. For example: shcm-1.0.0.1-cluster-51

Kill the current node

rhino/rhino.sh kill

Link the rhino path to the new cluster

rm rhino
ln -s <new cluster> rhino

where <new cluster> is the path identified above. Example: shcm-1.0.0.1-cluster-51

Start the node and check for connection

./rhino/rhino.sh start

Wait some minutes and check if you can connect to the node using rhino-console.

Previous page