Online management
Once a cluster has been established following the procedures in initial setup, online management of the scattercast endpoints becomes possible. There are four basic management commands, to get, add, delete, or update scattercast endpoints.
Each command applies the result to all currently executing nodes.
If there is a node not currently executing that requires the new endpoints set,
this must be done manually.
To provide an up-to-date endpoints set to an offline node, copy {$NODE_HOME}/config/savanna/scattercast.endpoints
from any up-to-date node to the matching path for the new node. All currently running nodes should have an up-to-date
copy of this file. This can be verified by using the getscattercastendpoints command.
|
scattercast.endpoints cannot be manually edited.
Nodes will not boot with a manually edited scattercast.endpoints.
|
Multicast, localhost, and wildcard addresses are not permitted in scattercast endpoints. As an endpoint address is used to both send and recieve, the localhost address confines the cluster to a single host. Wildcard addresses are only usable to listen on all interfaces. These addresses cannot be sent from, and thus are invalid in scattercast endpoints.
Inconsistent states
When manually copying scattercast endpoint sets to all cluster members, the cluster will reject all write-management commands until it is rebooted. This occurs because persistent and in-memory state are not identical across all nodes.
This may also occur in other ways, and can be resolved with the following steps
-
If the disk state is correct on all nodes, reboot the cluster.
-
If the disk state is not correct or not the same on all nodes:
-
If the disk state incorrect, use the
recover-scattercast-endpoints.shtool in repairing to create a new, correct file; and copy to all nodes before reboot. -
If disk state is correct on some but not all nodes, copy the file from correct nodes to all other nodes.
-
Repair
In most cases where the scattercast configuration is inconsistent, the faulty nodes can be restored by copying the
scattercast.endpoints file from an operational node. If the node has been deleted from the current configuration, it
should first be re-added using the addscattercastendpoints rhino-console command. The
configuration file can be found at $RHINO_HOME/$NODE/config/savanna/scattercast.endpoints.
If no nodes are operational, such as after a major change to network addressing, the tool recover-scattercast-endpoints.sh
can be used to rebuild the configuration from scratch.
After running recover-scattercast-endpoints.sh, you must copy the generated file to
$RHINO_HOME/$NODE/config/savanna/scattercast.endpoints for each node.
Recovering scattercast endpoints
The recover-scattercast-endpoints.sh script is used to rebuild the scattercast config file after a major network
change or configuration data loss. It can be run interactively, prompting for node,IP,port tuples, or using the
new configuration on the command line. Options must be provided before the list of endpoints. If you want to use
automatic port assignment, you must provide the baseport and offset options to allow calculation of a valid port set.
Usage:
$ cd rhino
$ ./recover-scattercast-endpoints.sh -?
Usage: ./recover-scattercast-endpoints.sh [options] [node,ip-address[,port]]*
Creates a seed scattercast endpoints file. The generated file needs to be copied to {$NODE_HOME}/config/savanna/scattercast.endpoints in all cluster nodes.
If no endpoints are provided, enters interactive mode.
arguments:
-f, --file Relative path to output file.
-b, --baseport The scattercast base port, used to derive a port when no port is specified in endpoints.
-o, --offset The scattercast port offset, used to derive a port when no port is specified in endpoints.
-?, --help Displays this message.Example:
$RHINO_HOME/recover-scattercast-endpoints.sh -b 19000 -o 100 101,192.168.1.1,19000 102,192.168.1.2 103,192.168.1.2,19003|
|
If baseport and offset are provided, they are used only for the recovery tool. Nodes added or updated with
management commands will continue to use values in cluster.properties.
|
