Rhino Element Manager (REM) Guide

About this document

This document explains how to install and start the Rhino Element Manager (REM). Documentation about use of the element manager is in the "online help" screens on every within REM itself.

About the Rhino Element Manager

The Rhino Element Manager (REM) is a web-based console for monitoring, configuring, and managing a Rhino Service Logic Execution Environment (SLEE). REM provides a graphical user interface (GUI) for many of the management features documented in the Rhino Administration and Deployment Guide.

You can use REM to:

monitor a Rhino element (cluster nodes, activities, events, SBBs (service building blocks), alarms, resource adaptor entities, services, trace notifications, statistics, logs)
manage a Rhino element (SLEE state, alarms, deployment, profiles, resources), instances available in REM, and REM users
configure threshold rules, rate limiters, licenses, logging, and object pools
inspect activities, SBBs, timers, transactions, and threads
scan key information about multiple Rhino elements on a single screen.

This release of REM can be used with Rhino versions 3.1, 3.0, 2.7, 2.6, 2.5, 2.4 and 2.3.

There is a special embeddable version of REM that you can install into a Rhino SDK. This special version runs as part of the SDK runtime instead of as a standalone web application. For more information, see Embedding REM in the Rhino SDK.

Topics

This document includes the following topics:

Topic	Explains …
Installing, starting and logging in to REM	how to get started with REM
REM functional areas	the functional areas in REM
Using REM tools	how to use tools built in to REM
REM plugins and extensions	how to install REM plugins and extensions
Upgrading REM	how to upgrade your REM installation to a new version
Appendixes	known issues, system properties, SSO authentication, and running REM over HTTPS.

Topic

Explains …

Installing, starting and logging in to REM

how to get started with REM

REM functional areas

the functional areas in REM

Using REM tools

how to use tools built in to REM

REM plugins and extensions

how to install REM plugins and extensions

Upgrading REM

how to upgrade your REM installation to a new version

Appendixes

known issues, system properties, SSO authentication, and running REM over HTTPS.

Audience and Scope

Intended Audience

This document is for system administrators, developers, and operators interested in using a web-based GUI to monitor, configure, or manage instances of a Rhino SLEE.

This document assumes a basic knowledge of core concepts about Rhino, JAIN SLEE and Java Management Extensions (JMX).

Scope

This document covers installing, running, configuring, and using the Rhino Element Manager.

This document does not focus on…

installing and starting the SLEE — see the Rhino (Production) Getting Started Guide
commands and operations, with links to javadocs, for administering and deploying a Rhino SLEE — see the Rhino Administration and Deployment Guide
tools for Rhino maintenance — see Rhino Management Tools
troubleshooting — see the Rhino Troubleshooting Guide

Notices

This manual is issued on a controlled basis to a specific person on the understanding that no part of the Metaswitch Networks product code or documentation (including this manual) will be copied or distributed without prior agreement in writing from Metaswitch Networks.

Metaswitch Networks reserves the right to, without notice, modify or revise all or part of this document and/or change product features or specifications and shall not be responsible for any loss, cost, or damage, including consequential damage, caused by reliance on these materials.

Metaswitch and the Metaswitch logo are trademarks of Metaswitch Networks. Other brands and products referenced herein are the trademarks or registered trademarks of their respective holders.

Installing, Starting and Logging in to REM

Quick start for testing and development

To run the Rhino Element Manager:

1

Ensure the JAVA_HOME environment variable points to Java 11 or 17 (Refer to the Rhino Compatibility Guide).

2

Run the rem.sh script.

3

Open http://localhost:8080/rem/. The Rhino Element Manager login page displays.

login

Derby database and Jetty web server

REM uses an Apache Derby instance to store Rhino instances and REM users. If it does not already exist, the REM installation creates this instance in a directory called rem_data (in the current working directory).
The database runs in embedded mode, so it does not accept incoming TCP connections.
REM runs in a Jetty web server. You can configure the web server (for example, if you need to change the port number), by editing jetty.xml.

Logging in to REM

To login to REM:

1

Enter your username and password, and click Login. The REM menu displays.

The default administrator login (which you’ll want to change) is username emadm and password password.

Depending on your permissions, buttons display for up to four types of REM operations:

Manage a Rhino Element
Monitor all Rhino Elements
Edit Rhino Instances
Edit Element Manager Users.

2

If this is the first time REM has been used you will probably need to either:

add a Rhino instance; or
add a server certificate to the automatically created 'Local' Rhino instance.

Running REM as a production service

When installing REM as a production service, ensure REM availability and security by:

Running REM as a daemon, by either:
- Running REM as a daemon using the built-in Jetty web server.
- Running REM on Apache Tomcat by deploying the REM web application resource (WAR) file.
Using only HTTPS to reach REM, either by Running REM over HTTPS directly, or by putting REM behind an HTTPS terminating application like Tomcat, nginx, or the Apache web server.
Using a centralized LDAP service for authentication to provide centralized account management and policy enforcement.

REM’s communication with Rhino is always protected by TLS, and REM will insist you provide a TLS server certificate to trust before allowing connection to Rhino.

REM Functional Areas

The Rhino Element Manager has the following main functional areas:

REM Management
Rhino Management
Network Monitor

More detailed instructions for each area can be found in the built-in help in REM.

REM Management

Rhino instances

REM can monitor and manage multiple Rhino instances at once.

The instances can be:

RhinoSDKs
Production Rhino clusters
different versions (as long as listed as a supported version in the REM documentation)
local or remote (as long as they are accessible by the REM server)

To manage Rhino instances in REM, select Edit Rhino Instances from the home page.

You must be logged in as a REM user that is assigned a role with the REM Feature: Edit Rhino Instances permission.

REM users and roles

REM has its own users and roles separate from those in Rhino.

REM users can be assigned one or more roles. Roles can be assigned permissions for various high-level REM features and given access to specific Rhino instances.

To manage REM users and roles, select Edit Element Manager Users and Roles from the home page.

You must be logged in as a REM user that is assigned a role with the REM Feature: Edit Users and Roles permission.

Rhino Management

To manage aspects of a specific Rhino instance:

When configuring a Rhino instance the admin password is located in <rhino installation>/client/etc/client.properties

1

Select Manage a Rhino Element from the home page.

2

Connect to a configured Rhino instance using the Connect to … dropdown in the main area or by clicking the New Connection link in the upper right.

You must be logged in as a REM user that is assigned a role with the REM Feature: Manage Element permission.

SLEE Management

REM allows one to view and modify a Rhino instance’s SLEE management state.

To view or modify SLEE management state:

1

Ensure you are in the Rhino Management area and connected to a Rhino instance.

2

Select an item from the Management menu.

This includes:

Item	To
SLEE State	View current SLEE state; start/stop SLEE, services, and RA entities.
Alarms	View current alarms; clear alarms; log all alarms; rebroadcast all alarms.
Deployment	View deployed components; install new DUs; uninstall DUs; change component install levels.
Profiles	View profile tables, profiles, and profile data; create/delete profile tables; create/edit/delete profiles.
Resources	View resource adaptors and RA entities; create/edit/delete RA entities.
Services	View/start/stop services.
Usage	View/create usage parameter sets; view/enable/disable usage notifications.
Bindings	View/add/remove service bindings.
Component Priorities	View/edit component start/stop priorities.

Item

To

SLEE State

View current SLEE state; start/stop SLEE, services, and RA entities.

Alarms

View current alarms; clear alarms; log all alarms; rebroadcast all alarms.

Deployment

View deployed components; install new DUs; uninstall DUs; change component install levels.

Profiles

View profile tables, profiles, and profile data; create/delete profile tables; create/edit/delete profiles.

Resources

View resource adaptors and RA entities; create/edit/delete RA entities.

Services

View/start/stop services.

Usage

View/create usage parameter sets; view/enable/disable usage notifications.

Bindings

View/add/remove service bindings.

Component Priorities

View/edit component start/stop priorities.

Rhino Configuration

REM allows one to view and modify a Rhino instance’s runtime configuration.

To view or modify Rhino configuration:

1

Ensure you are in the Rhino Management area and connected to a Rhino instance.

2

Select an item from the Configuration menu.

This includes:

Item	To
Threshold Rules	View/edit/create/delete/enable/disable threshold rules.
Rate Limiters	View/edit/delete rate limiters.
Licenses	View/install/remove licenses.
Logging	View/edit log levels; view/edit/create/remove appenders.
Pools	View/edit object pools.
Staging Queue	View/edit staging queue settings.
SNMP	View/edit SNMP configuration; enable/disable SNMP agent; view/edit OID and counter mappings.
Auditing	View/edit auditing level; view opcodes.
Domains	View replication domains.
Security Policy	View Rhino security policy.
Alarm Catalog	View information on alarms that could be raised.
Access Control	Manage Rhino roles and permissions; enable/disable permission logging.
Persistence	View/edit/delete persistence instances, persistence resources, and JDBC resources.

Item

To

Threshold Rules

View/edit/create/delete/enable/disable threshold rules.

Rate Limiters

View/edit/delete rate limiters.

Licenses

View/install/remove licenses.

Logging

View/edit log levels; view/edit/create/remove appenders.

Pools

View/edit object pools.

Staging Queue

View/edit staging queue settings.

SNMP

View/edit SNMP configuration; enable/disable SNMP agent; view/edit OID and counter mappings.

Auditing

View/edit auditing level; view opcodes.

Domains

View replication domains.

Security Policy

View Rhino security policy.

Alarm Catalog

View information on alarms that could be raised.

Access Control

Manage Rhino roles and permissions; enable/disable permission logging.

Persistence

View/edit/delete persistence instances, persistence resources, and JDBC resources.

Inspecting state

REM allows one to inspect the state of a Rhino instance.

To inspect Rhino state:

1

Ensure you are in the Rhino Management area and connected to a Rhino instance.

2

Select an item from the Inspections menu.

This includes:

Item	To
Activities	View and remove activities.
SBBs	View and remove SBBs.
Timers	View and remove timers.
Transactions	View and rollback long-running transactions.
Threads	View SLEE event router threads.
AC Bindings	View activity context bindings.

Item

To

Activities

View and remove activities.

SBBs

View and remove SBBs.

Timers

View and remove timers.

Transactions

View and rollback long-running transactions.

Threads

View SLEE event router threads.

AC Bindings

View activity context bindings.

Monitoring

REM allows one to monitor the state of a Rhino instance.

To monitor a Rhino instance:

1

Ensure you are in the Rhino Management area and connected to a Rhino instance.

2

Select an item from the Monitoring menu.

This includes:

Item	To
Dashboard	See a general overview instance state and statistics.
Tracing	Configure trace levels and monitor trace messages.
Statistics	Subscribe to and monitor statistics.
Logging	Monitor log messages.

Item

To

Dashboard

See a general overview instance state and statistics.

Tracing

Configure trace levels and monitor trace messages.

Statistics

Subscribe to and monitor statistics.

Logging

Monitor log messages.

Network Monitor

REM has a network monitor that can be used to monitor the health of multiple Rhino instances on one screen.

Select Monitor all Rhino Elements from the home page.

You must be logged in as a REM user that is assigned a role with the REM Feature: Monitor Elements permission.

Using REM tools

Tools for annotating and displaying the log, and a command line

The Element Manager comes with tools for annotating the REM log, opening a command-line interface, and displaying the audit log.

Log Controls

REM sends log messages to four destinations:

Mini Status Window — the pane at the bottom of REM pages
Audit Window — the Audit Log page
Debug Window — the debug log; display by pressing Ctrl+Alt+` or from the Audit Log page
Remote logging — logs that go to the server, to be written to the server log.

To control log levels for the four destinations:

1	From the Monitoring Dashboard, select Tools > Log Control. Log control options display.
2	Click to select a particular handler level. A popup menu displays. Click to select a logging level. CAUTION: REM maps levels `FINE`, `FINER`, and `FINEST` to the `DEBUG` level on the server — so to see them on the server, the log level for the `rem.client` logger in the server-side logging configuration must also be changed in the `log4j.properties` (it defaults to INFO).
3	To see how each of the four handlers treats a log message at a particular level: * Enter it in the Message field, and click Log Message Only or Log with Exception. Results displays in the message area at the bottom of the page.
4	When you are done controlling logging options, click Done. The log control window closes.

Annotate Rhino Log

To manually add a message to the Rhino log:

1

From the Monitoring Dashboard, select Tools > Annotate Rhino Log.
An Annotate Rhino Log dialog box displays.

2

Enter your message, and click Annotate.
REM adds the message to at level INFO, on logger rem, on all nodes.

Command Line

REM provides a command-line interface, identical to the rhino-console.

To use the REM command-line interface:

1

From the Monitoring Dashboard, select Tools > Command Line.
The command-line interface displays.
commandline

2

Enter Rhino console commands.

Full help is available online.

Audit Log

Use this option to view the audit log in the main area of the screen (instead of just at the bottom):

To clear the log display, click clearlog (at left).

The log area clears.

You can also access the debug window from here by clicking the icon in the upper right corner of the audit log.

REM Plugins and Extensions

The Rhino Element Manager supports plugins and extensions.

These can add server-side functionality such as custom servlets as well as contribute client-side elements to the REM web interface.

The new REM plugin framework has superseded the older extension framework. Development of new extensions is deprecated in favour of plugins, but installing existing REM extensions is still fully supported.

REM plugins and extensions can coexist in a REM installation.

Installing REM Plugins

To install plugins into REM:

1	Copy all the plugin jar files that you want to install into the `plugins` directory in REM.
2	Restart REM or whatever servlet container you are running REM in. Next time you refresh your browser and connect to REM your plugins should be present.

1

Copy all the plugin jar files that you want to install into the plugins directory in REM.

2

Restart REM or whatever servlet container you are running REM in.

Next time you refresh your browser and connect to REM your plugins should be present.

Installing REM Extensions

To install extensions into REM:

1	Open a terminal, and `cd` into an installation of your standalone REM installation.
2	Copy all the extension zip files that you want to include with REM into the `admin/extensions` directory. For example: cp my-extension.zip admin/extensions/ cp my-other-extension.zip admin/extensions/
3	Run the `admin/install-extensions.sh` script. The script bundles the extensions in `admin/extensions` with REM, to produce a new `rem.war` file (`admin/target/rem.war`).
4	If you are using the default bundled Jetty server, you will need to: Copy the new `rem.war` from `admin/target/rem.war` to `webapps/rem.war`. Restart REM. If you are using Tomcat or some other servlet container you will need to: Copy the new `rem.war` from `admin/target/rem.war` to your servlet container’s `webapps` directory. Restart your servlet container. Next time you refresh your browser and connect to REM your extensions should be present.

Upgrading REM

Upgrading REM to a new version

The Rhino Element Manager stores some configuration data locally.

This includes:

rem_data

Derby database storing Rhino instances, saved credentials, users, roles, and permissions

rhino-ems.ks

keystore containing trust certificates for Rhino instances and TLS

.encryption.password

file containing the password used to encrypt and decrypt the Rhino credentials stored in rem_data

You may also have:

Site-specific configuration for things like using LDAP for authentication or running REM over HTTPS.
REM Plugins — installed in the plugins directory
REM Extensions — placed in admin/extensions and installed using the admin/install-extensions.sh script

Plugins and extensions may store their own data in REM’s home directory. Refer to the documentation for each plugin and extension you have installed for information on what additional files need to be considered when upgrading.

The upgrade process differs slightly depending on how you run REM:

Upgrading REM in the standard distribution
Upgrading REM in Apache Tomcat

Upgrading REM in the standard distribution

To upgrade REM when running it from the standard distribution package (using the embedded Jetty web server):

1

Locate the data files mentioned above in your existing REM installation.

By default they will all be stored in the root of your existing REM installation (rhino-element-manager-<old_version>).

2

Copy the files mentioned above into the root of your new REM installation (rhino-element-manager-<new_version>).

cp -R rhino-element-manager-<old_version>/rem_data \
    rhino-element-manager-<old_version>/rhino-ems.ks \
    rhino-element-manager-<old_version>/.encryption.password \
    rhino-element-manager-<old_version>/ldapauth.properties \
    rhino-element-manager-<old_version>/rem.jaas \
    rhino-element-manager-<new_version>/

Your new REM installation should now contain all the same data as your previous one.

3

Depending on your installation, you may need to:

enable JAAS in stand-alone REM to restore LDAP-based authentication
migrate REM Extensions
migrate REM Plugins
update the database schema

Migrating REM Extensions

If you have REM Extensions installed:

1	Verify that they are compatible with the new version of REM by referring to the documentation for each extension.
2	Copy them (or newer versions of them) to `admin/extensions` in your new REM installation. cp rhino-element-manager-<old_version>/admin/extensions/*.zip \ rhino-element-manager-<new_version>/admin/extensions/
3	Run the `admin/install-extensions.sh` script in your new REM installation. The extensions will be installed into the new version of `rem.war`
4	Following the instructions output by the script, copy `admin/target/rem.war` to `webapps/rem.war`.
5	If any of the extensions store their own data in REM’s home directory, copy those files to the new REM installation. cp -r rhino-element-manager-<old_version>/<extension_data> \ rhino-element-manager-<new_version>/

Migrating REM Plugins

If you have REM Plugins installed:

1

Verify that they are compatible with the new version of REM by referring to the documentation for each plugin.

2

Copy them (or newer versions of them) to plugins in your new REM installation.

cp rhino-element-manager-<old_version>/plugins/*.jar \
    rhino-element-manager-<new_version>/plugins/

3

If any of the plugins store their own data in REM’s home directory, copy those files to the new REM installation.

cp -r rhino-element-manager-<old_version>/<plugin_data> \
    rhino-element-manager-<new_version>/

Updating the database schema

The schema for REM’s Derby database (rem_data) may require updates when upgrading between major versions of REM.

REM includes a schema update tool that can apply any required schema updates automatically. All existing data will be maintained, with some data being migrated when necessary.

To enable automatic updating of REM’s database schema:

1

Uncomment this line in REM’s startup script:

#JVM_ARGS="${JVM_ARGS} -Drem.schema.update.enabled=true"

2

Start REM (stopping it first if it was already running).

Any database schema updates found to be required will be applied.

Refer to rem.log for messages regarding the update process.

Upgrading REM in Apache Tomcat

To upgrade REM when running it in Apache Tomcat:

1	Locate the data files mentioned above in your `${rem.home}` directory. If you followed the original instructions for running REM in Apache Tomcat, this will be `rem_home` in the root of your Tomcat installation. cd apache-tomcat-<version>
2	Make a backup copy of the entire `rem_home` directory. cp -R rem_home rem_home_OLD
3	Make a backup of your existing `rem.war`. cp webapps/rem.war rem.war_OLD
4	Copy the new version of `rem.war` into your Apache Tomcat `webapps` directory. cp /full/path/to/rhino-element-manager-<new_version>/admin/resources/rem.war webapps/rem.war
4	Copy the new version of `rem-rmi.jar` to `bin/rem-rmi.jar` in your Tomcat installation. cp /full/path/to/rhino-element-manager-<new_version>/rem-rmi.jar bin/rem-rmi.jar
5	Review the instructions for running REM on Apache Tomcat to check if there are any additional changes or steps required.
6	Depending on your installation, you may need to: migrate REM Extensions migrate REM Plugins update the database schema

Migrating REM Extensions

If you have REM Extensions installed:

1	Verify that they are compatible with the new version of REM by referring to the documentation for each extension.
2	Copy them (or newer versions of them) to `admin/extensions` in the new REM distribution package.
3	Run the `admin/install-extensions.sh` script in the new REM distribution package. The extensions will be installed into the new version of `rem.war`
4	Following the instructions output by the script, copy `admin/target/rem.war` to `apache-tomcat-<version>/webapps/rem.war`.

Migrating REM Plugins

If you have REM Plugins installed:

1

Verify that they are compatible with the new version of REM by referring to the documentation for each plugin.

2

Replace any that require upgrading with newer versions of them (making backups of the old ones if desired).

Updating the database schema

The schema for REM’s Derby database (rem_data) may require updates when upgrading between major versions of REM.

REM includes a schema update tool that can apply any required schema updates automatically. All existing data will be maintained, with some data being migrated when necessary.

To enable automatic updating of REM’s database schema:

1

Add rem.schema.update.enabled=true to CATALINA_OPTS in bin/setenv.sh in your Apache Tomcat installation.

# Enable automatic REM database schema updates
CATALINA_OPTS="$CATALINA_OPTS -Drem.schema.update.enabled=true"

2

Start Apache Tomcat (stopping it first if it was already running).

Any database schema updates found to be required will be applied.

Refer to your Tomcat logs for messages regarding the update process.

Appendixes

REM includes the following appendixes:

Appendix A. Known issues in REM
Appendix B. Configuration parameters
Appendix C. SSO Authenticator
Appendix D. Running REM over HTTPS
Appendix E. Running REM on Apache Tomcat

Appendix A. Known issues in REM

This page lists several known issues in REM.

Issue: The refresh timer sometimes causes pages to refresh while you’re editing.

Workaround: Disable refresh (through the Refresh menu).

Issue: Serialization errors when shutting down REM.

Explanation: Depending on which screens have been used, some objects stored in the servlet session cannot be persisted to disk when REM shuts down. These sessions will not be restored when REM restarts, but the error is otherwise harmless.

Issue: Inspection filter form values don’t reset when cancel is pressed.

Workaround: Change the filter values back manually, to avoid having them applied next time the list is refreshed.

Issue: Embedded Rhino console fails to load with a JavaScript error terminal.proposeGeometry is not a function.

Explanation: The JavaScript Terminal implementation was upgraded, but your browser may have the old version cached.

Workaround: Clear your browser’s cache and reload the page.

Appendix B. Configuration parameters

REM system properties

REM supports the following system properties that affect its runtime behaviour:

Property Valid values (default in bold) Description

Property	Valid values (default in bold)	Description
`rem.nosessionlogin`	true, false	If set to true, disables login from a cookie. Normally authentication tokens in a cookie will be checked when the app is reloaded and can continue to be used. Disabling this feature increases security but forces users to re-login when the app is restarted (with F5 for example.)
`rem.norestart`	true, false	If set to true, disables the feature that allows the administrator to restart the REM server (if started by `rem.sh`) from the upload certificate dialog box.
`rem.home`	${JETTY_HOME}	Path to the directory where REM should store its working files (Derby database, keystore, etc.)
`rem.plugins.dir`	${rem.home}/plugins	Path to the directory where REM should search for plugins.
`rem.plugins.work.dir`	${rem.plugins.dir}	Path to the directory where REM should expand plugins to.
`rem.schema.update.enabled`	true, false	If set to true, enables automatic database schema updates to `rem_data` (if required) during REM startup.

rem.nosessionlogin

true, false

If set to true, disables login from a cookie. Normally authentication tokens in a cookie will be checked when the app is reloaded and can continue to be used. Disabling this feature increases security but forces users to re-login when the app is restarted (with F5 for example.)

rem.norestart

true, false

If set to true, disables the feature that allows the administrator to restart the REM server (if started by rem.sh) from the upload certificate dialog box.

rem.home

${JETTY_HOME}

Path to the directory where REM should store its working files (Derby database, keystore, etc.)

rem.plugins.dir

${rem.home}/plugins

Path to the directory where REM should search for plugins.

rem.plugins.work.dir

${rem.plugins.dir}

Path to the directory where REM should expand plugins to.

rem.schema.update.enabled

true, false

If set to true, enables automatic database schema updates to rem_data (if required) during REM startup.

REM servlet parameters

In addition to the system properties above, REM uses the following servlet parameters to control the loading of additional configuration:

Parameter name Valid values (default in bold) Description

Parameter name	Valid values (default in bold)	Description
`rem.config-file`	${rem.home}/rem.properties	Path to the REM configuration file. Stores sensitive configuration properties such as the truststore configuration.
`rem.config-file.password`	Any sequence of printable ASCII characters i.e. characters from the range from ' ' to '~'	Password used to encrypt sensitive properties in the REM configuration file such as the truststore password.

rem.config-file

${rem.home}/rem.properties

Path to the REM configuration file. Stores sensitive configuration properties such as the truststore configuration.

rem.config-file.password

Any sequence of printable ASCII characters i.e. characters from the range from ' ' to '~'

Password used to encrypt sensitive properties in the REM configuration file such as the truststore password.

Appendix C. SSO Authenticator

REM Single Sign-on Authenticator

What is the SSO Authenticator?

Single sign-on (SSO) infrastructure stores a username and password, and uses a simple HTTP request to login.

REM’s SSO authenticator allows authentication using a simple HTTP POST instead of the login dialog box in the Element Manager user interface.

The REM SSO authenticator uses the following fields:

URL — http:// host : port /rem/rem/sso_authenticator
Username form field — username
Password form field — password
Instance ID form field (optional) — instance

Opening the URL in a browser displays a simple form that can be submitted for sign on (though typically the SSO infrastructure takes care of the submission). The authenticator authenticates the submitted username and password against the Element Manager user database. If successful, it returns an HTTP redirect (containing a URL) to the main UI page. A browser can then be launched with that URL. The URL is valid for 60 seconds and can be used only once.

If an instance ID is supplied, the Element Manager automatically connects to the instance, provided:

the instance has a stored Rhino username and password
the user has admin rights to that instance.

Appendix D. Running REM over HTTPS

HTTPS for added security

By default REM runs over HTTP in the bundled Jetty web server. If additional security is required, there are a couple ways in which REM can be run over HTTPS.

Changing Jetty to Use HTTPS

An example configuration for HTTPS is provided in recent versions of REM. To enable this, open the start.ini file and uncomment the ssl and https modules:

-# --module=ssl
+--module=ssl

-# --module=https
+--module=https

You will also need to provide or create a proper web keystore (by default located at etc/keystore inside the main rhino-element-manager directory).

For more detailed instructions on creating a keystore and configuring HTTPS in Jetty, see Configuring Security in the Jetty documentation.

Running REM behind Apache

If you have an existing Apache installation that is already configured to use HTTPS, then you may wish to serve REM through that.

The setup can be a bit complicated on the Apache side and a couple different options are available.

For detailed instructions see How to use Jetty with Apache, AJP, mod_jk, mod_proxy_balancer

Appendix E. Running REM on Apache Tomcat

REM on Apache Tomcat

By default REM runs on the Jetty web server bundled in the REM distribution package. The REM application is packaged as a standard war though, so running it on a different servlet container, like Apache Tomcat, is quite straightforward.

You will need:

the REM distribution package — rhino-element-manager-<version>.zip; expanded to a location of your choice.
an Apache Tomcat installation — either downloaded and configured manually, or installed via a package manager. Use the latest release from the Tomcat 9.x series.

Setting up Tomcat to run REM

1	REM requires a few System Properties to be set in order to function properly. In your Apache Tomcat installation, edit (or create) `bin/setenv.sh` and add the following lines: REM_HOME="$CATALINA_BASE/rem_home" CATALINA_OPTS="-Drem.home=$REM_HOME -Dderby.stream.error.file=$REM_HOME/derby.log -Drem.norestart=true" CLASSPATH="$CATALINA_BASE/bin/rem-rmi.jar" If you wish to have REM store its working files in a different location, you can change the `REM_HOME` value above.
2	The `rem-rmi.jar` is required to be on the System class path for REM Plugins to function correctly. You can find the jar in the root of the REM distribution package (`rhino-element-manager-<version>/rem-rmi.jar`). Copy it to `$CATALINA_BASE/bin/rem-rmi.jar` (to match the path specified in the `CLASSPATH` entry above).
3	Copy `rem.war` from `rhino-element-manager-<version>/admin/resources/rem.war` to `$CATALINA_BASE/webapps/rem.war`.
4	Edit `$CATALINA_BASE/conf/server.xml` and add a `Context` element to the `Host` element serving the REM application. In this `Context` element, set initialisation parameters `rem.config-file` and `rem.config-file.password` similar to the example below: <Context path="/rem"> <Parameter name="rem.config-file" value="${rem.home}/rem.properties"/> <Parameter name="rem.config-file.password" value="You should choose a unique password & it should be strong"/> </Context> You should now be able to start Apache Tomcat and access REM at `<tomcat_base_url>/rem`. If running Apache Tomcat locally using the default port `8080` then that would be http://localhost:8080/rem.

Customize REM logging

1	Unzip `log4j2.properties` from `rem.war`: cd apache-tomcat-<version> mkdir rem-tmp cd rem-tmp unzip ../webapps/rem.war WEB-INF/classes/log4j2.properties
2	Edit `WEB-INF/classes/log4j2.properties`, customizing the configuration as desired. The default logging configuration is: # log4j 2 configuration status=error name=PropertiesConfig appender.rolling.type=RollingFile appender.rolling.name=FILE appender.rolling.fileName=${sys:catalina.base}/logs/rem.log appender.rolling.filePattern = ${sys:catalina.base}/logs/rem.log.%i appender.rolling.policies.type = Policies appender.rolling.policies.size.type = SizeBasedTriggeringPolicy appender.rolling.policies.size.size=10MB appender.rolling.strategy.type = DefaultRolloverStrategy appender.rolling.strategy.max = 10 appender.rolling.layout.type=PatternLayout appender.rolling.layout.pattern=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p <%t> [%c] %m{nolookups}%n logger.rolling.name=rem logger.rolling.level=INFO rootLogger.level=INFO rootLogger.appenderRef.file.ref=FILE
3	Replace `WEB-INF/classes/log4j2.properties` in `rem.war`: zip ../webapps/rem.war WEB-INF/classes/log4j2.properties
4	Remove temporary files: cd .. rm -rf rem-tmp

See the Log4j 2 documentation for more information on logging configuration.

Appendix F. Running REM as a daemon

In most production deployments it is desirable to run REM as a daemon.

Remd

The remd.sh script starts the Rhino Element Manager as a background service and provides basic control actions. This script does not provide control of REM instances started with the rem.sh script, even if they share the same $REM_HOME.

Usage: ./remd.sh {start|stop|restart|check}

Command Action

Command	Action
start	Start REM as the `$REM_USER`
stop	Stop the REM server process
restart	Restart the REM server
check	Check the REM process is running

start

Start REM as the $REM_USER

stop

Stop the REM server process

restart

Restart the REM server

check

Check the REM process is running

Init script

A sample init script is provided in init.d/rhino-element-manager, offering similar functionality to remd.sh for operating systems using a SystemV type init system. This script is written for LSB compliant Linux systems and must be modified for other systems.

Command Action

Command	Action
start	Start REM as the `$REM_USER`
stop	Stop the REM server process
restart	Restart the REM server
status	Check the REM process is running

start

Start REM as the $REM_USER

stop

Stop the REM server process

restart

Restart the REM server

status

Check the REM process is running

Configuration of the init script may be provided in the system environment or by editing the values at the top of the file.

Variable

Value

REM_USER

Username of the user to run the REM server as.

REM_HOME

Path to the rhino-element-manager directory. This directory contains the REM executables and log files.

JAVA_HOME

Path to the base directory of the JDK used to run REM. Typically this will be a path like /opt/jdk11.0.4

Systemd unit file

A sample systemd service unit file is provided in init.d/rem.service for RedHat Enterprise Linux and similar operating systems using a systemd init system.

To use the systemd service on system start, first edit the unit file to set the following variables:

Variable

Value

REM_USER User

Username of the user to run the REM server as. Both the REM_USER environment variable and the User service option should be set to the same value.

REM_HOME

Path to the rhino-element-manager directory. This directory contains the REM executables and log files.

JAVA_HOME

Path to the base directory of the JDK used to run REM. Typically this will be a path like /opt/jdk11.0.4

Once configured correctly, the unit file can be copied to /etc/systemd/system. The REM service can be manually started using systemctl start rem and stopped using systemctl stop rem, or enabled for automatic start on system reboot using systemctl enable rem.

Appendix G. Using LDAP for authentication

We strongly recommend that you use LDAP as the authentication backend for REM to enhance security.

A centralized LDAP service for user accounts, which provides security features such as password policy enforcement and automatic account locking, should be used.

Before getting to configuration, we need to discuss how REM users and roles interact with LDAP, and the two methods that can be used to authenticate users using LDAP.

REM users and roles when LDAP is used

REM uses LDAP strictly as an authentication service. Authorisation is handled solely by REM’s understanding of user roles, as follows:

LDAP users without corresponding REM user accounts are entirely unauthorized and cannot log in to REM. User accounts must be created in REM using the 'Users and Roles' management interface for every LDAP user who needs access to REM.
REM user accounts, not LDAP user accounts, provide the set of REM roles used to enable or disable particular REM views and actions. Any required REM role changes must be made via the REM 'Users and Roles' management interface.

LDAP authentication methods

REM can use either "direct mode" or "search mode" to authenticate users against LDAP. The direct mode is simpler, while the search mode is more flexible.

Direct mode

The direct mode is the simplest, safest, but most restricted LDAP authentication method. REM is configured with a DN template string, and when a user tries to log in, the user name provided by the user is inserted into that template to generate a DN. REM then attempts an LDAP bind against the generated DN with the password provided by the user. If the bind succeeds, the user is successfully authenticated. If the bind fails, REM denies access to the user.

Search mode

The search mode provides more flexibility at the cost of additional configuration. In this mode, REM is configured with LDAP authentication credentials of its own plus some search parameters for locating users. When a user tries to log in, REM binds to LDAP with its configured authentication credentials and searches for the correct DN by combining search parameters with the user name provided by the user. If a DN is successfully located, REM attempts an LDAP bind using that DN and the password provided by the user. If both the DN search and a bind using that DN succeed, then the user is successfully authenticated. If either the DN search or the bind fails, REM denies access to the user.

Configuring REM to use LDAP authentication

Follow these steps to configure REM to use LDAP authentication. These instructions assume a fresh, empty REM installation.

1. Create an LDAP-backed REM administration account

Before doing anything else, create an LDAP-authenticated REM administration account. Pick one of these two approaches:

Create an LDAP user entry matching the default REM administrator account name emadm. To do so:
1. Create an LDAP user entry for emadm. Step-by-step instructions for doing this are beyond the scope of this manual; for those instructions, see your LDAP server vendor’s documentation.
2. Start REM.
3. Log in using the default REM administrator credentials: user emadm with the password of password.
4. Select Edit Element Manager Users and Roles.
5. Change the password for the emadm user to a strong password. This is a precaution in case LDAP authentication is accidentally disabled at some point in the future and REM is left using its own internal password database.
6. Either delete the user user or change its password to a strong password.
7. Stop REM.
Replace REM’s default administrator account with one already present in your LDAP service. To do so:
1. Start REM.
2. Log in using the default REM administrator credentials: user emadm with the password of password.
3. Select Edit Element Manager Users and Roles.
4. Click Add User and set:
  1. Username to match the desired LDAP entry, after template expansion (for direct mode), or search (for search mode).
  2. Real name as you wish.
  3. New Password and New Password (again) to a strong password. This is a precaution in case LDAP authentication is accidentally disabled at some point in the future and REM is left using its own internal password database.
  4. Roles to include at least em-adm so that this account can manage other REM users. Add any other roles as desired.
5. Save the new user.
6. Select and delete the default REM accounts: emadm and user. This is a precaution to prevent unintended access if your LDAP service already contains those user names.
7. Stop REM.

2. Add your LDAP CA root certificate to rhino-ems.truststore

The only TLS certificates which will be trusted by REM are those that appear in rhino-ems.truststore, so the root certificate for your LDAP server’s TLS certificate must be added to it. Acquiring the root certificate is beyond the scope of this manual, but possibilities include:

Acquire the certificate from the certificate authority who provided your LDAP server’s TLS certificate, either as a download or through their support team.
Extract the certificate from your JDK’s lib/security/cacerts keystore.

Use the LDAP server’s certificate directly, or one of the intermediate certificates in its chain.

This approach is prone to causing REM to reject the LDAP server whenever the LDAP server gets a new TLS certificate.

Once you’ve acquired the certificate in the PEM format, locate your rhino-ems.truststore file and change the current directory to where it is:

For stand-alone REM, the rhino-ems.truststore will be in the top-level Rhino element manager installation directory.
For Tomcat-based REM, the rhino-ems.truststore will be in the rem_home directory within Tomcat’s top-level installation directory.

Then use this command, with $PATH_TO_YOUR_CA_CERT replaced with the correct path to your CA certificate, to add the certificate to the rhino-ems.truststore file:

keytool -importcert -noprompt -alias ldap-server-ca-cert -file $PATH_TO_YOUR_CA_CERT -keystore rhino-ems.truststore

You will be prompted for a keystore password, which can be found in the rhino.remote.ssl.trustStorePassword property of the rem.properties file.

The password in the properties file may include backslash escape sequences that you must manually unescape before entering the passphrase into keytool. If you see a lengthy error message that ends with Failed PKCS12 integrity checking, then you have entered the wrong password, possibly because you unescaped the password incorrectly, or forgot to unescape it.

3. Enable JAAS in REM

The Java Authentication and Authorization Service is a Java-standard mechanism for configuring flexible authentication and authorisation services, and is how REM is instructed to use LDAP.

The procedure for enabling JAAS in REM differs for stand-alone REM installations and Tomcat-based installations. Select the relevant procedure from the two below.

Enable JAAS in stand-alone REM

Edit the script you are using to start REM, typically one of:
- remd.sh, as documented in Appendix F. Running REM as a daemon.
- init.d/rhino-element-manager, as documented in Appendix F. Running REM as a daemon.
- rem.sh, as documented in Installing, Starting and Logging in to REM.

Find this line, which is present in all the files listed above:

#JVM_ARGS="${JVM_ARGS} -Djava.security.auth.login.config=${REM_HOME}/rem.jaas"

and uncomment it by removing the # at the start of the line so that it becomes:

JVM_ARGS="${JVM_ARGS} -Djava.security.auth.login.config=${REM_HOME}/rem.jaas"

Save the file, overwriting the original copy.

Enable JAAS in Tomcat-based REM

These instructions assume you have created a bin/setenv.sh script while following the instructions in Setting up Tomcat to run REM.

You may need to adapt these instructions if your Tomcat instance requires JAAS for other applications and/or itself.

Edit Tomcat’s bin/setenv.sh script, adding the following line at the end of the file:

CATALINA_OPTS="$CATALINA_OPTS -Djava.security.auth.login.config=${REM_HOME}/rem.jaas"

Save the file, overwriting the original copy.

4. Use LDAP, and only LDAP, via JAAS policy

JAAS policy controls which authentication and authorization types are used.

Edit or create the file rem.jaas, whose location varies by installation type:
- For stand-alone REM the rem.jaas will be in the top-level Rhino element manager installation directory.
- For Tomcat-based REM the rem.jaas will be in the rem_home directory within Tomcat’s top-level installation directory.

Set the content of rem.jaas to:

rhino-element-manager {
    com.opencloud.rem.server.security.auth.LdapLoginModule REQUIRED
    properties = "${rem.home}/ldapauth.properties";
};

Then save the file, overwriting the original.

5. Define REM’s LDAP querying behaviour

REM needs to know how to reach the LDAP service, and how to use it. This is controlled by the properties in ldapauth.properties.

Edit or create the ldapauth.properties file, whose location varies by installation type:
- For stand-alone REM the ldapauth.properties will be in the top-level Rhino element manager installation directory.
- For Tomcat-based REM the ldapauth.properties will be in the rem_home directory within Tomcat’s top-level installation directory.
Set the following properties within the ldapauth.properties file:
ldap.url
to the URL for your LDAP service, in the either the form:
ldaps://host[:port]/basedn, or

ldap://[host[:port]]/basedn if STARTTLS should be used for TLS.
Here are some examples:
# Connect to local directory server ldap.url=ldap:///dc=example,dc=com
# Connect to remote directory server (ldap.usetls should be set to true) ldap.url=ldap://remoteserver/dc=example,dc=com
# Connect to remote directory server using SSL ldap.url=ldaps://remoteserver/dc=example,dc=com
ldap.usetls
to true, unless an ldaps URL was used for ldap.url, in which case this property should be left unset:
ldap.usetls=true

Decide whether to use Direct mode or Search mode, and continue editing ldapauth.properties as directed by the relevant subsection below. REM cannot use both modes at the same time. If you configure both modes, REM will use direct mode.

Configuring direct mode

In ldapauth.properties set:
ldap.userdnpattern
to a DN pattern that can be used to directly login users to LDAP. This pattern is used for creating a DN string where the pattern is relative to the base DN in the ldap.url. The string {0} will be replaced with the user name submitted in the login prompt.

Here’s an example:
# This is relative to the ldap.url base DN given ldap.userdnpattern=uid={0},ou=People
Save ldapauth.properties, overwriting the original, and proceed to step 6. Start REM and provision additional users.

Configuring search mode

In ldapauth.properties set:
ldap.managerdn

to the LDAP account DN which should be used to search for the LDAP DN of the user attempting to log in to REM. This may be blank if the LDAP server allows anonymous connections to search adequately.

ldap.managerpw

to the password for the ldap.managerdn configured above. This may be blank if the LDAP server allows anonymous connections to search adequately.

ldap.searchfilter
to the LDAP filter expression that should be used to locate the DN of the user attempting to log in to REM. The string {0} will be replaced by the submitted username. The scope of this search may be adjusted using ldap.searchbase as documented below.

Here’s an example:
ldap.searchfilter=(uid={0})
ldap.searchbase
optionally, to the context name to search in, relative to the base DN given in the ldap.url. This property may be omitted if not required.

Here’s an example:
ldap.searchbase=ou=People
Save ldapauth.properties, overwriting the original.
Ensure that only the REM (or Tomcat, as appropriate) user can read ldapauth.properties if you set ldap.managerpw, because it now contains a password in clear text that is available to anyone who can read the file.
Proceed to step 6. Start REM and provision additional users.

6. Start REM and provision additional users

Finally:

Start REM.
Log in as the LDAP-authenticated REM administration user you chose in 1. Create an LDAP-backed REM administration account.
Create any additional REM user and/or REM administration accounts that you require using REM’s Users and Roles page. Remember:
- No LDAP user will have access to REM unless there is a corresponding REM user account defined within REM itself.
- User permissions in REM are controlled by the REM roles assigned to the REM user via the REM management interface. REM does not use LDAP to determine user roles.

Troubleshooting LDAP authentication problems

You may need to troubleshoot if LDAP authentication isn’t working after following the steps above. Here are the two most useful things to try, in order.

Enable debug logging in REM

To enable debug logging in REM:

Edit log4j2.properties and

Comment out the line:

logger.rolling.level=INFO

so that it becomes:

#logger.rolling.level=INFO

Change the line:

rootLogger.level=INFO

to

rootLogger.level=DEBUG

Save log4j2.properties, overwriting the original.
Restart REM.

Enable JDK TLS debugging

If the REM debug logging, or anything else, indicates a TLS problem, add the second line shown below just below the first line shown below in your REM startup script:

JVM_ARGS="${JVM_ARGS} -Djava.security.auth.login.config=${REM_HOME}/rem.jaas"
JVM_ARGS="${JVM_ARGS} -Djavax.net.debug=all"

(For Tomcat installations, edit setenv.sh and use CATALINA_OPTS instead of JVM_ARGS.)

This enables the JDK’s built in TLS debugging, producing a lot of output that should help you diagnose the cause of the TLS problem.