The DB Query Resource Adaptor lets SLEE services send SQL queries to an RDBMS and receive the results as SLEE events or return values.
The resource adaptor:
-
allows both synchronous and asynchronous operations
-
continuously checks connections are valid and fails fast if no connections are available
-
has been tested with Oracle, MySQL, and PostreSQL databases
-
includes scripts and examples for these databases
-
should work with any other JDBC datasource.
JDBC drivers are deployed as JAIN SLEE Libraries. The resource adaptor references the libraries in its deployment descriptor.
Topics
How to set up a build environment, deploy the resource adaptor and dependencies |
|
Configuring the resource adaptor using profiles and properties |
|
Enable and disable a datasource, enable and disable tracing, monitor statistics |
|
Install and run the example services for the DB Query Resource Adaptor |
Other documentation for the DB Query Resource Adaptor can be found on the DB Query Resource Adaptor product page.
Usage and execution model
The DB Query Resource Adaptor receives SQL queries from the invoking service as QueryInfo objects containing the query and associated metadata, including the type of query and if and how it may be retried. These are added to the worker pool queue for the primary datasource and control is returned to the service. Synchronous queries return a future for the service to wait for a result, asynchronous queries deliver the result in a SLEE event. The sample statistics in the "Query" parameter set measure times from query submission to return of a result. They provide an overview of query performance in the RA. Statistics in the "DataSource" parameter sets contain finer timing data for execution of queries against individual data sources.
When a query is added to the worker queue the submitting thread starts the query timeout timer. Measurement of the query performance in the datasource starts at this point.
-
The worker threads for the data source take queries from the queue and execute them against the database. The time in the queue is measured in both the
WorkerPool.queueTime
andDataSource.queueTime
sample statistics. If the query has timed out in the queue, this is recorded in theDataSource.queueTimeout
counter statistic. -
The thread attempts to obtain a connection from the current transaction, or from the connection pool, creating one if the pool is empty and the connection limit is not yet reached. This is measured using the
DataSource.connectionWait
sample statistic. Within this time, theDataSource.poolWait
andDataSource.newConnectionTime
statistics record the time spent waiting for a connection to be available in the pool or creating a new one. If the query has timed out while getting a connection, this is recorded in theDataSource.poolWaitTimeout
counter statistic. -
The query is submitted to the database for execution. The time spent creating the prepared statement and executing it is recorded in one of the
DataSource.databaseTimeX
sample statistics, whichever is appropriate for the task type (query, update, commit, rollback). If the query has timed out while the statement was executing, this is recorded in theDataSource.resultTooLate
counter statistic and the time taken to execute the query is recorded in theDataSource.databaseTimeLate
statistic. -
When the result is received the query timeout timer is cancelled.
-
The result is checked and, if an error occurred, the connection is released.
-
If one of these steps fails and the query may be retried, the execution task calls back to the RA to reschedule the query on the next available datasource.
-
The query timeout timer is restarted for the next datasource.
-
This repeats until the query completes or all datasources have been tried.
-
When the query has completed, or no more datasources are available to try, the result returned is sent to the service.
If the query timeout timer fired at any point in this process, or, for synchronous queries the service stopped waiting on the result future, the result set is released and, the enclosing transaction is rolled back if applicable.
Connection pooling
The DB Query Resource Adaptor implements a connection pool to reduce the connection load on the database and minimise the time spent waiting for a connection. The pool may be configured with minimum and maximum sizes and is pre-filled to the minimum size. A separate connection limit may be configured to limit the maximum number of connections in use. The pool shrinks when connections have been idle for a configured period. Connections are replaced automatically when they exceed a configured age.
When a query is processed a connection is taken from the pool if present, otherwise a new connection is created unless the maximum number of connections has already been made. The connection remains in use until both the service has released the database result set and the transaction enclosing the query has completed. Once released the connection is added to the pool unless the pool already contains the configured maximum number of connections.
Notices
Copyright © 2024 Microsoft. All rights reserved
This manual is issued on a controlled basis to a specific person on the understanding that no part of the Metaswitch Networks product code or documentation (including this manual) will be copied or distributed without prior agreement in writing from Metaswitch Networks.
Metaswitch Networks reserves the right to, without notice, modify or revise all or part of this document and/or change product features or specifications and shall not be responsible for any loss, cost, or damage, including consequential damage, caused by reliance on these materials.
Metaswitch and the Metaswitch logo are trademarks of Metaswitch Networks. Other brands and products referenced herein are the trademarks or registered trademarks of their respective holders.
Deploying the resource adaptor
Deploying the resource adaptor involves setting up a build environment, DU, and SLEE Library JARs.
The download package includes: du/ - deployable units lib/ - compilation dependencies docs/ - documentation examples/ - an example service etc/ - library JAR descriptors and library-ref XML fragments createradu.macro.xml - a reusable Ant macrodef for assembling an RA DU build.xml - build files build.properties - build properties |
Below are instructions for setting up a local build environment, creating a deployable unit, and building JDBC driver SLEE Library JARs.
Setting up a local build environment configuration
The file build.properties
must be edited to provide values for the properties specific to the local environment. The required properties are marked at the top of the file.
Creating the resource adaptor deployable unit
A deployable unit for the resource adaptor implementation is not included pre-assembled in the distribution. This is because the resource-adaptor-jar.xml
descriptor within the component JAR must be updated to reference the appropriate library JARs via <library-ref>
elements before the DU can be deployed.
The included build.xml
contains targets (oracle
, mysql
, and postgresql
) to build the database-specific deployable units. They use the <createradu>
macro to update the resource-adaptor-jar.xml
descriptor within the dbquery-ra-x.y.z.jar
, and then create dbquery-ra-x.y.z.du.jar
.
The create-dbquery-ra
macro in build.xml
creates a DU using the supplied library-ref fragments.
Building JDBC driver SLEE Library JARs
Supported databases
The resource adaptor should work with any JDBC DataSource that uses standard configuration methods. A SLEE Library JAR must be created for the driver and the resource adaptor’s deployment descriptor updated to reference that library. The package contains Ant build scripts to perform this task. The resource adaptor has been tested with recent versions of these databases:
|
Database JDBC drivers are not distributed with this package. The libraries will be built when the targets mentioned above are run (oracle
, mysql
, and postgresql
). The JDBC driver locations must be specified by the properties at the top of build.properties
. If the JDBC driver is distributed as a native library, then the path to the driver must be specified in the LD_LIBRARY_PATH
environment variable.
Configuring the resource adaptor
DB Query resource adaptor configuration
You configure the DB Query resource adaptor using SLEE profiles. A single configuration property is used to point to a master configuration profile for the resource adaptor entity. This configuration profile in turn references a number of other profiles that configure the datasources. For each datasource configuration profile specified, the resource adaptor will create a DataSource
object using the given class name, then set the other profile attributes as datasource properties.
To configure DB Query resource adaptor entities, use the following types of resource adaptor configuration:
-
Elements of the Datasource properties profile attribute array
-
Per-node configuration when nodes in a cluster require different configurations.
And see the configuration example, which sets up two different datasource configurations.
You should set a network timeout to avoid possible long or indefinite waiting for a response from the database in the event of a network or database failure. See Network timeout configuration. |
Resource adaptor configuration properties
The DB Query resource adaptor has the following resource adaptor configuration properties:
Name | Type | Description |
---|---|---|
|
|
The name of the profile to configure this resource adaptor entity |
|
|
Flag to indicate that per-node configuration profiles should be used (see details below) |
|
|
Enables improved RA behaviour under overload conditions by caching results and closing all resources immediately |
Resource adaptor configuration profile
The DbQueryConfigProfile
profile has the following attributes:
Name | Type | Description |
---|---|---|
|
ProfileID[] |
List of DataSource profiles. These will always be tried in the order given. |
|
String |
An SQL statement to be executed against the datasources to test for availability |
|
long (ms) |
Time between polling for changes to datasource profiles |
|
long (ms) |
Maximum time a synchronous transaction can exist before assuming the service has neglected to end it and it is marked as 'lost' |
|
long (ms) |
Maximum time a transaction will stay in the 'lost' transaction list before being forcibly ended |
|
long (ms) |
Time between scanning sync and lost transactions |
Datasource configuration profiles
The DbQueryDataSourceProfile
profile has the following attributes:
Name | Type (units) | Description |
---|---|---|
|
String |
A description for this datasource, will be used in log messages and statistics parameter set names |
|
boolean |
Flag to enable/disable this datasource administratively |
|
String |
The Java class name for the datasource, will be instantiated via reflection |
|
String |
Value for the |
|
String |
Value for the |
|
String |
Value for the |
|
String |
Value for the ServerName property if used by this datasource (not needed if |
|
int |
Value for the PortNumber property if used by this datasource (not needed if |
|
String |
Value to be passed to the |
|
String |
Value to be passed to the |
|
String |
Value for the DatabaseName property if used by this datasource (not needed if |
|
String[] |
Arbitrary values to be passed to |
|
String |
Properties to be passed to the |
|
long (ms) |
Timeout for a query, including getting a connection, preparing and executing the statement. On |
|
long (ms) |
How frequently to poll for availability when a datasource is marked unavailable (default: 2000ms) |
|
int |
The number of threads in the worker thread pool used to invoke JDBC operations (default: 30) |
|
int |
The maximum number of work items in a queue before they are rejected (default: 600) |
|
long (ms) |
Timeout for retrieving a connection from the pool (default: 5000ms) |
|
long (ms) |
The maximum time a physical connection may be used before it will be closed instead of being returned to the pool (default: 60000ms) |
|
int |
The maximum sum of connections currently in use and available in pool (default: 30) |
|
int |
The maximum number of available connections kept in pool (default: 30) |
|
int |
The minimum number of available connections that a pool maintains |
|
long (ms) |
Maximum idle time for a connection before it is closed and removed from the pool (default: 30000ms) |
|
long (ms) |
How frequently to check for idle connections (default: 500ms) |
|
long (ms) |
How frequently to scan the connection pool for idle connections (default: 35000ms) |
|
long (ms) |
How long overload alarms will stay active before being cleared (default: 30000ms) |
The Url /URL attribute exists with both title case and upper case because some datasources use one form and some use the other. |
Datasource properties profile attribute
The elements of the array for the DataSourceProperties
profile attribute have the following format:
Name
/Type
/Value
where
-
Name
is the DataSource property name -
Type
is the property type (a Java primitive or a Java class which has a constructor that takes a singlejava.lang.String
argument) -
Value
is the String representation of the value.
For example, to set the loginTimeout
property on an Oracle datasource to 10, the array element would be:
loginTimeout/java.lang.Integer/10
Type can be java.util.Properties
, in which case the Value must be a java.lang.String
that can be parsed by the java.util.Properties.load(InputStream)
method ("name: value" separated by line breaks). Line breaks in Ant build files can be specified by using the XML entity 

.
Oracle connection properties
The Oracle datasource has another means of setting connection properties: the setConnectionProperties(java.util.Properties)
method. These can be specified by the ConnectionProperties
attribute.
The value of this property must be a java.lang.String
that can be parsed by the java.util.Properties.load(InputStream)
method ("name: value" separated by line breaks). Line breaks in Ant build files can be specified by using the XML entity 

.
Per-node configuration
Per-node configuration (when nodes in a cluster require different configurations) is supported by having separate DbQueryConfigProfile
profiles for each node. The resource adaptor must be configured with the configuration property ConfigProfileAddNodeSuffix
set to true and a profile created for each node, named as follows:
-
ConfigProfile
__NodeID
For example, if the resource adaptor configuration specified ConfigProfile=MySQLConfig
and ConfigProfileAddNodeSuffix=true
on a two-node cluster with nodes 101 and 102, then two profiles would be needed:
-
MySQLConfig__101
-
MySQLConfig__102
If a new node is added to the cluster and the resource adaptor is using per-node configuration, an appropriately named profile must be created before activating the resource adaptor entity on the new node.
Configuration example
The following Ant commands would create one MySQL DataSource and one Oracle DataSource.
To use this configuration, the resource adaptor entity would be created with the configuration properties ConfigProfile=dbquery-config/MySQL-Ora-Config,ConfigProfileAddNodeSuffix=false
.
<?xml version="1.0"?>
<project name="dbquery-ra-oracle-mysql-examples" default="deploy-oracle-mysql-example" basedir="./..">
<import file="${basedir}/examples/common.build.xml"/>
<import file="${basedir}/examples/oracle.build.xml"/>
<import file="${basedir}/examples/mysql.build.xml"/>
<target name="deploy-oracle-mysql" depends="oracle, deploy-oracle-library, deploy-mysql, deploy-mysql-library, deploy-dbquery-ra,
create-oracle-mysql-dbquery-profiles, create-dbquery-ra-entity"
description="Deploy Oracle+MySQL RA"/>
<target name="create-oracle-mysql-dbquery-profiles" depends="init, login" description="Create profiles for Oracle/MySQL configuration">
<slee-management>
<createprofile tablename="dbquery-ds" profilename="MysqlDataSource">
<profilevalue name="Description" value="${mysql.host} (MySQL)"/>
<profilevalue name="DataSourceClass" value="com.mysql.cj.jdbc.MysqlDataSource"/>
<profilevalue name="Url" value="jdbc:mysql://${mysql.host}:3306/${mysql.database}"/>
<profilevalue name="User" value="${mysql.user}"/>
<profilevalue name="Password" value="${mysql.password}"/>
<profilevalue name="AdminEnable" value="true"/>
<profilevalue name="WorkerThreads" value="5"/>
<profilevalue name="QueryTimeout" value="10000"/> <!-- milliseconds -->
</createprofile>
<createprofile tablename="dbquery-ds" profilename="OraDataSource">
<profilevalue name="Description" value="${oracle.host} (Oracle)"/>
<profilevalue name="DataSourceClass" value="oracle.jdbc.pool.OracleConnectionPoolDataSource"/>
<profilevalue name="ConnectionProperties" value="oracle.jdbc.ReadTimeout: 30000
"/>
<profilevalue name="URL" value="jdbc:oracle:thin:@//${oracle.host}:1521/${oracle.database}"/>
<profilevalue name="User" value="appuser"/>
<profilevalue name="Password" value="${oracle.password}"/>
<profilevalue name="AdminEnable" value="true"/>
<profilevalue name="WorkerThreads" value="5"/>
<profilevalue name="QueryTimeout" value="10000"/> <!-- milliseconds -->
</createprofile>
<createprofile tablename="dbquery-config" profilename="MySQL-Ora-Config">
<profilevalue name="DataSourceProfileIDs" value="[dbquery-ds/MysqlDataSource,dbquery-ds/OraDataSource]"/>
<profilevalue name="TestSql" value="SELECT short_code FROM mappings WHERE short_code IS NULL"/>
<profilevalue name="ProfilePollTime" value="5000"/> <!-- milliseconds -->
<profilevalue name="MaxSyncTransactionAge" value="120000"/> <!-- milliseconds -->
<profilevalue name="MaxLostTransactionAge" value="120000"/> <!-- milliseconds -->
</createprofile>
</slee-management>
<property name="dbqueryra.properties" value="ConfigProfile=dbquery-config/TT-Ora-Config,ConfigProfileAddNodeSuffix=false"/>
</target>
<target name="undeploy-oracle-mysql" depends="undeploy-dbquery-ra, undeploy-oracle-library, undeploy-mysql-library"
description="Undeploy Oracle+MySQL RA"/>
<target name="deploy-oracle-mysql-example" depends="deploy-oracle-mysql, deploy-example-service"
description="Deploy Oracle+MySQL RA with example"/>
<target name="undeploy-oracle-mysql-example" depends="undeploy-example-service, undeploy-oracle-mysql"
description="Undeploy Oracle+MySQL RA with example"/>
</project>
MySQL Configuration
Later MySQL databases and drivers (8.x +) will use TLS connections by default. However there can be difficulties getting the connections to work due to interactions with other aspects of Rhino.
If you get the following or similar Exceptions, in particular Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: unknown_ca
this page provides some solutions.
Exception in thread "main" com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:836)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:456)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:246)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:197)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:416)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:128)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:113)
at com.mysql.cj.jdbc.MysqlConnectionPoolDataSource.getPooledConnection(MysqlConnectionPoolDataSource.java:48)
<snip>
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
at com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:538)
at com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:702)
at com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:679)
at com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:128)
at com.mysql.cj.protocol.a.NativeAuthenticationProvider.proceedHandshakeWithPluggableAuthentication(NativeAuthenticationProvider.java:466)
at com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:171)
at com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1342)
at com.mysql.cj.NativeSession.connect(NativeSession.java:157)
at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:956)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:826)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:456)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:246)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:197)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:416)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:128)
at com.mysql.cj.jdbc.MysqlDataSource.getConnection(MysqlDataSource.java:113)
at com.mysql.cj.jdbc.MysqlConnectionPoolDataSource.getPooledConnection(MysqlConnectionPoolDataSource.java:48)
<snip>
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: unknown_ca
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:313)
at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:186)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:171)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1406)
at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1371)
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:958)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:133)
at com.mysql.cj.protocol.FullReadInputStream.readFully(FullReadInputStream.java:64)
at com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:63)
at com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:45)
at com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:532)
... 24 more
Cause
This happens when MySQL JDBC driver and MySQL database connect in their default connection mode, AND the client’s JVM has a PrivateKeyEntry using RSA Public Key Algorithm (and probably no other PrivateKeyEntry - but we haven’t verified that) in the keystore used by the SSL protocol stack.
Although two way authentication is not configured, the MySQL server asks the client for a certificate anyway. If the client returns an RSA Certificate then the connection is closed by the server, resulting in the above Exception chain. If the client sends no certificate, or a DSA Certificate the connection succeeds.
Unfortunately, Rhino’s rhino-server.keystore
, which is configured as the SSL keystore, has a single PrivateKeyEntry using RSA Public Key Algorithm.
Solution 1 - Disable TLS connections
To disable TLS, set the following connection properties in the datasource profile’s DataSourceProperties
<profilevalue
name="DataSourceProperties"
value="[sslMode/java.lang.String/DISABLED,allowPublicKeyRetrieval/java.lang.Boolean/true]"
/>
Solution 2 - Supply MySQL with an empty keystore
An empty keystore can be used specifically for the MySQL connection which overrides the default one Rhino installs for the SSL protocol.
To create an empty keystore, you create a keystore with a single private key then delete that key.
keytool -genkeypair -storetype pkcs12 -alias dummy -storepass secret -keypass secret \
-keystore empty.keystore \
-dname "CN=IDontCare, OU=someunit, O=someorg, L=somecity, ST=less, C=WORLD"
keytool -delete -alias dummy -storepass secret -keystore empty.keystore
using you own password to replace secret
.
Now put this in rhino home directory adjacent to the rhino-server.keystore
file.
Finally, set the following connection properties in the datasource profile’s DataSourceProperties
to tell MySQL to use this keystore.
<profilevalue name="DataSourceProperties"
value="[clientCertificateKeyStoreUrl/java.lang.String/${empty.keystore.url},clientCertificateKeyStorePassword/java.lang.String/${empty.keystore.pass},clientCertificateKeyStoreType/java.lang.String/JKS]"
/>
setting or replacing ${empty.keystore.pass}
with the keystore password, and ${empty.keystore.url}
with the absolute path to the empty.keystore
file as a URL.
Solution 3 - Enable 2 way authentication - untested
Follow MySQL documentation to enable two way authentication, and get a client certificate compatible with the MySQL server.
Import that certificate into an empty.keystore
file, and install that as explained for solution 2. You must either use the empty.keystore
file name, or modify the dbquery-ra permissions to allow dbquery-ra and the packaged driver to read another keystore. Other DataSourceProperties
will need to be added to configure the client for two-way authentication. This may also work if the certificate is imported into the default rhino-server.keystore
without setting an explicit keystore in the DataSourceProperties
.
Other untested potential solutions
-
Modify the MySQL server to accept the RSA certificate in the
rhino-server.keystore
. -
Modify the MySQL server to not ask for a client certificate.
-
Generate rhino keystores with DSA signing algorithm (note there are security implications).
-
A very similar problem was observed with 8.0.18 driver, with DSA keys also, but that was fixed in 8.0.21, so maybe this problem will also disappear some time.
Network timeout configuration
Set a network timeout to avoid possible long or indefinite waiting for a response from the database in the event of a network or database failure.
The query timeout limits execution time of a query; the network timeout controls the underlying communication channel between the JDBC driver and the database.
Full support for timeouts requires the network timeout to be configured on the JDBC driver. This feature is JDBC driver dependent.
For example, when a query timeout is detected and the JDBC driver sends a notification to the database, a communication timeout may occur. This will not be detected (or the detection can take arbitrary time) unless the network timeout is explicitly configured.
Not configuring the network timeout causes either:
|
Below are details for configuring the network timeout for some common database drivers.
Database, JDBC driver | Where to configure | Property name | Unit | Example |
---|---|---|---|---|
Oracle, Thin driver, v10.1.0.5+ |
|
|
ms |
|
Oracle, Thin driver, older |
|
|
ms |
|
Oracle, OCI driver |
Not supported. |
|||
MySQL, v3.0.1+ |
|
|
ms |
|
PostgreSQL, v8.4+ |
|
|
s |
|
Always check the vendor’s documentation for the right property name, place of configuration, and any other restrictions or guidelines. |
As a general guideline: the value for network timeout should be greater than maximum time of a query or transaction. The resource adaptor internally sets the query timeout for the JDBC driver (more precisely, for each issued query statement) equal to 1.5 times DbQueryDataSourceProfile
QueryTimeout
. The network timeout should not be less than 1.5 times the QueryTimeout
, as otherwise the JDBC driver may end a query prematurely with a failure.
Managing the resource adaptor
Enabling, disabling, reconfiguring, and monitoring
The DB Query resource adaptor supports management procedures to enable, disable, reconfigure, and monitor datasources.
Administratively enabling or disable a datasource
The profile attribute AdminEnable
can be used to disable a configured datasource. If this attribute is set to false
using a management interface (such as the Rhino command-line console), then the data source will be treated as though it is not available. The profile is polled for a change to this attribute with a period specified by attribute ProfilePollTime
in the main DbQueryConfigProfile
profile.
Reconfiguring datasources
Most aspects of datasource configuration can be modified while a datasource has been administratively disabled. When the datasource is re-enabled, the actual DataSource
implementation class will be re-instantiated and reconfigured according to the new profile values.
Enabling or disabling tracing
The DB Query Resource Adaptor has several tracers that behave in the usual way, and whose trace levels can be modified using the Rhino management tools.
There is one special case. The trace level of the tracer dsm.driver
determines whether or not setLogWriter()
will be called on the DataSource
class. If JDBC driver tracing is required, this tracer’s (or a parent’s) trace level should be set to FINEST
. If JDBC driver tracing is no longer required, this tracer’s trace level should be unset.
Tracing can affect performance
|
Monitoring resource adaptor statistics
The DB Query Resource Adaptor creates several statistic parameter sets that can be monitored using the Rhino Stats tool.
Statistics parameter sets
The root parameter set is named DatabaseQuery
.entity-name
and has three child parameter sets:
The Query
parameter set has general statistics that are not specific to a given datasource. The DataSource
and WorkerPool
parameter sets each have child parameter sets named for each datasource configured in the profiles, and contain statistics specific to the datasource. The child parameter set uses the value of the Description
profile attribute as its name.
A list of statistics with descriptions can be displayed with the rhino-stats -l
command:
Query
parameter set
This parameter set contains statistics relating to the overall execution of queries. Information such as the number of queries of a given type and the execution time including queuing and retries when failing over between datasources.
Parameter Set: /DatabaseQuery.dbquery-0.Query Parent Parameter Set: /DatabaseQuery.dbquery-0 Parameter Set Type: /ResourceAdaptorID[name=Database Query,vendor=OpenCloud,version=2.1].DatabaseQueryStats Description: Database Query RA statistics Counter type statistics: Id: Name: Short name: Label: Description: 0 activeSyncTxn activeSyncTxn n/a Active sync transactions 1 asyncQueries asyncQ n/a Total number of async queries 2 asyncTxnQueries asyncTxnQ n/a Total number of transactional async queries 3 asyncTxnUpdates asyncTxnU n/a Total number of transactional async updates 4 asyncUpdates asyncU n/a Total number of async updates 5 failed fail n/a Total number of query failures 6 syncQueries syncQ n/a Total number of sync queries 7 syncTxnQueries syncTxnQ n/a Total number of transactional sync queries 8 syncTxnUpdates syncTxnU n/a Total number of transactional sync updates 9 syncUpdates syncU n/a Total number of sync updates 10 txFailed txFail n/a Total number of failures during transaction end Sample type statistics: Id: Name: Short name: Label: Description: 11 commitTime commitT n/a Milliseconds taken to execute commit (including queue time) 12 failureTime failureT n/a Milliseconds taken by queries and other requests that failed (including retries and queue time) 13 queryTime queryT n/a Milliseconds taken to execute query (including retries and queue time) 14 rollbackTime rollbackT n/a Milliseconds taken to execute rollback (including queue time) 15 updateTime updateT n/a Milliseconds taken to execute update (including retries and queue time)
DataSource
parameter set
The individual data sources parameter sets under this parameter set contain statistics relating to a single data source. These report the execution status and time spent in each part of the execution sequence of queries, and the status of the connection pool.
Parameter Set: /DatabaseQuery.dbquery-0.DataSource Parent Parameter Set: /DatabaseQuery.dbquery-0 Parameter Set Type: /ResourceAdaptorID[name=Database Query,vendor=OpenCloud,version=2.1].DataSourceStats Description: Database Query RA DataSource statistics Counter type statistics: Id: Name: Short name: Label: Description: 0 closedIdle closeI n/a Total number of connections closed due to being idle 1 closedOld closeO n/a Total number of connections closed due to being too old 2 closedPoolMax closeP n/a Total number of connections closed due to pool max being exceeded 3 connectionsInProgress connecting n/a Total number of connection attempts that are in progress 4 databaseTimeout dbTimeout n/a Total number of query timeouts while waiting for database execution 5 getConnectionTimeout connTimeout n/a Total number of attempts to get a connection from the pool that timed out due to the pool timeout 6 inPool inPool n/a Number of available connections in the pool 7 inUse inUse n/a Number of connections in use 8 newConnectionCreated created n/a Total number of connections created 9 newConnectionFailure connectFail n/a Total number of connection attempts that failed (including timeouts) 10 newConnectionTimeout connectTimeout n/a Total number of connection attempts that failed due to a timeout 11 poolWaitTimeout pTimeout n/a Total number of query timeouts while waiting for a connection from the pool 12 pooledConnectionUsed used n/a Total number of pooled connections used 13 queueTimeout qTimeout n/a Total number of query timeouts while waiting for worker thread 14 resultTooLate late n/a Total number of ResultSets received too late (after query timeout) Sample type statistics: Id: Name: Short name: Label: Description: 15 connectionWait connWait n/a Milliseconds spent waiting for a connection from the pool (including establishing a new connection if needed) 16 databaseTimeCommit dbTimeC n/a Milliseconds spent calling commit 17 databaseTimeLate dbTimeL n/a Milliseconds spent creating and executing PreparedStatement that was abandoned after the query timeout 18 databaseTimeQuery dbTimeQ n/a Milliseconds spent creating and executing PreparedStatement for query 19 databaseTimeRollback dbTimeR n/a Milliseconds spent calling rollback 20 databaseTimeUpdate dbTimeU n/a Milliseconds spent creating and executing PreparedStatement for update 21 newConnectionTime newConnT n/a Milliseconds taken to establish a new connection to the database server 22 poolWait poolWait n/a Milliseconds spent waiting until a pooled connection is available (not including establishing a new connection). 23 queueTime queueTime n/a Milliseconds spent waiting in executor queue
The databaseTimeLate
statistic differs from the other databaseTimeX
statistics in that it records the time taken by all database tasks which completed after the execution timeout. It is provided separately from the task specific time samples to assist in determining if there are database tasks that take an unusually long time to execute.
WorkerPool
parameter set
Thw worker pool parameter sets contain statistics relating to the pool of worker threads that perform database operations. Each data source has two worker pools - the main query execution thread pool and the connection pool management thread pool.
Parameter Set: /DatabaseQuery.dbquery-0.WorkerPool Parent Parameter Set: /DatabaseQuery.dbquery-0 Parameter Set Type: /ResourceAdaptorID[name=Database Query,vendor=OpenCloud,version=2.1].WorkerPoolStats Description: Database Query RA worker pool statistics Counter type statistics: Id: Name: Short name: Label: Description: 0 queueSize qSize n/a Number of tasks waiting in worker pool's queue 1 rejects rejects n/a Total number of tasks rejected because queue was full 2 tasksExecuted tasksTotal n/a Total number of tasks executed 3 threadsAvailable availThrd n/a Number of available worker threads 4 threadsTotal totalThrd n/a Total number of worker threads Sample type statistics: Id: Name: Short name: Label: Description: 5 processingTime procT n/a Milliseconds spent processing a task (internal processing time and database execution time) 6 queueTime queueT n/a Milliseconds spent in queue waiting for a thread
Running the example service
To configure and deploy the example, please refer to MySQL setup instructions. Once all the steps are completed, the following scenario can be followed to test it.
If building and deploying the example on Rhino version 2.6 and onwards, the reload4j jar required in the
|
Running the example
To run the example, Telnet to the machine running Rhino, using port 9999
.
Then try these commands:
Enter: | To… |
---|---|
|
send a query using the synchronous API (if a transaction exists, it will be used) |
|
send a query using the asynchronous API (if a transactional activity exists, it will be used) |
|
start or use an existing transaction and send an update using the synchronous API |
|
start or use an existing transactional activity and send an update using the asynchronous API |
|
start or use an existing transaction and execute a stored procedure synchronously to extend the mapping with the prefix |
|
start or use an existing transactional activity and execute a stored procedure asynchronously to extend the mapping with the prefix |
|
commit the transaction |
|
rollback the transaction |
|
close the Telnet connection |
|
see brief help |
$ telnet localhost 9999 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. help COMMANDS: q-s [short] - query short code synchronously (using current transaction if one exists) q-a [short] - query short code asynchronously (using current transaction if one exists) up-s [short] [long] - update short -> long mapping synchronously (starting a transaction if needed) up-a [short] [long] - update short -> long mapping asynchronously (starting a transaction if needed) p-s [short] [prefix] - execute stored procedure synchronously to extend a mapping with the prefix (starting a transaction if needed) p-a [short] [prefix] - execute stored procedure asynchronously to extend a mapping with the prefix (starting a transaction if needed) co - commit current transaction ro - rollback current transaction close - close the telnet connection q-s 0000 Short code '0000' maps to long code '11111111111' q-a 0000 Submitted SELECT long_code FROM mappings WHERE short_code = '0000' on non-transactional activity Short code '0000' maps to long code '11111111111' up-s 0000 66666666666 Transaction for synchronous requests created: DatabaseTransaction[101.100003,age=0ms,conn=none] Updated mapping of short code '0000' to long code '66666666666' (There is an open transaction, it must be committed or rolled back) q-s 0000 Short code '0000' maps to long code '66666666666' (There is an open transaction, it must be committed or rolled back) ro Rolled back DatabaseTransaction[101.100003,age=9940ms,conn=DbConnectionHandle[id=4,state=CLOSED,transactional=true,connection=com.mysql.jdbc.ProxyConnection@608148fe]] q-s 0000 Short code '0000' maps to long code '11111111111' up-s 0000 555555555555 Transaction for synchronous requests created: DatabaseTransaction[101.100005,age=0ms,conn=none] Updated mapping of short code '0000' to long code '555555555555' (There is an open transaction, it must be committed or rolled back) co Committed DatabaseTransaction[101.100005,age=4698ms,conn=DbConnectionHandle[id=6,state=CLOSED,transactional=true,connection=com.mysql.jdbc.ProxyConnection@6cbd7ea6]] q-s 0000 Short code '0000' maps to long code '555555555555' q-a 0000 Submitted SELECT long_code FROM mappings WHERE short_code = '0000' on non-transactional activity Short code '0000' maps to long code '555555555555' up-a 0000 333333333333 Transactional activity created: DatabaseQueryActivity[101.100008],I Submitted UPDATE mappings SET long_code = '333333333333' WHERE short_code = '0000' on activity DatabaseQueryActivity[101.100008],P (There is an open transaction, it must be committed or rolled back) Updated mapping of short code '0000' to long code '333333333333' (There is an open transaction, it must be committed or rolled back) q-s 0000 Short code '0000' maps to long code '555555555555' (There is an open transaction, it must be committed or rolled back) q-a 0000 Submitted SELECT long_code FROM mappings WHERE short_code = '0000' on activity DatabaseQueryActivity[101.100008],P (There is an open transaction, it must be committed or rolled back) Short code '0000' maps to long code '333333333333' (There is an open transaction, it must be committed or rolled back) co Submitted commit request on transactional activity DatabaseQueryActivity[101.100008],P Transaction committed q-a 0000 Submitted SELECT long_code FROM mappings WHERE short_code = '0000' on non-transactional activity Short code '0000' maps to long code '333333333333' up-a 0000 44444444444 Transactional activity created: DatabaseQueryActivity[101.100011],I Submitted UPDATE mappings SET long_code = '44444444444' WHERE short_code = '0000' on activity DatabaseQueryActivity[101.100011],P (There is an open transaction, it must be committed or rolled back) Updated mapping of short code '0000' to long code '44444444444' (There is an open transaction, it must be committed or rolled back) ro Submitted rollback request on transactional activity DatabaseQueryActivity[101.100011],P Transaction rolled back q-a 0000 Submitted SELECT long_code FROM mappings WHERE short_code = '0000' on non-transactional activity Short code '0000' maps to long code '333333333333' p-s 0000 0 Transaction for synchronous requests created: DatabaseTransaction[101.100013,age=0ms,conn=none] Updated mapping of short code '0000' to long code '0333333333333' (There is an open transaction, it must be committed or rolled back) q-s 0000 Short code '0000' maps to long code '0333333333333' (There is an open transaction, it must be committed or rolled back) q-a 0000 Submitted SELECT long_code FROM mappings WHERE short_code = '0000' on non-transactional activity (There is an open transaction, it must be committed or rolled back) Short code '0000' maps to long code '333333333333' (There is an open transaction, it must be committed or rolled back) ro Rolled back DatabaseTransaction[101.100013,age=16806ms,conn=DbConnectionHandle[id=14,state=CLOSED,transactional=true,connection=com.mysql.jdbc.ProxyConnection@441e8828]] p-a 0000 00 Transactional activity created: DatabaseQueryActivity[101.100015],I Submitted CALL EXTEND_LONG_CODE_WITH_PREFIX(short_code = '0000' and prefix = '00') on activity DatabaseQueryActivity[101.100015],P (There is an open transaction, it must be committed or rolled back) Updated mapping of short code '0000' to long code '00333333333333' (There is an open transaction, it must be committed or rolled back) q-a 0000 Submitted SELECT long_code FROM mappings WHERE short_code = '0000' on activity DatabaseQueryActivity[101.100015],P (There is an open transaction, it must be committed or rolled back) Short code '0000' maps to long code '00333333333333' (There is an open transaction, it must be committed or rolled back) q-s 0000 Short code '0000' maps to long code '333333333333' (There is an open transaction, it must be committed or rolled back) co Submitted commit request on transactional activity DatabaseQueryActivity[101.100015],P Transaction committed q-s 0000 Short code '0000' maps to long code '00333333333333' close Goodbye! Connection closed by foreign host.
MySQL setup
Below are the steps required to set up the MySQL example: create and fill a test database, and deploy and undeploy the service.
Creating and filling in the database
To run the example, a test database should be created in your MySQL installation, containing the correct schema and data. One way of doing this is to execute the SQL script included in the examples:
$ mysql -u root -p < examples/src/com/opencloud/slee/example/dbquery/MySqlTestDB.sql
CREATE DATABASE mappings;
CREATE USER 'appuser' IDENTIFIED BY 'appuser';
GRANT ALL PRIVILEGES ON mappings.* TO 'appuser';
FLUSH PRIVILEGES;
USE mappings;
DROP TABLE IF EXISTS mappings;
CREATE TABLE mappings (
short_code VARCHAR(10) PRIMARY KEY,
long_code VARCHAR(64) NOT NULL
);
INSERT INTO mappings (short_code, long_code) VALUES ('0000', '11111111111');
COMMIT;
DROP PROCEDURE IF EXISTS extend_long_code_with_prefix;
DELIMITER //
CREATE PROCEDURE EXTEND_LONG_CODE_WITH_PREFIX(
OUT p_concat VARCHAR(64),
IN p_short_code VARCHAR(10),
IN p_prefix VARCHAR(64))
BEGIN
SELECT concat(p_prefix, long_code)
INTO p_concat
FROM mappings
WHERE short_code = p_short_code;
UPDATE mappings
SET long_code = p_concat
WHERE short_code = p_short_code;
END //
DELIMITER ;
The same host, database, user, and password must be set in examples/build.properties . |