About the Rf Control Resource Adaptor
The Rf Control Resource Adaptor (Rf Control RA) is a component that interfaces Rhino TAS applications, with a Charging Data Function (CDF), for offline charging, using the Diameter Rf protocol. It provides applications a full implementation of the "Client accounting state machine" as per RFC 6733. Applications create Accounting Requests and pass them to the Rf Control RA. The Rf Control RA is responsible for delivery of Accounting Requests to the CDF.
Topics
This document includes the following topics:
Topic | Explains… |
---|---|
How to configure an Rf Control RA entity |
|
How to monitor the behaviour of an Rf Control RA entity |
|
How to manage an Rf Control RA entity |
|
How to use the Rf Control RA API in an application |
Service Assurance Server (SAS) Tracing
The Rf Control resource adaptor is integrated with the Metaswitch Service Assurance Server (SAS). SAS provides a mechanism to record and search detailed end-to-end tracing of call handling.
See the SAS Facility section in the Rhino extended API documentation for more information about SAS tracing.
SAS tracing is supported on Rhino versions 2.6 and later. The Rf Control resource adaptor can be used with earlier Rhino versions if the Rhino API Compatibility SLEE library is deployed. |
The resource adaptor automatically performs SAS tracing, if it is enabled in Rhino.
Rf Control sessions (outgoing requests)
For Rf Control sessions, the resource adaptor will create a new SAS trail and will associate it with the trail on the invoking thread.
SAS markers
When creating a SAS trail, the resource adaptors will raise a SAS marker for trail correlation. They raise a GENERIC_CORRELATOR_MARKER
containing the Session-Id
AVP.
SAS events
---
version: 0.1
events:
FAILED_SESSION_ENDING:
summary: Failed to send Rf ACR as the session is ending
level: 40
details: | Session ID: {{var_data[0]}}
FAILED_START_NOT_SENT:
summary: 'Failed to send Rf {{var_data[0]}} ACR as a START_RECORD ACR has not yet been sent'
level: 40
details: | Session ID: {{ var_data[1] }}
FAILED_STOP_SENT:
summary: 'Failed to send Rf {{var_data[0]}} ACR as a STOP_RECORD ACR has already been sent'
level: 40
details: | Session ID: {{ var_data[1] }}
FAILED_START_ALREADY_SENT:
summary: 'Failed to send Rf START_RECORD ACR as a Diameter session has already been established'
level: 40
details: | Session ID: {{ var_data[0] }}
FAILED_EVENT_ON_EST_SESSION:
summary: 'Failed to send Rf EVENT_RECORD ACR as the Diameter session has already been used for session charging'
level: 40
details: | Session ID: {{ var_data[0] }}
BUFFER_FAILED_REQUEST:
summary: 'Buffering failed Rf {{ var_data[0] }} ACR'
level: 40
details: | The request will be reattempted later.
BUFFER_REPLAY:
summary: 'Replaying buffered Rf request on session: {{var_data[0]}}'
level: 40
FAILED_TO_ADD_TO_BUFFER:
summary: 'Failed to store Rf request in buffer'
level: 20
details: | Error details: {{var_data[0]}}
Configuring the Rf Control Resource Adaptor
Rf Control Resource Adaptor configuration
The Rf Control Resource Adaptor (Rf Control RA) has 2 types of configuration:
-
General Resource Adaptor configuration
-
Additional configuration, which includes the Diameter configuration that extends the general Diameter Resource Adaptor configuration
Configuration of an Rf Control RA Entity
The Rf Control RA has the following configuration parameters.
Parameter | Description | Default value |
---|---|---|
ConfigurationProfile |
The diameter configuration profile. See Configuring Diameter Resource Adaptors |
RfControlDiameterConfiguration/client |
ConfigurationProfilePollTime |
The time (milliseconds) between checking for configuration changes in the diameter configuration profile (as indicated by the |
30000L |
3GPPVersion |
The diameter version used for messages between the Rf Control RA and the CDF. See Diameter 3GPP configuration properties. |
Vcb0 |
ExtensionAvpSet |
The diameter extension AVP configuration profile. See Configuring Extension AVPs for more information on how to add extension AVPs. |
DiameterExtensions/Charging |
ExtensionAvpSetPollTime |
The time (milliseconds) between checking for configuration changes in the extension AVP profile (as indicated by the |
0L |
IOClientWorkers |
Number of IO threads serving outgoing connections. |
0 |
ConnectTimeout |
Timeout (milliseconds) to wait for an outgoing transport connection to be established |
30000L |
RequestTimeout |
Timeout (milliseconds) to wait for a response before firing a Timeout event |
2000L |
ReconnectDelay |
Delay (milliseconds) before reconnecting (RFC6733 timer Tc) |
5000L |
ForceReconnectAfterDPR |
if true, always attempt to reconnect even if the server sent a DPR with Disconnect-Cause set to |
false |
ThreadPoolSize |
Size of thread pool for decoding incoming diameter messages. |
-1 |
DestinationRealm |
Destination realm to use for sending messages, if not already set by client. |
'opencloud' |
DestinationHost |
Destination host to use for sending messages, if not already set by client. |
empty |
See Configuring Diameter Resource Adaptors for details on the Diameter configuration. |
Rf Control RA Diameter Configuration
The ConfigurationProfile
configuration parameter identifies the profile that contains additional configuration for the Rf Control RA and includes the Diameter configuration. The default value of the ConfigurationProfile
parameter (RfControlDiameterConfiguration/client
) means the client
profile in the RfControlDiameterConfiguration
profile table is used. The profile has the following fields.
Field Name | Description | Default Value |
---|---|---|
OnDiskBufferDirectoryName |
Name of directory that the SQLite database will store the records in when no CDF is reachable. The Rf Control RA requires write permission on the path specified and the path has to be present in the |
'${rhino.dir.home}/rf-control' |
MaxRecordsInMemoryQueue |
Maximum number of records waiting to be written in the in-memory queue. |
34000L |
MaxDbCommitIntervalInMs |
The interval (milliseconds) the Persist thread will wait, after successfully writing all records from the memory queue to the disk buffer. |
5000L |
DbErrorRetry |
The interval (milliseconds) the Replay thread and the Persist thread will wait, before trying to access the disk buffer, after a previous access failure. |
30000L |
ReplayRunInterval |
The time (milliseconds) the replay thread will wait, before trying to access the disk buffer, after sending all previous records to the CDF. |
30000L |
Product |
The product name |
OpenCloud Diameter |
ProductVendorID |
Vendor ID; must be |
19808L |
ApplicationId |
Diameter Application ID |
3L |
ApplicationVendorID |
Diameter Application ID. |
19808 |
Version |
Version number to use in all Firmware-Revision AVPs generated by the Diameter Rf stack. |
1 |
Host |
Host to use in all Origin-Host AVPs. |
diameterclient |
Realm |
Realm to use in all Origin-Realm AVPs. |
example |
Peer Table |
A valid XML document for peer table configuration. |
|
Realm Table |
A valid XML document for realm table configuration. |
|
PeerPort |
Port of remote Diameter peer |
5898 |
PeerHost |
Host of remote Diameter peer |
CDF |
PeerConnectAtStartup |
Whether to connect to the remote peer at startup. must be true or false |
true |
Action |
Must be LOCAL for this version of the RA |
LOCAL |
Transports |
Transport(s) to listen on. Values can be tcp, sctp, or tcp,sctp |
tcp |
Configuration for Multiple CDF Nodes
Failover between CDF nodes is supported by the Diameter Rf stack. You configure the Rf Control RA by creating realm and peer tables in the Rf Control RA Diameter Configuration appropriate for your deployment. For further information on configuration and diameter stack routing see:
Monitoring the Rf Control Resource Adaptor
Statistics
The health of the Rf Control Resource Adaptor entity can be monitored using the Rhino statistics tools. For example, to monitor the statistics from the command line (<entity-name>
is the Resource Adaptor entity name):
rhino-stats -m SLEE-Usage.RAEntities.<entity-name>.(default)
The Rf Control Resource Adaptor updates the default parameter set. |
The statistics collected by the Rf Control Resource Adaptor are:
Name | Description | Incremented when … |
---|---|---|
recordsReceived |
Number of records received via provider interface |
The client post a record to the RA |
recordsFailedToPersist |
Number of record write failures that occur |
The Persist thread fails to put a record in the disk buffer |
recordsPersisted |
Number of records persisted to database on CDF failure |
The Persist thread successfully writes a record to the disk buffer |
recordsPending |
Number of records persisted waiting to be sent to the CDF |
The Persist thread write records to the disk buffer. Decremented when the Replay thread deletes the record from the disk buffer |
recordsReadFromDb |
Number of records read from database |
The Replay thread reads a record from the disk buffer |
recordsRetransmitted |
Number of records retransmitted due to failover |
The Replay thread receives an ACA, for an ACR, that has the potentially retransmitted flag set in the header |
successfulAnswers |
Number of successful answers |
The RA receives an ACA for an ACR |
sendFailures |
Number of temporary send failures |
The Diameter Rf stack
|
failedAnswers |
Number of permanent failed answers with record discarded |
The Diameter Rf stack
|
recordsDiscarded |
Number of records discarded due to non-failed-answer issues |
The Persist thread cannot write the message to the memory buffer as it is full. The Replay thread cannot read or create an ACR from the stored messages. This indicates that there are corrupt records. |
processingTime |
Processing times for ACR/ACA pairs. It is the time (milliseconds) between sending an ACR and receiving an ACA in response. |
The Diameter Rf stack received an ACA or error message |
FailedToStartActivity |
The amount of times the Rf Control RA failed to start an activity |
The Rf Control RA was unable to start a SLEE activity |
FailedToSubmitEvent |
The amount of times the Rf Control RA failed to submit an event |
The Rf Control RA fails to submit an event for delivery |
You can define your own threshhold alarms that are a function of the statistics the Rf Control Resource Adaptor collects. See Threshold Alarms in the Rhino TAS Admin Guide for more information about defining threshold based alarms. |
Alarms
The following alarms may be raised with a source of OpenCloud <entity-name> <version>
(for a Resource Adaptor entity <entity-name>
, of version <version>
).
Alarm | Level | Raised when … | Cleared when … |
---|---|---|---|
|
WARNING |
A configuration property (such as the host or realm) is changed while the RA entity is active |
The RA entity is deactivated |
|
CRITICAL |
The memory queue reaches the limit set by the configuration parameter |
The Request handler is able write a record to the queue. |
|
MAJOR |
The Persist thread cannot write to the disk buffer. |
The Persist thread is able to write to the disk buffer. |
|
CRITICAL |
The Persist thread caught an unexpected exception |
The RA is restarted |
|
MAJOR |
The Replay thread can’t read records from the disk buffer |
The Replay thread is able to read records from the disk buffer |
|
CRITICAL |
The Replay thread caught an unexpected exception |
The RA is restarted |
Managing the Rf Control Resource Adaptor
CDF connection
The Rf Control Resource Adaptor (Rf Control RA) will connect to the CDF when it is active. An alarm is raised if the Rf Control RA fails to connect to the CDF. There are several possible reasons connecting to the CDF may fail, including:
-
Incorrect diameter configuration of the Rf Control RA
-
There is a problem with the network
-
The CDF is not accepting connections
-
The host machine the Rf Control RA is running on is not properly configured
The alarm is automatically cleared once the failure is resolved, and the Rf Control RA successfully connects to the CDF.
Disk buffer management
The Rf Control RA uses an SQLite database as a disk buffer when the CDF is not reachable. SQLite is a simple, embedded, SQL database engine ideally suited to be used as an on-disk buffer.
For more information on SQLite see www.sqlite.org |
The database name is: buffer-<ra entity>-<rhino node>.db
. Example: buffer-rf-control-ra-101.db
.
The location of the database is defined by the Rf Control RA Diameter Configuration profile parameter OnDiskBufferDirectoryName
(default value is ${rhino.dir.home}/rf-control
)
Accounting Request (ACR) messages are stored in the database when the CDF is not reachable.
As a rule of thumb, assume 3kB per ACR (large but reasonable) and 3 ACRs per call (START, INTERIM, STOP). Assuming 500 call attempts per second per Rhino node, the database will grow at a rate of:
-
4.5 MB per second
-
270 MB per minute
-
approximately 16 GB per hour
If the Rf Control RA fulls the entire disk partition it is buffering to, an alarm of type write-failure
or write-thread-failure
will be raised (depending on the cause). The write-failure
alarm will be cleared when the database is available for writing. The write-thread-failure
alarm will require a Rf Control RA restart.
The platform operator is responsible for increasing the available disk space |
During the write-failure
and write-thread-failure
condition the Rf Control RA will store ACR records in a memory buffer. If the database or the CDF connection issue is not resolved, the memory queue will eventually become full and the Rf Control RA will start discarding records.
When the CDF becomes available again, the Replay thread
will attempt to transmit stored ACR records. The database will automatically re-use the disk space that successfully replayed ACR records had occupied.
The number of stored and transmitted records can be monitored by viewing the RF Control RA’s statistics. |
The SQLite database supports VACUUMING, which rebuilds the database file, repacking it into a minimal amount of disk space. VACUUMing is only necessary if the disk partition that the database is stored in is too small for normal operation use. For example, the database fits into the partition but the database size is deemed "too large". This procedure is nevertheless attempted automatically after all the records have been replayed by the replay thread, that is the database is nominally empty, since the impact on performance in this case is negligible. However, this attempt may fail if there happens to be a concurrent access to the database by another thread. This means that manual intervention may be necessary if disk space is getting low.
The database has incremental vacuum support enabled, since running an incremental vacuum should have less of a performance impact than a full one if there are a lot of records in the database, but it may be less effective.
An incremental vacuum procedure can be performed as follows:
$ sqlite3 buffer-rfcontrol-101.db "PRAGMA incremental_vacuum;" or $ sqlite3 buffer-rfcontrol-101.db sqlite> PRAGMA incremental_vacuum; sqlite> .quit
If this doesn’t produce the desired results, a full VACUUM procedure can be performed as follows:
$ sqlite3 buffer-rfcontrol-101.db "VACUUM;" or $ sqlite3 buffer-rfcontrol-101.db sqlite> VACUUM; sqlite> .quit
The vacuum procedure may fail if there is concurrent access to the database. To guarantee a successful vacuuming the Rf Control RA may have to be deactivated before performing the procedure. |
For more information on VACUUM see SQLite VACUUM.
We recommend that the disk partition is large enough to not require VACUUMing.
If the operator has an independent alarm configured to monitor the disk partition use, then it is possible that a VACUUM may be necessary to reduce the reported disk usage (after a CDF outage and replay completion). Such an alarm should clear once the VACUUM has completed.
Using the Rf Control API
The Rf Control RA provides a Rhino TAS API that allows a client application to send Diameter Rf messages. The following example shows a Service with a single SBB using the API.
@SleeService( id = @ComponentId(name="ExampleRfControl", vendor="OpenCloud", version="1.0"), rootSbb = @RootSBB(sbb = @ComponentId(name="ExampleRfControl", vendor="OpenCloud", version="1.0")) ) @SBB( id = @ComponentId(name="ExampleRfControl", vendor="OpenCloud", version="1.0"), sbbClasses = @SBBClasses(), raTypes = { @RATypeBinding( raType = @ComponentId(name = "rf-control-ratype", vendor = "OpenCloud", version = "1.0.0"), activityContextInterfaceFactoryName = "rf-control-ra/activitycontextinterfacefactory", resourceAdaptorEntityLink = "rf-control-ra", resourceAdaptorObjectName = "rf-control-ra/provider" ) } ) public abstract class ExampleRfControl implements Sbb { @Override public void setSbbContext(SbbContext context) { tracer = context.getTracer("example.rf-control"); this.context = context; try { /* Get Rf control facilities */ final Context env = (Context) new InitialContext().lookup("java:comp/env"); this.profileFacility = (ProfileFacility) env.lookup(ProfileFacility.JNDI_NAME); this.rfControlProvider = (RfControlProvider) env.lookup("rf-control-ra/provider"); this.rfControlACIFactory = (RfControlActivityContextInterfaceFactory) env.lookup("rf-control-ra/activitycontextinterfacefactory"); } catch (NamingException e) { throw new RuntimeException("Got NamingException trying to locate facilities", e); } } private void exampleCall() { final RfMessageFactory rfMessageFactory = rfControlProvider.getRfMessageFactory(); tracer.fine("Creating the the message"); final AccountingRequest startAcr, interimAcr1, interimAcr2, stopAcr; try { /* Create simple Rf messages */ startAcr = buildACR(roMessageFactory, rfMessageFactory, AccountingRecordType.START_RECORD); interimAcr1 = buildACR(roMessageFactory, rfMessageFactory, AccountingRecordType.INTERIM_RECORD); interimAcr2 = buildACR(roMessageFactory, rfMessageFactory, AccountingRecordType.INTERIM_RECORD); stopAcr = buildACR(rfMessageFactory, AccountingRecordType.STOP_RECORD); } catch (Exception e) { tracer.warning("Failed to send ACR due to an exception", e); sendHttpResponse(requestActivity, aci, HTTP_SERVER_ERROR, "Internal Error", "Failed to send ACR due to an exception: " + e.toString()); return; } tracer.fine("Sending the the message"); try { /* Send Rf messages */ final RfControlActivity rfControlActivity = rfControlProvider.newSession(); rfControlActivity.sendAccountingRequest(startAcr); rfControlActivity.sendAccountingRequest(interimAcr1); rfControlActivity.sendAccountingRequest(interimAcr2); rfControlActivity.sendAccountingRequest(stopAcr); } catch (Exception e) { tracer.warning("Failed to send ACR due to an exception", e); return; } } private AccountingRequest buildACR(RfMessageFactory rfMessageFactory, AccountingRecordType type) throws Exception { final AccountingRequest acr = rfMessageFactory.createRfAccountingRequest(type); final ServiceInformation serviceInformation = rfMessageFactory.createServiceInformation(); final ImsInformation imsInformation = rfMessageFactory.createImsInformation(NodeFunctionality.AS); final SubscriptionId subscriptionId = rfMessageFactory.createSubscriptionId(SubscriptionIdType.END_USER_E164, "tel:1234"); serviceInformation.setSubscriptionId(subscriptionId); serviceInformation.setImsInformation(imsInformation); acr.setServiceInformation(serviceInformation); return acr; } }
The client application needs to have an RA Type Binding and RA Entity link binding to retrieve the Rf Control Provider via JNDI. Once this setup has occurred the primary interfaces for the application are the:
-
RfMessageFactory
(to create Rf requests), and -
RfControlActivity
(to send requests on a logical Rf session) .
The RfMessageFactory
exposes the underlying Diameter Rf factory to create all necessary requests and AVPs. Once an AccountingRequest
has been created and populated, it can be sent via an RfControlActivity
.
For more detailed information see: