This book contains performance benchmarks using the Sentinel Authentication Gateway

Topics

Benchmark Scenarios

Descriptions of each of the benchmark scenarios, and notes on the benchmark methodology used

Hardware and Software

Details of the hardware, software, and configuration used for the benchmarks

Benchmark Results

Summaries of the benchmarks and links to detailed metrics.

Other documentation for the Sentinel Authentication Gateway can be found on the Sentinel Authentication Gateway product page.

Benchmark Scenarios

This page describes the scenarios and methodology used when running the benchmarks.

In all benchmarks, the OpenCloud Scenario Simulator performs the roles of UE and HSS.

BSF Scenarios

Successful bootstrapping scenario

In this scenario, a UE completes a typical bootstrapping procedure with the BSF.

  1. The UE sends an initial, unauthorized bootstrapping request to the BSF, containing the IMPI.

  2. The BSF sends a Diameter Zh Multimedia-Auth-Request to the HSS.

  3. The HSS responds with a Multimedia-Auth-Answer, containing an Authentication Vector and the user’s GUSS.

    • The Authentication Vector and GUSS are written to the BSF’s Cassandra database.

  4. The BSF sends a challenge to the UE in the HTTP 401 response.

  5. The UE sends a new HTTP request to the BSF, containing the challenge response in the Authorization header.

  6. The BSF validates the challenge response, and sends an HTTP 200 OK response to the UE containing the B-TID and lifetime of the security association, completing the bootstrapping procedure.

    • The BSF reads the Authentication Vector and GUSS from Cassandra, and inserts a new entry for the security association.

Successful bootstrapping call flow

successful bootstrapping scenario

Response time is measured at the UE, from when the initial HTTP request is sent to the arrival of the 401 response from the BSF. This includes the time taken for the BSF to contact the HSS and its Cassandra database.

Test Setup

Each test run consists of a 10 minute ramp-up period where load is increased from zero to the target rate, then a 60 minute measurement period at peak load.

The ramp-up period is included as the Oracle JVM provides a Just In Time (JIT) compiler. The JIT compiler compiles Java bytecode to machine code, and recompiles code on the fly to take advantage of optimizations not otherwise possible. This dynamic compilation and optimization process takes some time to complete. During the early stages of JIT compilation/optimization, the node cannot process full load. JVM garbage collection does not reach full efficiency until several major garbage collection cycles have completed.

Only latency measurements during the measurement period are used; latency measurements during the ramp-up period are ignored.

Load is not stopped between ramp up and starting the test timer.

Hardware and Software

This page describes the hardware and software used when running the benchmarks.

Hardware

benchmarks hardware

Software

Vendor Software Version

OpenCloud

OpenCloud Sentinel Authentication Gateway

1.0.0.5

OpenCloud

Rhino

2.4.0.14

OpenCloud

Scenario Simulator

2.3.0.8

OpenCloud

HTTP Scenario Pack

1.0.1.0

OpenCloud

Diameter Scenario Pack

2.6.0.3

Apache

Apache Cassandra

2.0.17

Configuration

Parameter Value

heap_size

2048M

new_size

512

gc_type

cms

Additional GC parameter

-XX:+ParallelRefProcEnabled

Benchmark Results

This page summarises the results for the Sentinel Authentication Gateway benchmarks. Detailed metrics follow the summary tables.

The benchmark scenarios has more information on how these configurations are defined.

Benchmarks

3000 sessions per second (12,000 messages per second)

1066% across 24 threads

Note Maximum theoretical CPU usage is 2400%

939MB average heap

50th percentile 90th percentile 95th percentile 99th percentile

4.1ms

43.2ms

58.4ms

115.5ms

Detailed metrics

Call Rate

60at3000 sim rate

CPU usage

60at3000 rhino cluster 101 cpu

Heap usage

60at3000 rhino cluster 101 heap

Scenario latencies

60at3000 latency