Synthetic User Monitoring - Setup & Configuration

Synthetic User Monitoring (SUM) configuration is to be performed via the application itself.

The configuration involves the following components:

  • Runners - to define and configure where the monitoring scenarios are executed.
  • Resources - to upload and handle the Selenium IDE files
  • Scenarios - to define and configure the monitoring scenarios
  • Business Services - to define and configure the Synthetic User Monitoring business services. Here you can also define the events to emit in case of Availability issue.
  • Application Settings - to define the Synthetic User Monitoring housekeeping settings.

Information

  • Several monitoring scenarios can be created from a given Selenium IDE file.
  • Several scenarios can be associated to one or several runners.
  • One scenario is associated to at least one or several business services.
  • On business service is associated to at least one or several LMS cloud services or LMS systems.

Prerequisites

To set up Synthetic User Monitoring you need:

  • At least one Selenium remote webdriver infrastructure to configure the runners (You can find more information about Selenium remote webdriver here)
  • At least one Selenium SIDE file to configure the scenarios. (refer to Creating Synthetic User Monitoring Scripts)

Typical Setup Flow

The typical setup consists of six steps:
  1. Configure a new Runner: configure its endpoint to connect to your Selenium remote webdriver infrastructure
    (see Configure Runners section below).
  2. Import the SIDE file into SUM. This creates a new resource listing the scripts defined in the imported SIDE file
    (see Configure Runners section below).
  3. Create and configure a new scenario (choose its name, indicate the corresponding script, ...,)
    (see Create and Maintain Scenarios section below).
  4. Associate and activate the Scenario to your Runner. The SUM schedules the Scenario executions
    (see Activate Monitoring section below).
  5. (optionally) Adjust the Scenario configuration: schedule period, variables, performance monitoring thresholds, ...
    (see FAQ section below).
  6. (optionally) Configure the Scenario events: Alerts, Notifications, ...
    (see Configure Scenario Events section below).

Configure Runners

The first step during the setup is the creation of a SUM Runner. A runner is the component in charge of the execution of the monitoring scenarios.

Every runner has a Runner Endpoint. A runner endpoint is the Selenium remote webdriver infrastructure that Synthetic User Monitoring (SUM) accesses to pass the scenario commands to execute.

It also hosts the web browser used to access the monitored applications.

To access the endpoint, SUM uses the connection information defined in the runner configuration. Two types of configuration are possible:

InternetThe endpoint is access via the HTTPS URL specified in the configuration.
On-premise
The endpoint is access via the Cloud Connector information specified in the configuration

Important

For Internet type, only HTTPS URLs are supported.

A runner endpoint can be located in the cloud or on-premises.

To get a Selenium remote webdriver infrastructure, you have several possibilities:

  • Some companies are specialized in providing such infrastructure (SauceLabs, BrowserStack, LambdaTest, ...). 
    This is often the easiest way to obtain a reliable and efficient execution infrastructure.

    Please note: SAP does not recommend any specific Selenium remote webdriver infrastructure provider.

  • You can also create your own infrastructure.
    For instance, Selenium provides samples docker images containing the minimum required elements for such an infrastructure. You can build your own images out of these templates.

You can find more information about Selenium remote webdriver here.

Important

The runner endpoint URL must be a trusted infrastructure. SUM will send the scenario commands to this URL, including sensitive data maintained in the scenario configuration. 

Declare a SUM Runner

  1. Open the SUM Configuration tray
  2. Expand the Runners section
  3. Click the pen icon to open the Configure Runners dialog.
  4. Click the "Add" button on top of the table. 
  5. Fill-in the General and Endpoint fields
  6. Click the "Add" button.


Edit a Runner Endpoint (optional)

If necessary you can adjust the endpoint of the SUM Runner later.

  1. Open the SUM Configuration tray
  2. Expand the Runners section
  3. Click the pen icon to open the Configure Runners dialog
  4. Select the runner you want to edit in the table
  5. Navigate to the Endpoint tab.

  6. Perform the required changes

    Once you enter changes the "Save" and a "Discard" button will appear. Changed fields will be highlighted.

  7. Click the "Save" button to save your changes.

Hint: After saving, you can test your settings using the Test Connection button (for details see FAQ - How to test the connectivity of a Runner infrastructure?).

Import Resources

The second step is to import the Selenium SIDE files that contain your scripts. For more information on how to create scripts, please read Creating Synthetic User Monitoring Scripts
  1. Open the SUM Configuration tray
  2. Expand the Resources section
  3. Click the pen icon to open the Configure Resources dialog.
  4. Click the "Upload" button on top of the table. 
  5. Select the SIDE file to import using the "Browse" button
  6. Click the "Upload" button.

When a new SIDE file is imported, it becomes a new Resource entry in the SUM configuration.

During the import process, the SIDE file content is analyzed and:

  • Each test suite becomes a SUM Selenium script.
  • Each test in the test suite becomes a step of the SUM script.
  • The enclosed SUM annotations are evaluated to:
    • Create the additional SUM steps.
    • Create the declared variables.
  • If the SIDE file already existed in SUM, it is updated. For more information refer to the FAQs section.


Relationship between Elements in the SIDE File and SUM Scripts Elements

Script Names and IDs

Names and IDs come directly from the SIDE file.

Script ElementSIDE ElementInternal Identifier
ScriptName of the test suiteIdentifier of the resource (SIDE file) + GUID of the Selenium test suite
Step coming from a testName of the testIdentifier of the Script + GUID of the Selenium test 
Step coming from an annotationAnnotation textIdentifier of the Script + GUID of the Selenium command

Nested Tests

Via the "run" Selenium command, a test can call another Selenium test (in the SIDE file).

In this case, the commands inside the called test belong to the current SUM step. Moreover, if a called test contains SUM step annotations, they are ignored.

Step Thresholds

When the steps are created, they are automatically configured to:

  • Monitor the availability
  • Temporarily ignore the performance (because the thresholds values cannot be provided by the SIDE file)

You can set up thresholds later as described in the section "Scenario Maintenance"

SUM Variables

When the SIDE file is imported, SUM detects the SUM variable annotations and automatically creates the corresponding SUM script variables.

By default, the value of the created variable will be the "value" field of the corresponding Selenium command.

If during the SUM scenario creation good practice was followed, the secured variables don't have their correct value. It will be necessary to appropriately set them via the SUM configuration in the parameters section of the Scenario details as described in the section "Scenario Maintenance".

Create and Maintain Scenarios

The third step is to create a scenario for your SUM scripts.

  • You can create several scenarios from the same Selenium IDE script.

  • One runner can run several scenarios.

  • Each scenario can be assigned to several runners.


Create a new Scenario

  1. Open the SUM Configuration tray
  2. Expand the Scenarios section.
  3. Click the pen icon to open the Configure Scenarios dialog.
  4. Click the "Add" button on the top of the table.
  5. In the General section, enter a meaningful Name and Description for your Scenario.
  6. In the Execution section, select the Selenium script your Scenario has to execute.
  7. In the Business Service section, associate your Scenario to the convenient SUM Business Service.
    (If you don't know yet which Business Service to use, you can associate your Scenario to the Default SUM Business Service)
  8. Click the "Add" button.

The creation will automatically open the Scenario Details view. You can also access this later by clicking on the line of the scenario in the scenario table. 


Additional Configuration

Additionally, you can:

  • Adjust the steps to consider for the performance monitoring.
  • Adjust the Execution Period.
  • Adjust the SUM variables declared in the Script (mandatory step if the script declares secured variable).
  • Adjust the Selenium settings (e.g. screenshots capture).

For more information refer to the FAQs section.

Activate Monitoring

The forth step (last required step) is to activate the scenario on a runner. Please note that you can assign several runners to a scenario. Also, you can use a same runner for several scenarios.

  1. Open the SUM Configuration tray
  2. Expand the Scenarios section.
  3. Click the pen icon to open the Configure Scenarios dialog.
  4. Access the corresponding Scenario Details by clicking on the line of the scenario.
  5. Select the General tab.
  6. In the Runners section, click the "Assign" button.
    This opens the Assign Runners popup.
  7. In the popup, select the runner to use and click the "Assign" button. This could take a few seconds.
  8. Set the toggle button to ON to activate the scenario execution.
  9. Click the "Save" button.

Configure Scenario Events

SUM supports events for both the availability and performance issues.

SUM scenarios can be assigned to several business services (at minimum one), therefore, you need to decide which business service to configure. Refer to the FAQs to learn how to assign/unassign a scenario to a business service.

  1. Open the Configuration area
  2. Expand the Scenarios section.
  3. Click the pen icon to open the Configure Scenarios dialog.
  4. Access the corresponding Scenario Details by clicking on the line of the scenario.
  5. Select the Business Services / Events tab.
  6. Jump to the business service to configure by clicking on it.

  7. In the business service configuration, click on the Events tab.
    A table indicates which event configuration is active.

  8. Open the Event Configuration by clicking on the event name you want to configure: "Availability Event" or "Performance Event".
  9. Use the toggle buttons to enable/disable the actions to perform when the event is emitted.
  10. For the enabled action, enter the corresponding configuration (e.g. for the "Send Email To" action, enter the recipients).
  11. Click the "Save" button

Housekeeping

By default, SUM keeps the scenarios executions metrics for 14 days. This retention period is configurable. the minimal value is 7 days. The maximum is 30 days.

The alerts triggered by SUM, are automatically deleted when the execution, that initially triggered the event, is housekept.

SUM also stores hourly-aggregated executions for 31 days. This can be configured. The retention time for the aggregated data cannot be smaller than the retention time for the single executions. The maximum is 190 days. The retention of aggregated executions can be also be disabled.

For the screenshots and the metrics captured during the scenarios executions, SUM has specific retention periods:

  • For the screenshots, the default is 3 days. It's configurable with a minimum of 1 day and a maximum of 10 days.
  • For the metrics, the default is 6 days. It's configurable with a minimum of 1 day and a maximum of 30 days.

Screenshots and metrics can consume a lot of space. This is something to consider when configuring their retention period.

To adjust SUM housekeeping setting:

  1. Open the Configuration tray.
  2. Expand the Application Settings section.
  3. Click the pen icon to open the Application Settings dialog.
  4. If checked, uncheck the "Use default" flag for the retention period to adjust.
  5. Adjust the housekeeping settings and save.

The change will be taken into account the next time the housekeeping job is executed. This happens once a day.

Frequently Asked Questions (FAQs)

TermDescription
ScenarioMonitoring scenario - A Selenium Script scheduled to be regularly executed by a Runner
Step

Monitoring scenario step - Sub-part of a monitoring scenario focusing on a specific action.
e.g., Logon

Script

Selenium IDE test suite describing a sequence of actions (a.k.a. commands) to perform on the application to monitor.

This is what simulates the end-user actions on the monitored application.

SIDE file

Selenium IDE file containing one or several test suites that Synthetic User Monitoring will use as Scripts.

SIDE files are produced by the Selenium IDE plugin when a Selenium test project is saved.

ResourceExternal file uploaded in the SUM database.
Currently, the only resource SUM supports is Selenium IDE (SIDE) files.
Selenium

Selenium is an open-source framework dedicated to Web application tests.

AnnotationSynthetic User Monitoring specific keyword added into the comment section of a Selenium command.
RunnerThe component in charge of the execution of the monitoring scenarios.
Runner endpointSelenium remote web driver infrastructure: the entity hosting the web browser and the web driver used to access the monitored applications
Business ServiceLogical component associating a set of SUM Scenarios with a set of LMS Cloud Services (and/or Systems).The Business Service is also the configuration point for the SUM events.

Selenium is an open-source framework dedicated to web application tests.

It consists of several components.

In the context of the Synthetic User Monitoring, the involved Selenium components are mainly:

  • Selenium IDE – Integrated Development Environment for Selenium. This is the tool that is used to create the Selenium scripts.
  • Selenium WebDriver – This is the component that "operates" the Web browser. It interprets the Selenium commands and converts them into action on the Web browser. The WebDriver is Web-browser-specific.

Important: Selenium is not developed by SAP.

Usually designed with the Selenium IDE component, a Selenium IDE script (or SIDE file) corresponds to a Selenium IDE project.

It describes the sequences of actions (also referred to as commands) to be performed on a web browser, such as clicks, setting texts, waiting for elements, etc.

These sequences are organized into:

  • Tests: a sequence of actions to be executed one after the other.
  • Test suites: a sequence of tests to be executed one after the other.

Each action has a set of attributes to indicate:

  • Command: the action to be performed.
  • Target: the element on which the action has to be performed.
  • Value: the value to be applied for the command. Some commands have no value.
  • Comment: a free text to describe the expected action.

Selenium IDE proposes several different commands, most of which are supported by Synthetic User Monitoring. Please refer to the complete list of Selenium commands supported by SUM for details.

To help with the script creation, the Selenium IDE proposes a recorder.
It also gives the possibility to play back the script.

When such a script is saved via the Selenium IDE, it generates a ".side" file.
This file is the JSON representation of the sequence of actions.

SUM identifies a SIDE file via both its Project name and the main ID (GUID) in the .side file.

If a SIDE file with the same project name and GUID is re-imported, SUM will not create a new resource entry. It will update the existing one.

Please note: Only the resource owner can update it.

During the import, the following changes are performed:

At Script level

  • Scripts no longer in the SIDE file are deleted. If SUM scenarios exist for the script, then the update is rejected. I.e. only scripts without scenarios can be removed.

  • New scripts in the SIDE file are added to the resource.
  • Modified scripts (e.g. change of the test suite name) are updated.

At Step level

  • Deleted steps are removed. Their threshold configuration is deleted. The corresponding monitoring metrics remain until deleted by the housekeeping.
  • New steps are created. Their threshold configuration is created with empty values and a disabled state.
  • Modified steps (e.g. change of the step name is the annotation) are updated.

Please note: If a step annotation is moved from one command to another - i.e. with the same step name - SUM considers that it is a new step. (Steps coming from an annotation are identified by the Script identifier plus the GUID of the Selenium command)

At SUM Variable level

  • Deleted variables are currently not removed.
  • New variables are added.
  • Changed variables values are updated.

SUM identifies a SIDE file via both its Project name and the main ID (GUID) in the .side file.

If when you re-import a SIDE file, you want to create a new resource - i.e. new Scripts - then you need to change one of the two above elements.

The easiest is to change the project name. For instance:

  1. Open the SIDE file in a Selenium IDE
  2. Click on the Project name field (at the top of the IDE UI) to change the name
  3. Save the project.
    By default, the proposed name for the .side file will be the new project name.
  4. Then, you can import this new SIDE file in SUM.
    This will create a new resource containing a copy of the scripts contained in the previously imported SIDE file. 

The scenario availability depends on the success of the  Selenium IDE commands during the scenario execution.

If a command fails during execution, the scenario availability will be rated as failure (Red).

A command failure can be caused by:

  • A command target not found preventing the command execution,
  • A failed assert or verify command,
  • An invalid evaluation of a property, ...

If the script defines steps, then SUM will also evaluate the availability at the step level.

For a given execution, the step availability depends on the success of the commands in the step.

Info: Generally, when a failure happens, the SUM scenario execution is interrupted.
Except for the verify commands. In this case, the scenario availability is rated as failed but the execution continues.

If the scenario script defines steps, then SUM will also evaluate the availability at step level. For a given scenario execution, the step availability depends on the success of the commands in the step.

The scenario performance depends on the response time (i.e. duration) of the scenario and scenario steps execution.

The response time is compared to a pair of thresholds: Green-to-Yellow and Yellow-to-Red.

For each scenario, performance thresholds can be defined at:

  • Scenario level,
  • Step level (for each scenario step).

For each execution, SUM provides performance metrics for:

  • The scenario level
  • The step level
  • The overall scenario (considers both the scenario level and the step level performance)

Scenario-level Performance

It considers the total response time of the execution (i.e. all the executed commands, in all the steps).

The scenario level performance is evaluated only if both:

  • The scenario thresholds are enabled in the scenario configuration
  • The scenario availability is successful

The scenario level performance is:

  • Red - If the total response time of the execution exceeds the Yellow-to-Red threshold.
  • Yellow - If the total response time of the execution exceeds the Green-to-Yellow threshold.
  • Green - if the total response time of the execution is below the Green-to-Yellow threshold.
  • Grey - if the scenario performance monitoring is not enabled
  • Blue - if the performance cannot be evaluated because the scenario availability has failed (red).

Step-level Performance

Considers the response time of the step execution (i.e. all the executed commands in the step).

For a given step, the step level performance is evaluated only if both:

  • The step thresholds are enabled in the scenario configuration
  • The step availability is successful

The step level performance is:

  • Red - If the step response time exceeds its Yellow-to-Red threshold.
  • Yellow - If the step response time exceeds its Green-to-Yellow threshold.
  • Green -If the step response time is below the Green-to-Yellow threshold.
  • Grey - if the step performance monitoring is not enabled
  • Blue - if the performance cannot be evaluated because the step availability has failed (red).

Overall Scenario Performance

For an execution of the scenario the overall performance will be:

  • Critical (Red), if at least one Yellow-to-Red threshold is crossed (for either the scenario level or one of the step level)
  • Poor (Yellow), if at least one Green-to-Yellow threshold is crossed and no Yellow-to-Red threshold is crossed.
  • Good (Green), if no Green-to-Yellow or Yellow-to-Red threshold is crossed.
  • Not Enabled (Grey), if all the scenario thresholds are disabled.
  • Not Rated (Blue), if the overall availability has failed.

Performance thresholds are dynamically calculated. You don't have to maintain them.
They are automatically defined by SUM, based on the historical executions of the scenario on the runner.

The dynamic thresholds follow 3 stages:

  • Not Available,
  • Learning,
  • Predictive.

Not Available

This stage corresponds to the period when SUM cannot determine the thresholds because there is no valid historical execution.

A scenario-on-runner is on this stage when:

  • It is first activated (or re-activated after a long period of inactivity).
  • It received a major update.
  • It suffered a very long succession of executions in unavailable status (executions with an unavailable status are not considered valid executions).
  • Its executions were interrupted because an associated Service/Systems suffered a long maintenance or disruption period.

It generally stay "Not Available" for one valid execution. Then the scenario-on runner enters in Learning stage.


Learning

This stage corresponds to the period when SUM doesn't have enough valid historical execution to have the "predictive" algorithm running: SUM is still learning.

During this intermediate period, the thresholds are computed from simple statistical formulae.
So the performance evaluation is less accurate. 

This period generally last 3 to 5 days depending on the scenario scheduling and the ratio of valid historical executions.


Predictive

This stage corresponds to the normal situation.
A scenario-on-robot is in Predictive stage when SUM has enough valid historical executions for it.
This means SUM can use its "predictive" algorithm to determine what should be its "normal" duration for the future execution.


Prediction calculation

The predictive algorithm is based on SAP HANA Predictive Analysis Library (SAP HANA PAL).

It is expected to detect the daily, and / or weekly seasonality, if existed.

For every scenario-on-runner in predictive stage, SUM tries predict what should be the normal executions duration for the next 3 days.

Every 6 hours, SUM re-evaluated the accuracy of the prediction. If it deviates to much, then it triggers a recalculation of the prediction.

The current stage is indicated in the Scenario configuration:

  1. Open the Configuration (wheel icon) tray
  2. Expand the Scenarios section.
  3. Click the pen icon to open the Configure Scenarios dialog.
  4. Access the corresponding Scenario Details by clicking on the line of the scenario.
  5. Select the Performance tab.
  6. In the Advanced section, select the corresponding runner.
    The Quality fields indicate the current stage for the scenario-on-runner and for each of its steps.In this section, SUM also indicates the current value for the dynamic thresholds.

How to know what is a scenario-on-runner current stage?

The current stage is indicated in the Scenario configuration:

  1. Open the Configuration (wheel icon) tray
  2. Expand the Scenarios section.
  3. Click the pen icon to open the Configure Scenarios dialog.
  4. Access the corresponding Scenario Details by clicking on the line of the scenario.
  5. Select the Performance tab.
  6. In the Advanced section, select the corresponding runner.
    The Quality fields indicate the current stage for the scenario-on-runner and for each of its steps.In this section, SUM also indicates the current value for the dynamic thresholds.

By default, SUM evaluates the performance of each scenario and each of its steps. However, you can disable the Performance evaluation:

  • Globally: for the scenario and all its steps,
  • Independently for the scenario-level or for the step-levels.

Globally

  1. Open the Configuration (wheel icon)
  2. Expand the Scenarios section.
  3. Click the pen icon to open the Configure Scenarios dialog.
  4. Access the corresponding Scenario Details by clicking on the line of the scenario.
  5. Select the Performance tab.
  6. Change the Global Performance Ratings field to Enabled or Disabled.

Independently

  • Open the Configuration (wheel icon)
  • Expand the Scenarios section.
  • Click the pen icon to open the Configure Scenarios dialog.
  • Access the corresponding Scenario Details by clicking on the line of the scenario.
  • Select the Performance tab.
  • Expand the Advanced section
  • In the Activation sub-tab, use the On/Off toggle to Enable/Disable the the Performance evaluation at Scenario-level or at Step-level.

By default, SUM executes the scenarios every 5 minutes (300 seconds).

You can configure the time interval at which a scenario is executed.

  1. Open the Configuration tray.
  2. Expand the Scenarios section.
  3. Click the pen icon to open the Configure Scenarios dialog.
  4. Access the corresponding Scenario Details by clicking on the line of the scenario.
  5. Select the Scheduling tab.
  6. Adjust the scheduling settings
    1. If the default value is currently used, uncheck the "Use default" flag.
    2. Change the "Execution Period".
      Note: To revert to the default value, check the "Use default" flag.
  7. Click the "Save" button to save your changes.

If you defined SUM variables in your Selenium test script SUM uses the default values defined in the script during its execution.

However, you can specify different values in the context of a scenario.

  1. Open the Configuration tray.
  2. Expand the Scenarios section.
  3. Click the pen icon to open the Configure Scenarios dialog.
  4. Access the corresponding Scenario Details by clicking on the line of the scenario.
  5. Select the Variables tab.
    Here you find all the variables defined by the script, along with their default value.
    If a wrench icon is displayed next to the Value, it indicates that the default value was overridden for the scenario.
  6. Enter the new value in the Value field.
    1. A green "+" icon indicates that the scenario previously used the default value.
    2. A wrench icon indicates that the scenario previously used a modified value.
  7. (optional) To revert a variable value to its default, check check the checkbox in front of the variable row and click the "Restore to default" button above the table.
    1. A red "-" icon indicates that the variable will be set back to its default value.
  8. Click the "Save" button to save your changes.

SUM allows you to adjust some properties related to the execution of the Selenium script associated with the scenario.
For instance, the web browser to use, the selenium IDE "execution speed", etc.

You can adjust the following properties:

PropertyDescription
Browser > BrowserThe web browser to use.
Currently, SUM supports Chrome (default choice) and Edge Chromium. 
Browser > Webdriver argumentsThe webdriver arguments to pass to the runner endpoint. (e.g., --headless, --incognito, etc.)
Execution > Execution speed

This is similar to the Selenium IDE "Test Execution Speed". It corresponds to a delay introduced after the execution of each command.
Possible values are:

  • Fastest (default) = zero delay
  • Fast = 0.5 seconds delay
  • Medium = 1.5 seconds delay
  • Slow = 2.5 seconds delay
  • Slowest = 3 seconds delay
Execution > Element Wait timeout [s]When a command has several targets, this is the maximum wait timeout for each of the targets.
Does not apply to the commands with an explicit timeout (e.g. the wait commands)
Execution > Global Wait timeout [s]Maximum time wait timeout to find the target of a command.
Does not apply to the commands with an explicit timeout (e.g. the wait commands)
Screenshots > Automatic screenshots

Possible options are:

  • None (default) = no automatic screenshot is taken.
  • At each step = a screenshot is automatically taken at the end of each step execution.
  • On error only = in case of execution error, a screenshot is taken when the error is encountered.
    A screenshot is also taken at the end of each step execution but, if the execution is successful, screenshots are not stored.
Screenshots > Enable explicit screenshotsWhen enabled, a screenshot is taken after each command with the @sap.sum.screenshot annotation.
Metrics > Active metrics

Enable or disable the capture of the browser-side metrics. Possible options are:

  • None (default) = no browser-side metric is captured.
  • network:navigation = capture the URL, size, round-trip duration of the navigation requests emitted by the browser.
  • network:resources = capture the URL, size, round-trip duration of the resource requests emitted by the browser.

  1. Open the SUM Configuration tray.
  2. Expand the Scenarios section.
  3. Click the pen icon to open the Configure Scenarios dialog.
  4. Access the corresponding Scenario Details by clicking on the line of the scenario
  5. Select the Selenium tab.
  6. For the properties you want to change:
    1. If the default value is currently used, uncheck the "Use default" flag.
    2. Change the property value.
    3. To revert to the default value, check the "Use default" flag.
  7. Click the "Save" button to save your changes.

With SUM Business Services, you can group your scenarios monitoring a same application.

You can also indicate which LMS Cloud Service (or System) is involved in the monitored application.

Last but not least, the events SUM can send are configured at business service level.

  1. Open the Configuration tray.
  2. Expand the Business Services section.
  3. Click the pen icon to open the Configure Business Services dialog.
  4. Click the "Add" button on the top of the table.
  5. In the General section, enter a meaningful Name and Description for your Business Service.
  6. In the Services / Systems section,
    1. Use the "Search" field to find the LMS objects to associate to your Business Service.
    2. Check them to add them in the "Selected Services / Systems" list.
  7. Click the "Add" button.

  1. Open the Configuration tray.
  2. Expand the Business Services section.
  3. Click the pen icon to open the Configure Business Services dialog.
  4. Access the corresponding Business Service Details by clicking on the line of the business service.
  5. Select the General tab.
  6. To assign a scenario:
    1. Click the "Assign" button above the Assigned Scenarios table.
      This opens the Assign Scenarios popup.
    2. Check the row corresponding to the scenario to add
      This adds the object to the Selected Scenarios field. 
    3. Click the "Assign" button.
  7. To unassign a scenario:
    1. Check the row corresponding to the scenario to unassign in the Assigned Scenarios table.
    2. Click the "Unassign" button above the table.

    3. Confirm the unassign.

Please note:

A scenario must be associated to at least one business service.
If this is the only assignment of the scenario, SUM will reject the unassign.
To unassign it, it will be required to first assign it to another business service (for instance, the "Default SUM Business Service").

  1. Open the Configuration tray.
  2. Expand the Business Services section.
  3. Click the pen icon to open the Configure Business Services dialog.
  4. Access the corresponding Business Service Details by clicking on the line of the business service.
  5. Select the Services/Systems tab.
  6. To assign an LMS cloud service/system:
    1. Click the "Assign" button above the Assigned Services / Systems table.
      This opens the Assign Services / Systems popup.
    2. Check the row corresponding to the LMS object to add.
      This adds the object to the Selected Services / Systems field.
    3. Click the "Assign" button.
  7. To unassign an LMS cloud service/system:
    1. Check the row corresponding to the LMS object to unassign in the Assigned Services / Systems table.
    2. Click the "Unassign" button above the table.
    3. Confirm the unassign.

Please note: 

A business service must be associated to at least one LMS cloud service or system. If this is the last assignment, SUM will reject the unassign.

To be able to delete an LMS Cloud Service (or an LMS System), it has to be unassigned from any SUM Scenario.

SUM scenarios are assigned to LMS object via SUM Business Services.
Therefore, to dissociate SUM scenarios from an LMS object, you can:

  1. Access the Cloud Service (or System) in LMS.
  2. In the "Supported Use Cases" section, check the "Configuration Status" for Synthetic User Monitoring.
  3. If it is "Active" or "Inactive", click on the glasses icon to display the list of associated Scenarios-on-Runners.
    (If the status is "Not Configured", no SUM Scenario is associated. There is nothing to do).
  4. In SUM, open the Configuration (icon) and expand the Scenarios section.
  5. Click the pen icon () to open the Configure Scenarios dialog.
  6. Access the corresponding Scenario Details by clicking on the line of the scenario.
  7. Select the Business Services / Events tab.
    This shows all the Business Services associated to this scenario.
  8. Jump to the Business Service Details by clicking on the Business Service.
  9. In the Service / Systems tab, if the Cloud Service / System to dissociate is listed:
    1. Check the corresponding row.
    2. Click the "Unassign" button above the table.
  10. Repeat the operations for all the scenarios listed in LMS.

If your Runner infrastructure requires some specific webdriver capabilities, you can define them in the Advanced section of the Runner endpoint configuration. Each capability is composed of a name and a value. For the value, SUM supports the following types: 

  • Boolean,
  • Integer,
  • String,
  • Secured: String capability but the value is stored by SUM in a secured way.
  • JSON: JSON string that SUM will convert in a JSON object.

To set a new capability:

  1. Open the Configuration (wheel icon) 
  2. Expand the Runners section.
  3. Click the pen icon to open the Configure Runners dialog.
  4. Access the corresponding Runners Details by clicking on the line of the runner.
  5. Select the Endpoint tab and expand the Advanced section.
  6. In the Custom WebDriver Capabilities sub-section, click the Add link.
    This opens the Add Capability popup.
  7. Indicate:
    1. The capability name,
    2. The capability value type,
    3. The capability value.
  8. Click the "Add" button to save your changes.

In the SUM configuration, you can test that your Runner infrastructure is properly set up: can be accessed by SUM and can reach the URL you want to monitor. For that:

  1. Open the Configuration (wheel icon)
  2. Expand the Runners section.
  3. Click the pen icon to open the Configure Runners dialog.
  4. Access the corresponding Runners Details by clicking on the line of the runner.
  5. Select the Endpoint tab and click the Test Connection button in the Connection Check section.

This will trigger a new test.

The result is visible in the Runner Details header and is represented by a chain of a 4 icons where each icon corresponds to one step of the test.
Steps are successively testing that:

  • SAP Cloud ALM Check: the SUM infrastructure is up-and-running
  • Runner Check: the runner definition can be accessed
  • WebDriver Check: the infrastructure WebDriver can be accessed and instantiated for the selected Browser
  • URL Check: the runner infrastructure can access the Checked URL

In the configuration, you can indicate:

Property Description
Connection Check → Browser
The web browser to use for the connection test.
Here you should indicate the most relevant Browser for your runner.
Connection Check → Checked URL
The URL to try to access during the connection test (step 4 - URL Check).
Here you should typically indicate the entry URL of one of your Scenario to ensure that the runner will be able to access it.
 
Advanced → Initial Connection Timeout
The maximum time (in second), waited by SUM when it tries to initiate a new session on the runner infrastructure.
This property is used both for the Connection Test and at runtime when a scenario is executed.
The minimum value is 5 seconds.The maximum value is 60 seconds.
 


To change a property:

  1. For the properties you want to change:
    1. If the default value is currently used, uncheck the "Use default" flag.
    2. Change the property value.
    3. To revert to the default value, check the "Use default" flag.

  2. Click the "Save" button to save your changes.

Customer Support

In case you encounter problems using or setting up Real User Monitoring in SAP Cloud ALM, please report a case using one of the following components:

  • SV-CLM-OP-SUM - Synthetic User Monitoring

Please select your SAP Cloud ALM tenant when reporting the case.