Synthetic User Monitoring - Selenium IDE Scripts

How you can contact us:

Technical Assistance

Request product support from SAP
Non-Technical Assistance

Request non-product support or provide feedback on SAP Support Portal site

Creating Synthetic User Monitoring Scripts

A Synthetic User Monitoring (SUM) Selenium test is a Selenium IDE (SIDE) script that is potentially enriched with SUM-specific annotations.

This page does not intend to explain in detail how to create a Selenium IDE test. (Plenty of documentation can be found on the internet about the ways to create an efficient Selenium IDE test.)

It intends to explain how such a test is integrated with the SUM data model.

Create Selenium Test Scripts for Synthetic User Monitoring

The easiest method of creating a SUM Selenium test scripts is:

Use Selenium IDE to record the main sequence of your scenario. The IDE is a browser plug-in available for numerous different browsers. You can find the documentation for the Selenium IDE here.
In Selenium IDE, replay the created script and make the necessary adjustments. For instance:
- Adjust the "Target" fields of commands to ensure that they properly identify the targeted elements.
- Add "wait for xxx" commands to wait for the UI rendering.
- Add "assert xxx" and "verify xxx" commands to ensure that the monitored application displays the expected information.
Make sure your Selenium IDE script only contains commands that are supported by SUM. You find the full list at the end of this page.
When your Selenium IDE script is ready, add the SUM-specific annotations (as described in the next sections).
For instance:
- Add SUM secured variable annotations to store sensitive command values (e.g. a password) in SUM secured variables.
- Add SUM variable annotations to store command values (not sensitive) in SUM variables.
- Add SUM step annotations to group script commands in SUM steps.
For each command with a sensitive value (e.g. password):
1. Replace the values with a dummy text (e.g. "dummy").
2. Make sure that you added a SUM secured variable. At execution time, SUM will replace the command value with the value of the SUM secured variable.
Save your Selenium IDE script. This will create a SIDE file that you can import into SUM to create the SUM Selenium script.

Please note: Websites containing Shadow DOM structures are not supported by the Selenium IDE. Since we need the Selenium IDE to create a SIDE file for SUM, we also do not support the monitoring of websites containing Shadow DOM structures.

Define SUM Variables in Selenium Tests

When you create a Selenium script, you may want to have some Selenium commands value/target set from SUM variables. This is typically the case for user names or sensitive data like passwords.

For that, you can add a SUM variable annotation in the "Description" field of the Selenium command.

Such a description is ignored by Selenium. However, when the SIDE file is imported into SUM, the content is parsed and the annotations are detected.

Syntax

SUM supports four types of variable annotations:

@sap.sum.variable=<variable_name> – to have the command value field replaced by the specified SUM standard variable value.
@sap.sum.secure.variable=<variable_name> – to have the command value field replaced by the specified SUM secured variable value.
@sap.sum.variable.target=<variable_name> – to have the command target field replaced by the specified SUM standard variable value.
@sap.sum.secure.variable.target=<variable_name> – to have the command target field replaced by the specified SUM secured variable value.

The following character can be used for variable names:

Alphanumeric ([a-z], [A-Z], [0-9])
Underscore ('_')

Behavior during SIDE file import

During the SIDE file import, SUM will create the corresponding variables in the SUM script.

The created variable will be of the following types:

String for the @sap.sum.variable and @sap.sum.variable.target annotations.
Secured for the @sap.sum.secure.variable and @sap.sum.secure.variable.target annotations.

The SUM variable default value is the value of the "value" field of the Selenium command in Selenium IDE.

IMPORTANT: For the secured variables, do not keep the sensitive value in the command value field. (It would be stored in clear.) Replace it with a dummy value.

If the same variable annotation is declared several times, only one variable is created. Its default value are those of the last declaration. You cannot use the same variable name for both a standard and a secured variable. If your SIDE contains such a situation, its import will fail.

If the Selenium command is disabled, the annotation is ignored.

Please note: If the SIDE file is re-imported and a variable annotation is changed or removed, the SUM script will be updated but the deprecated variable will not be removed from the Script variables.

Behavior during the SUM Scenario Configuration

When a scenario is created for the SUM script, it will inherit all the variables of the script. These variables are listed in the SUM Configuration, in the Variable section of the scenario details.

IMPORTANT: It is required to set the value of the secured variables. The default value should not be the correct one.

Behavior at Runtime

If a command has a variable annotation, SUM replaces its value with the value of the SUM variable before the command execution.

You can change the variable value during the scenario configuration. This is described on page Synthetic User Monitoring - Setup & Configuration

Add SUM Steps to Selenium Tests

Motivation

SUM steps are a concept that only exists in the SUM scripts but not in the Selenium test script. The reason to create SUM steps is to gain more insight into the execution and performance of your SUM script.

SUM captures its monitoring metrics at the step level.

By defining steps, you can "group" the Selenium test script commands that are related to the same activity. This way, you can better evaluate which activity creates a problem.

For instance, imagine a monitoring script executing the following commands:

Open an application URL
Wait for the UI to display the logon screen
Enter a user credentials to log in the application
Wait for the UI to load and display a list of elements
Use a search functionality to find something
Wait for the search to return a result
Click the search result
Wait for the UI to display the result details
Logout
Wait for the confirmation that the session is now closed

With SUM steps you can group these commands in logical steps:

Open Website = grouping all the Selenium commands 1 and 2
Logon = grouping the Selenium commands 3 and 4
Search = grouping the Selenium commands 5 and 6
Show Details = grouping the Selenium commands 7 and 8
Logout = grouping the Selenium commands 9 and 10

Create SUM Steps

There are two ways to define SUM steps in a Selenium test script:

By adding a Selenium test in the Selenium test suite
By adding the SUM step annotation in the description field of a command in a Selenium test script

Adding SUM Steps as Selenium Tests in a Test Suite

During the SIDE import, a SUM step is created for each test added to the test suite.

The step starts with the first command of the test and ends with the first of the following events:

A SUM step annotation creates a new step.
The end of the Selenium test.

The created step name is the name of the test.

The created step is identified by the identifier of the test script plus the GUID of the Selenium test.

Adding SUM Steps using the SUM step annotation

To explicitly define a new step, you can add a SUM step annotation in the "Description" field of a Selenium command.

Such a description is ignored by Selenium. However, when the SIDE file is imported into SUM, the content is parsed and the annotations are detected.

If the Selenium command is disabled, the annotation is ignored.
If the Selenium command is in a nested test (a test called via the "run" command), the annotation is ignored.
If the Selenium command is in a conditional block (IF, ELSEIF, ELSE), the annotation is ignored.

Syntax

@sap.sum.step=<step_name>
@sap.sum.step="<step name with space>"

You can use the following characters for the step name:

Alphanumeric ([a-z], [A-Z], [0-9])
Underscore ('_')
Space if name is enclosed with " "

Behavior

The step starts with the command containing the step annotation.

It ends with the first of the following events:

A SUM step annotation creates a new step.
The end of the test.

The created step is identified by the identifier of the test script plus the GUID of the Selenium command with the step annotation.

Add an Explicit Screenshot

When you create a Selenium script, you may want to instruct SUM to take a screenshot after a specific Selenium command. This is an "explicit screenshot".

For that, you can add a SUM screenshot annotation in the "Description" field of the Selenium command.

Such a description is ignored by Selenium.

Syntax

@sap.sum.screenshot

This annotation does not take parameter.

Behavior during the import

During the SIDE file import, the screenshot annotations have no impact.

Behavior during the SUM Scenario configuration

When a scenario is created for the SUM script, the screenshot annotations have no direct impact. However, in the "Selenium" section of the created scenario, a dedicated property gives the possibility to enable/disable the explicit screenshots capture.

Indicate a Sensitive Scenario Command

When you create a Selenium script, some commands may perform sensitive actions for which you want to ensure that SUM:

Never takes a screenshot of the monitored UI
Never displays the command "target" and "value"

Even in case of Scenario execution error.

For that you can add the SUM sensitive annotation in the "Description" field of the Selenium command.

Syntax

@sap.sum.sensitive.action

This annotation does not take parameter.

Behavior during the import

During the SIDE file import, the sensitive annotations have no impact.

Behavior During the SUM Scenario Configuration

When a scenario is created for the SUM script, the sensitive annotations have no impact.

Behavior at runtime

When SUM executes a Selenium command with the sensitive annotation:

No screenshot is taken after the command, e.g.
- With the scenario configured to automatically take screenshots on error, if the command fails, no automatic screenshot will be taken for the command.
- With the scenario configured to automatically take screenshots after each step, if the command is the last of the step, no screenshot will be taken at the end of the step.
The command "target" and "Value" are obfuscated in the execution results. E.g. if the command execution fails, the SUM UI will not indicate the "target" or the "value" of the command

Best Practices for Synthetic User Monitoring Script Creation

Here is a set of recommendations to keep in mind when you design an SAP Cloud ALM – Synthetic User Monitoring (SUM) scenario.

Synthetic User Monitoring is a monitoring application

SUM is not intended for application testing or for task automation.

Keep your scenario simple

Complex scenarios are costly to create and maintain. It is better to have 2–3 simple scenarios rather than 1 complex one.

Do not create "disruptive" scenarios

The execution of the SUM scenario should not negatively impact the monitored application.

Try to anticipate what could be the impact of not fully executing a scenario through to the end (e.g. in the event of availability issues).

For instance, monitoring a procurement application, a scenario booking actual items, and saving a booking could jeopardize the production system.

What happens if the scenario ends unexpectedly, leaving a fake booking record?
What happens if the scenario is executed too often, creating a shortage of booked items for the actual consumers?

For such monitoring, we recommend that you manipulate fake elements.

Do not manipulate actual user data

The SUM infrastructure is not designed to manipulate sensitive or personal user information within the created scenarios.

For instance, when you monitor a sales application, you should not navigate through real sales operations. Especially if this implies navigating through personal or sensitive information related to the sales operations (such as names, credit card numbers, or account balances).

Do use monitoring users

If the scenario requires authentication when accessing the application to be monitored, do not connect with an actual end-user account.

We strongly recommend that you use specifically created monitoring users with no associated personal or sensitive data.

Selenium Commands Supported by Synthetic User Monitoring

Selection Commands

Command Name	Description
add selection	Adds a selection to the set of options in a multi-select element.
select frame	Selects a frame within the current window.
select window	Selects a popup window using a window locator. Once a pop-up window has been selected, all commands will go to that window.
remove selection	Remove a selection from the set of selected options in a multi-selected element using an option locator.
select	Select an element from a drop-down menu using an option locator. Option locators provide different ways of specifying a select element. If no option locator prefix is provided, a match on the label will be attempted.

Assert / verify / check Commands

Command Name	Description
assert alert	Confirm that an alert has been rendered with the provided text. The test will stop if the assert fails.
assert	Check that a variable is an expected value. The variable's value will be converted to a string for comparison. The test will stop if the assert fails.
assert checked	Confirm that the target element has been checked. The test will stop if the assert fails.
assert confirmation	Confirm that a confirmation has been rendered. The test will stop if the assert fails.
assert editable	Confirm that the target element is editable. The test will stop if the assert fails.
assert element present	Confirm that the target element is present somewhere on the page. The test will stop if the assert fails.
assert element not present	Confirm that the target element is not present somewhere on the page. The test will stop if the assert fails.
assert not checked	Confirm that the target element has not been checked. The test will stop if the assert fails.
assert not editable	Confirm that the target element is not editable. The test will stop if the assert fails.
assert not selected value	Confirm that the value attribute of the selected option in a dropdown element does not contain the provided value. The test will stop if the assert fails.
assert not text	Confirm that the text of an element does not contain the provided value. The test will stop if the assert fails.
assert selected value	Confirm that the value attribute of the selected option in a dropdown element contains the provided value. The test will stop if the assert fails.
assert selected label	Confirm that the label attribute of the selected option in a dropdown element contains the provided value. The test will stop if the assert fails.
assert text	Confirm that the text of an element contains the provided value. The test will stop if the assert fails.
assert value	Confirm the value of an input field (or anything else with a value parameter). For checkbox/radio elements, the value will be "on" or "off" depending on whether the element is checked or not. The test will stop if the assert fails.
verify	Soft assert that a variable is an expected value. The variable's value will be converted to a string for comparison. The test will continue even if the verify fails.
verify checked	Soft asserts that a toggle button has been checked. The test will continue even if the verify fails.
verify editable	Soft assert whether the specified input element is editable. The test will continue even if the verify fails.
verify element present	Soft asserts that the specified element is somewhere on the page. The test will continue even if the verify fails.
verify element not present	Soft asserts that the specified element is not somewhere on the page. The test will continue even if the verify fails.
verify not checked	Confirm that the target element has not been checked. The test will stop if the assert fails.
verify not editable	Soft assert whether the specified input element is not editable. The test will continue even if the verify fails.
verify not selected value	Soft asserts that the expected element has not been chosen in a select menu by its option attribute. The test will continue even if the verify fails.
verify not text	Soft asserts the text of an element is not present. The test will continue even if the verify fails.
verify selected label	Soft assert the visible text for a selected option in the specified select element. The test will continue even if the verify fails.
verify selected value	Soft asserts that the expected element has been chosen in a select menu by its option attribute. The test will continue even if the verify fails.
verify text	Soft assert the text of an element is present. The test will continue even if the verify fails.
verify title	Soft asserts the title of the current page contains the provided text. The test will continue even if the verify fails.
verify value	Soft assert the value of an input field. For checkbox/radio elements the value will be "on" or "off" depending on whether the element is checked or not. The test will continue even if the verify fails.
assert prompt	Confirm that a JavaScript prompt has been rendered. The test will stop if the assert fails.
assert title	Confirm the title of the current page contains the provided text. The test will stop if the assert fails.

Click Commands

Command Name	Description
click	Click on a target element.
check	Check a toggle-button (checkbox/radio).
uncheck	Uncheck a toggle-button (checkbox/radio).

Storage Commands

Command Name	Description
store	Save a target string as a variable for easy re-use.
store text	Gets the text of an element and stores it for later use. This works for any element that contains text.
store window handle	Store the handle of the current page.
store value	Store the value of an element.
store title	Store the title of the current page.
store attribute	Store and get the value of an element attribute.
store xpath count	Store the number of nodes that match the specified xpath.

Wait Commands

Command Name	Description
wait for element present	Wait for a target element to be present on the page.
wait for element editable	Wait for an element to be editable.
wait for element not editable	Wait for an element to not be editable.
wait for element not present	Wait for a target element to be not present on the page.
wait for element not visible	Wait for a target element to not be visible on the page.
wait for element visible	Wait for a target element to be visible on the page.
wait for text	Wait for a target text to be visible on a specified element on the page.
pause	Wait for the specified amount of time.
set speed	Set execution speed. By default, there is no such delay, e.g., the delay is 0 milliseconds. This setting is global and will affect all test runs until changed.

Actions Commands

Command Name	Description
open	Opens a URL and waits for the page to load before proceeding. This accept both absolute and relative URLs.
send keys	Simulates keystroke events on the specified element, as though you typed the value key-by-key. This simulates a real user typing every character in the specified string; it is also bound by the limitations of a real user, like not being able to type into a invisible or read only elements.
type	Sets the value of an input field, as though you typed it in. Can also be used to set the value of combo boxes, check boxes, etc. In these cases, value should be the value of the option selected, not the visible text.
set window size	Set the browser's window size, including the browser's interface.
run	Run a test case from the current project.
run script	Creates a new "script" tag in the body of the current test window, and adds the specified text into the body of the command. Beware that JS exceptions thrown in these script tags aren't managed by Selenium, so you should probably wrap your script in try/catch blocks if there is any chance that the script will throw an exception.
execute script	Executes a snippet of JavaScript in the context of the currently selected frame or window. The script fragment will be executed as the body of an anonymous function. To store the return value, use the 'return' keyword and provide a variable name in the value input field.
execute async script	Executes an async snippet of JavaScript in the context of the currently selected frame or window. The script fragment will be executed as the body of an anonymous function and must return a Promise. The Promise result will be saved on the variable if you use the 'return' keyword.
edit content	Sets the value of a content editable element as if you typed in it.
echo	Prints the specified message into the third table cell in your Selenese tables.
choose cancel on next confirmation	Affects the next confirmation alert. This command will cancel it. If the alert is already present, then use "webdriver choose cancel on visible confirmation" instead.
choose cancel on next prompt	Affects the next alert prompt. This command will cancel it. If the alert is already present, then use "webdriver choose cancel on visible prompt" instead.
choose ok on next confirmation	Affects the next confirmation alert. This command will accept it. If the alert is already present, then use "webdriver choose ok on visible confirmation" instead.
webdriver answer on visible prompt	Affects a currently showing alert prompt. This command instructs Selenium to provide the specified answer to it. If the alert has not appeared yet then use "answer on next prompt" instead.
webdriver choose cancel on visible confirmation	Affects a currently showing confirmation alert. This command instructs Selenium to cancel it. If the alert has not appeared yet then use "choose cancel on next confirmation" instead.
webdriver choose cancel on visible prompt	Affects a currently showing alert prompt. This command instructs Selenium to cancel it. If the alert has not appeared yet then use "choose cancel on next prompt" instead.
webdriver choose ok on visible confirmation	Affects a currently showing confirmation alert. This command instructs Selenium to accept it. If the alert has not appeared yet then use "choose ok on next confirmation" instead.
answer on next prompt	Affects the next alert prompt. This command will send the specified answer string to it. If the alert is already present, then use "webdriver answer on visible prompt" instead.

Control-flow Commands

Command Name	Description
if	This is the opening command for a conditional block. Along with it, you provide a JavaScript expression you would like to evaluate. This can consist of variables created from prior JavaScript expressions in your test. This all goes in the target input field of the if command. If the expression evaluates to true then the test will execute the commands that follow it up until the next conditional control flow command (e.g., else if, else, or end). If the expression evaluates to false it will skip the commands that follow and jump to the next relevant conditional control flow command (e.g., else if, else, or end).
elseIf	This command is used within an if command block. Just like with if it takes a JavaScript expression in the target input field to evaluate, executing either the command branch that follows it, or skips to the next relevant control flow command (e.g., else or end).
else	Else is the final condition you can have in an if block. When none of the prior conditions are met, this command branch will be executed. After it's done it will jump to the end command.
end	This command terminates the conditional command block. Without it the command block is incomplete and you'll receive a helpful error message letting you know when trying to run your test.