OpenTelemetry@SAP

The below image summarizes the observability with OpenTelemetry@SAP through SAP Cloud ALM for operations.

SAP built SAP SaaS applications as well as customer built SAP PaaS applications are instrumented with the same Data Collection Instrumentation Libraries, which produce monitoring data based on standard OpenTelemetry. Furthermore, SAP has defined certain OpenTelemtry extensions.

The data from SAP built applications is transferred by the Data Collector Runtime service (which is completely managed by SAP) to SAP Cloud ALM. The data from customer built applications is transferred directly to SAP Cloud ALM, where the Queueing is taking care to store the data in the SAP Cloud ALM database. In SAP Cloud ALM you have to switch-on the monitoring for the respective service per use case.

As OpenTelemetry follows an open standard inbound API's can be used to integrate non-SAP data providers with SAP Cloud ALM. OpenTelemetry enabled Outbound API's can be used to integrate non-SAP data consumers. Please check the following link to find more information about these SAP Cloud ALM Raw data APIs.

Additionally, it is possible to also transfer the data to a SAP Focused Run system, in case the customer himself or its service provider has a SAP Focused Run system in use.


SAP Cloud ALM Applications that can Monitor Custom-built App

The following applications are able to consume data provided by OpenTelemetry@SAP. 

  • Real User Monitoring (supported for Java and Node.js)
  • Health Monitoring (supported for Java and Node.js)
  • Exception Monitoring (supported for Java and Node.js)
  • Job & Automation Monitoring (SAP Job Scheduler Service jobs in Java and Node.js)


Supported Environments

  • SAP BTP, Cloud Foundry Environment
  • SAP BTP, Kyma runtime
     

Why using OpenTelemetry?

  • OpenTelemetry is an open-source observability framework with a collection of tools, APIs, and SDKs.
  • OpenTelemetry is the new industry standard for adding observable instrumentation to cloud-native applications both inside and outside SAP.
  • OpenTelemetry enables us to instrument, generate, collect, and export telemetry data for analysis and to understand the application's performance and behavior.
  • OpenTelemetry is vendor-agnostic instrumentation library (available per language) with support for both automatic and manual instrumentation.
  • OpenTelemetry makes observability effortless, inexpensive, and insightful.

 

High Level Architecture

In this documentation we focus on, how you can instrument your custom-built application in SAP BTP, Cloud Foundry environment and SAP BTP, Kyma runtime with our Data Collection Instrumentation Libraries to monitor this in SAP Cloud ALM.

The graphic below depicts the high-level architecture of the OpenTelemetry data collection. In this simple scenario, we assume, that we want to monitor a full-stack application built on SAP BTP, Cloud Foundry environment or in SAP BTP, Kyma runtime. The front end consists of SAP UI5. The UI requests will be routed to the application backend via the Application Router. Here in our example, the application backend consists of three different microservices.

The numbers in the picture correspond to the sections described below.

Not all steps are required for all use cases.

  • If you want to get the end-user response times to be displayed in Real User Monitoring you need to go through all steps. 
  • If you only want to get data from your back-end services for Health Monitoring or Integration & Exception Monitoring you can skip step 1 and can start with step 2.  

Prerequisites

The cloud application to be monitored may consist of several microservices and run on SAP BTP, Cloud Foundry environment and SAP BTP Kyma runtime. In order to execute the steps described below you need some development knowledge and deployment authorization in SAP BTP, Cloud Foundry environment and SAP BTP Kyma runtime.

Configure Settings with Download Link

The OTEL ALM Libraries are shipped by SAP Repository Based Shipment Channel (RSBC). Request the technical user name and password in the RBSC UI. Please also check the corresponding documentation for User Authentication, for a full documentation of the RBSC please see the RBSC documentation.

Java

Required Java version:

  • Runtime needs to be Java 11 or Java 17

Required library versions:

  • otel-agent-ext-java: Version 1.5.20
  • calm-crun-client-api-java*: Version 1.5.22
  • fesr-to-otel-java: Version 1.5.20 (for Spring Boot 2)
  • fesr-to-otel-java: Version 2.0.16 (for Spring Boot 3)

* only relevant for Integration Monitoring

For complete version history, please check the following Version Update for Data Collection Instrumentation Libraries (SAP Help).

For the Java OTEL ALM Libraries, an integration in the SAP CF Java build pack is available. In that case, no integration into the Maven build is required as the libraries are already included in the build pack itself. If you do not use the Java build pack, you have to include the library as a Maven dependency:

  • Add artifactory as repository in pom.xml

<repositories>

<repository>

<id>***your-artifactory-id***</id>

<url>https://73555000100200018064.mavensrv.base.repositories.cloud.sap</url>

</repository>

</repositories>

  • Update settings.xml pointing to technical user credentials for the artifactory

    Note - It is recommended to store the password in a vault and reference it in the settings, so that it can be read during maven build and pipeline deployment!

<?xml version="1.0" encoding="UTF-8"?>

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">

<servers>

<server>

<id>**your-artifactory-id**</id>

<username>***technical-user-name read from vault***</username>

<password>***password read from vault***</password>

</server>

</servers>

</settings>

Node.js or CAP (JavaScript) based

Required library versions:

  • xotel-agent-ext-js: Version 1.5.20
  • fesr-to-otel-js: Version 1.5.11

Additionally for Exception Monitoring:

  • cf-nodejs-logging-support: Version 7.4.0-beta.4
  • @sap/opentelemetry-exporter-for-sap-cloud-logging: Version 0.1.1

For complete version history, please check the following Version Update for Data Collection Instrumentation Libraries (SAP Help).

In your .npmrc file, specify the credentials for the repository containing the OTEL ALM Libraries:

//73555000100200018064.npmsrv.cdn.repositories.cloud.sap/:_auth=<base64 encoded version of username:password>

The value for the _auth property is Base64 encoding of the combination of 'username:password' where the user name is the user name of the technical user. Respectively, the same holds for password.

Note - It is recommended to store the password in a vault and reference it in the settings, so that it can be read during build and pipeline deployment!

Implement Observability in a Full-Stack CAP Application Following SAP BTP Developer's Guide

You can find a node.js implementation sample based on our guides as part of implementing Observability in a Full-Stack CAP Application following this SAP BTP Developer's Guide in Github.

Step 1: Enabling Frontend Data Collection (only for Real User Monitoring)

You only need to perform the steps described in this section if you want to get the end-user response times to be displayed in Real User Monitoring. 

As of now, the only offered frontend measurements come from SAP UI5. For this, we leverage the Front End Subrecords measurement provided by the UI5 framework. Currently, we only support "standalone" UI5 applications for front-end measurements.

To instrument your own application with Frontend measuring there are three steps:

  • Configure your UI to gather frontend statistics and send them to a receiving endpoint
  • Configure your reverse proxy (e.g. approuter) to forward the FESR beacons from UI5
  • Provide an FESR receiver endpoint that will transfer the data to SAP Cloud ALM


1a Configure UI5

The first point can be achieved with a simple meta tag in your root index.html of your UI5 application. We will send the beacon to the /fesr endpoint (Note: depending on your project and routing setup this may contain an application specific prefix like in this example):

<meta name="sap-ui-fesr" content="/<optional app-specific-prefix>/fesr">

Here, the prefix is /<optional app-specific-prefix> is an application specific prefix example. It may or may not be necessary in your case. The endpoint /fesr is mandatory.


1b Configure Application Router (Reverse Proxy)

Since your UI application will likely use a reverse proxy to communicate with your backend you need to configure the /fesr route to be routed to the specific endpoint which will be configured in the next step. Usually, this is done by a central approuter. You must configure your central approuter to route the /fesr routes:

xs-app.json 
{

  "routes": [

    {

      "source": "^/fesr$",

      "target": "/fesr",

      "destination": "<destination pointing to your fesr receiver backend>",

      "csrfProtection": false,

      "authenticationType": "xsuaa"

    }

  ]

}

Note that csrfProtection must be set to false.


1c Instrument Receiver with FESR Receiver Library and DCI

Now, that UI5 is sending the FESR beacons to /fesr they need to be received and transmitted to SAP Cloud ALM. For this, you need to include both the FESR receiver library as well as the OpenTelemetry data collection instrumentation:

FESR Receiver

At least one Java (Spring Boot) application or one Node.js (express) application must exist acting as a handler for Front-end Statistical Records (FESR).

Step 2: Enabling Backend Data Collection

To configure the data collection instrumentation for backend services you only need the respective autoinstrumentation:

Step 3: Authentication

In order to export towards SAP Cloud ALM you have to setup the destination via OAuth based authentication.

  • Bind to Destination Service
  • Add the below lines to mta.yaml

Service Binding (for SAP BTP, Cloud Foundry environment)

mta.yaml 
modules:

- name: <your application srv module>

  requires:

   - name: <destination-service-name>

resources:

 - name: <destination-service-name>

   parameters:

     service-plan: lite

     service: destination

   type: org.cloudfoundry.managed-service


Service Binding (for SAP BTP, Kyma runtime)

  • Create destination service instance

apiVersion: services.cloud.sap.com/v1alpha1

kind: ServiceInstance

metadata:

  name: <destination-name>

spec:

  serviceOfferingName: destination

  servicePlanName: lite

  • Create destination service binding

apiVersion: services.cloud.sap.com/v1alpha1

kind: ServiceBinding

metadata:

  name: <destination-name>

spec:

  serviceInstanceName: <destination-name>

  • Configure K8S Deployment YAML to use Service Binding
  • Service Binding and KYMA Namespace must be added to the environment

apiVersion: apps/v1

kind: Deployment

spec:

  template:

    spec:

      containers:

        - env:

            - name: KYMA_NAMESPACE

              valueFrom:

                fieldRef:

                  fieldPath: metadata.namespace              

            - name: SERVICE_BINDING_ROOT

              value: /bindings              

          volumeMounts:

          - mountPath: /bindings/destination

            name: <destination-name>

            readOnly: true          

          ports:

            - containerPort: <port>

          resources: {}

      restartPolicy: Always

      volumes:

      - name: <destination-name>

        secret:

          defaultMode: 420

          secretName: <destination-name>


Configure Destination at the Subaccount

  • Create a destination configuration using Destination Service for persisting the base URL and retrieving it.
  • Destination Configuration fields should be maintained as below:
    • Name : CALM_datacollector
    • Type : HTTP
    • URL : The Destination endpoint should be your SAP Cloud ALM endpoint "Api" URL taken from the API service key. Details on how to retrieve the API service key, can be found here.
      E.g. https://eu10.alm.cloud.sap/api
    • Proxy Type: Internet
    • Authentication Type: OAuth2ClientCredentials
    • Client ID : Enter from API service key
    • Client Secret : Enter from API service key
    • Token Service URL: API service key parameter "url"
  • Please note:
    • The Destination must be named as 'CALM_datacollector'.
    • The Destination must be created at provider account level
    • SAP Cloud ALM data push endpoints will be formed dynamically by retrieving the base URL from Destination.

The destination should look like this:

When clicking on button Check Connection you might receive http status code 404.

This concludes the configuration in your SAP BTP, afterwards you need to activate the respective use cases (see also next section).

Activate the Use Cases

After you have done the instrumentation as described above, you may do the final activation in the respective use cases:

Customer Support

In case you encounter problems using or setting up the Next Generation – Data Collection Infrastructure in SAP Cloud ALM, please report a case for a scenario using the following components:

  • SV-CLM-OP-RUM for Real User Monitoring
  • SV-CLM-OP-HM for Health Monitoring
  • SV-CLM-OP-EXM for Exception Monitoring
  • SV-CLM-OP-JM for Job & Automation Monitoring

Please select your SAP Cloud ALM tenant when reporting the case.