SAP Cloud Integration for Data Services

SAP Cloud Integration for data services allows you to efficiently and securely use ETL (extract, transform, load) tasks to move data between on premise systems and the cloud. With SAP Cloud Integration for data services you can replicate the data and define how the data is mapped to your database schema.

SAP public cloud application like SAP Integrated Business Planning for Supply Chain use SAP Cloud Integration for data services to integrate with the on premise SAP system.

Technical Prerequisites

The following technical prerequisites have to be met in order to monitor this scenario:

  • Requires at least SAP Solution Manager 7.2 SP03

Available Monitoring Content

SAP Cloud Integration for data services represents the integration layer of a hybrid scenario. Monitoring it is relevant for all hybrid scenarios using this integration form. For SAP Cloud Integration for data services we can collect exceptions in the tasks running in the service.

Which tasks are relevant depends on the integrated solution and the data that is integrated.

Monitoring Template: Cloud (SAP Cloud Integration DS) 

Metric NameDescriptionMAI CategorySince SP

Number of erroneous tasks in SAP Cloud Integration DS (Delta)

Number of erroneous tasks detected in SAP Cloud Integration for data services since the last data collection. The exceptions are retrieved from SAP Cloud Integration for data services via the central Exception Management in SAP Solution Manager.

Exceptions

7.2 SP03

Number of erroneous tasks in SAP Cloud Integration DS (last 24h)

Number of erroneous tasks detected in SAP Cloud Integration for data services during the last 24h. The exceptions are retrieved from SAP Cloud Integration for data services via the central Exception Management in SAP Solution Manager.

Exceptions

7.2 SP03

Connect Cloud Service

Preparations in SAP Cloud Integration for data services

Create Connection User

The user will be maintained in the HTTP endpoint and must be must have the role Production Operator to be able to monitor running and finished tasks and processes.

For more information about default role assignments, please refer to SAP Cloud Integration for data services security guide: https://help.sap.com/viewer/f9f202d1c5924753833b487ac9306384/latest/en-US/d627c9c23a1c4311bd0f88a611ac2aa6.html

Connect Cloud Service to SAP Solution Manager

The Cloud Services configuration is now accessible from the SAP Solution Manager Configuration (SOLMAN_SETUP) → Managed Systems Configuration → Tab 'Cloud Services'.

Create Cloud Service:

  1. Select 'Cloud Service Operations' → Create Cloud Service
  2. Select Service Type: 'SAP Integration Suite for data services' from the drop-down list.
  3. Define Service:
    • Organization name: The SAP Cloud Integration for Data Services organization name
  4. Define Root URL: Root URL (e.g. https://<myhost>.hana.ondemand.com)
    • You can also adjust the extended service ID (The ID used as SID in LMDB) here.
  5. Confirmation: Check your input values and click 'Finish'

The second step is to create the end-point for the cloud service.

Create the End-Point:

  1. Select your SAP Cloud Integration for data services cloud service from the list and click 'Configure Cloud Service' 
  2. Configure Endpoints: Click the 'Add' button on the 'HTTP Endpoints' tab.
  3. Enter the following information:
    • Description: A meaningful description
    • Repository: Select if this end-point points to the 'Production' or 'Sandbox' repository
    • User: Connection user
    • Password: Password of the user
    • If necessary add proxy information to connect to SAP Cloud Integration for data services

Monitoring Configuration

Step 1: Configure Exception Management

After the basic configuration you now have to configure Exception Management, as the exceptions collected via Exception Management and stored in the central exception store are the basis for the Integration Monitoring data collection.

Configuration for SAP Cloud Integration for data services

To access the Exception Management setup please go to transaction SOLMAN_SETUP → Application Operations → Exception Management 

  1. In the step 'Define Scope' select the tab 'Cloud Services' and your SAP Cloud Integration for data services service. If you didn't configure the system before make sure to select 'All Cloud Services' in the view drop-down. Click on 'Configure' or click 'Next' two times. You will enter the step 'Exception Configuration'.
  2. Click 'Add / Configure Log Store'. Select 'SAP BTP Integration DS Errors'. Select the correct end-point for the SAP Cloud Integration for Data Services if you have more than one.
  3. Select the fields for filtering for your log store. The following filter fields are available:
    • ProjectName: Name of the workflow project
    • TaskName: Name of the project task
  4. Maintain the filter values for the fields in the Filter Definition panel under the log stores table. You can also add, remove or change filter fields and values later on in the area. You can use different operators for your filter value. You can add more than one filter filed combination per log store. To add an additional filter field combination click on the button 'Add Filter' in the 'Configured Log Stores' panel.
  5. You should not set up the monitoring for the exceptions in the Exception Management. If you activate the monitoring here, you will receive an alert (and if set up this way an email) for every single exception in the application log. Instead we will use Interface and Connection Monitoring to monitor the central exceptions store for exceptions.

Step 2: Configure Interface and Connection Monitoring

The last step is the configuration of Interface and Connection Monitoring. The Interface and Connection Monitoring setup can be accessed via SAP Solution Manager Configuration (SOLMAN_SETUP). 

Configuration for Monitoring Template: Cloud (SAP Cloud Integration DS)

To access the Integration Monitoring setup please go to SAP Solution Manager Configuration (SOLMAN_SETUP) → Application Operations → Integration Monitoring → Interface and Connections.

Navigate to the step 'Define Scope'. You can create a new scenario for the SAP Cloud Integration for DS monitoring or use an existing one. Make sure the on premise system for the SAP Cloud Integration for DS scenario and the Cloud Service created for SAP Cloud Integration for DS is part of the Interface and Connection Monitoring scenario.

Create the Interface Channel:

  1. Select the scenario and click 'Next'
  2. In step 'Preparation' perform all relevant manual activities and run all automatic activities.
  3. In step 'Configuration' click the 'Add' button.
    • Channel Name: Enter a meaning full name (max. 30 characters)
    • Type: Select 'Cloud'
    • Monitoring Template: Select 'Cloud (SAP Cloud Integration DS)'
    • Description: Enter a description for the channel
  4. Click Next.
  5. Source type:
    • If the source system is the on premise system, please select 'Technical System'
    • If the source system is the SAP Cloud Integration for DS service please select 'External Service'
  6. Source: Select the on premise system resp. the SAP Cloud Integration for DS Cloud Service from the drop-down list
  7. Target Type:
    • If the target system is the on premise system, please select 'Technical System'
    • If the target system is the SAP Cloud Integration for DS service please select 'External Service'
  8. Target: Select the on premise system resp. the SAP Cloud Integration for DS Cloud Service from the drop-down list
  9. The measuring point is selected automatically
  10. If more than one client are connected for the on premise system please select the correct client for the monitoring
  11. Click Next.
  12. Click Finish.

Maintain the Interface:

  1. Select the interface channel you created
  2. On the 'Interfaces' tab click the 'Add' button.
  3. Provide the following information:
    • Interface name: Enter a name for the interface
    • Project Name: Name of the  workflow  project. This field accepts  wildcards .
    • Task name (mandatory): Name of the project task. This field accepts  wildcards .

Select Metrics:

  1. On the tab 'Metrics' select the metrics you want to monitor
  2. Save your interface channel by clicking on the 'Save' button.

Filter Values in Interface and Connection Monitoring

Since the data collection is based on the content of the global exceptions store, you can always check the collected exceptions in Exception Management. All filter parameters are part of the collection context of the exceptions. The filters applied in Interface and Connection Monitoring allow you to filter the exceptions collected by the Exception Management further. However if the filters in Exception Management do not collect an exception, it is not available in the central exception store and you will not be able to find it in Interface and Connection Monitoring!

 

You can maintain attributes as described in the Interface and Connection Monitoring Setup on the tab 'Attributes'.

Thresholds and the collection schedule are maintained in the next step of the guided procedure. Once you have maintained all your channels, click 'Next' in the main guided procedure to move to the step 'Activation'.

Maintain Thresholds and Schedule:

  1. Select the Alert for the interface channel (the alert is the line with the red flash icon next to it)
    • On alert level you can maintain notification and incident message creation
    • Do not remove the flag for 'Do not Group Individual Occurrences' as this flag ensures that an alert each time new exceptions are found during a data collection interval
  2. Select the Metrics
    • Do not change the data collector type or data collector name on the tab 'Data Collection' as the monitor will not work anymore if this is changed.
    • Do not change the threshold if it is set to 'Already Rated'.
  3. Click 'Apply and Activate' → <Choose one option> to activate the monitoring

If not active you can also activate the reporting for the channels by checking the box in the column 'Reporting'. This way you can see a history of the number of exceptions that happened and also place the metric on the OCC Dashboard in Focused Insights.