Job & Automation Monitoring

Scope & Content

Scope

The purpose of Job & Automation Monitoring is to provide a central Job Monitoring solution for all SAP products to complement local job scheduling. It gives transparency about the current status of the different automation processes (for simplification called “jobs” in the later text) regarding execution status, application status, start delay and run time. 

The basic concept is to collect individual job execution data and related exceptions and give a rating for the execution (the runtime is rated based on historical data). The rating of the latest execution is propagated to the job definition level. You have the option to drill down from the job to the list of executions. Finally, you have a direct navigation to the corresponding job instance in the executing service for a more detailed analysis, which gives you the option to stop a not ending execution or restart a failed execution. To offer these two fundamental job management actions directly from the central monitor is planned.

Value add: Job & Automation Monitoring helps - with a unified user experience using a common look-and-feel and handling pattern - to ensure all automation processes across the different products are running successfully. This is especially valuable in integration scenarios of multiple products of the SAP Intelligent Suite, where each product runs customer-specific jobs and you do not use a central job scheduler (as then you could do the monitoring using the central scheduler). 

Content

Job & Automation Monitoring in SAP Cloud ALM collects data for these job types: 

  • SAP Application Job (preview)
  • SAP ABAP Job (preview)
  • SAP BW Process Chain (preview)
  • SAP Build: Automation Job / SAP Intelligent RPA Job (preview)
  • SAP Build: Process and Workflow (preview)
  • SAP Business Workflow (preview)
  • SAP Job Scheduling Service Job (of a BTP customer built application) (preview)
  • SAP Cloud Identity Provisioning Job (preview)
  • SAP Digital Manufacturing Processes (preview)

Supported Solutions

Job & Automation Monitoring in SAP Cloud ALM collects data for these solutions and products.

  • SAP S/4HANA Cloud Private Edition
  • SAP S/4HANA Cloud Public Edition
  • SAP S/4HANA and SAP Business Suite 
  • SAP BW/4HANA (i.e. SAP NetWeaver AS for ABAP 7.40 and higher)
  • SAP Build Process Automation (SAP Intelligent Robotic Process Automation)
  • SAP Marketing Cloud
  • SAP Integrated Business Planning for Supply Chain
  • SAP SuccessFactors Employee Central Payroll
  • SAP BTP, ABAP Environment 
  • SAP BTP, Cloud Foundry (customer built application)
  • SAP Cloud Identity Services
  • SAP Digital Manufacturing

Features

Monitoring

The Overview provides a summary status regarding job execution for the services in scope. It shows the status of the latest execution of every job with regards to the Execution Status, Application Status and Run Time. On top it displays the number of alert situations, i.e. jobs for which an alert had been created, that is not yet confirmed.

 

When clicking on a card for a service, you are navigated to the Monitoring view and see the list of jobs/automations for the service sorted by Execution Status rating in a descending order. If you click on a number within a card you will see the same list, but sorted and filtered by the aspect.

If you directly navigate to the Monitoring view, you see the list of jobs/automations (for all services in scope) rated based on the status of their latest execution with regards to the Execution Status, Application Status, Start Delay and Run Time. For every line you see the number of open alert situations.

  • The Execution Status is rated green, if a job finished successfully and red, if the job execution did not finish, i.e., aborted. It is rated yellow, if a job finished with warnings or errors without aborting.
  • The Application Status is rated green, if a job successfully processed the application data. It is rated red, if e.g., an ABAP job execution writes errors into the application log and yellow, if there are warnings, but no errors.
  • The Start Delay rating is rated green, if the technical delay of a job (actual start time vs. planned start time) did not exceed the threshold defined.
  • The Run Time is rated green, if the runtime of a job did not exceed the threshold defined. 

By clicking on a name, you drill down into the list of executions of the selected job/automation sorted by planned start time. By clicking on the information icon next to each execution, you can view further details and navigate directly to the job in the managed cloud service or on-premise system* by clicking on the Run ID value.

*Note: If you use a SAP Web Dispatcher please maintain the URL of the Web Dispatcher as Logon URL for the respective System in SAP Cloud ALM Landscape Management. 

Defining List of Jobs as Favorite

In Monitoring in the job list view you can filter the list for specific jobs and have the option to define this list as favorite. After you have created the favorite by pressing the * icon, go to the home page and set a speaking title and subtitle (via the star icon) for the card. 

From the favorite card you can forward navigate to monitoring, namely to the list of jobs that you have defined as favorite.

Analysis

Use the Analysis view to identify those jobs that have the longest response times and highest number of exceptions. A trend icon indicates, if the failure rate or response time is up or down in the time frame selected taking the mid point of the time frame as comparison point.

For any job it offers aggregated information on total and average run time, number of executions and number of failed executions (regarding Execution Status or Application Status) and the resulting failure rate. By clicking on the arrow at the end of every line, you can drill down into various trend charts for the metrics. 

Exceptions

Use the Exceptions view to check for exceptions of the services and systems in scope.

To get exception data collected in general please follow the instructions provided in Integration & Exception Monitoring. In case of issue with the view, please report it via component.

  • SV-CLM-OP-EXM – Exception Monitoring Service

To get exception data collected related job exceptions, no additional configuration is needed as those are collected together with the job execution data. Currently the following job types are supported:

  1. SAP Intelligent RPA Job executions send notifier execution failures and the job execution gets a red Application Status rating. 
  2. SAP ABAP Job executions send *
    • Job Log exceptions and the job execution gets a red or yellow Execution Status rating. 
    • Application Log exceptions and the job execution gets a red or yellow Application Status rating. 

On click on the rating icon you are forward navigated to the list of exceptions related to the job execution.

*Prerequisite: ST-PI 7.40 SP20 or higher is implemented in the managed on-premise system and in /n/sdf/alm_setup use case Exception Monitoring is activated.

Alerting

Use the Alerting view to check the job execution related alerts for the services in scope. You can configure to get alerted, if a job has a red or yellow Executions Status, a red or yellow Application Status, if its Run Time or Start Delay exceeds a defined threshold.

You can adjust the Alert Name for example to indicate what you have configured, e.g. the filters used, but note that in the alert Object Details the name of the job is contained and in the alert list UI, you can filter for the content of the object details, i.e. the job name. The name of the job is appended into the notification mail subject. The notification body contains information on the alerted job run and a link to the alert.

Job execution data that has arrived at SAP Cloud ALM Job & Automation Monitoring is evaluated every minute, so that alerts are raised almost immediately. (How quickly data arrives depends on the job type. For ABAP Jobs and S4 HANA CE Application Jobs you can configure in the sending system the periodicity of collection (could let it run every minute). For other use cases like SAP Intelligent Robotic Process Automation and SAP Build Process Automation monitoring data is sent instantly, i.e. event based.)

Alerts are raised per job and alert type and contain all failed executions of a specific job. From the Run Id value, you can navigate to the corresponding job execution in the managed cloud service, just like from the job execution popup in the monitoring application.

By default a downstream action (like email) is triggered "At new occurrence", i.e. when there is a change to the event rating (to red or yellow). As a consequence no new notification is sent on consecutive failures of the same job. If you need a new notification at every job execution failure, you can configure to get the event action triggered "At every occurrence".

FAQs

Question: In the Overview view, I see for every managed component in scope one card. What is the motivation for this?

Answer: Monitoring provides you a central overview on the job ( automation execution status of all services / systems that you have selected into your scope. Consequently, you can quickly understand, if there are issues in specific managed components. From the cards you can forward navigate to the monitoring view, which is then filtered by the service. 

Using Favorites you can define groups of jobs (cross service / system) if you want to have a special focus on a subset of jobs / automations.

Question: In the Monitoring view, I see the list of jobs. Why has every job four ratings?

Answer: Monitoring groups the different executions of a job and rates the job definition according to its latest execution. The ratings indicate if the latest execution 

  • did finish successfully
  • did process its data successfully
  • started not more than x minutes after the planned start time
  • did not have an exceptionally long run time

Question: I like the monitoring application, but I want to look at it only, if there is an issue with a job. How can I get alerted or notified in such cases?

Answer: Expand the Configuration pane and there the section Managed Components. For every individual managed component you can create events and maintain filter conditions (e.g. restrict on Job or Executable names) and the "Event Actions" in order to get alerted and / or notified

  • if a job has a red Execution Status, which is the case, if it aborts/cancels
  • if a job has a yellow Execution Status, which is the case for ABAP jobs, if they write warnings or errors into the job log
  • if a job has a red or yellow Application Status, which is the case, 
    • if an ABAP or Application Job writes an error or warning message into the application log 
    • if an SAP Intelligent RPA job or SAP Build Process Automation job execution has a notifier failure (see here).
  • if a job runs longer than a threshold 
  • if a job start delay is above a threshold 

In general an alert is raised immediately after the event occurs, for example:

  1. "Critical Execution Status" event is raised at the end time of a job execution, which is typically immediately after a job cancels.*
  2. "Critical Runtime" event is raised when the defined threshold is passed during the execution of a job. 
  3. "Critical Delay" event is raised when the defined threshold is passed (comparing current time with planned start time)

* the "Critical Application Status" event is raised also at the end time of a job execution, even though the critical message might have occurred during job execution without the job ending.  

Question: Can I get alerted, if an SAP Intelligent RPA job or SAP Build Process Automation job execution has a notifier failure?

Answer: Yes, such a case is indicated by a red Application Status Rating, which can be alerted on (see here).

Question: Can I see exception message details that occurred at job execution in the monitoring application and quickly navigate to the message details?

Answer: Yes, if exception details have been collected (subject to job type and configuration settings) then this is indicated by a slightly different icon on job execution level in columns Execution Status or Application Status. On clicking the icon you drill down into into Exceptions view and see the details (e.g. ABAP Job Log or ABAP Application Log messages).

Question: Can I do long term trend analysis on job run times and failure rates?

Answer: Yes in the Analysis view you can see

  • which jobs have long run time or a high failure rate
  • if there are jobs that deteriorate regarding run time or failure rate 

Question: Can I show long term trend on job run times and failure rates in an external dashboard?

Answer: Yes, you can use analytics-api for Job & Automation Monitoring to get aggregated data and see

  • job run time over time per service / job type / job name
  • job failures over time per service / job type / job name 

See here a Grafana Job & Automation Monitoring dashboard with 4 panels and its configuration.

Question: What information is offered for SAP Business Workflow monitoring? How to use the application?

Answer: See here a short demo on how to use the application for SAP Business Workflow monitoring.

Question: Currently we use SAP Solution Manager and SAP BPA by Redwood for Job Management. However, according to SAP Note 3216013 end of Mainstream Maintenance for SAP BPA by Redwood is 31.12.2024 and SAP Solution Manager maintenance will end 31.12.2027. Will SAP Cloud ALM offer a similar functionality?

Answer: SAP Cloud ALM offers standardized ALM functions focused on operational efficiency and is free of charge for SAP customers (see here). For process automation SAP offers SAP Build Process Automation, but this product is not a successor or replacement for SAP BPA by Redwood or another external job scheduling tool like Tivoli, which have a very strong focus and a broad range of features for efficient batch process (chain) execution. To replace an external scheduler, you need to look in detail at the features and functions to decide, if you can manage job scheduling as follows: 

  • use product inbuilt job scheduling tools (e.g. SAP Application Jobs in case of S/4HANA) for those processes, where SAP BPA by Redwood is used only for triggering batch execution 
  • use SAP Build Process Automation (or alternative SaaS automation tools) for those processes, where SAP BPA by Redwood is used in a complex way, i.e. not only triggering the execution of a job, but based on the result triggering follow on executions 
  • use Job & Automation Monitoring with SAP Cloud ALM that is retrieving job execution data directly from all SAP services and systems for monitoring.

For a high level overview on how SAP Cloud ALM and SAP Build Process Automation work together see here.

Note, that:

  • for first level issue resolution (e.g. stop of long running jobs, restart of aborted job) Job & Automation Monitoring with SAP Cloud ALM offers direct access to the local monitoring tools.
  • for a controlled implementation of automation processes SAP Cloud ALM for Implementation plans to offer functionality in the context of Requirements Management and Solution Documentation.

Question: We use the IBM Tivoli Workload scheduler (TWS) and would like to explore the option to replace TWS with the SAP Job Scheduler Service. Is this possible?  

Answer: External schedulers offer a wide range of connectors to job executing systems and capabilities to build job chains and react on the result of jobs within a chain. The focus of the SAP Job Scheduler Service is different. It basically offers to trigger the execution of jobs on a regular basis. For this a cloud application needs to expose an action endpoint (HTTP/REST endpoint) that can be called from the SAP Job Scheduler Service.

Long-running jobs can be separated from the cloud application process by using Cloud Foundry Tasks. The service offers a REST API and a dashboard for managing jobs and tasks.

SAP Cloud ALM for Operations plans to offer central monitoring of the jobs and tasks that are scheduled using the SAP Job Scheduler Service.

Further details regarding the SAP Job Scheduling Service are available at:

Question: We use SAP Solution Manager for Job Scheduling Request Management and document the scheduled jobs (using the versioning option offered). We have integration with an external ticketing system and external scheduling. Will we be able to use SAP Cloud ALM to replace SAP Solution Manager?

Answer: It is not on the roadmap of SAP Cloud ALM standard to replace all of SAP Solution Manager's Job Management capabilities, but partners are welcome to extend SAP Cloud ALM via APIs. 

The core capability of SAP Cloud ALM for operations is to offer central monitoring. As of now ABAP based jobs (cloud as well as on-prem), BW process chains as well as SAP Build Process Automation automations can be monitored. Further job and automation types will follow and it is also planned to offer basic (central) Job Management:

  • SAP Cloud ALM for Implementation plans to offer the option to document jobs and automations and link those documents to the implemented business processes
  • SAP Cloud ALM for Operations (Job & Automation Monitoring)  
    • plans to offer a visualization in monitoring, if a job / automation is documented
    • plans to offer direct access from monitoring the related job / automation documentation 
    • already today offers central monitoring of all jobs running in the various systems and services
    • already today offers the option to directly access the local monitoring UI to react in case of an exception situation (e.g. to stop a long running / hanging job or restart a failed job)

We believe that the above sketched basic job management offering covers the operation requirements of a SaaS solution, where customers are responsible for a subset of jobs only, namely those that process customer specific application data and which business end users can schedule locally.

If central scheduling is still needed, we think, that the (request) management of the centrally scheduled jobs should be handled by the central scheduling tool.

In a nutshell we recommend to use

Question: We use SAP Solution Manager for Job Monitoring. Will we be able to use SAP Cloud ALM to replace SAP Solution Manager?

Answer: See this document that goes along Configuration Guide Job Monitoring 7.2 and tells, if such monitoring is offered in SAP Cloud ALM or not. If not, if it is planned.

Please note, that ​SAP Cloud ALM for Operations differs from SAP Solution Manager Operations in so many ways that it can hardly be compared, especially as SAP Cloud ALM for Operations continues to evolve. ​Job & Automation Monitoring aims to be simple to use and provide out of the box generic monitoring and analysis for the various SAP cloud and on-prem products regarding the execution of jobs and automation artifacts. If you need, you can configure events/alerts for specific jobs and trigger various downstream actions.

Setup & Configuration

For more details on prerequisites, setup, housekeeping, and customer support, please check the Job & Automation Monitoring -  Setup & Configuration page.