Extractor Framework Troubleshooting

Extractor Errors

The EFWK admin UI offers a lot of information that help with the analysis of extractor errors. It tells you the status of the last extractor run, the status of the single phases and the application area of the extractor. Best case you even get a meaningful error message that tells you what is wrong.


Error is the phases Extractor, Postprocessor or Dataloader are most likely not related to the Extractor Framework but have to be solved by the corresponding application area.

If the error happens in the phase Main Extractor, please check, which main extractor is used in this extractor. You find this information on the Extractor Detail tab.


If the extractor uses one of the following generic main extractors, the error is in EFWK responsibility. The other MEs are not in EFWK responsibility.

Generic main extractors are:

  • E2E_ME_RFC_ASYNC
  • E2E_ME_STAPI_ASYNC (using required records algorithm)
  • E2E_ME_RFC_ASYNC_NO_RR
  • E2E_ME_RFC_ASYNC_NO_RR_255
  • E2E_ME_ST_API_ASYNC_NO_RR

For each phase you also have a return code. The following two tables show the most common error codes that can be found in the Extractor log.

Extractor

Error codes are defined by the the extractor. Common error codes are:

96  Communication Failure

97  System Failure (there will be most likely a corresponding dump in the managed system)

Data Loader

0  Infocube not found

2  Illegal input

3  Rollback error

4  Duplicate records

5  Request locked

6  Infocube not transactional

7  Write others

8  Target data error

9  DDIC component not found

10  Others

11  Create data error

96  Communication Failure

97  System Failure

Also not all Data Loaders are in the responsibility of the EFWK. Only the data loaders SMD_DATA_LOADER100 and SMD_BI_DATALOADER (Twin Cubes) are in the responsibility of EFWK. All others are in the responsibility of the application.

The following database tables contain the logging and status information for the EFWK:

  • E2E_EFWK_STATUS:      Current Status, Priority
  • E2E_EFWK_LOG:              Logging Table

The extractors in the table E2E_EFWK_STATUS can have the following status:

Status Name Description Instance
Q Queued

Set when extractor is taken into global worklist

 Resource Manager
S Started

Initialization of Mainextractor

 Worklist API
R Running

Initialization of Mainextractor

 Worklist API
M Banned

Set in case timeout / exception occurred

 Worklist API
N Inconsistent Set in case of wrong configuration EFWK Housekeeper / Worklist API


and priorities:

Priority Description
0 Normal Extractor
100 Extractor in „time based chunk“ mode
999 MAI Extraktors
N+1 RMGR was not able to inject WLI during runtime

All extractors started by the EFWK always run under the user SM_EFWK or SOLMAN_BTC. If you experience problems in the EFWK you should also check for dumps (ST22) for these two users and if they are related to the EFWK.

The dump details tell you if the dump is related to the extractor or to the EFWK Resource Manager.


Last but not least you can check the tab “Exceptions” or the application log (SGL1) directly for errors in extractors. On the tab exceptions the entries from the application log are pre-filtered.

If you want to use SLG1 directly please use the Object E2E_ALERTING and the sub-object EFWK (for EFWK runtime log entries) and EFWK_SETUP (for EFWK setup log entries). The setup log entries tell you important EFWK Setup information, like which content was supplied (System / PPMS IDs), which implementation / setup type is executed (RCA, CCDB, …) and were there any errors.


Inconsistent Extractors

Every time the EFWK Housekeeping runs and also during the runtime of an extractor the consistency of an extractor is verified. Extractors are scheduled for a specific technical system and for a specific Product / Software Component Version. Mismatches with LMDB information will lead to inconsistencies. The most common inconsistencies are listed below.

Message: PPMS Modelling is not valid. Check PPMS Keys.

This inconsistency usually occurs if the managed system is upgraded without rerunning the managed system setup. The consistency of the PPMS keys is evaluated by the EFWK housekeeper, that checks the PPMS keys maintained in the extractor versus the ones known in the landscape browser.

Please note, that PPMS modelling differs among the different extractors, i.e, extractors can be modelled, e.g., in the context of a software component version only, others might run in the context of a specific product instance and some extractors do not require a PPMS modelling at all. For comparison with information in the landscape browser all maintained PPMS keys need to be considered.

Message: RFC Destination not maintained/invalid

The RFC destination is evaluated during the runtime or from the housekeeper. Check if the RFC destination maintained on tab “Configuration” of the Extractor details.

Message: Technical System not found in Landscape

In this case the extractor is scheduled for a system that does not longer exist with the Long SID. These inconsistencies can arise, when systems are deleted from LMDB without de-scheduling the extractors or if the Long-SID is changed. Check if the systems exist in the Landscape Browser. If not you can delete the extractor.


Message: Technical System not Diagnostic Relevant

This means that the system is not marked as diagnostics relevant. If all systems for which extractors are scheduled are diagnostics relevant is checked by the EFWK housekeeper.

Check in the Landscape Browser if the system is marked as Diagnostics Relevant.


If not make sure to mark it as diagnostics relevant and rerun the Managed Systems Setup for the system.


Depending on the outcome of the analysis you can either release inconsistent or you can delete them. After deleting an inconsistent extractor you'll have to reschedule them. For RCA extractors this is done by rerunning the Managed System Setup, for all other extractors this is done by running the respective setup scenario depending on the setup type.


Banned Extractors

Banning extractors is a safety mode of the EFWK to make sure that an extractor that blocks resources for a very long time cannot cause harm to the managed system or hinder other extractors. Usually extractors are banned if there are multiple very long runs and a specific time interval. Technically it is banned if it causes the RFC exception SYSTEM_ERROR (RC = 97) for several times.

To analyze the banned extractor check the log for RC 97:


Most likely you will find also a corresponding dump in the managed system. The error message in the Extractor Log and the dump should help you identify and solve the problem.

After you have solved the issues you can release the banned extractor.


Extractors with Backlog (Time Based Chunks)

Sometimes you will find the message like “Info: Time based chunks. Retrieving 2014012105. Extractor will be registered again”. This means this extractor is working on reducing a backlog. This is a normal procedure of the EFWK and in general no need to worry. But you might want to consider some tuning in the EFWK to prevent future backlogs.

The timestamp tells you how big the backlog is. You will always notice that this extractor is scheduled more often, than its scheduling period.


The scheduling period can be found in the table E2E_ACTIVE_WLI in the field DTRUN.


To find the right entry in this table use the Worklist ID that you can find on the Extractor Detail information.

Performance Tuning

Analyzing the Performance

If you experience a lot of backlog or the data in the E2E applications seems not to be current, you might have a performance issue in EFWK.

The first thing to check is the EFWK self-monitoring. You find this information in the following location:

  1. SAP Solution Manager Administration workcenter
  2. Self-Monitoring
  3. Start the 'Self-Monitoring Overview'
  4. Under the 'Extractors' and then 'Performance' section, select the 'Resource Manager Runtime' and/or 'Resource Manager Throughput'.

The following picture describes the location:


Remark: As a prerequisite the Self-Monitoring of SAP Solution Manager must be configured and activated. In case it is not, activate it via SOLMAN_SETUP -> Technical Monitoring -> SolMan Self-Monitoring.

The Runtime of the Resource Manager gives you a hint, which time periods might be interesting, when you analyze the RMGR job logs later on. Check for phases with duration of over 40 seconds. This typically implies that the maximum number of injection cycles was used. The more RMGR executions with such a long duration, the higher is the resource utilization. The EFWK RMGR runs every minute. Constant long runtime indicates a high load but not necessarily a resource bottle neck.


The throughput tells you how many percent of the planned worklist items, where actually processed during the RMGR run. The RMGR has a list of N worklist items he tries to process during its run. The throughput is “Number of actually started WLIs / N”. If this value is constantly below 1, it indicates a resource bottle neck.


Once you have identified if you actually have a resource bottleneck, you now have to analyze the job log of the RMGR runs. That's with the ones longer than 40s.

Analyzing the Resource Manager Job Logs

The Resource Manager Job logs contain information which resources were allocated and how successful this allocation went. Please remember the log on one job is only a snap shot. You should evaluate the job log on multiple job instances.

How to interpret Throughput Statistics

The throughput is the throughput within the lifetime of the Resource Manager instance. The less iterations are needed for a successful injection, the higher is the throughput.


To calculate the throughput for a resource you divide the number of successful injections by the total number of tries (In this example: 4 / (4+84) = 4.5%).

In particular for a freshly setup system, temporarily low throughput values are normal.  Resource competition leads to shifts in execution times. The resource consumption will automatically spread over time (Load Peaks “melt”).


What does “DIA WP Resource Cap Exceeded” mean?

This log entry refers to the local resource protection in SAP Solution Manager. The resource cap which is exceeded here is the resource cap for the resource SOLMAN_DIALOG_WORKPROCESS.


What does “DIA WP Resource Failure” mean?

In this case the resource cap is not exceeded, but still there are not enough dialog WPs available. The reason usually has nothing to do with the EFWK, but shows a general resource bottle neck. The asynchronous RFC call from the RMRG to the local dialog WP checks if a dialog work process is really available when performing the call. If not, the RESOURCE_FAILURE exception is thrown.


This message in the log should be uncritical. EFWK resource management is “logical” and only handles request to resources used by EFWK. At times of high dialog load, the logical EFWK resource might be free, but the “real” resource is blocked by a non-EFWK dialog process.

Tuning the Performance

The tuning of the EFWK is performed using the configuration options described in the section “Setup & Administration”.

Known Errors and Problems

Always check if your problem might be already fixed with SAP Note 1717403.

Error: The Extractor Log of a selected Extractor shows 'No Authorization'

Solution: Authorization object /SDF/E2E is missing for the user SM_EFWK. Please refer to SAP note 1901619.

Multiple Resource Manager Jobs run in parallel

After an upgrade it is possible that the old resource manager job from SAP Solution Manager 7.01 was not de-scheduled correctly. The new (correct) resource manager job is called "EFWK RESOURCE MANAGER" and runs under the user SM_EFWK or SOLMAN_BTC. The old (deprecated) resource manager job is called "EFWK Resource Manager (01 Minute". This one needs to be de-scheduled!


Missing data in application XYZ

Follow the error analysis chart:


Error: No corresponding template entry found

This error comes from the EFWK housekeeping. This means that there is no template for this extractor in the template directory. All delivered templates can be found in table E2E_TEMPLEXTR or on the tab "Extractors" in the Root Cause Analysis content browser.  


Check to which application area (setup type) the extractor belongs to and open a ticket on its component.

Resource Manager Job canceled

Check Job Log and determine whether there is a general Batch Job problem or not. There might be a lot of extractors in status Q (check table E2E_EFWK_STATUS), this not a problem, as they are removed by timeout mechanism, once RMGR is running again. It is also possible that no Dialog WP was available (job log message: No Lock Could be obtained on Resource Counter Table for SOLMAN_DIALOG_WORKPROCESSES.)


All SAP Solution Manager Workprocesses are used by SM_EFWK or SOLMAN_BTC

Sometimes SAP Solution Manager can't be accessed. Investigation show in SM50 that all dialog processes are used up - user SOLMAN_BTC using many of the total work processes. If this is the case it could be that the settings for the EFWK are not correct. Please refer to SAP Note 1851551 for recommendations how to set up EFWK resources correctly.


The Activity "Setup Extractor" in the "Managed System Setup" is very slow or times out

With SAP Solution Manager 7.1 SP06, SP07 & SP08, the "Setup Extractor" activity in solman_setup, scenario "Managed System Configuration" for Step "Configure Automatically" is very slow or times-out. You receive the error: CX_AI_SYSTEM_FAULT : SOAP:1.023 SRT: Processing error in Internet Communication Framework: ("ICF Error when receiving the response: ICM_HTTP_TIMEOUT").

You also experience very long runtime for some extractors (e.g. ‘CONFIGURATION ADD(ABAP)') in the managed system.

Please apply SAP Note 1833976 to the managed system.