11.07.2022
Michael Paternoster
Author: Michael Paternoster

Dynamic Data Extraction

The evolution of one-click data export

We could say we went to work with a well thought-out plan and optimised our processes quite strategically. In fact, the idea for one of the greatest improvements of the dab:AnalyticSuite came from a spontaneous idea.

If you want to carry out evaluations on SAP data with an external analytics tool, you must define in advance which raw data are to be extracted from the SAP system as a basis for this. This definition contains metadata in the form of a list of tables or table fields, in combination with filters to be applied during extraction. The type and scope of this metadata depends on the questions to be analysed. We and our customers describe them as "export or download packages" - the definition of the respective quantity of SAP data to be extracted.

For many years, we had to manually create these export packages for our customers so that they could extract the desired data from their SAP system and analyse it in ACL for Windows or Robotics. "It must be easier", this sentence at one of our team meetings was at the beginning of the development of the dynamic data extraction. 

 

dab:Exporter: reliable, but time-consuming in package management

With our product "dab:AnalyticSuite" we offer a large portfolio of predefined analytics for various SAP processes and categories, such as audit, internal control system, cash recovery or SAP authorization check. Since these analytics, as explained above, require a certain set of basic data from SAP, each customer usually receives a predefined export package from us, which contains the necessary SAP tables for the analytics they have licensed. This saves companies from having to manually search for the SAP transactions and tables that are relevant for their respective analytic target. However, the extraction of the data from SAP must take place correctly.  

Originally, we used the dab:Exporter for this: an extraction solution for SAP through which users could access the desired SAP data. Depending on which analytic areas companies had acquired with the dab:AnalyticSuite, they received from us an export package tailored to them in the form of a file with this meta-information. They uploaded this to the dab:Exporter, which extracted the SAP data according to these instructions. 

The problem: users do not always want to run the entire set of analytics available to them. For example, if companies only want to analyse supplier master data, it does not make sense to also extract all SAP data from the purchasing area for reasons of data protection and data economy. Customers could therefore request individual export packages for analytics from their purchased modules from us, and we prepared this "blueprint for extraction" for them in the dab:Exporter. 

Integration of the export function in ACL Robotics

However, the procedure had several disadvantages: On the one hand, it was time-consuming to always have to take the step of using a separate tool to export the data from SAP, while the actual analytic was done with another software tool. On the other hand, a precise data extraction was often associated with an unacceptable effort due to the tedious preliminary work - even the absence of a single field out of hundreds could prevent a successful analytic run. 

Because as soon as companies wanted to analyse a new combination of data within their purchased modules, they had to contact us again: Only we could generate the new package for extraction in the dab:Exporter. Alternatively, companies were left to work with oversized data exports with the associated challenges in terms of data sparsity and runtime behaviour. 

To reduce this effort, we integrated the data extraction directly into the analytic tool ACL for Windows/Robotics as a first step. Users could now perform the extraction directly in their analytic program and, with the help of the ACL Connector for SAP, a fully integrated interface, saved themselves the step via another tool. However, this did not solve the problem of potentially oversized data exports: 

"We only want to do the analytics for double payments and bank account changes." This type of request caused extra work not only for the client, but also for us. In a separate internal tool, our developers had to compile each combination of analytics individually, generate an individual export package and then send them to the customer. Later, we were able to create a module-specific package with just one click and automatically generate typical individual analytics. Whether the package would be used by the customer in this form? We could neither know nor influence that. It was only logical to take the idea of our colleague further, who had demanded: "This has to be easier.
 

As if it had never been any different 

And we had all the prerequisites to simplify the process: The information about which SAP tables and fields are assigned to the individual analytics or dab:AnalyticSuite modules was already implicitly available. So was the solution that made it possible to compile these packages individually, but until now it had only been used internally - but that could be changed. Today, these elements are integrated into the code of dab:AnalyticSuite.

Analytics with a touch on the button

The result: Users simply select in the Robotics interface which analytic modules they want to run (of course, it is also possible to select individual analytics). Based on this, the information needed to export the required SAP data is generated at runtime - without any additional effort for us or the customer, unnoticed in the background and at the push of a button. Static export packages are just as much a thing of the past as support requests to have individual export packages put together. Today, the extraction is generated precisely at the push of a button.

For more than two years now, dynamic data extraction has been a standard feature of dab:AnalyticSuite. Only our long-standing customers know about the technical evolution. For new customers, SAP data export means ticking the right box from the start. And in a few years, we will probably be able to say the same about other features. After all, we are continuously developing our tools - and many things can be solved even more easily with the right approach or new technologies.


Comments (0)
Be the first who comments this blog entry.
Blog login

You are not logged in. Please log in to comment this blog entry.

go to Login