dab: AnalyticIntelligence - New wine in new wineskins
As you may have seen, we have added a new product, dab:AnalyticIntelligence, to our website. This blog post describes the product and how it differs from our existing solutions with which you are familiar. The blog has been written by Ernst-Rudolf Töller, with whom we have a longstanding, trusting and collaborative relationship in research and development matters. I hope you enjoy reading the blog – please contact us at any time if you have any questions.
Stefan Wenig, Managing Director of dab: Daten – Analysen & Beratung GmbH
Starting point: Traditional data analytics is not the solution to all problems.
Many companies still envisage fundamental problems when transitioning from data analytics to real data mining:
- What does data mining actually look for; can it really show things in company data that can’t be found with SQL queries, for example?
- Even if new data analytics approaches can be found, can innovative concepts really be implemented using traditional hardware and software?
- And ultimately, don't all ideas of data-mining fail sooner or later due to skill shortages?
Processing data more intelligently
At dab we have confronted these challenges and carried out intensive development work to search for new approaches. We soon realised that we first needed totally new perspectives on data. Existing IT technology can by and large “process” data without really “understanding” it. However, modern search engines are an example of how users can be given the almost perfect impression that the search engine has actually “understood” their query. This is, among other things, achieved through intelligent sorting of the search results. Consequently, data analytics, too, needs to use the corridor between innovative concepts and what can still be implemented using conventional IT approaches. We focus on an alternative representation of data based on proven mathematical processes, systematically visualised using modern graphical methods.
Describing data mathematically as an autonomous reality
Data are no longer merely a reflection of reality but are instead understood as an autonomous reality per se. The mathematical description of this reality is the key to a whole new perspective on the topic of data that is not only applicable to search engines.
Transforming data into mathematical objects – moving away from traditional SQL-type queries or trivial process mining graphs
Using these deliberations as a basis, dab has developed the concept of the “data derivative”. Company data such as the data contained in individual or linked tables of ERP systems are transformed into mathematical objects. These objects permit new forms of “endogenous data analytics”. Now, data are no longer queried on a strictly external basis as happens with SQL queries, for example:
Example of traditional SQL query:
SELECT * FROM LIEFERANT WHERE Adresse IS NULL AND BankVerb IS NOT NULL
Here, a specific constellation is filtered in the supplier data: Suppliers without an address but with a bank account. This constellation is based on meta information/meta data of the user behind, who combines e.g. increased data criticality with this data constellation.
Example of a result based on endogenous data analytics:
Here, the data (one point = one customer account) are divided into groups, entirely without any external specifications by a user. The blue circle shows a group of divergent cases. The large mass of cases forms a coherent group. This classification of customer accounts is performed on an “endogenous” basis, i.e. it is not based on externally set filters, thresholds, etc. In this form, the classification is calculated from the totality of all data. Whether the graphical presentation selected here shows the data contiguously or whether there are additional individual “islands” is a characteristic of the overall data volume, which only becomes visible at all in this presentation. Of course, filters can subsequently also be defined which describe the individual islands. What is important is that these filter criteria/statements about their selectivity only become available AFTER such an analysis.
Example – derivative formation:
In the present example, important semantic aspects relating to the composition of the documents for a customer account are included in the definition of the derivative.
ERP system: SAP
Relevant tables: KNA1, BKPF, BSEG
Key data attributes: KTOKD, Account group (KNA1), BLART Document type (BKPF), TCODE Transaction code (BKPF), BSCHL Post key (BSEG)
Key: KUNNR, Customer number
In this case, a derivative is defined with the customer number as unique key. In this derivative, the composition of the booking documents is coded to a customer number based on the specified attributes/fields. One could also use only part of these attributes/fields or even totally different fields, etc. This means all important semantic aspects of the data always remain under the control of the user. However, no critical forms of attributes/fields, thresholds, etc. need to be defined at all.
The use of derivatives comprehensively opens up the syntax of the data for algorithm-based analysis. The necessary semantic considerations occur separately during definition of the derivative. Here, it is necessary to decide which tables, fields, etc. are to be used to analyse a certain correlation. In this sense, therefore, the analysis of derivatives cannot be considered “correlation bingo”.
This brief overview reveals very interesting, positive answers to the first two of the three questions asked at the start.
1. What does data mining actually look for; can it really show things in company data that can’t be found with SQL queries, for example?
In answer to this first question it could be said that in principle, the analysis of data using derivatives could also be undertaken in parallel using SQL queries. However, these queries would be so complex that they would be unmanageable from a user’s perspective and possibly also in IT terms.
2. Even if new data analytics approaches can be found, can innovative concepts really be implemented using traditional hardware and software?
In answer to the second question: the hardware and software systems required for processing derivatives are available through AnalyticIntelligence from dab. The software solution from dab is based on stringent mathematical penetration of the overall topic; this and also the subsequent processing can be implemented using traditional resources.
3. Ultimately, don’t all ideas of data-mining fail sooner or later due to skill shortages?
This also brings us to the third question: The entire process under discussion here can only be visualised through managed services. In principle, the comparison with modern search engines is appropriate here, too: The question at the start and the result at the end of the process are always also understandable for the user. As with search engines, the technical bit in the middle is on a completely different level.
Summary
Our product dab:AnalyticIntelligence is breaking new ground, combining the strengths of semantic knowledge and years of experience with the impartial view of data which accounts for data mining's appeal, without incorporating the many identified weaknesses. We are convinced that in a world in which machine learning and artificial intelligence are gaining increasing significance, the mathematical description of data and the associated data analysis will assume an important role. Deterministic queries such as those mapped by SQL-type queries are not obsolete – the more innovative and more efficient method of tackling large and complex data volumes irrespective of their source lies in approaches such as dab:AnalyticIntelligence.
We hope you have found this brief outline of the principles of dab AnalyticIntelligence interesting. Please contact us at any time if you have questions on this article, about data analytics in general, or about products from dab and ACL™.