SAP HANA SQL Data Warehousing relies onmodel-driven development and and, thanks to this approach, offers a high degree of structural flexibility and process agility.
Table of Contents
1. Model-driven development in the software industry
1.1 Approaches
1.2 Objectives
1.3 Characteristics
1.4 Advantages
2. Model-driven SAP HANA SQL data warehousing
2.2 Data warehouse architecture
2.4 Advantages
3. Conclusion
In this blog, we want to explain the concept of model-driven development in more detail. To do this, we will first look at the goals, characteristics, and advantages of this software development approach. In a second step, we will specify the methodology for setting up the SAP HANA SQL Data Warehouse and show you our specific approach and the advantages it offers.
Model-driven development in the software industry
The use of models when dealing with complex contexts and systems is an approach that is common to all technical disciplines in science and practice. In IT and the development of information and software systems, too, the possibility of abstraction through models has long been used extensively. At the beginning of the 21st century, the idea of model-driven development became established, which is intended to improve software development on the basis of models and the subsequent automated generation of source code.
approaches
There are different approaches to model-driven development. The most widely recognized approach is Model Driven Architecture (MDA) of the Object Management Group (OMG), which is also a standardization initiative and is supported by various well-known companies and universities. For a basic understanding, a reference to MDA or the MDA Guide is therefore sufficient and, due to its widespread use, most useful.
objectives
The main goal of MDA is to use models to better deal with the complexity and interdependencies of complex systems. The added value is reflected primarily in five aspects:
- Models as a means of communication: Models can be used to create a common basis for communication between development teams and other individuals involved in the development process. Defined semantic model properties and an appropriate level of abstraction enable individuals from different departments to quickly achieve a uniform understanding of a specific subject area. Models can also be easily exchanged and further developed on the basis of version statuses and model levels.
- Automated transformations: The generation of source code and the implementation of software based on models can be automated in whole or in part. Automation reduces the effort required to implement a model, make changes, and perform maintenance, and ensures greater consistency in the artifacts developed. The emphasis on "model-driven"highlights the aspect of accelerating the development process, particularly through automation.
- Model simulation and execution: Simulations can be performed on the basis of models to provide insights into functions or the consequences of changes to a system. In conjunction with the automated transformation mentioned above, abstract and non-technical models can also be executed directly. Model-driven development thus offers increased validation potential and enables high-quality developments in a short time.
- Information gathering: Model-driven development opens up further opportunities for gathering and analyzing information. Process chains and responsibilities can be identified, and much of the documentation can be handled by integrated version and change management. In addition, metrics can be developed around the execution of models to improve quality assessment and facilitate decisions about specific changes and further developments. Furthermore, due to the importance of metadata in the context of modeling, there is increased potential for structuring unstructured information.
- Standardization: Jointly developed model structures and rules enable development processes within an organization to be standardized.
characteristics
The basis of MDA is the capture of models as data that can be queried, analyzed, validated, simulated, and transformed into other useful formats (such as executable source code). The following features are important here:
- Modeling: The main task in model-driven development is to organize the data behind a diagram or graphic into a clearly defined structure. This data concept or semantics provides information about what the data means in a specific technical context (domain) and enables its further use. A model can represent any context-specific aspects such as business processes, information carriers, or resources. Various modeling languages are now established for representation, in particular Business Process Model and Notation (BPMN)What does BPMN mean? BPMN is the abbreviation for Business Process Model... More, Unified Modeling Language (UML)UML (Unified Modeling Language) is a standardized, graphical modeling language that... More, Entity-Relationship Modeling (ER) or Extensible Markup Language (XML).
- Abstraction: Abstraction is an essential feature of every model. The more a model is tailored to a specific system or technical solution, the less abstract it is considered to be. In general, three levels of abstraction have become established here.
- A business model/conceptual model contains the characteristics and relationships of the real objects of the domain to be modeled and no technical information. In MDA, this model is called a computation-independent model (CIM).
- Based on the business model, the logical model describes how the object-related system components interact to achieve the goals of the system or organization. In MDA, this model is called the Platform Independent Model (PIM).
- The implementation model contains information on how the specific system components interact to fulfill their overall function. It is therefore usually tied to a specific technology (in MDA Platform). In MDA, it is referred to as a Platform Specific Model (PSM).
- Model transformation: One key added value offered by MDA is the automation of the path from abstract, stakeholder-focused models (CIM) to executable information systems. This goal cannot always be fully achieved. Nevertheless, it is the main advantage of MDA due to the reduction in effort and development risks and should therefore be pursued at all costs. As a rule, the three-step process involves moving from the conceptual or business model (CIM) to the logical, platform-independent model (PIM) and finally to the platform-specific model (PSM). In the context of automation, the transformation patterns between the individual stages must be defined, for example in XML.
- System life cycle: MDA has a major influence on the design of the system development life cycle (SDLC), which is an important prerequisite for a holistic management approach to the creation and maintenance of an information system.
Advantages
Model-driven development offers many advantages, which have already been mentioned in the previous paragraphs. In summary, the following points can be noted:
- By using the business model, stakeholders are better integrated into the development of information systems and can contribute more. This reduces the complexity of the task and better highlights the technical expertise of the domain. The final system corresponds to a greater extent to the business interests of the organization.
- The potential for automation can reduce costs and enable solutions to be developed more quickly. Operational benefits can be achieved sooner.
- Platform-independent modeling can be used for various technologies.
- The transformation ensures a high degree of consistency between the individual models. Changes can thus be comprehensively documented.
- Impact assessments can be easily carried out using simulations.
- The approach supports agile software development methods, as systems with a strong technical influence can be developed incrementally and iteratively in short sprints.
- By generating source code, less time needs to be spent on coding or debugging.
Model-driven SAP HANA SQL data warehousing
Although data warehousing has become an established part of decision support systems (DSS) over the past 25 years, data warehouse projects are still often complex, time-consuming, and costly. It is therefore not surprising that the ideas of model-driven development have also found favor in the data warehouse environment and that specific forms of model-driven data warehousing exist in literature and practice. SAP HANA SQL Data Warehousing also relies on a model-driven approach that reflects many elements of MDA.
Requirements Management
For development work, the requirements for what is to be developed are decisive. This applies all the more to a model-driven approach, as the specific requirements are ideally already incorporated into a model. For a data warehouse, defining requirements is no easy task, as is the case with other information systems. As a DSS, it should provide information for business decisions and enable business intelligence. This essentially results in three influencing factors that need to be taken into account. On the one hand, the requirements of the technical decision-makers or ultimate users of business intelligence in the form of reports, dashboards, and other visual presentations of data are relevant for decision-making. The needs of the various departments must also be incorporated into the strategic corporate orientation, as the classic enterprise data warehouse generally attempts to place the data in a context that is valid throughout the company. The third influencing factor is the existing data sources from which the information must be extracted. SAP HANA SQL data warehousing focuses primarily on the first two aspects, as the comprehensive system openness of the SQL approach means that details of the system landscape are not so relevant in the first step. In our consulting practice, we focus in workshops with users on the technical insights and metrics that are to be gained in the end, and in this way we develop an initial conceptual data model. This business model is extremely valuable as a means of communication when exchanging information with technical experts and forms the central requirements profile that is used for all further steps.
Data warehouse architecture
A data warehouse is characterized by an architecture of interdependent layers. For a model-driven approach, the characteristics of these layers are of great importance, as the defined structure is reflected in the related models. Basically, the following picture emerges for the SAP HANA SQL DWH (Figure 01): Data from various source systems is integrated into the data warehouse via ETL routes or virtually. In the SAP HANA SQL DWH, which is modeled according to the data vault approach, the data for BI and analytics front ends is prepared in multidimensional or specific presentation forms. The SAP HANA platform offers the advantage that, thanks to the powerful performance of the SAP HANA database, complex calculations can already be performed comprehensively and quickly at the database level.
Figure 01: Basic DWH architecture
development approach
As described above, the business model forms the starting point for the model-driven development of the SAP HANA SQL DWH at the conceptual level and corresponds to the Computation Independent Model (CIM) of the MDA system. It is created in SAP PowerDesigner, which is used as the main modeling tool and displays the model in a very simple ER diagram (information on all SAP HANA SQL DWH tools can be found here). In the next step, the business model is converted into the physical data model of the data warehouse (core data vault). The technical creation is based primarily on the structures of the existing data sources, as these can be used to automatically generate a model with the help of SAP PowerDesigner. From this physical reverse model of the data sources, essential parts of the Core Data Vault are generated via the intermediate step of the staging model (data integration) and enriched and designed with the technical information from the business model. The logical level and thus the Platform Independent Model (PIM) are bypassed, as SAP HANA database artifacts are to be created. However, SAP PowerDesigner also offers functions for generating a logical data model if required. The physical data model of the data warehouse is structured in the form of a Data Vault, as this offers the greatest advantages in terms of flexible further development and agile data warehousing in line with the DevOps philosophy .
The physical data vault model ultimately serves as the basis for the required analytical model (virtual analytical model). Here, too, the technical considerations of the business model play an important role, and the model objects are linked to each other. In general, the possibility of model transformations and the automatic generation of models and source code is used as often as possible, and at least all models are linked to each other. This creates a coherent model structure for the SAP HANA SQL DWH (Figure 02). The source code can be generated automatically from this structure and imported into the SAP HANA platform and executed using the distributed source code management system Git.
Figure 02: Model-driven SAP HANA SQL data warehousing
Advantages
The explanations show that many advantages of the MDA standard can also be achieved in model-driven SAP HANA SQL data warehousing. The following points are worth highlighting:
- Communication with specialist departments: Based on the business model, the requirements of specialist decision-makers for the data warehouse can be incorporated in the best possible way, and subsequent transformations ensure that the technical expertise is not lost in the course of the increasingly technical work steps.
- Automation: Automatic generation of models and source code is possible at many stages of SAP HANA SQL DWH development, reducing the amount of work involved and the risk of errors.
- Quality: The schematic overall process and associated rules, such as modeling and naming conventions, as well as the use of transformations and the automatic generation of models and source code, increase quality by reducing the risk of errors. Lineage, impact, and usage analyses also offer the possibility of simulating development steps and running through scenarios.
- Transparency: Due to the comprehensive interrelationships, SAP HANA SQL DWH offers a high level of transparency. Developments can be easily documented in the background and are traceable at any time.
- Standardization: Model-driven SAP HANA SQL data warehousing offers significant potential for standardization.
Conclusion
In this blog, we have introduced you to the concept of model-driven development and presented the basic concept of Model Driven Architecture (MDA). Many useful features of this fundamentally software-related standard can be used for SAP HANA SQL data warehousing. SAP HANA SQL data warehousing can thus be implemented in a highly agile manner with a strong focus on business requirements, quickly generating usable business intelligence for operational decision-makers. This is also due to the fact that the model-driven approach can be easily combined with the other features of SAP HANA SQL data warehousing, which we have introduced in this blog . If you have any further questions about SAP HANA SQL data warehousing or model-driven development, please do not hesitate to contact us.
Authors: Tim Andzinski, Martin Peitz
Marius Wimmers
Service Manager
SAP Information Management
marius.wimmers@isr.de
+49 (0) 151/42205-434
