Unit 4 - Overview of Modeling with SAP HANA Cloud

Developing Data Models with SAP HANA Cloud

 Develop a data model using SAP Business Application Studio.

Old Approach - Data Processing in the Application

In a legacy application, the role of the database is to simply store and provide raw data to the application.

The raw data is sent from the database directly to the application. The application then begins to process the data by combining it,
aggregating it, and performing calculations in order to generate meaningful output.
We can find ourselves moving a lot of raw data between the database and the application. Unfortunately, when we move raw data
to the application layer, we make the application code very complex. This is because the code has to deal with the data processing
tasks as well as managing all of the other parts of the application including process flow control, business logic, user Interface (UI)
operations, integrating data from multiple sources, and so on.

New Approach - Data Processing in the Database

With SAP HANA Cloud, we can build a sophisticated data processing layer on top of the database tables, right inside the database.
This means we can first process the raw data and turn it into something meaningful in the database before passing the results to the
application for consumption.
With SAP HANA Cloud, the main type of modeling object is called a calculation view. A calculation view can handle a wide
variety of data processing tasks. For example, calculation views can combine data from multiple tables or other calculation views,
apply filters, calculate new data and aggregate the data at any level. A calculation view does not store data, it simply calculates
results on-the-fly in memory. The calculation views are developed in SAP HANA Cloud using easy-to-use graphical modeling
tools, and are stored in SAP HANA Cloud database.
So, instead of the application including complex code to process the data, the application now calls the required calculation views
and the processing is pushed down to SAP HANA Cloud database and the results are passed to the application for further
processing. This is efficient in the following ways:

 The application code is simplified, as it does not have to deal with many data processing tasks. These tasks are
pushed down to SAP HANA Cloud database where in-memory processing takes place.

 The processing on the data is carried out where the data resides, so we do not have to move raw data from the
database to the application. We only move the results of the data processing to the application.

 The calculation views can be reused in multiple applications so we avoid redundancy.

In traditional applications, there is a high degree of redundancy in the application code. Developers find themselves continually
creating the same code to process data. SAP HANA Cloud database can take over all data processing tasks from the application.
SAP HANA Cloud database takes care of complex calculations and data flow logic, including executing aggregations and
disaggregation. SAP HANA Cloud calculation views can contain dynamic placeholders. This means a business user can provide
input values to the calculation view at run-time using a pop-up prompt. Typical uses cases for this are to provide filter values to
reduce the data set.
Calculation views can consume the results of other calculation views. This encourages a high degree of modularization and reuse of
calculation views. A calculation view is usually part of a stack of calculation views that make up a complete data model. It is good
practice to design a calculation view considering future uses of that view so that it can easily be reused. This avoids redundancy of
models and encourages easier maintenance.
Advanced Data Processing Capabilities

Types of Calculation Views

The calculation view is the main modeling object in SAP HANA Cloud.

There are three types of calculation view:

 Dimension

 Cube

 Cube with star join

Calculation views are created using a graphical editor with the tool SAP Business Application Studio. To develop a
calculation view you should be familiar with basic data modeling concepts.
Modeling Dimensions
The purpose of a dimension type of calculation view is to define a group of related attributes. For example, a material dimension
might include attributes such as material description, material color, material weight, and material price. Although this type of
calculation view can be directly consumed by an application, it is most likely to be found as a consumable view in a calculation
view of the type Cube with star schema to provide the cube's dimensions (see later for details).
It might be helpful to think of calculation views of type dimension as central master data views. You define them once and reuse
them many times. To get started with calculation views of type DIMENSION, you need to set the data category to DIMENSION.
You then proceed to define the source tables and joins if they are needed. You define filters, and then identify the output columns
that are to be exposed by the view. It is also possible to define additional derived attributes. An example of this could be a new
column to generate a weight category based on a range of weights. You could use an IF expression to check each weight and
assign a category.
A dimension calculation view can consume other dimension calculation views. For example you could create a new dimension
called partners that joins customer dimension and customer dimension.

Modeling Cubes
The next type of calculation view is the type Cube. This type of calculation view is used to define a data set comprised of attributes
and measures. This means they are ideal for transactional data modeling. For each measure you are able to define its aggregation
behavior so that business users can slice and dice in an ad-hoc manner and all measures are aggregated according to their behavior.
A Cube calculation view can consume other Cube calculation views or dimension calculation views. A cube calculation view is a
relational model not a dimensional model.
Applications can directly access this type of calculation view. They can also be accessed via SQL statements.

To get started, set the data category to CUBE.

Do not select the Star Join flag.

You then select the table, or tables, which are to be included in the model. Typically, you choose a transaction table so that you
have columns from which you can define attributes and measures. It is possible to include more than one transaction table. For
example, you may need to include a header and a line item table to form the complete picture of a sales transaction. In this case,
you simply join the tables. You can also merge transaction tables by implementing a union. Then, select the columns from the
tables that are to be exposed. You can optionally set filters and define additional calculated columns. Finally, rename any columns
to provide meaningful names to the business user.

Modeling Cubes with Star Joins

Now we come to the most powerful of all the calculation views: the Cube type, but with an additional setting — star join. This is
how we model a star schema. This model is based on the cube type of calculation view, but with one or more dimension calculation
views joined to the cube. This provides a very powerful and flexible dimensional data model that enables drill-down, slice and dice
analysis over any combination of attributes and measures. Adding multiple dimensions dramatically enhances the analysis
possibilities compared to cube type calculation views.
To get started, make sure that you set the data category to CUBE and select the Star Join flag.
First, select the transaction table(s). Use joins to combine tables if needed. Then, choose the columns to expose, set any filters, and
create any calculated columns. What you are doing up to this point is forming a central fact table that is used as the hub of the star
schema.
The next step is to form the star schema by joining suitable calculation views of type DIMENSION to the fact table.
Finally, adjust the labels of any columns to make them more meaningful by using the rename function in the Semantic node.
Cube with star join type calculation views can consume other Cube with star join calculation views and also
simple Cube calculation views.

Modeling with Business Application Studio

The Business Application Studio is a multi-purpose integrated development environment (IDE) tool that is used to create all
artifacts used in a full-stack application. This includes the UI, application code and database objects.
Web IDE for SAP HANA was introduced in 2016 for creating development artifacts for SAP HANA on-premise. This interface can
still be used with SAP HANA Cloud but it is recommended to use Business Application Studio as this is not only more feature rich,
all new modeling features will only appear in the Business Application Studio and not in the Web IDE.
The Business Application Studio also includes a terminal console so commands can be entered directly using a command line
interface (CLI) instead of using the GUI.

With Business Application Studio you create all data modeling artifacts including:

 Calculation Views

 Procedures
  Table and Scalar Functions

 Flowgraphs

 Analytic Privileges

 Local and Virtual Tables

 Replication Tasks

The artifacts are created using either graphical editors or text editors. You are able to swap between the types of editors. The
artifacts are stored in simple text files which makes them very portable. Apart from SQL, the format of the files is usually JSON or
XML.
Watch this video to identify the key areas of the Business Application Studio.
For version control of the database artifact files, the open-source Git framework is used. Git is fully integrated in Business
Application Studio. This means as you develop artifacts you can commit them to the team repository. You can also go back to older
version of any artifact. Git allows developers to clone a project and work in parallel with other developers on sections of the project
without affecting others. When development is complete, the artifacts from different branches of the project can be merged together
using Git.

Extending Calculation Views with SQLScript

Calculation views are built using a graphical approach and no coding is required. However, sometimes the graphical approach does
not provide all the functions and data flow logic you require for a complex calculation view. This is when you use functions.
Functions are built using SQLScript which is the native database language of SAP HANA Cloud.
In SAP HANA Cloud modeling, table functions are typically used to generate a tabular data set that is used as a data source in a
calculation view. Table functions can be used in SQLScript in a from clause of a select statement (in other words wherever a
standard table identifier is used). A table function encapsulates the logic in a reusable form so that it can be used many times in
different artifacts.
Table functions can accept one or more input parameters. Table functions are read-only; that is, they cannot be used to change data.
Table functions produce exactly only one tabular output. Table functions can also call other functions.

Add SQL to Models Using Procedures

Procedures define reusable data processing logic that can be used to enhance a calculation view. Procedures are very similar to
functions in that they are written in SQLScript and can have one or more inputs and they always have outputs. However,
procedures can produce multiple output data sets of different structures, whereas a table function can only return one tabular output
structure. Procedures have a limited role in data modeling. Procedures cannot be used as data sources to calculation views but they
are used in other places in a calculation view, for example, to automatically derive values for an input parameter.
A procedure can be called directly from SQLScript, which means it can be called from a function or even another procedure.
Procedures used within modeling must be set to read-only. In that case they are called stateless (or side-effect free), because they
don't alter any data or meta data in the database. However, procedures can also be used to update, insert, and delete data if required.
These procedures are called stateful, and these type are not allowed when called from calculation views. Stateful procedures are
more likely to be used by developers who build applications that change data, rather than modelers.

Securing Data Models

There are three levels of access to consider. Let's take a closer look at them.
There are 3 level of access to consider:

1.level (access role): veri modelinde yer alan bizim geliştirdiğimiz nesnelere olarak erişim. Bunlar CV içinde yer verdiğimiz TF,
Tables, Functions vb... nesnelerdir. Bunlara erişim db access permision ile mümkün oluyor, bunun anlamı bu nesnelere SELECT
yetkisi vermemiz gerekiyor. Bu yetki ilk olarak role verilirse, role dahil edilen her user için de geçerli olmuş olacak. Bu şekilde CV
içinde yer alan DB Nesnelerine erişim hakkımız olacak ama verileri göremeyeceğiz.

2.level (analytic privileges): you now need data access at the row level (satır ve/veya sütün bazında verileri görüntülemek/erişmek
için yetki gerekiyor). Verilere erişim “analytic privilage” ile mümkün oluyor. Analytic Privilage, user’a veya role assign edilmesi
yeterli oluyor.

Analytic Privilage = is an object created to describe for each CV roles to allow data access (CV bazında verilere erişmek için
oluşturulan role nesneleridir). Roller, sadece ŞK1000 verilerinin görüntülenmesi ve/veya Year2017-2018 dönemine ait ŞK1000
verilerinin görüntülenmesi için oluşturulabilir. A user can be assigned to multiple analytic privileges.

3.level (masking role): herhangi bir row veya column’da yer alan verileri bazı user’lar görüntüleyebilirken bazı user’lar
görüntüleyemez. Onlara maskeleme uygulanmıştır.

Advanced Analytics with SAP HANA Cloud

 Describe advanced analytics with SAP HANA Cloud.
SAP HANA Cloud Spatial
Many organizations already rely on spatial data processing and use specialist applications alongside, but separate from their
business process applications. For example, a gas storage tank is leaking and an emergency repair order is raised in the SAP ERP
system by a coordinator. Next, the coordinator uses a separate geo-based application and manually enters the asset number of the
tank. The tank shows up on a map, and the coordinator then uses yet another application to identify the nearest available, qualified
engineer, who is then dispatched. Multiple, unconnected applications are needed in this business process.

Missed Opportunities using Different Applications

Beyond business applications, there are more exciting use cases for spatial analysis in the sports environment. SAP has developed a
series of applications that provide deep analysis of player performance. For example, in golf, by adding a sensor to the ball and pin,
we can create a graphical history to illustrate the improvements in accuracy of the shot. These types of applications are already in
use by major sports organizations around the world.
There are many applications that could be dramatically enhanced with the integration of spatial data.
SAP Spatial provides new data types for storing geometrical data such as points, lines, and polygons. These can be used to
represent precise locations, roads, and regions. SAP HANA Cloud Spatial uses open standards and so can easily be integrated with
well-known, leading geospatial providers such as ESRI, OGC, OpenStreepMap, GoogleMap.
As well as storing spatial data, SAP HANA Cloud also provides spatial query functions that can easily be included in SQL Script.
Here are some examples of the functions:

 Within — which customers are in my region?

 Distance — what is the longest distance a high-value customer has to travel to reach my sales outlet?

 Crosses — where does truck route A cross truck route B?

SAP HANA Spatial also provides algorithms that can determine clusters. This helps an organization to locate precise locations that
might be lucrative based on income data and other interesting attributes associated with consumers = SAP HANA Spatial ayrıca
kümeleri belirleyebilen algoritmalar da sağlar. Bu, bir kuruluşun gelir verilerine ve tüketicilerle ilişkili diğer ilginç özelliklere
dayalı olarak kazançlı olabilecek kesin konumları bulmasına yardımcı olur.
Predictive Analysis
Watch this video to learn about predictive analysis approaches.

Predictive modelleme/uygulama geliştirebilmemiz için SAP 2 yaklaşım sunuyor. Bunlar yukarıdakilerdir.

SAP HANA Predictive Analysis Library (PAL)

Predictive Analysis Library (PAL) contains a large number of algorithms that can be used to develop predictive and
machine learning models. Some of these algorithms are used for data mining pre-processing tasks such as:

 Sampling — select a few records from large data sets (for example, we need 1000 people from each country).

 Binning — grouping records into basic categories (for example, age ranges).

 Partitioning — creating sets of data for training, testing, and validation used to train models and check their
predictive accuracy.

The majority of the algorithms are used for scoring or predictive modeling. There are many algorithms provided for all
major data mining categories including:

 Association

 Classification

 Clustering

 Regression

 Time Series

 Neural Networks

PAL algorithms are access can be called directly from procedures in SQLScript or they can be integrated into an SAP HANA
flowgraph which is built using a graphical editor in the Web IDE. A flowgraph defines the data inputs, data processing, and
outputs and parameters used in the predictive model.
Using PAL requires knowledge of statistical methods and data mining techniques. This is because the choice of which algorithm to
use must be made by the developer. So it is important that the developer initially understands the differences between the
algorithms used for data preparation, scoring, and prediction. But they must also know how to fine-tune the algorithms to reach to
desired outcome. For example, a developer would need to decide when to use the time series algorithm for double exponential
smoothing versus triple exponential smoothing and then how to adjust the parameters to consider trend or to eliminate outliers.
Developers who work with SAP HANA PAL are typically already working in predictive analysis projects or have a reasonable
understanding of the topic.

Automated Predictive Library (APL)

The Automated Predictive Library (APL) is aimed at developers who do not have (or desire to have) detailed knowledge of the
algorithms and the maths behind the models. The selection of algorithms for data preparations, scoring, and predictions is
completely automated (hence the name). All the developer has to do is to provide the data, and APL analyzes that data to identify
the best data preparation (cleaning up the data) and predictive models.

Graph Modeling
Graphs are used to model data that is best represented using a network. Examples include supply chain networks, transportation
networks, utility networks, and social networks. The basic idea behind graph modeling is that it allows a modeler to define a series
of entities (nodes) and link them with lines (edges) to create a network. This network represents how each node relates to all other
nodes.

Graph models can also indicate flow direction between entities and also any number of attributes can be added to nodes or the lines
that connect them. This means that additional meaning can be added to the network and queries can be executed to ask questions
relating to the network.
Imagine a complex supply chain mapped using a graph, where all manufacturers, suppliers, distributors, customers, and consumers
are represented with information stored along the connections. The benefit to this form of modeling is that it makes it easy to
develop applications that can traverse huge graphs at speed. As a result you can ask questions such as the following:
  How many hours has the product traveled between two specified points in the network?

 Where are all the possible points of origin of this product?

 Describe the entire journey of a product by listing all of the stop-off points in the path

 What is the shortest path between point A and point B?

Graph processing allows you to discover hidden patterns and relationships in your network data, and all in realtime.
SAP HANA Graph provides tools for graph definition and additional language for graph querying to ensure that model
development is more natural and simplified. It also guarantees that the processing is flexible, and of course, optimized for in-
memory processing using a dedicated in-memory graph engine right inside the database.