TOPIC 1: HISTORY OF forerunner of contemporary data

FUNDAMENTAL DATA warehouse technology).

WAREHOUSING 5. 1983 – Teradata introduces the
Content Discussion: DBC/1012 database computer
specifically designed for decision
What is fundamental data of warehousing? support.

• The concept of data warehousing dates 6. 1984 – Metaphor Computer

back to the late 1980s when IBM Systems, founded by David Liddle
researchers Barry Devlin and Paul Murphy and Don Massaro, releases a
developed the "business data warehouse". In hardware/software package and GUI
essence, the data warehousing concept was for business users to create a
intended to provide an architectural model database management and analytic
for the flow of data from operational system. 1985 - Sperry Corporation
systems to decision support environments. publishes an article (Martyn Jones
and Philip Newman) on information
• The concept attempted to address the centers, where they introduce the
various problems associated with this flow, term MAPPER data warehouse in the
mainly the high costs associated with it. In context of information centers.
the absence of a data warehousing
architecture, an enormous amount of 7. 1988 – Barry Devlin and Paul Murphy
redundancy was required to support multiple publish the article "An architecture for
decision support environments. In larger a business and information system"
corporations, it was typical for multiple where they introduce the term
decision support environments to operate "business data warehouse".
independently. 8. 1990 – Red Brick Systems, founded
• Though each environment served different by Ralph Kimball, introduces Red
users, they often required much of the same Brick Warehouse, a database
stored data. The process of gathering, management system specifically for
cleaning and integrating data from various data warehousing.
sources, usually from long-term existing 9. 1991 – Prism Solutions, founded by
operational systems (usually referred to as Bill Inmon, introduces Prism
legacy systems), was typically in part Warehouse Manager, software for
replicated for each environment. Moreover, developing a data warehouse.
the operational systems were frequently re-
examined as new decision support 10. 1992 – Bill Inmon publishes the
requirements emerged. Often new book Building the Data Warehouse.
requirements necessitated gathering,
cleaning and integrating new data from "data 11. 1995 – The Data Warehousing
marts" that was tailored for ready access by Institute, a for-profit organization that
users. promotes data warehousing, is
founded.
Key developments in early years of data
warehousing: 12. 1996 – Ralph Kimball publishes the
book The Data Warehouse Toolkit.
1. 1960s – General Mills and Dartmouth
College, in a joint research project, 13. 2000 – Dan Linstedt releases in the
develop the terms dimensions and public domain the Data vault
facts.[11] modelling conceived in 1990 as an
alternative to Inmon and Kimball to
2. 1970s – ACNielsen and IRI provide provide long-term historical storage
dimensional data marts for retail of data coming in from multiple
sales. operational systems, with emphasis
3. 1970s – Bill Inmon begins to define on tracing, auditing and resilianc9e to
and discuss the term Data Warehouse. change of the source data model.

4. 1975 – Sperry Univac introduces 14. 2012 – Bill Inmon develops and
MAPPER (maintain, Prepare, and makes public technology known as
Produce Executive Reports), a "textual disambiguation". Textual
database management and reporting disambiguation applies context to
system that includes the world's first raw text and reformats the raw text
4GL. It is the first platform designed and context into a standard data base
for building Information Centers (a format.
What Is a Data Warehouse? enhances decision-making efficiency.
• A data warehouse is a database designed • A data warehouse usually stores many
to enable business intelligence activities: months or years of data to support historical
analysis. The data in a data warehouse is
• it exists to help users understand and typically loaded through an extraction,
enhance their organization's performance. transformation, and loading (ETL) process
from multiple data sources.
•It is designed for query and analysis rather
than for transaction processing, and usually • Modern data warehouses are moving
contains historical data derived from toward an extract, load, transformation (ELT)
transaction data, but can include data from architecture in which all or most data
other sources. transformation is performed on the
database that hosts the data warehouse. It
• Data warehouses separate analysis is important to note that defining the ETL
workload from transaction workload and process is a very large part of the design
enable an organization to consolidate data effort of a data warehouse. Similarly, the
from several sources. speed and reliability of ETL operations are
the foundation of the data warehouse once it
This helps in: is up and running.
1. Maintaining historical records. • Users of the data warehouse perform data
analyses that are often time-related.
2. Analyzing the data to gain a better Examples include consolidation of last year's
understanding of the business and to sales figures, inventory analysis, and profit
improve the business. by product and by customer. But
• In addition to a relational database, a data timefocused or not, users want to "slice and
warehouse environment can include an dice" their data however they see fit and a
extraction, transportation, transformation, well-designed data warehouse will be
and loading (ETL) solution, statistical flexible enough to meet those demands.
analysis, reporting, data mining capabilities, Users will sometimes need highly
client analysis tools, and other applications aggregated data, and other times they will
that manage the process of gathering data, need to drill down to details. More
transforming it into useful, actionable sophisticated analyses include trend
information, and delivering it to business analyses and data mining, which use
users. existing data to forecast trends or predict
futures. The data warehouse acts as the
• To achieve the goal of enhanced business underlying engine used by middleware
intelligence, the data warehouse works with business intelligence environments that
data collected from multiple sources. The serve reports, dashboards and other
source data may come from internally interfaces to end users.
developed systems, purchased applications, • Although the discussion above has
third-party data indicators and other sources. focused on the term "data warehouse", there
It may involve transactions, production, are two other important terms that need to
marketing, human resources and more. In be mentioned. These are the data mart and
today's world of big data, the data may be the operation data store (ODS).
many billions of individual clicks on web
sites or the massive data streams from • A data mart serves the same role as a data
sensors built into complex machinery. warehouse, but it is intentionally limited in
• Data warehouses are distinct from online scope. It may serve one particular
transaction processing (OLTP) systems. department or line of business. The
With a data warehouse you separate advantage of a data mart versus a data
analysis workload from transaction warehouse is that it can be created much
workload. Thus data warehouses are very faster due to its limited coverage. However,
much read-oriented systems. They have a data marts also create problems with
far higher amount of data reading versus inconsistency. It takes tight discipline to
writing and updating. This enables far better keep data and calculation definitions
analytical performance and avoids impacting consistent across data marts. This problem
your transaction systems. A data warehouse has been widely recognized, so data marts
system can be optimized to consolidate data exist in two styles. Independent data marts
from many sources to achieve a key goal: it are those which are fed directly from source
becomes your organization's "single source data. They can turn into islands of
of truth". There is great value in having a inconsistent information. Dependent data
consistent source of data that all users can marts are fed from an existing data
look to; it prevents many disputes and warehouse. Dependent data marts can avoid
the problems of inconsistency, but they 4.Non-volatile
require that an enterprise-level data
warehouse already exist. Once data is in the data warehouse, it will
not change. So, historical data in a data
• Operational data stores exist to support warehouse should never be altered.
daily operations. The ODS data is cleaned
and validated, but it is not historically deep: it
may be just the data for the current day.
Rather than support the historically rich Data Warehouse Functions:
queries that a data warehouse can handle,
the ODS gives data warehouses a place to 1. Staging- Staging is used to store raw data
get access to the most current data, which for use by developers.
has not yet been loaded into the data
warehouse. 2. Integration- The integration layer is used
to integrate data and to have a level of
• The ODS may also be used as a source to abstraction from users.
load the data warehouse. As data
warehousing loading techniques have 3. Access- The access layer is for getting
become more advanced, data warehouses data out for users.
may have less need for ODS as a source for With data marts it stores subsets of data
loading data. Instead, constant trickle-feed from a warehouse, which focuses on a
systems can load the data warehouse in specific aspect of a company like sales or a
near real time. marketing process. This definition of the
A common way of introducing data data warehouse focuses on data storage.
warehousing is to refer to the The main source of the data is cleaned,
characteristics of a data warehouse as set transformed, catalogue and made available
forth by William Inmon: for use by managers and other business
professionals for data mining, online
1. Subject Oriented analytical processing, market research and
decision support.
2. Integrated
3. Nonvolatile
TOPIC 2: Business Process Modeling
4. Time Variant
Content Discussion:
• Business process modeling (BPM)- In
business process management and systems
engineering is the activity of representing
processes of an enterprise, so that the
current process may be analyzed, improved,
Characteristics of a data warehouse: and automated.
1. Subject-Oriented • BPM is typically performed by business
analysts, who provide expertise in the
A data warehouse can be used to analyze a modeling discipline; by subject matter
particular subject area. For example, “sales” experts, who have specialized knowledge of
can be a particularbsubject. the processes being modeled; or more
commonly by a team comprising both.
2. Integrated Alternatively, the process model can be
derived directly from events' logs using
A data warehouse integrates data from process mining tools.
multiple data sources. For example, source
A and source B may have different ways of
identifying a product, but in a data
warehouse, there will be only a single way of
identifying a product.
3. Time-Variant
Historical data is kept in a data warehouse.
For example, one can retrieve data from 3
months, 6 months, 12 months, or even older
data from a data warehouse. This contrasts
with a transactions system, where often only
the most recent data is kept. • Business process - A business process is a
collection of related, structured activities or
tasks that produce a specific service or
product (serve a particular goal) for a
particular customer or customers.
There are three main types of business
processes:
1. Management processes, that govern
the operation of a system. Typical 2.UML Diagrams
management processes include
corporate governance and strategic • Is a modeling language mainly used for
management. specification, visualization, development and
documenting of software systems. But
2. Operational processes, that business professionals have adapted it as a
constitute the core business and powerful business process modeling
create the primary value technique.
stream.Typical operational processes
are purchasing, manufacturing, 3. Flowchart Technique
marketing, and sales.
• Are probably the most popular diagram
3. Supporting processes, that support type in the world. Because it has few
the core processes. Examples include standard symbols it can be easily
accounting, recruitment, and technical understood by many. Also, most drawing
support. software support creation of flowcharts it is
used by a much wider audience as well.
Some business process modeling
techniques are:
1.Business process modeling notation
(BPMN)
2.UML diagrams
3.Flowchart technique
4.Data flow diagrams
5.Role activity diagrams
4. Data Flow Diagrams – Yourdon’s
6.Role interaction diagrams Technique
• Data flow diagrams (DFD) show the flow of
data or information from one place to
another. DFDs describe the processes
showing how these processes link together
through data stores and how the processes
relate to the users and the outside world.
They are used to record the processes
analyzed as a part of the design
documentation.
• A DFD can be seen as a method of
organizing data from its raw state. DFDs are
7.Gantt charts the backbone of structured analysis that was
developed in the early sixties by Yourdon.

1.Business Process Modeling Notation

(BPMN)
• Is a graphical representation of your
business process using standard objects. If
you want to get more technical It can also be
defined as a set of graphical objects and
rules defining available connections between
the objects.
5. Role Activity Diagrams – RAD TOPIC 3: Warehousing Design and
• Roles are abstract notations of behavior Functions
describing a desired behavior within the
organization. They are often organizational Content Discussion:
functions. They also include software
systems, customers and suppliers. Warehouse Management
Warehouse management System- Is the
control of the day-to-day operations of a
warehouse, such as the shipping, receiving,
putaway and picking of goods.
• The WMS receives orders from the
overlying host system, mostly an ERP
system,
• manages these in a database and, after
appropriate optimization, supplies them to
the connected conveyor control systems.
6. Role Interaction Diagrams – RID
Warehouse management function:
• Activities are connected to roles in a type
of matrix. Activities are shown vertically on 1. Planning
the left axis and the roles are shown
horizontally at the top. Text and symbols are • Finalizing the daily plan for receiving dock
used together in order to represent the activity, selecting the workload to be
process. processed in the day or shift, calculating an
estimate of the labor and vehicles required
to pick and ship the orders to ensure the
staffing is appropriate, and to notify carriers
regarding to load and depart to meet
customer requirements.
2. Organizing
• Sequencing the orders to be picked.
Organizing orders for picking can be
accomplished in many ways, meeting the
needs of the user. The initial way of
7. Gantt Charts: organizing was called wave planning or wave
picking, with two objectives, a. to minimize
• A Gantt chart is a matrix that lists on the need for dock staging space b. to create an
vertical axis all the tasks or activities to be order of flow that will support monitoring the
performed in a process. Each row contains a progress through the day and reduce last
single activity identification, which usually minute requests for overtime or delay of
consists of a numaber and a name. carrier departure.
• The horizontal axis is headed by columns 3. Staffing
indicating estimated activity duration, skill
level needed to perform the activity, and the • Assign staff to work functions and areas,
name of the person assigned to the activity, by wave, to minimize staging.4. Directing
followed by one column for each period in
the project’s duration. • Ensuring the documented processes and
procedures are embedded in the WMS and
are consistently applied, used and
appropriate for the nature of the work and
service level intentions of the company5.
Controlling
• Providing milestones for management to
monitor progress through the day, providing
the opportunity to respond to problems in a
timely way, and report data for performance
analysis.
Advantages of using a Warehouse and postimplementation stages.
Management System (WMS)
3. Granting unsuitable levels of
A WMS will enable you to analyze current authorization to employees within the
procedures and performance, and system.
implement improvement that will:1.Reduce
picking errors: • In the case of custom solutions, it is the
client who decides which users will be able
• Picking errors are costly at any stage of the to view, edit, add, or delete data.
process, if they are spotted before dispatch,
there are the labor costs of re-picking. If Granting authorization to wrong people may
incorrect items are shipped to the customer, prevent your processes from running
not only is it expensive to accept the return smoothly.
and process the replacement, but there is
the hidden cost of damage to the customer 4. Potentially unsuccessful implementation.
relationship. Like every investment, also this one entails
certain risk.
2.Optimize stock control:
• To avoid unsuccessful implementation, it is
There must be a balance between having important to make a detailed analysis of
stock available for customers, and trying up your needs, choose a competent software
too much capital.3.Maximize use of space house, consult your project with your
employees in terms of how the application
• The right proportion of floor storage, should look like, as well as commit time to
vertical storage, racks and pallets for your properly train your staff on how to use the
warehouse will depend on the types of good application. Working with our clients, we pay
stored and the patterns of distribution. attention to each of these elements so that
you get a guarantee of the success of the
4.Improve worker productivity project.
• Your workforce is likely to be your biggest Warehouse value-adding roles.
cost. How much would you save if you
improved productivity.
5. Ensure compliance with Health and
Safety regulations
• A WMS can guide workers through risk
assessments and flag up warehouse safety
requirements, protecting your company,
workers and customers alike.
Dis-Advantages of Warehouse Management
System (WMS): Transportation consolidation:
1.Many processes are integrated into a
system
• An error in one place entails errors in
others. In this way, a human error can cause
the whole system to be inaccurate. To
prevent such situations, there are certain
operations that the system will not allow you
to perform (e.g. issuing more goods than
what is available in the warehouse).
2. Risk of choosing incorrect parameters
before use. SUPPLY AND PRODUCT MIXING:
• If the designed modules, categories, and
processes are not well-tailored to your
company, the system will gather data in an
inefficient way. This may mean that your
solution will not work as it was supposed to.
To avoid this scenario, we ensure you get
our support not only while we build the
software but also during the implementation
 Flexibility
Basic warehouse decisions: a cost trade-off Public Warehousing Service
framework:
Bonded warehousing
Ownership
Field warehouse
Public versus contract versus private
Public warehousing regulation
Centralized or Decentralized
Warehousing Liability
How many Receipt
Location Public warehousing rates based upon:
Size Value
Layout Fragility
What products where Potential damage to other goods
Volume and regularity
Basic warehousing decisions: Weight density
Services required Contract
Warehousing:
The ownership decision: Up 23% per year in 2000 to $20.4
billion.
Public warehousing costs mostly all
variable. Compensation for seasonality in
products.
Private warehousing costs have a
higher fixed cost component. Increased geographical coverage.
Thus private warehousing virtually Ability to test new markets.
requires a high and
Managerial expertise and dedicated
constant volume. resources.
Less strain on the balance sheet.
The ownership decision: Possible reduction of transportation
costs.
Factors to consider:
The Number of Warehouses
Throughout volume
Factors affecting the number of
Stability of demand Warehouses.
Density of market area to be served Inventory costs
Security and control needs Warehousing costs
Customer service needs Transportation costs
Multiple use needs of the firm Cost of lost sales
Maintenance of customer service
levels
Public warehousing:
Service small quantity buyers
Rationale for Public Warehousing
Limited capital investment