BIT4440_ICT4445

CLOUD COMPUTING AND BIG DATA

MR LISTONE KAPUTULA
 Cloud Computing and Big Data

• Introduction
• Cloud Computing Foundation
• Virtualization and Containerization
• Big Data Technologies
• Data Management in the Cloud
• Big Data Analytics
• Stream Processing
• Cloud Security
• Cloud Cost Management
• Cloud Project Management
 Introduction
Introduction
• Cloud computing refers to the delivery of
shared computing resources over the internet.
• Shared pool of computing resources are
provided on-demand which includes servers,
storage, databases, networking, software,
and applications.
• The key attributes of the cloud are:
✔ On-demand access to compute and storage,
✔ Limitless scaling,
✔ Flexible pricing
 Introduction
Evolution of Cloud Computing
 Introduction
Evolution of Cloud Computing
• Data Centers are physical facilities within
organizations where they house their IT
infrastructure.
✔ Including servers, storage systems, networking
equipment, and software applications.
• Organizations manage and maintain these data
centers on-premises.
✔ This required significant capital investment, space,
and resources.
✔ Complex maintenance, and the need for skilled IT
staff to manage the infrastructure
 Introduction
Evolution of Cloud Computing
• Virtualization technology emerged as a solution to
address the limitations of traditional data
centers.
• Virtualization enables the creation of multiple
virtual instances or virtual machines (VMs) on a
single physical server.
✔ Decoupling the hardware from the software.
• Hypervisor software, such as VMware, Hyper-V, and
KVM.
✔ Manages the virtualized environment by
allocating resources to VMs and isolating them
from each other.
 Introduction
Cloud Computing – Characteristics & *…
Models
• Cloud
 Introduction
Cloud Computing – Characteristics & *…
Models
• On-demand self-service: Users can provision
and access computing resources without
requiring human intervention from the service
provider.
• Broad network access: Cloud services are
accessible over the network through standard
mechanisms, enabling users to access resources
using a variety of devices.
 Introduction
Cloud Computing – Characteristics & *…
Models
• Resource pooling: Computing resources are
pooled together to serve multiple users
simultaneously.
• Users typically have no control or knowledge
over the exact physical location of the resources.
• Rapid elasticity: Computing resources can be
rapidly scaled up or down based on demand,
allowing users to access additional resources
when needed and release them when no longer
required.
 Introduction
Cloud Computing – Characteristics & *…
Models
• Measured service: Cloud providers monitor and
control the usage of computing resources,
enabling users to be billed based on their
consumption.
• Examples include pay-per-use models.
 Introduction
Ecosystem of Cloud Computing
• Consumers of services: These are the
end-users that use cloud services in their
day-to-day business activities.
• Provider of services: These cloud providers
offer a variety of functions ranging from
infrastructure services to applications and tools.
• Designer of services: These companies build
applications and tools. Often services are
intended to work within a specific cloud
ecosystem or can augment a packaged cloud
application.
 Introduction
Benefits of Cloud Computing
• Cost savings
✔ Pay-as-you-go pricing and reduced capital
expenditure on hardware and infrastructure.
• Scalability
✔ Ability to scale resources up or down dynamically
to meet fluctuating demand.
• Flexibility
✔ Access to a wide range of services and resources
on-demand, tailored to specific needs.
 Introduction
Benefits of Cloud Computing
• Reliability
✔ Redundant infrastructure, automated backups, and
disaster recovery capabilities ensure high availability and
reliability.
• Security
✔ Cloud providers implement robust security measures,
including encryption, access controls, and compliance
certifications.
• Innovation
✔ Access to cutting-edge technologies and services,
enabling rapid experimentation, development, and
deployment of new applications and features.
 Introduction
Big Data definition
• Big data refers to a vast volume of data that
is too large, complex, and diverse to be
processed using traditional data
management and analysis methods.
• Initially referred to 3V (Volume, Velocity,
Variety)
• Being constantly extended to n ‘Big Data Vs‘
(Veracity, Validity, Value…)
 Introduction
Big Data definition
• Volume: Big data involves massive amounts of
data generated from various sources, including
social media, sensors, websites, transactions, and
more.
• Velocity: The speed at which big data is generated
and collected is significant. Data is continuously
streaming in real-time, requiring efficient processing
and analysis.
• Variety: Big data comes in various formats,
including structured, semi-structured, and
unstructured data. It includes text, images, videos,
audio, log files, and other data types.
 Introduction
Big Data definition
• Other Vs:
• Veracity: Refers to the quality and reliability of
the data. Veracity important due to the massive
volume and diverse sources of data, which can
lead to data inaccuracies, noise, and uncertainty.
• Validity: Concerned with whether the data
accurately reflects what it is supposed to
represent.
• Value: The usefulness and significance of the
insights and knowledge derived from big data.
 Introduction
Significance of Big Data
• Big Data plays a pivotal role in transforming
industries and driving innovation across various
sectors.
• Big Data has the ability to provide actionable
insights from vast and diverse datasets, which
enables organizations to:
✔ Make informed decisions,
✔ Foster innovation,
✔ Gain a competitive edge.
 Introduction
Cloud Service Providers
• There are several prominent cloud service
providers in the industry, each offering a suite of
cloud services.
• Some of the leading providers include:
• Amazon Web Services (AWS)
✔ AWS is a comprehensive cloud computing
platform offering a wide range of services,
including compute, storage, databases,
networking, AI/ML, analytics, and more.
 Introduction
Cloud Service Providers
• Google Cloud Platform (GCP)
✔ GCP provides a variety of cloud services,
including computing, storage, databases, AI/ML,
networking, and analytics.
✔ It also offers unique services like BigQuery and
Kubernetes.
 Introduction
Cloud Service Providers
• IBM Cloud
✔ IBM Cloud provides a robust set of infrastructure
and platform services, including compute,
storage, networking, AI/ML, blockchain, and IoT
solutions.
✔ It also offers industry-specific solutions.
 Introduction
Cloud Service Providers
• Microsoft Azure
✔ Azure is a comprehensive cloud platform offering
services for compute, storage, databases,
networking, AI/ML, analytics, and more.
✔ It also integrates well with Microsoft's other
products and services.
 Introduction
Cloud Service Providers
• Alibaba Cloud
✔ Alibaba Cloud offers a full range of cloud
products and services, including elastic
computing, databases, networking, security,
analytics, big data, domains, website
management, application services, media
services, middleware, and more.
 Introduction
Cloud Adoption Trends
• Accelerated migration to the cloud
✔ Accelerated migration to the cloud refers to the
rapidly increasing pace at which
organizations are moving their workloads,
applications, and data from on-premises
infrastructure to cloud environments.
✔ Organizations are increasingly adopting cloud
services to leverage scalability, flexibility,
agility, and cost-efficiency offered by cloud
providers.
 Introduction
Cloud Adoption Trends
• Hybrid Cloud Strategy
✔ Integrating on-premises infrastructure with public
cloud services to create a unified, flexible, and
scalable IT environment.
✔ Organizations adopt a hybrid cloud approach to
leverage the benefits of both on-premises and
cloud environments, including data sovereignty,
regulatory compliance, and workload
optimization.
 Introduction
Cloud Adoption Trends
• Multi-Cloud Strategy
✔ Involves using services from multiple cloud
providers to distribute workloads, applications,
and data across different cloud environments.
✔ They adopt this to avoid vendor lock-in,
optimize performance, increase resilience,
and leverage best-of-breed services from
different providers.
 Introduction
Cloud Adoption Trends
• Focus on security and compliance
✔ As organizations transition their workloads,
applications, and data to cloud environments,
they ensure the security and compliance of
cloud-based resources becomes paramount.
✔ Organizations are increasingly recognizing the
importance of protecting sensitive data,
maintaining regulatory compliance, and mitigating
cybersecurity risks in the cloud.
 Introduction
Cloud Adoption Trends
• Embrace of cloud-native technologies
✔ Adoption of cloud-native technologies such as
containers, microservices, and serverless
computing to build and deploy applications
optimized for cloud environments.
✔ By embracing cloud-native technologies,
organizations can unlock the full potential of
cloud computing to build and deploy scalable,
resilient, and innovative applications that drive
digital transformation and competitive advantage.
 Introduction
Cloud Adoption Trends
• Rise of industry-specific clouds
✔ Emergence of industry-specific cloud solutions
tailored to the unique requirements of vertical
markets such as healthcare, finance, and
government.
✔ Industry-specific clouds provide specialized
services, capabilities, and compliance
frameworks designed to address the challenges
and requirements of specific verticals.
 Cloud Computing and Big Data

• Introduction
• Cloud Computing Foundation
• Virtualization and Containerization
• Big Data Technologies
• Data Management in the Cloud
• Big Data Analytics
• Stream Processing
• Cloud Security
• Cloud Cost Management
• Cloud Project Management