Scribd
Upload
125 views

OCE K8s Architecture

1) OCE is building their architecture to support cloud-native applications on Kubernetes. 2) Kubernetes is an application orchestrator that allows deploying, scaling, updating, and healing applications on containers. It avoids vendor lock-in. 3) A cloud-native application is designed to be self-healing, scalable, and support rolling updates through features like containers, services, and horizontal pod autoscaling.

Uploaded by

manojattal2646
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
125 views

OCE K8s Architecture

1) OCE is building their architecture to support cloud-native applications on Kubernetes. 2) Kubernetes is an application orchestrator that allows deploying, scaling, updating, and healing applications on containers. It avoids vendor lock-in. 3) A cloud-native application is designed to be self-healing, scalable, and support rolling updates through features like containers, services, and horizontal pod autoscaling.

Uploaded by

manojattal2646
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

1

OCE Kubernetes Architecture

20th October 2021

Igor Stepin, R&D Team


Goal of today’s session 2

● Why should we support Cloud?


● How is Kubernetes related to Cloud?
● Why do we need to create Cloud-native apps?
● How is OCE being built to support Cloud?
3

Cloud and Kubernetes


History: Evolution of software deployment approaches 4

Bare-metal VMs Cloud Serverless Cloud

HW Utilization Low Medium High High

Central Management Low Medium High High

Anti Apps Vendor Low (software vendors Medium (Ops can High High
Lock-in sells closed HW-boxes) backup and restore any
VM)

Immutability Low Low Medium High

Granularity Low Low Medium High

Scalability Low Low Medium High

Owning Option High High Medium Low (for example, DBs)

Renting Option Medium High High High


(* as a service)

Kubernetes No No Yes Yes


Why Cloud? 5

• Changes of a cloud approach:


• Pipelines to re-create everything automatically (level 1: Lift & Shift approach)
• More parts are centralized (level 2: Cloud Native approach)
• Advantages:
• Cheaper -- centralized, automated, self-service, cheaper HW, autoscaling from 0, ...
• Faster reconfigurations -- self-service, pipelines, ...
• Better quality -- dedicated small apps, monitoring, self-healing, ...
• Disadvantages:
• Cost of transaction. Especially if products / vendors are not ready for a cloud.
Public vs Private Clouds 6

• Public Cloud -- state of art technology but:


• expensive
• fragile aka bad backward compatibility: teams needs to continuously upgrade scripts for the same tasks
• instability introduced by new releases are not aligned with company's internal schedule
• possible security and legal problems
• vendor-lock
• Private Cloud tries to repeat the same in own DCs.
• Main technology to avoid cloud lock-in is Kubernetes
What is Kubernetes 7

● Kubernetes is an application orchestrator.


○ An orchestrator is a system that deploys and manages application.
■ For the most part, it orchestrates containerized cloud-native microservices apps.
● 1980-1990: Apps ran on physical servers in the open-system era
● 2000-2010: Apps ran in virtual machines in the virtualisation era
● Today: Apps run in containers in the cloud-native era
○ Kubernetes can:
■ Deploy your application
■ Scale it up and down dynamically based on demand
■ Self-heal it when things break
■ Perform zero-downtime rolling updates and rollbacks

● Reference
○ The Kubernetes Book, Nigel Poulton
Supported Kubernetes-based platforms 8

• GKE (Google Kubernetes Engine)


• RedHat OpenShift IPI (Installer-provided infrastructure) on GCP (Google Cloud Platform)
• RedHat OpenShift UPI (User-provided infrastructure) for bare-metal
• RedHat OpenShift UPI (User-provided infrastructure) for ProxMox (our VMs in AMS DC)
• (maybe in future) VMWare Tanzu for VMWare VMs
• (maybe in future) RedHat OpenShift IPI (Installer-provided infrastructure) for bare-metal
• (maybe in future) RedHat OpenShift IPI (Installer-provided infrastructure) for ProxMox

OKD is open source name of OpenShift.

K8s don’t have installation scripts, monitoring, Docker registry, support for storages, and so on.
It’s like Linux kernel (Kubernetes) and Linux distribution (OpenShift).
K8s Terms 9

• Pod is set of containers with shared network (other parts like list of processed and file system also can be
shared but it requires configuration).
• Node is host for pods, it’s b-m server or VM.
• Any 1 node or pod can go down at any time. So, to recover pods different controllers are used: Deployment
(for apps), StatefulSet (for DBs), DaemonSet (for technical apps that should start on each node), CronJob (job
that should start by some schedule), and Job (pod that should finish successfully).
• Service allows to access ports from pods in a stable way (stable IP and DNS name). It’s implemented as iptable
rules. Service has different ways to expose to external world.
• Ingress is a way to expose http(s) ports to external world with configuration for domain names and subpathes.
• NetworkPolicy is internal K8s firewall.
• K8s is configured in a declarative way by standard or custom resources (YAML or JSON files).
• K8s resources can be namespaced or cluster-wide. Most of resources are namespaced (like pod or service).
• Helm is a templating engine for K8s resources.
• Helm chart is a package of YAML templates (something like rpm).
• Helm release is an installed helm chart (chart + values for templates).
What is a cloud native app? 10

It offers self-healing, scalability, rolling updates


1. 1 process/app per container (pod can have several containers)
2. Only required code is inside Docker image
3. Small pods (cpu < 4 and memory < 8Gi)
4. Specify requests and limits for CPU and Memory
5. Use service and readiness check
6. Use liveness check
7. Use K8s Services for discovery
8. Export metrics in Prometheus format
9. Storage-related items
a. Print logs into stdout and stderr
b. Don’t save to filesystem (aka read-only root filesystem)
c. Don’t use NFS protocol (it’s not compatible with K8s)
d. No volumes or emptyDir (with specified limit)
e. Use ConfigMaps for configs
f. Use Secrets for secrets
g. Use S3 API for backups
10. Horizontal scalability and autoscaling
11. Rolling updates without downtime
12. Random non-root user inside container
13. No special privileges for user or container
14. Use OpenTracing
15. Support 3-zones setup instead of 2 DCs setup
11

OCE
12
Artifacts Folder 13

/opt/offline_dir:
• bootstrap -- scripts to work with artifacts (load and check)
• git -- git bundles to be imported into GitTea
• images -- Docker images to be imported into local Docker registry
• images.csv -- Docker images description in csv format for scripts
• images.yaml -- Docker images description in YAML format for scripts
• openshift-installation -- binaries for OpenShift
• services -- scripts and configuration files for services started on Ship & Deploy server
Ship & Deploy Services 14

• Prerequisites for OpenShift (if we can’t use external service):


• NTP
• DNS
• Http-server for UPI OpenShift installation
• LB (HAProxy)
• Docker registry
• Services for our deploy and test pipelines:
• GitTea
• Jenkins
• Ansible with pipelines:
• artifacts -- import of git bundles and docker images
• deployment -- deployment into OpenShift
• smoke -- product smoke tests
• OTF -- project-specific tests
• IPS server
Installation Procedure 15

1. Ask and receive HW


2. Network planning — in spreadsheet, then in configuration files for DNS, ISOs, HAProxy, and Ansible hosts file.
Also, we prepare TPD for CDB.
3. Ship & Deploy Server Installation with all 8 services on it
4. OpenShift installation
5. RedKnee Install Server (RK IS) installation.
6. CDB (CustomerDB aka Oracle) installation from RK IS
7. Jenkins (and Ansible with Helm inside) is used to run deployment pipeline that installs everything into
OpenShift
8. Manual bring-up procedure (it was automatic in GCP)
9. (optionally) Manual BRS (Backup&Recovery Server) aka NetWorker installation
10. Manual F5 LBs installation
Challenges 16

• Move to a cloud-native approach for all apps and dependencies


• Support of Kubernetes clusters installed by 3d-parties
• Better rollbacks (maybe use of ArgoCD)
Thank you

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy