Oracle Clusterware Architecture

2 - 2
Objectives
Describe the Oracle Clusterware architecture
Describe the components of Grid Infrastructure
High Availability Services
Clusterware
Automatic Storage Management
Understand why ASM and Clusterware are combined in GI
2 - 3
Oracle Grid Infrastructure For a Cluster
Oracle Clusterware:
Platform-independent facility for starting, stopping, and
managing clusterwide resources
Oracle Clusterware comprises two physical stacks:
Cluster Ready Services stack
Oracle High Availability Services stack
Oracle Automatic Storage Management (ASM):
Platform-independent shared storage solution providing
storage for the Clusterware, databases, and most other
application or system requirements
2 - 4
Oracle Cluster Registry (OCR)
Node1
OCR cache
CRSD
process
Client
process
Node2
OCR cache
CRSD
process
Node3
OCR cache
CRSD
process
Client
process
OCR
primary
file
Shared
storage
OCR
secondary
file
2 - 6
CSS Voting Disk Function
Node1 Node2
CSS CSS
Voting disk
Node3
CSS
Node1 Node2
CSS CSS
Voting disk
Node3
CSS
Nodes can
see each other.
Node3 can no longer
communicate through
private interconnect.
Others no longer see
its heartbeats and
evict that node by using
the voting disk.
I do not see 3.
Node1: I see 1&2.
Node2: I see 1&2.
=>
We should
evict 3!
I see 1&2.
I see 3.
Ive been
evicted!
Id better stop!
I see
1, 2, & 3.
I see
1, 2, & 3.
I see 1, 2, & 3.
Split-brain
(RMN)
2 - 7
Oracle Local Registry and High Availability

The Oracle Local Registry (OLR) is a registry similar to
OCR that is located on each node in a cluster.
The OLR contains Clusterware manageability information,
including dependencies between services.
Oracle High Availability Services uses the OLR to maintain
Clusterware resources.
To check the integrity of the Oracle Local Registry:
# ocrcheck -local
Status of Oracle Local Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2644
Available space (kbytes) : 259476
ID : 250248496
Device/File Name : /u01/app/11.2.0/grid/cdata/host01.olr
Device/File integrity check succeeded
Local registry integrity check succeeded
Logical corruption check succeeded
2 - 8
Oracle Clusterware is started by the OS init daemon.








Oracle Clusterware installation modifies /etc/inittab to
restart ohasd in the event of a crash.
Oracle Clusterware Initialization
/etc/init.d/init.ohasd
ohasd.bin cssdmonitor
octssd.bin crsd.bin
oraagent.bin gipcd.bin
diskmon.bin mdnsd.bin
ocssd.bin gpnpd.bin
evmd.bin evmlogger.bin
cssdagent oraagent.bin
ologgerd.bin osysmond.bin
ons orarootagent.bin
/sbin/init
Clusterware
startup script
Oracle Clusterware processes
Operating system
init daemon
# cat /etc/inittab
..
h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1 </dev/null
2 - 9
Clusterware Startup Details
init
oraagent
cssdagent
ohasd
cssdmonitor orarootagent
cssd
2 - 11
Clusterware Startup: The OHASD orarootagent
ohasd
ctssd
osysmond diskmon
ACFS
Drivers
ologgerd
oraagent
cssdagent
cssdmonitor orarootagent
crsd
2 - 13
osysmond
Clusterware Startup Details:
The CRSD orarootagent
ohasd
ctssd
diskmon
ACFS
Drivers
ologgerd
oraagent
cssdagent
cssdmonitor orarootagent
crsd
network GNS vip SCAN vip Node vip
orarootagent oraagent
2 - 14
osysmond
Clusterware Startup Details:
The CRSD oraagent
ohasd
ctssd
diskmon
ACFS
Drivers
ologgerd
oraagent cssdagent cssdmonitor orarootagent
crsd
SCAN
Listener
Node
Listener
ASM
Instances
ons
orarootagent oraagent
Database
Instances
2 - 15
Clusterware Startup Details:
The OHASD oraagent
ohasd
gpnpd
gipcd evmd ASM mdnsd
oraagent
cssdagent
cssdmonitor orarootagent
2 - 16
The crsctl utility is used to invoke certain OHASD functions.
To start or stop Oracle Clusterware on all nodes:



To enable or disable Oracle Clusterware for automatic
startup on a specific node:



To check the status of CRS on the local node:
# crsctl start cluster
# crsctl stop cluster
# crsctl enable crs
# crsctl disable crs
Controlling Oracle Clusterware
# crsctl check cluster
2 - 17
Verifying the Status of Oracle Clusterware
The crsctl utility can be used to verify the status of
Oracle Clusterware on all nodes:

$ crsctl check cluster -all
***********************************************************
host01:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
***********************************************************
host02:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
***********************************************************
host03:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
***********************************************************
2 - 18
Viewing the High Availability Services Stack
$ crsctl stat res -init -t
---------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
---------------------------------------------------------------
Cluster Resources
---------------------------------------------------------------
ora.asm
1 ONLINE ONLINE host01 Started
ora.cluster_interconnect.haip
1 ONLINE ONLINE host01
ora.crf
1 ONLINE ONLINE host01
ora.crsd
1 ONLINE ONLINE host01
ora.cssd
1 ONLINE ONLINE host01
ora.cssdmonitor
1 ONLINE ONLINE host01
ora.ctssd
1 ONLINE ONLINE host01 OBSERVER
ora.evmd
1 ONLINE ONLINE host01
...
2 - 19
How GPnP Works: Client Database Connections
Database
client
DNS
SCAN
listener
Listener1
Listener2
Listener3
SCAN
listener
SCAN
listener
GNS
2 - 20
ASM is a volume manager and file system built in to the
Oracle Database server.
ASM provides a complete shared storage solution for:
Clusterware files
Voting files
Oracle Cluster Registry (OCR)
Database files
ASM permits Clusterware to access the shared OCR file.
The Oracle ASM partnership and status table (PST) is
replicated on multiple disks and is extended to store OCR.
Oracle ASM reserves several blocks at a fixed location on
every Oracle ASM disk for storing the voting disk.
Automatic Storage Management
2 - 21
Summary
In this lesson, you should have learned how to:
Describe the Oracle Clusterware architecture
Describe the components of Grid Infrastructure
High Availability Services
Clusterware
Automatic Storage Management
Understand why ASM and Clusterware are combined in GI