Oracle 11gR2 Node Addition into Cluster post Host Reimage When a host is reimaged, we assume that there

will be no Oracle software on it and the addNode process will take care of adding Oracle stuff to reimaged host. When these errors were encountered, noticed that the reimaged host had the Oracle Home ( GI and Database with all contents. !his implies that Oracle Homes were copied o"er from the a"ailable host to this reimaged host. #o, first thing I did was remo"e the Oracle Home folders on the reimaged host. $ctual steps should be % &. Delete the information about reimaged host from the in"entor' of e(isting node and also the O)* a. )learing the O)* ke's for reimaged host + crsctl delete node +n g,&+stgdnadb+--& .ooks like this was alread' done successfull' prior to raising the #* , since olsnodes command was not returning an' info about node&. /root0g,&+stgdnadb+--1 234 olsnodes +n g,&+stgdnadb+--1 1 b. *emo"e the 5I6 information of reimaged host from O)* 7(ecute the following on e(isting node % 8oragrid8product8&&.1.-.18bin8sr"ctl remo"e "ip +i g,&+stgdnadb+--&+" +f c. )learing the in"entor' info for reimaged host /oracle0g,&+stgdnadb+--138oragrid8product8&&.1.-.18oui8bin9 .8runInstaller +updateNode.ist O*$).7:HO;7<8oragrid8product8&&.1.-.1 =).>#!7*:NOD7#<g,&+stgdnadb+--1= )*#<!*>7 +silent +local #tarting Oracle >ni"ersal Installer... )hecking swap space% must be greater than ?-- ;@. $ctual &1&AB ;@ !he in"entor' pointer is located at 8etc8oraInst.loc !he in"entor' is located at 8oracle8oraIn"entor' C>pdateNode.istC was successful. NOTE !he error for which #* was raised was % .8addNode.sh +silent =).>#!7*:N7W:NOD7#<Dg,&+stgdnadb+--&E= =).>#!7*:N7W:5I*!>$.:HO#!N$;7#<Dg,&+stgdnadb+--&+"E= 6assed

6erforming tests to see whether nodes g,&+stgdnadb+--&,g,&+stgdnadb+--& are a"ailable ............................................................... &--9 Done. 7rror ocurred while retrie"ing node numbers of the e(isting nodes. 6lease check if clusterware home is properl' configured. #757*7%7rror ocurred while retrie"ing node numbers of the e(isting nodes. 6lease check if clusterware home is properl' configured. ++ !his clearl' indicates that e"en though O)* ke's were cleared, step &+c was not done properl'F as we see that g,&+stgdnadb+--& was still seen as a"ailable node in in"entor' but the node number information was missing from the O)*. Steps to add reimaged Node <<<<<<<<<<<<<<<<<<<<<<<<<< &. *un the )luster 5erification >tilit' .8clu"f' stage +pre nodeadd +n g,&+stgdnadb+--& +"erbose If possible, redirect the output of abo"e to some file so that it can be re"iewed and an' issues reported can be rectified. Gor this case, the most impacting errors were % 7**O*% 6*5G+AHIA % 6ath location check failed for% =8oragrid8product8&&.1.-.1= *esult% #hared resources check for node addition failed 7**O*% 6*5G+?AAJ % )heck of 5oting Disk location =O*).%DI#KI(O*).%DI#KI = failed on the following nodes% g,&+stgdnadb+--&%No such file or director' 6*5G+?AB& % Oracle )luster 5oting Disk configuration check failed 1. Now run addNode.sh from e(isting node /oracle0g,&+stgdnadb+--138oragrid8product8&&.1.-.18oui8bin9 .8addNode.sh +silent =).>#!7*:N7W:NOD7#<Dg,&+stgdnadb+--&E= =).>#!7*:N7W:5I*!>$.:HO#!N$;7#<Dg,&+stgdnadb+--&+"E= /oracle0g,&+stgdnadb+--138oragrid8product8&&.1.-.18oui8bin !his completed without gi"ing an' messages. $ctuall' the addNode.sh didnot run at all. )ause % !he error seen in clu"f' 6*5G+?AAJ % )heck of 5oting Disk location =O*).%(O*).% = failed

+ !his is because we use $#;.I@ Solution : IGNO*7:6*7$DDNOD7:)H7)K#<L e(port IGNO*7:6*7$DDNOD7:)H7)K# Now run addNode.sh again. ( !his issue discussed in % 6*5G+?AAJ % )heck of 5oting Disk location =O*).%(O*).% = failed /ID &1IM?IJ.&3 Output +++++++++ oracle0g,&+stgdnadb+--138oragrid8product8&&.1.-.18oui8bin9 .8addNode.sh +silent =).>#!7*:N7W:NOD7#<Dg,&+stgdnadb+--&E= =).>#!7*:N7W:5I*!>$.:HO#!N$;7#<Dg,&+stgdnadb+--&+"E= #tarting Oracle >ni"ersal Installer... )hecking swap space% must be greater than ?-- ;@. $ctual &1&AB ;@ erforming tests to see whether nodes g,&+stgdnadb+--& are a"ailable ............................................................... &--9 Done. )luster Node $ddition #ummar' Global #ettings #ource% 8oragrid8product8&&.1.-.1 New Nodes #pace *e,uirements New Nodes g,&+stgdnadb+--& Instantiating scripts for add node (!uesda', December 1&, 1-&- B%B?%&I $; 6#! . &9 Done. Instantiation of add node scripts complete )op'ing to remote nodes (!uesda', December 1&, 1-&- B%B?%&H $; 6#! ............................................................................................... Home copied to new nodes JI9 Done.

#a"ing in"entor' on nodes (!uesda', December 1&, 1-&- B%BM%?M $; 6#! . &--9 Done. #a"e in"entor' complete W$*NING% !he following configuration scripts need to be e(ecuted as the =root= user in each cluster node. 8oragrid8product8&&.1.-.18root.sh 4On nodes g,&+stgdnadb+--& !o e(ecute the configuration scripts% &. Open a terminal window

1. .og in as =root= B. *un the scripts in each cluster node !he )luster Node $ddition of 8oragrid8product8&&.1.-.1 was successful. 6lease check C8tmp8silentInstall.logC for more details. B. *un root.sh from GI Home on the reimaged host to startup GI stack. /root0g,&+stgdnadb+--& &&.1.-.134 .8root.sh *unning Oracle &&g root script... !he following en"ironment "ariables are set as% O*$).7:OWN7*< oracle O*$).7:HO;7< 8oragrid8product8&&.1.-.1 ..... 7ntries will be added to the 8etc8oratab file as needed b' Database )onfiguration $ssistant when a database is created Ginished running generic part of root script. Now product+specific root actions will be performed. >sing configuration parameter file% 8oragrid8product8&&.1.-.18crs8install8crsconfig:params )reating trace director' .O)$. $DD ;OD7 )reating O)* ke's for user CrootC, pri"grp CrootC.. Operation successful. O.* initialiNation + successful $dding daemon to inittab $)G#+J1--% #upported $)G#+JB--% $D5;8$)G# distribution files found. $)G#+JB-M% Installing re,uested $D5;8$)G# software. $)G#+JB-H% .oading installed $D5;8$)G# dri"ers. $)G#+JB1&% )reating ude" for $D5;8$)G#. $)G#+JB1B% )reating module dependencies + this ma' take some time. $)G#+JB1M% 5erif'ing $D5;8$)G# de"ices. $)G#+JB-J% $D5;8$)G# installation correctness "erified. )*#+AA-1% !he )## daemon was started in e(clusi"e mode but found an acti"e )## daemon on node g,&+stgdnadb+--1, number 1, and is terminating $n acti"e cluster was found during e(clusi"e startup, restarting to Ooin the cluster clscfg% 7PI#!ING configuration "ersion ? detected. clscfg% "ersion ? is &&g *elease 1. #uccessfull' accumulated necessar' O)* ke's. )reating O)* ke's for user CrootC, pri"grp CrootC.. Operation successful. 6reparing packages for installation... c"u,disk+&.-.J+& )onfigure Oracle Grid Infrastructure for a )luster ... succeeded !his will complete addition of GI Home to the new node.

A. 6roceed to run addNode.sh for D@ Home. .8addNode.sh +silent =).>#!7*:N7W:NOD7#<Dg,&+stgdnadb+--&E= ?. Instance ma' not be up b' default. >sed sr"ctl to start the instance of newl' built node and also the ser"ices.