insta:- @mustudentsunited A big thanks to NAME :- Shreeganesh Taralekar From: G.V.Acharya BRANCH :- Computer For providing notesMs) Data Wage hous & Mining Page No. ss Bis beb Dato DOLTEL Stends fog ra ; 3 Online Tyansachion Processin DOLTP Woyks aon Cuyent | JAPA @OLAP works O” Hisroys cal dara ) It has Noymalize) data - Sryucruyes @It has Stay Schema oy Snowflake Schema @ Te has Simelé Que ejes @Tt has complex Queries | S)OLTP uses Tyajitional DBMS 1@ OLAP uses the dara ay€house, © tt 15 Used b4 Front Line Emplous Managev U © It isusey ot Analyst ExeCubivés , Decisron - Maken Require fost ces ponse byme @ Cn _ have. longe, | b “CSpon se Hye | | @ Dara in ol TP SY srems is 18) Data in olpp 5 Shem} UpdaHd SiN Cq\ Hime. Peviodical Cf eCShre J 1 - in @) It is CuShomey o tcient) POCESS + EX, Tableau, Powey BI Scanned with OKEN ScannerPageNo, Date ne wl piffevence between Stay Schema an) Snow Stay Schema Snowflake Schema GIt io Cas to unieysanlO LE io vifficupto Ondevstand- ete weguire 655 Space Ott kequiveé Moye Space Que GMplex) ) 18 Queyy Gmple x) i low . 1S highs. : (4) Bottom-up approach ~_|@ Top-down App voach ts used 15 Use). | __ © Tt hos De- hoymalizel (©) 1t has Nomalizes Fables - fables . © Fewey joins © High, numbey_ of joins @ Weyks beSr IN aie Waylhousé | dara may Berrey’ Foy Sma)! warthouse! / satea_mayt j H @) Consume move_time | iN queyy Cxecu ron _ @Gnosurne tess time IN huey 2x ecuHon _@ Let ig easy fo Jesian | @ It is Gmplex +9 design || O u | — | —_|©1t @nvains ony Single Tt nay Lave move Hoon ane ———__dimension tavle foy Cath jimnension fable For each If ——L_dimension dimension. im! = /ifFference betwen Top Top -Jown LAppyoarCh 3 puapcies pA Tot op- down a ia L —— i aa Fivgr the cote oy data axknsuse is d2Signet i | ___—_—— 7 = @® Then the vélational maiatogel 2205 yaar — 19 @nsreuched Usin a i __ = = (4) The. Cube i Jeployes.. ——\- —— e ay Je { Gin Toph down appyoth dai ie ex ryaciey Promey Ji Fhestins dara Sotycés- | © Tn this Dak coarthouse 15 bul th’ 1p wife ERLE ee ANY Summay cation pe yfovm | CO” | ES i +(~e JAtA L ee Por Foyme) LO loa} gala INto Ja koa Mayes - ; = | Advan ages 2 a= F fle ‘= li} Represemrs dara: Vieto Fell the” Pres pecive © de L entreyPyI Se. | i} Dara avour the Content io Short] in 4 Singie , cent fe : location Wid Contsalized Contrvol any cules. bs M 15%) yar ra.ges © oF WJ Expenses art high without pyook of Concepr.Page No. | Date ane ea - 32 Source ; oa <—__/ —— e HM System 2 Ss stem 4 a KE 4 Extvact tyansforn an) load Enterprise dara warthouse Croymatizes) i ‘. ne a AN BI Mata meerh 4 Dart mayk 2 Stay Schema == ot Schema | SDato poocoonel ——— — Bottom = por, 2 ——___\ © Bottom - up Appyeath iS sag Bem O Lt 1S algo calleJ a5 kimball prothodolej4— a a A Lt (5 Albov@ngryucte} using Stay Schema \ a The Wbe is Lesigned — a 5 Finale tte data (6 joaded ints da ba_cxor Chase = © Ib is REVERSE of pop- down i ee ae ee tea ee @ Dara mayts ane’ divecry | forages , tlhe Jara 1S cater) fyom various jJafa. Sources (B) Thyough ETL this data From Jara marr j5 loadel into Jara warthouse | {22 Date fyom Jaa Merve id A494 Keqared IN Summa MO 14n loaje)\ jn Jarra wavehouse i | Agyontanes & - 2 | 1) Iimpementa Hon of small portions 7S fers rey am} sie TO Tevet ork low Chance of Fajhiye, | ii} ACS project fam te QvOw 4) develop. = ES = =a DiSayventeges s - i) Every dara Weyy Ks iB wey Floods j wih, & Junjanr jn Pownst ) Incatoses fl Nuwvoey of aNmet ne ee Lie mM bevfa ces . — Lee ae Le—=—— ——i Paga No. 5 Data a | ae SSS ee ~~ ~~! \7 Ql | what aye. Hee basic build ing box of daPcere)— Anos |@ Aychikecturt is tLe pyopey avrangement © ie elements. @we buily a data wanthouse wiht, opfrwave IMs hay swrerye Gmponenrs, < - = o @ To suit de equi yemen Ss of ouy evga nize OMhs ee ayyan tres & pang wem want HG Goost uw Shaiache itt Crt £2015 ap 5 2Vi CES Source data Externg | \ i Management an) Cenrro! [metadata a" | : io oh ee f | Data werthouse_| | Dimenana — | DAMS wk LPiaise | OAR — A Toni i | 13 i ‘S}OYAGP Repti a Wr 4 oe $ Reet 3 te, ed im Data Sragng Builving Blocks of Data AVC LouseData | Een tal cca — SSS SS ee Gouyce Data Com ponent e B ~ v0) UCHOM Pars: 2 of dara Comes Fyom ite Jitfoyent opevaring Fh ansot enrerprise. paseh on fhe dari _pguiemints oa Choose seqment’ of the data n qhe Jaya Warthouse , we ¢ om 1A Yavious opéyarional Modes. ryor per a) Inkernal Dara ° - et ach OY anitaron —~T ee useful 1 a dara want house. oa Pythived Darr s- I In every operariona| Sts rem we peviolicaliy prke HOR ol) ove (KM archieved Files ne os _ | gota ony SF Ti} Exrernal Dare 2° al most executives Jepens on jn forge percentage of He jnformar gourees for a [ons 4 (@ Dara Staging ¢ | ea CDSP)IS a temporary Srova e ay Cor i] A data sta4 ing ay herwoen HAL data Sources an) A Jafa wank pouse - formation from &xiey nal jon aa 6 quiclel extracel ii) Te oft in anea_19 mainly Uses f | niny zai the impact data Fyrom rs Jara Souvees, M) OF Hye Sowrces. Componéy vr Dara Extyachon ° Darr Lowy iMetadata % - Mw fA AtY ryt icut, EL J ney fox examiplé aud thov A dake éaked are mofiFie any Fre Sjze art examples of rery baste Oe metadata I.| Explain issues mining " ) eee ge ee lok OF Chation €s any DP {Souk in FOV World. Some of Bae Ain't: eens: ee ming — |methosology avn — | recoy Inte vachion © Mining methodology % mining diffeyenr “ing of knowledge iN Jarabase ay evev Usey might not Want the same yesuly _ Fyom tre data mining. Hence roa) yange aie know ledge diigionieorss Should be possible. ~__|it) the Process ShoulJ be inteyactive ar qn} tii level of AbstyacHion To quid discovery process & fo Cxpyess discovered patterns, He _backavoun) knowledge Can be uses i Visuatization oF dara Shout) be easi\ v\ Dara Cleaning _1 Unjervstanda ble nComplere 5 Manda toe and 1" omp JOra Should be semoves Af sone Ei,Page No. > Peyformance 1S5UC5 0° ee ¥ Efficiency & Scala tilike of data rin = Ith Shoul) be high as fhe acess 19 —dS8E On 4 ‘a len : cnn one _MUIhiPle tine mine) faster than normal —— @ Diverse Dara Ape issues $- : | Tie Jara bace may contin @mple Obj ku, Multimédia data senjoving ov mining pak oA one” 5 Nor Possible iJ Mining dara From heterogeneous database will also ajJ buyien Fo data miningx mining pyoress thal involves cleaning ny transforming yaw Jato fo make jt Suitable cone 5iaet : ome Commo HW doa Po Pri processing | Clu Data_cleaning 8: 15 involves ijentiFring an) Covrecring exryors = So the Jara Such as Missing values ANY) dup) Cates: ee Vaylous fechniques Can be Use) fox dara Cleaning , ete Buch aS i puyation , removal qn) ryans Fovmating ~_|@Dara In leq vation - 24> OU This involves combining Jala from muriple yo Cetatk a Unifie) data ser. Dara jnre4 carion an be Challenqing ag jt (eq uives handling data with different fowmaks ,— S7Yyucrurts ad Semantics - 4 @ Data _Twansformarion 3° This_invelves Cnvevring the Jaye inre 9 Suitable fovmar foy MAYS iS - Common Techniques Used in Japa tyansfovmarion InclUye NowMalizarion , Gran dongi zation Qn) discvelrzyrio @ Data Reduction 3: This jnvolves Yeducing fle Size of the da taser WR prdsewing rt impovian* in fovmatlon Tata yejuction Cam be Acieve though, echn) Suc, ay feature selection any Fearart exreacHionPage No. Dato Data _Discve tization & a in “This involves diving conjinuous dara inte a alegoyies oy jnrevvals: \ 1SCyCti zation is of use) in dara Mining » ANd Machine leayning Agovithms thal ‘Cquire cakesoyi cad data. X, © Data Nowmatization 8- ao This involves Scaling the Jatal {2 a Commo ” Wye , Such 45 befween O am) 1 0 eden t Noymali zation is ofhen used fo handle daira | wi Hy different units an) Scales .| Heiijee Why aeeeanee ee Volw | | | AUR cn isa ataietelasy pags 4 Pie Kiel le siete feyeyyy. i ot [x plain Dit "Ty PES GF NET vibes yn ‘Daya qn, ae "They eye axe. or rg Pe 5 of PUG GNe aint ee ; (2 y Nominal ALY buy! sea duass YONA ALM bbe CHa) Pinay Ativibue oT a i enema iy -—T@ Nominal Alvibuie $2 aT TLE iS velo bey [9 name. -Syp. Be tha | a FZ] The values aye Seprtsenred b Vales. a ‘4 aioe Ov RewPagoNo, Dato a ———_| awk pes of Numey [os 2 te spt oF Nona Numeric A ore See mfsumiae ge [pa on equal sized y = Ae ine Overy Can be +ve ~ye a ae How US fo Compave am) Quantif 4 55 LEA rifY Hae [| sifertnct between alts Liar Reel 24 — Ratio- 6 ale.d 3: Ty Continuous Positive Measuytmenrs Or) O Novn.-tineav, Scale oy [22 Snrtiva Scare) and liweav Scaled measuyes : TS Aiton K& Subtyvachion 16 aAllowes i Ra FiO Scale _ buk Mutriplication & Divison 1S wor Allows) in | cans Scale). He: vat \ [LS] | Descvite tty, Ste ps involve _in data Put} -—Dnssi “Theve art vayjous is eps Heat iavele in daa da Fa Hi AVL AS follows —— © Dara Cleaning ——__ a (® Data Selection oa Er rt @) Data Pye pyocessing a, a ia (4) Data Transformation : a ay © Dare Minin KO) Pattevn Evatuation = i Knowledae Presentation. = et — ® Date Cleanin i} Removal of nojse ,incnsis+ ii} Stvategies to hanjle missing. dara Fiel)s. an) Ourle @ Data selection §: i] Relevenr Japa jd extvacte| fyeom Jarabase, ii) collecting ont Necessary information to the mojel. a. Data Pre pyovessing 8 Conve ing tee data jn @nsjsrent Sfate. 17 Remawal of unnecessary infovmatioy. Dara Tyans fovmation a » Gnvev ting fae Joba in Suitable format Using fVansfovmarion methods MVariant Yepresentatiols fox Hee data ig Foun) Data Mining 3 iJ Extyacting Use ful Patterns For data 4 apelying gout “4 Ch Mojel ay i Necising Whi— — = a at Evaluakion 3- - rj #4 COnvVe atteyn intro ow gets Do a now |e en Ono: 7 yigualizatov) aN) knowledge Yepeesentation technig.., Chant oy table. 2 / AT m1 Gdn ek Rae | Sek ay _ _ — __ Ht oS {— 2 eer eres aera - | } | - © scanned with OnENPage No. —— —— ey re Q\ | Discuss Data vViguaqlizatio Tempe) i AnS 8-|@ Data visualization 159 the _geaphical Xe psesenna OY of infoymarion an ata Coyle C4) forma . ara _visualizarfon tools prcavise an SOs EIS ee to See and undersran) tenis, parreyns 1A an) ouylievs. e Concept oF mas pictues 1a ro Understand Jaa Thar has been ase) fov Centuyiés- ———_ of dar visualize pron ave Chants . Table Maps , Dashboavds Arvap hs “Theye an€ yovious Hechniques in data visualizatron @ His tog yam 3° | VSualy ohows thy, Matviburion of values OF a Single Vayiabte H 2\ The height of each par ingrcares the Numbey i of objects. i 3) he of hisroqvam depenis on the numbey of bins | ex. Petal wis th TT ila Ss) oC [= =z > v ds _| | | Petal lenarh (om) T Histegra)ae oh - i) Invented 64 J Takes [a] pnottyr wo jo displaying Ht~e_distynbution of Jara % feltowing Fi uvt Shows H-~ basic pars of bax plot Tn revg av yite : __vanae cea) | Outtievs Outliers Minny) Q, Med iam Ws Maxwuwn [GD Scaney lors 3 | | | 1) Atryibures values debevmune to_pesirion : . bur Gin have —~] 9) Two - dimen sional Scatter plors most commen ; iS ‘, herbe - Jimensiona! Scattey ple { lk Pain XN = . ) welakioVy er ettat ees y's hlaads ive covxtlavio NO CONV | Po o! + Je is rth 3 Rjaad rl ) | jbure 1S measure) OM 4 SPA aks ie avon fet of Similay VAs. 2 2) The most coynmon example IS Cont 4) Gm also Jigplay dom pewatuv yainfatl, vr oyessure , EF: # ip vo Weedon nun maps Or elevation _ © scanned with OKEN se¥ \ What ayé the Vayious Methods fox EbliMating \ A classifievs Acuyacy- - Data Mining Cam he veferved [oA mining fyom Jara, knowledge extyachor-———_* ara /parreyn AMal4oi5 and Jara dredging. © Thert ayt vayious techni accuvac4y of Classifieys ave a5 follows 2 ———_ HoldJOUt Random Subsamplin __ 1 Cyoss~ ValjJation (4) Hootstvapping ® HolJ Out In the holdout method, the largest daraser ig yandomly divided into thyeé Subsets « =|" A taining set is a Subset of the Jarasel Which aye been Used fo build predictive models. The valijation Ser io a Subset of He Jatasef which io been used fo assess the performance of the mode built in the Feaining phase. it TOV JeS a_téor Plarfoxm foy Fine-tuning Of He maje)’s Paramehers 49 selecting Hy besr | pe foyming mode. Lt jo nor nee foy Ul modeling Algorithms £9" Nee) A VAliJyrion Ser. (@) Tesh S$ ers aye He Subset of fhe Jabaser fo asses, — - Hae future performance of She mojel, Now|,a CE \_N @® Random Subsameling 2- 7) Ramgom Su bsam ping iS a Vayiation OF fhe Jour method. The hollour metho) jg been eae K HM<€S a Splitting the Japa into O_frainin 4 Ser and Hest Sef. \\\ — 18) Cyos5 = Validation 6 - sJ Cyes5 — Validation ig been Used When there 15 only a timp ted amount of data available Fo Achieve unbfase 3 estimation of He _p) ow formance OF Hee moet Ti] Hee, we divie te data inte K subsers of equa Qi z€S ; | | | —_|— WLIE Kk equais the sample Size , then this 1g called a “Leave-one - our” 16 Boot stvapping & i) The hootsivapping mepho) involves the jrevahH ve Gesamplin4 of 4 datase Wi th placement. ii] Repeating this mutkiple Himes helps fro Obtain A yectoy of €otimales: | ii) Bootstrapping can Compute Variance , expecre | value, and offer elevanr Statistics of these Cstimath€s .Paga No. al a B or | | Lid y | & Diffevence Berween —— - a St aie Di Clus keY¢i Ser INSs: lomeyahve Clu iViSive CIus. iL = ethog MLE Z.0 a | O Tt is Top-down * | 7 Cale. fees @ It Stay with al i Objects in One | Object: custeyes 9) Og gi : ar @) Clushews ak mevge (8) Clugskeys Aye Subdivile ei] WhO lavae # wea ino _omatl & Sma Cet were © : Clu ber: @ It ger merges uori {O It get Sublivived unpil All Obsecrs an€ iN Single Cath obf€Ck foyvms A Clug ky. Clugeey On its OWN, ®Each Jata por starts |@ all Jato points Start in if> Own Clusty Ina SiGe Clustey . ‘ . \ | Q) Sciit-leayn proves © Diyisive Cluskeying is — a Methods fox Not Curren fh _mplemenHd — erate hve. cohen [JN Scikjf-leayn. —e Such a5 “Way”, “Corn ple re” ere | | = 2 Ex. Image Seqmenfation , QDE-* «Marker Seqmenia Hot Customey 5-44 neNtah on, Clenetic Biological Classi fycarion cs CG:yr IT coo Gi] k-means _clusie ving Algevithm — 728 ean Clusteving i @ _K-Means Clustéyin# is an Unsupervisey Machine oh \AChing | jeavning Algoyi thn , qvoups tp Unlabeled dara into diffevent Cluarevs - —6 UN Supervised Machine Leeming 19 the pyocess of taqching A @mpurey fo use unlabeled | a unclassrfiad dJara_an) enabling Het ls ; Opervare on Fhak data without supervvision. Q@ withour amu previous dara trainin the machyije’s ___| Job in this "case ip re organize lnsoyre) date ha ACCov dN avalle ig , Patterns AN) Vawiatl ons. 1@ The goal of Clusreving iS to divide the Population oy Set Of data Points inro 4 Numbey of gvoups So that the Jata points within Cath group ae Moe Comparable 10 one anotev any Jiffevenr Fyrom the para pois within tee otter 4youps: (5) The algoui fan will Caresoyize tha jhems into | k groups ov Clugbers OF Sinn lavi 4 . The atqoyi fam Works, aS follows ij Pixs b we faniomlA wiHeWet le points , Caltej means | i] we CAbe OY) 2. each [tem to ifs Closest meam AN) we Upjare Ha means Coovinares | iii) We (Cpear the Process for a iver Numbev of irevarions am) ab the En), we havd Ov Clusters. — ee 11-1 hic tSBefore K- Means AFKy K- Means .j Lae . = —— es FG, K 2 Medojis CMSEEViINg wigorithm oo | A edoid Cam be defined ag a point in tie | Clusrey , whose dissimilaviries with all the atthwr ' — | gonis in Re clu ank Minimum { Lee —_ lot Jisdimilayih of the mejod (Ci) an) object —|@ Te Cost in _K- Medoids a igokifinm in given as ae | eae —+ = a Page No. ——|; Date 7 — =a = J fea ees K-Medoids dlyor thw, was Proposed b+ Kaufman_an) Rousseuw in tte yea 1982 CPid 15 @ieulakes by using b= [Pi =i ees 2 Ah =.Gy | Ci Piece | Allaokithyn ¢- ; ‘Halize 2° Selecrk sanjom points out of then dake Poinks 45 [es Mmejoj)s - iJ AsSociale Catch Jara pain Haro Closest Me deid by using, avd Common di skance MeEHVIC rm efhein iJ Wile The Cosr Jecreases + For Cach pneiets m, for each jaka_o poiny which is ner 4 VYeEJOI) S- © Swap Man) &, Tsociare 2ACh (dara point ho hue Closes Medojid ,aNrcecompure The Cosh. _ OLE tw folal Cost io move than that in the Pyeviows S He P , UNlo Pte Swap.Page No. Date | -X plain Multilevel a) Mulhidieensiona aA - Asseciqtion yule Mining In Detail - Mself)/Mulrijevel Associaton Rutes 8- hd When tyansqachions Jata id taken for fink ns BL Lt is present ak the low level of absiraction that is él EL a | @ It NEY difficut to for association rales ce i | [ow (eye! of AbStyaction AO Jara Scary City 15 There [) | iso ce surat gues Gin nor ficient Used. @) Using nce pr higveaychies teansaction dara Can be Lepstconte) ap vowious levels of abstvaction. In Mulplevel SsouatHon Rules Associa hon rhe | Ink genvrate) at multiple Levels of abStyathign | | @ Insrea) of oiNng gt lowev leyd) of abstraction iy A S50CiA FjON) GUles Ve genera rtd From hicghey level 4 | of abstyaction Which Se Pyesents Common Sense knowlege s — | any cam be use) QFE icigntty | Nt te | Need of Multipie-Levél Associgtion Rute : le SomeHmes ak low data (evel, date JoeS nor Show any — Significant Patrevn. Burk flee are Useful in fory mation — hid ina behind - e Aim is to fin) tre hidjen infovmation ins Letweby jevels of alstracHon SSS aee ——. ee _—_ Bet of low) - [eve Conceprs fo highey eye [2 No + Date on boreal a if) Concept Hievaychy 8 - Tt i9 4 Sequence of Mappings fom a Pe A erg: [ Nonefes | Fa z pen Bectene rerwonic| | Elertyonic.| ion 1 Lfvesh bo | Paces | ff, Fasmericl [corted Mouitel Tapiopl Iminel | Beeng) [Sard bchoese [@ Puttigimensionas Association Ruses &- Tn Multidimensional aS3ociahion yule Qual Hes Can be gbsolure Ov quanti Fa HVS ° Quantitahve Chance yistics aye numey)c an ) GNSolidales Grier. eNumeyic haits Shout) be discretized. © Multi dimensional affiliation gu? Comprises of move Than ONR MEAS ureMENF. PAppvoaches in Mining mull dimensional affiliation Yules &- Tare Ap proacles in mining mult, dimensionas AFF lighion «shes ark AS following | | St = 11. 1 — ttt 4 4 << | NUsing Static diocyetizahion oF quantile quai) Dio crtHzebion io Sane and happens pseceding min i} Distrerized veces ave hesias-unnitjuel. SY ii] USe_apriovi calculation fo locahe al] K-24 ua Pre} Sers. Fath Subser of téqular cake Sek Should be ConriMNuous. a @ Using Powerful discveh zation of TuaMHEahive Haypce ® ‘J known a4 mining Quantitakive ASseciarto Rules. 9 it) Numeric properties ae Pryogyessivel”y discwerieved | aia —— Gri) FoR TUPLES $- LSing dig tance base) diScré Fizakiey) orth patna | i] fevbovn bunching fe discover 1h rime _povie) Meluja | i] Ger affiliation cures \gict Jooking for oe evings of | groups that happen) pogortov: J_ Mayke! Bagker finalysig in Dara Mining . Q ee Aldara mining: fee ique thar 15 Wsed ro : Uncoyey Puytlase Parreyns inan4 serail Setting 19 Knowy) AS Maykek bhaskel andl 5/5. > 33 In simple fteyms Basically [aver bagker analysis in dara mining jo Fo anajyee the Combination oF pyojucrs which been isonet S heniehtl CS | Pe eet ore Le This technique hat gives tht Careful stab oF purchases done a4 Cusfomey i a Supeymoaneer ig Concept identifies the partevn of frequent — | Purchase 1f@mS b4 Customers. | 1® this anaisia Can help fo promort deals , offers, Sale by Fae Companies am) dake mining Hecbniques, bolp fo achieve this analysis dae: 4 ©) Rigk agsessment ANI Frauy aréy Also USE pie data -mining Cneept fe ian fying in Appropriate oY unusual behayioy Cfc. There ave thee f1pes of Mavker Basket fneitasic ant A5 follows 8- ij Deacrip Hue May lath basket bNal4 Sis 3- : Tig Govt of = Jooks foxy pategn s | an) ConnecHons in the _jafa that @x5' bé fween | tht Components of A Marker baSleer | Ii] Poedictive Maykef Baskeek Analy Sis 3 ysis 1aar peelcls lutale Markey bagkef anal jaar leno) 2S PreJiCHiVR marker bas ii] Diffewenriat Marker Basuer Anat Differ riat Markel ana! sis analy ce vaypattonS between Hem. On | Benefirs of Mayket Basker Anatasis 2-——_| (® Prhancey Cuskomev anpeyspemaing © Improved Inventors oe © Bettey Pricing Strategies: a 1@ Sales Gvowth, eral | | : A ee of Mayket Baskek prianagi's 4 (2 E-Commerce = T Finance ® Telecovmunicarions (@)_ Manu factuvin q