Scribd
Upload
66 views3 pages

Setting Hadoop and Mysql 8.0

This document provides instructions to install Hadoop on an Ubuntu operating system in 11 steps: 1. Install Java JDK 8 and configure environment variables for Java home. 2. Download and extract the Hadoop tar file. 3. Configure Hadoop configuration files like core-site.xml, hdfs-site.xml and yarn-site.xml. 4. Format the HDFS file system and start all Hadoop daemons using start-all.sh.

Uploaded by

21020279 Trần Diệu Anh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
66 views3 pages

Setting Hadoop and Mysql 8.0

This document provides instructions to install Hadoop on an Ubuntu operating system in 11 steps: 1. Install Java JDK 8 and configure environment variables for Java home. 2. Download and extract the Hadoop tar file. 3. Configure Hadoop configuration files like core-site.xml, hdfs-site.xml and yarn-site.xml. 4. Format the HDFS file system and start all Hadoop daemons using start-all.sh.

Uploaded by

21020279 Trần Diệu Anh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
You are on page 1/ 3

https://codewitharjun-medium-com.translate.

goog/install-hadoop-on-ubuntu-operating-
system-6e0ca4ef9689?source=author_recirc-----8ef3d665e331----0---------------------
6d0ea0b1_45d1_4004_b330_964496b9d0d0-------
&_x_tr_sl=en&_x_tr_tl=vi&_x_tr_hl=vi&_x_tr_pto=sc

1.Install java jdk 8

✍️ sudo apt install openjdk-8-jdk

- To check it’s there cd /usr/lib/jvm

2. open .bashrc file and paste these commands

✍️ sudo nano .bashrc

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export PATH=$PATH:/usr/lib/jvm/java-8-openjdk-amd64/bin
export HADOOP_HOME=~/hadoop-3.3.5/
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
export HADOOP_STREAMING=$HADOOP_HOME/share/hadoop/tools/lib/hadoop-streaming-
3.3.5.jar
export HADOOP_LOG_DIR=$HADOOP_HOME/logs
export PDSH_RCMD_TYPE=ssh

3. install ssh

✍️ sudo apt-get install ssh

4. Go to hadoop.apache.org website download the tar file


(hadoop.apache.org — download tar file of hadoop.)

✍️ tar -zxvf ~/Downloads/hadoop-3.3.5.tar.gz


(Extract the tar file)

✍️ cd hadoop-3.3.5/etc/hadoop

- now open hadoop-env.sh


✍️ sudo nano hadoop-env.sh

JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 (set the path for JAVA_HOME)

5. Open core-site.xml
✍️ sudo nano core-site.xml //// gedit core-site.xml

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value> </property>
<property>
<name>hadoop.proxyuser.dataflair.groups</name> <value>*</value>
</property>
<property>
<name>hadoop.proxyuser.dataflair.hosts</name> <value>*</value>
</property>
<property>
<name>hadoop.proxyuser.server.hosts</name> <value>*</value>
</property>
<property>
<name>hadoop.proxyuser.server.groups</name> <value>*</value>
</property>
</configuration>

6. Open hdfs-site.xml
✍️ sudo nano hdfs-site.xml //// gedit hdfs-site.xml

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>

<property>
<name>dfs.client.block.write.locateFollowingBlock.retries</name>
<value>10</value>
</property>

7. Open mapred-site.xml
✍️ sudo nano mapred-site.xml //// gedit mapred-site.xml

<configuration>
<property>
<name>mapreduce.framework.name</name> <value>yarn</value>
</property>
<property>
<name>mapreduce.application.classpath</name>

<value>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/
hadoop/mapreduce/lib/*</value>
</property>
</configuration>

8. Open yarn-site.xml
✍️ sudo nano yarn-site.xml //// gedit yarn-site.xml

<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.env-whitelist</name>

<value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREP
END_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
</property>
</configuration>

9. Start ssh
✍️ ssh localhost
✍️ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
✍️ chmod 0600 ~/.ssh/authorized_keys
✍️ hadoop-3.3.5/bin/hdfs namenode -format

10. format the file system

✍️ export PDSH_RCMD_TYPE=ssh

11. To start

✍️ start-all.sh
(Start NameNode daemon and DataNode daemon)

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy