hadoop commands

Command Function Additional Info

Lists files and directories in If <dir> is omitted, lists the root

hdfs dfs -ls <dir>
the specified HDFS path. / directory.

Creates a new directory in Fails if the directory already

hdfs dfs -mkdir <dir>
HDFS. exists.

Creates an empty file in Used for testing and

hdfs dfs -touchz <file>
HDFS. placeholders.

hdfs dfs -put <local_path> Uploads a file from the local Ensure the HDFS destination
<hdfs_path> filesystem to HDFS. directory exists.

hdfs dfs -get <hdfs_file> Downloads a file from HDFS Local destination must exist and
<local_path> to the local system. be writable.

Displays the contents of a Useful for quickly checking file

hdfs dfs -cat <file>
file stored in HDFS. content.

Copies a file or directory Source ( <src> ) and Destination

hdfs dfs -cp <src>
from one HDFS location to ( <dest> ) must be valid HDFS
<dest>
another. paths.

hdfs dfs -mv <src> Moves (renames) a file or Works like cp , but removes the
<dest> directory in HDFS. original file after moving.

Use carefully; deleted files

hdfs dfs -rm <file> Deletes a file from HDFS.
cannot be recovered.

Removes an empty directory Fails if the directory contains files

hdfs dfs -rmdir <dir>
from HDFS. or subdirectories.

Recursively deletes a
Use with caution; deletes
hdfs dfs -rm -r <dir> directory along with its
everything inside the directory.
contents.

Displays disk usage statistics Shows size in bytes for each file
hdfs dfs -du <path> for files and directories in and directory within the specified
HDFS. path.

Displays the total size of the Summarizes the total size instead
hdfs dfs -du -s <path>
specified file or directory. of listing individual files.

hadoop commands 1
 Displays disk usage statistics Converts sizes into KB, MB, GB,
hdfs dfs -du -h <path>
in a human-readable format. etc., for better readability.

Shows the total size of a file Combines -s and -h for a

hdfs dfs -du -s -h <path> or directory in human- concise summary with easy-to-
readable format. read units.

Displays detailed disk usage

Provides additional metadata
hdfs dfs -du -v <path> information, including
about storage, if supported.
replication details.

Copies a file or directory Similar to -get , but specific to

hdfs dfs -copyToLocal
from HDFS to the local local file transfers. The local
<hdfs_path> <local_path>
filesystem. destination must be writable.

Copies a file or directory Similar to -put , but specific to

hdfs dfs -copyFromLocal
from the local filesystem to local files. The destination
<local_path> <hdfs_path>
HDFS. directory must exist.

Changes the group The group must already exist in the

hdfs dfs -chgrp
ownership of a file or system. User must be the file
<group> <path>
directory in HDFS. owner or have superuser privileges.

Recursively changes the

hdfs dfs -chgrp -R Useful when modifying permissions
group for a directory and all
<group> <path> for multiple files at once.
its contents.

hdfs dfs -chown <user> Changes the owner of a file Only the superuser can change
<path> or directory in HDFS. ownership.
hdfs dfs -chown
Changes both the owner and If only <user> is specified, the
<user>:<group>
<path> group of a file/directory. group remains unchanged.

hdfs dfs -chown -R Recursively changes the

Useful for applying changes to
<user>:<group> owner and group for a
<path>
entire directories.
directory and all its contents.

Permanently removes files Used to free up space after

hdfs dfs -expunge
from the HDFS trash. files have been deleted.

hadoop commands 2
 Immediately empties the
Requires superuser or
hdfs dfs -expunge -immediate trash without waiting for the
appropriate permissions.
regular cleanup cycle.

Lists files and directories in

<local_path> should be a
hdfs dfs -ls file:///<local_path> the specified local
valid local directory path.
filesystem path.

Lists contents of "C:\\New Ensure the path is correctly

hdfs dfs -ls file:///"C:\\New
folder" on the local formatted, especially in
folder"
filesystem. Windows.

The output consists of three

Displays the count of numbers: (1) Number of
hdfs dfs -count directories, files, and directories, (2) Number of
<path> total entries in the files, (3) Total number of
specified HDFS path. entries (directories + files +
symbolic links).

Shows additional details like

Displays the quota
hdfs dfs -count -q quota, remaining quota,
and count details for a
<path> space quota, and remaining
directory in HDFS.
space quota.

Displays the count in a Converts file sizes into KB,

hdfs dfs -count -h
human-readable MB, or GB for better
<path>
format. readability.

Command Function Additional Info

hdfs dfs - Appends data to an Cannot create a new file;

appendToFile existing HDFS file destination must exist

Displays file count, Useful for checking

hdfs dfs -count
directory count, and size directory size in HDFS

hdfs dfs - Merges multiple HDFS Does not merge

getmerge files into a single local file directories; only files

Checks if a file or Returns 0 if exists, 1 if

hdfs dfs -test -e
directory exists in HDFS not

Checks if the given path Returns 0 if directory, 1

hdfs dfs -test -d
is a directory otherwise

hadoop commands 3
 Checks if the given path Returns 0 if file, 1
hdfs dfs -test -f
is a file otherwise

Lists files and directories Shows permissions, size,

hdfs dfs -ls
in HDFS and modification date

Moves or renames Cannot move across

hdfs dfs -mv
files/directories in HDFS different HDFS clusters

Displays the first few Shows first 1 KB of the

hdfs dfs -head
lines of an HDFS file file content

Displays the last few lines Useful for checking

hdfs dfs -tail
of an HDFS file recent log entries

echo Prints the exit status of

0 = success, 1 = failure
%ERRORLEVEL% the last command

https://chatgpt.com/share/67d97934-ef90-8002-80f4-0410c7555df0

https://chatgpt.com/share/67d97934-ef90-8002-80f4-0410c7555df0

hadoop commands 4