HDFS Commands Part - I

Prequirement

Before start Hadoop shell have to install Hadoop.

File System Shell

1. Most of the commands in FS shell is like corresponding Linux commands.

2. The FileSystem (FS) shell is invoked by bin/hadoop fs <args>. All the FS shell commands take path URIs as arguments.

3. For HDFS the scheme is hdfs, and for the local filesystem the scheme is file. The scheme and authority are optional. If not specified, the default scheme specified in the configuration is
   used.

4. An HDFS file or directory such as /parent/child can be specified as hdfs://<namenodehost>/dir_1/dir_2 or simply as /dir_1/dir_2 (given that your configuration is set to point to hdfs://<namenodehost>).

5. Error information is sent to stderr and the output is sent to stdout.

Basic Commands

1. version  

This HDFS command prints the Hadoop version. 

     Example: hdfs dfs version

2. cat 

This HDFS command used to displays the contents of the filename on stdout/console.

     Usages: hdfs dfs -cat <file_path>

     Example: hdfs dfs -cat /dir_1/child_1/sample

3. ls

This HDFS command used to show all the file and directory on specified path.

For a file returns stat on the file with the following format:

<file_name> <replicas> <file_size> <modification_date>
<modification_time> <permissions> <userid> <groupid>

     Usages: hdfs dfs -ls <dir_path>

    Example: hdfs dfs -ls /dir_1

4. mkdir

This HDFS command used to create directory on specified path.

     Usages: hdfs dfs -mkdir <path> 

    Example: hdfs dfs -mkdir /dir_1

5. cp

This HDFS command used to copy the directory/file from source to destination on specified path. The directory/file will be available on both place source and destination.

     Usages: hdfs dfs -cp <src> <dst>

     Example: hdfs dfs -cp /dir_1/file.orc /dir_2/

6. mv

This HDFS command used to move directory/file from source to destination on specified path. The directory/file will available only on destination.

     Usages: hdfs dfs -mv <src> <dst>

     Example: hdfs dfs -mv /dir_1/file.orc /dir_2/

7. get

This HDFS command used to copy file or directory from hadoop file system to local file system.

     Usages: hdfs dfs -get <hdfs_path> <local_path>

     Example: hdfs dfs -get /dir_1/file.orc /home/user/Desktop/

8. put

This HDFS command used to copy file or directory from

local file system to hadoop file system.

     Usages: hdfs dfs -put <local_path> <hdfs_path>

     Example: hdfs dfs -put /home/user/Desktop/

/file.orc /dir_1/

9. rm

This HDFS command used to delete file or empty directory on specified path.

     Usages: hdfs dfs -rm <path>

     Example: hdfs dfs -rm /dir_1/file.orc 

10. rmr

This HDFS command will delete all the content inside the directory then the directory itself.

     Usages: hdfs dfs -rmr <hdfs_path>

     Example: hdfs dfs -rmr /dir_1/

11. rmdir

This HDFS command used to remove the directory.

     Usages: hdfs dfs -rmdir <hdfs_path>

     Example: hdfs dfs -rmdir /dir_1/

12. touchz

This HDFS command will create file with 0 byes.

     Usages: hdfs dfs -touchz <file_path>

     Example: hdfs dfs -touchz /dir_1/file.txt

What next?

In next session will see intermediate level commands.

 

Post your queries/issues in comment section :)