Find whether file exists or not in HDFS using shell script

You can try -test option to achieve the same.

hdfs dfs -test -[defszrw] HDFS_PATH
  • -d: if the path is a directory, return 0.
  • -e: if the path exists, return 0.

Since 2.7.0

  • -f: if the path is a file, return 0.
  • -s: if the path is not empty, return 0.
  • -r: if the path exists and read permission is granted, return 0.

since 2.8.0

  • -w: if the path exists and write permission is granted, return 0.
  • -z: if the file is zero-length, return 0.

Example:

if hdfs dfs -test -e $HDFS_PATH; then
    echo "[$HDFS_PATH] exists on HDFS"
    hdfs dfs -ls $HDFS_PATH
fi

Reference: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#test