## org.apache.hadoop.fs.s3native
Class NativeS3FileSystem

    java.lang.Object
      org.apache.hadoop.conf.Configured
          org.apache.hadoop.fs.FileSystem
              org.apache.hadoop.fs.s3native.NativeS3FileSystem

All Implemented Interfaces:
  Closeable, Configurable

    public class NativeS3FileSystem

extends FileSystem

A `FileSystem` for reading and writing files stored on Amazon S3. Unlike `S3FileSystem` this implementation stores files on S3 in their native form so they can be read by other S3 tools. A note about directories. S3 of course has no "native" support for them. The idiom we choose then is: for any directory created by this class, we use an empty object "#{dirpath}_$folder$" as a marker. Further, to interoperate with other S3 tools, we also accept the following: - an object "#{dirpath}/' denoting a directory marker - if there exists any objects with the prefix "#{dirpath}/", then the directory is said to exist - if both a file with the name of a directory and a marker for that directory exists, then the *file masks the directory*, and the directory is never returned.

See Also:
  `S3FileSystem`

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem
`FileSystem.Statistics`

Field Summary
`static org.apache.commons.logging.Log` `LOG`

Fields inherited from class org.apache.hadoop.fs.FileSystem
`FS_DEFAULT_NAME_KEY, statistics`

Constructor Summary
`NativeS3FileSystem()`
`NativeS3FileSystem(org.apache.hadoop.fs.s3native.NativeFileSystemStore store)`

Method Summary
`FSDataOutputStream` `append(Path f, int bufferSize, Progressable progress)` This optional operation is not yet supported.
`FSDataOutputStream` `create(Path f, FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress)` Opens an FSDataOutputStream at the indicated Path with write-progress reporting.
`boolean` `delete(Path path)` Deprecated.
`boolean` `delete(Path f, boolean recurse)` Delete a file.
`FileStatus` `getFileStatus(Path f)` Return a file status object that represents the path.
`URI` `getUri()` Returns a URI whose scheme and authority identify this FileSystem.
`Path` `getWorkingDirectory()` Get the current working directory for the given file system
`void` `initialize(URI uri, Configuration conf)` Called after a new FileSystem instance is constructed.
`FileStatus[]` `listStatus(Path f)` If `f` is a file, this method will make a single call to S3.
`boolean` `mkdirs(Path f, FsPermission permission)` Make the given file and all non-existent parents into directories.
`FSDataInputStream` `open(Path f, int bufferSize)` Opens an FSDataInputStream at the indicated Path.
`boolean` `rename(Path src, Path dst)` Renames Path src to Path dst.
`void` `setWorkingDirectory(Path newDir)` Set the working directory to the given directory.

Methods inherited from class org.apache.hadoop.fs.FileSystem

`addFileSystemForTesting, append, append, checkPath, clearStatistics, close, closeAll, closeAllForUGI, completeLocalOutput, concat, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, createNonRecursive, deleteOnExit, exists, get, get, get, getAllStatistics, getBlockSize, getCacheSize, getCanonicalServiceName, getCanonicalUri, getContentSummary, getDefaultBlockSize, getDefaultBlockSize, getDefaultPort, getDefaultReplication, getDefaultReplication, getDefaultUri, getDelegationToken, getFileBlockLocations, getFileChecksum, getHomeDirectory, getLength, getLocal, getName, getNamed, getReplication, getStatistics, getStatistics, getUsed, globStatus, globStatus, isDirectory, isFile, listStatus, listStatus, listStatus, makeQualified, mkdirs, mkdirs, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, open, printStatistics, processDeleteOnExit, setDefaultUri, setDefaultUri, setOwner, setPermission, setReplication, setTimes, setVerifyChecksum, startLocalOutput`

Methods inherited from class org.apache.hadoop.conf.Configured
`getConf, setConf`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### LOG

    public static final org.apache.commons.logging.Log LOG

Constructor Detail

### NativeS3FileSystem

    public NativeS3FileSystem()

### NativeS3FileSystem

    public NativeS3FileSystem(org.apache.hadoop.fs.s3native.NativeFileSystemStore store)

Method Detail

### initialize

    public void initialize(URI uri,
                           Configuration conf)
                    throws IOException

  Description copied from class: `FileSystem`
  Called after a new FileSystem instance is constructed.

  Overrides:
    `initialize` in class `FileSystem`
  Parameters:
    `uri` - a uri whose authority section names the host, port, etc. for this FileSystem
    `conf` - the configuration
  Throws:
    `IOException`

### append

    public FSDataOutputStream append(Path f,
                                     int bufferSize,
                                     Progressable progress)
                              throws IOException

  This optional operation is not yet supported.

  Specified by:
    `append` in class `FileSystem`
  Parameters:
    `f` - the existing file to be appended.
    `bufferSize` - the size of the buffer to be used.
    `progress` - for reporting progress if it is not null.
  Throws:
    `IOException`

### create

    public FSDataOutputStream create(Path f,
                                     FsPermission permission,
                                     boolean overwrite,
                                     int bufferSize,
                                     short replication,
                                     long blockSize,
                                     Progressable progress)
                              throws IOException

  Description copied from class: `FileSystem`
  Opens an FSDataOutputStream at the indicated Path with write-progress reporting.

  Specified by:
    `create` in class `FileSystem`
  Parameters:
    `f` - the file name to open
    `overwrite` - if a file with this name already exists, then if true, the file will be overwritten, and if false an error will be thrown.
    `bufferSize` - the size of the buffer to be used.
    `replication` - required block replication for the file.
  Throws:
    `IOException`
  See Also:
    `FileSystem.setPermission(Path, FsPermission)`

### delete

    @Deprecated
    public boolean delete(Path path)
                   throws IOException

  Deprecated.

  Specified by:
    `delete` in class `FileSystem`
  Throws:
    `IOException`

### delete

    public boolean delete(Path f,
                          boolean recurse)
                   throws IOException

  Description copied from class: `FileSystem`
  Delete a file.

  Specified by:
    `delete` in class `FileSystem`
  Parameters:
    `f` - the path to delete.
    `recurse` - if path is a directory and set to true, the directory is deleted else throws an exception. In case of a file the recursive can be set to either true or false.
  Returns:
    true if delete is successful else false.
  Throws:
    `IOException`

### getFileStatus

    public FileStatus getFileStatus(Path f)
                             throws IOException

  Description copied from class: `FileSystem`
  Return a file status object that represents the path.

  Specified by:
    `getFileStatus` in class `FileSystem`
  Parameters:
    `f` - The path we want information from
  Returns:
    a FileStatus object
  Throws:
    `FileNotFoundException` - when the path does not exist; IOException see specific implementation
    `IOException`

### getUri

    public URI getUri()

  Description copied from class: `FileSystem`
  Returns a URI whose scheme and authority identify this FileSystem.

  Specified by:
    `getUri` in class `FileSystem`

### listStatus

    public FileStatus[] listStatus(Path f)
                            throws IOException

  If `f` is a file, this method will make a single call to S3. If `f` is a directory, this method will make a maximum of (n / 1000) + 2 calls to S3, where n is the total number of files and directories contained directly in `f`.

  Specified by:
    `listStatus` in class `FileSystem`
  Parameters:
    `f` - given path
  Returns:
    the statuses of the files/directories in the given patch returns null, if Path f does not exist in the FileSystem
  Throws:
    `IOException`

### mkdirs

    public boolean mkdirs(Path f,
                          FsPermission permission)
                   throws IOException

  Description copied from class: `FileSystem`
  Make the given file and all non-existent parents into directories. Has the semantics of Unix 'mkdir -p'. Existence of the directory hierarchy is not an error.

  Specified by:
    `mkdirs` in class `FileSystem`
  Throws:
    `IOException`

### open

    public FSDataInputStream open(Path f,
                                  int bufferSize)
                           throws IOException

  Description copied from class: `FileSystem`
  Opens an FSDataInputStream at the indicated Path.

  Specified by:
    `open` in class `FileSystem`
  Parameters:
    `f` - the file name to open
    `bufferSize` - the size of the buffer to be used.
  Throws:
    `IOException`

### rename

    public boolean rename(Path src,
                          Path dst)
                   throws IOException

  Description copied from class: `FileSystem`
  Renames Path src to Path dst. Can take place on local fs or remote DFS.

  Specified by:
    `rename` in class `FileSystem`
  Throws:
    `IOException`

### setWorkingDirectory

    public void setWorkingDirectory(Path newDir)

  Set the working directory to the given directory.

  Specified by:
    `setWorkingDirectory` in class `FileSystem`

### getWorkingDirectory

    public Path getWorkingDirectory()

  Description copied from class: `FileSystem`
  Get the current working directory for the given file system

  Specified by:
    `getWorkingDirectory` in class `FileSystem`
  Returns:
    the directory pathname

Copyright © 2009 The Apache Software Foundation