Search in sources :

Example 86 with Configuration

use of org.apache.hadoop.conf.Configuration in project hadoop by apache.

the class TestDeprecatedKeys method testDeprecatedKeys.

//Tests a deprecated key
public void testDeprecatedKeys() throws Exception {
    Configuration conf = new Configuration();
    conf.set("topology.script.file.name", "xyz");
    conf.set("topology.script.file.name", "xyz");
    String scriptFile = conf.get(CommonConfigurationKeys.NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY);
    assertTrue(scriptFile.equals("xyz"));
}
Also used : Configuration(org.apache.hadoop.conf.Configuration)

Example 87 with Configuration

use of org.apache.hadoop.conf.Configuration in project hadoop by apache.

the class KeyShell method main.

/**
   * main() entry point for the KeyShell.  While strictly speaking the
   * return is void, it will System.exit() with a return code: 0 is for
   * success and 1 for failure.
   *
   * @param args Command line arguments.
   * @throws Exception
   */
public static void main(String[] args) throws Exception {
    int res = ToolRunner.run(new Configuration(), new KeyShell(), args);
    System.exit(res);
}
Also used : Configuration(org.apache.hadoop.conf.Configuration)

Example 88 with Configuration

use of org.apache.hadoop.conf.Configuration in project hadoop by apache.

the class FileSystem method copyToLocalFile.

/**
   * The src file is under this filesystem, and the dst is on the local disk.
   * Copy it from the remote filesystem to the local dst name.
   * delSrc indicates if the src will be removed
   * or not. useRawLocalFileSystem indicates whether to use RawLocalFileSystem
   * as the local file system or not. RawLocalFileSystem is non checksumming,
   * So, It will not create any crc files at local.
   *
   * @param delSrc
   *          whether to delete the src
   * @param src
   *          path
   * @param dst
   *          path
   * @param useRawLocalFileSystem
   *          whether to use RawLocalFileSystem as local file system or not.
   *
   * @throws IOException for any IO error
   */
public void copyToLocalFile(boolean delSrc, Path src, Path dst, boolean useRawLocalFileSystem) throws IOException {
    Configuration conf = getConf();
    FileSystem local = null;
    if (useRawLocalFileSystem) {
        local = getLocal(conf).getRawFileSystem();
    } else {
        local = getLocal(conf);
    }
    FileUtil.copy(this, src, local, dst, delSrc, conf);
}
Also used : Configuration(org.apache.hadoop.conf.Configuration)

Example 89 with Configuration

use of org.apache.hadoop.conf.Configuration in project hadoop by apache.

the class FileSystem method copyFromLocalFile.

/**
   * The src files are on the local disk.  Add it to the filesystem at
   * the given dst name.
   * delSrc indicates if the source should be removed
   * @param delSrc whether to delete the src
   * @param overwrite whether to overwrite an existing file
   * @param srcs array of paths which are source
   * @param dst path
   * @throws IOException IO failure
   */
public void copyFromLocalFile(boolean delSrc, boolean overwrite, Path[] srcs, Path dst) throws IOException {
    Configuration conf = getConf();
    FileUtil.copy(getLocal(conf), srcs, this, dst, delSrc, overwrite, conf);
}
Also used : Configuration(org.apache.hadoop.conf.Configuration)

Example 90 with Configuration

use of org.apache.hadoop.conf.Configuration in project hadoop by apache.

the class ChecksumFileSystem method copyToLocalFile.

/**
   * The src file is under FS, and the dst is on the local disk.
   * Copy it from FS control to the local dst name.
   */
@Override
public void copyToLocalFile(boolean delSrc, Path src, Path dst) throws IOException {
    Configuration conf = getConf();
    FileUtil.copy(this, src, getLocal(conf), dst, delSrc, conf);
}
Also used : Configuration(org.apache.hadoop.conf.Configuration)

Aggregations

Configuration (org.apache.hadoop.conf.Configuration)5973 Test (org.junit.Test)3243 Path (org.apache.hadoop.fs.Path)1602 FileSystem (org.apache.hadoop.fs.FileSystem)903 IOException (java.io.IOException)850 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)727 HdfsConfiguration (org.apache.hadoop.hdfs.HdfsConfiguration)517 MiniDFSCluster (org.apache.hadoop.hdfs.MiniDFSCluster)502 File (java.io.File)499 HBaseConfiguration (org.apache.hadoop.hbase.HBaseConfiguration)388 ArrayList (java.util.ArrayList)360 URI (java.net.URI)319 BeforeClass (org.junit.BeforeClass)275 Job (org.apache.hadoop.mapreduce.Job)272 Before (org.junit.Before)264 DistributedFileSystem (org.apache.hadoop.hdfs.DistributedFileSystem)219 FSDataOutputStream (org.apache.hadoop.fs.FSDataOutputStream)203 HashMap (java.util.HashMap)192 FileStatus (org.apache.hadoop.fs.FileStatus)190 Properties (java.util.Properties)187