Search in sources :

Example 11 with DistCpOptions

use of org.apache.hadoop.tools.DistCpOptions in project hadoop by apache.

the class TestOptionsParser method testTargetPath.

@Test
public void testTargetPath() {
    DistCpOptions options = OptionsParser.parse(new String[] { "-f", "hdfs://localhost:9820/source/first", "hdfs://localhost:9820/target/" });
    Assert.assertEquals(options.getTargetPath(), new Path("hdfs://localhost:9820/target/"));
}
Also used : Path(org.apache.hadoop.fs.Path) DistCpOptions(org.apache.hadoop.tools.DistCpOptions) Test(org.junit.Test)

Example 12 with DistCpOptions

use of org.apache.hadoop.tools.DistCpOptions in project hadoop by apache.

the class TestOptionsParser method testLogPath.

@Test
public void testLogPath() {
    DistCpOptions options = OptionsParser.parse(new String[] { "hdfs://localhost:9820/source/first", "hdfs://localhost:9820/target/" });
    Assert.assertNull(options.getLogPath());
    options = OptionsParser.parse(new String[] { "-log", "hdfs://localhost:9820/logs", "hdfs://localhost:9820/source/first", "hdfs://localhost:9820/target/" });
    Assert.assertEquals(options.getLogPath(), new Path("hdfs://localhost:9820/logs"));
}
Also used : Path(org.apache.hadoop.fs.Path) DistCpOptions(org.apache.hadoop.tools.DistCpOptions) Test(org.junit.Test)

Example 13 with DistCpOptions

use of org.apache.hadoop.tools.DistCpOptions in project hadoop by apache.

the class TestOptionsParser method testParseSyncFolders.

@Test
public void testParseSyncFolders() {
    DistCpOptions options = OptionsParser.parse(new String[] { "hdfs://localhost:9820/source/first", "hdfs://localhost:9820/target/" });
    Assert.assertFalse(options.shouldSyncFolder());
    options = OptionsParser.parse(new String[] { "-update", "hdfs://localhost:9820/source/first", "hdfs://localhost:9820/target/" });
    Assert.assertTrue(options.shouldSyncFolder());
}
Also used : DistCpOptions(org.apache.hadoop.tools.DistCpOptions) Test(org.junit.Test)

Example 14 with DistCpOptions

use of org.apache.hadoop.tools.DistCpOptions in project hadoop by apache.

the class TestOptionsParser method testExclusionsOption.

@Test
public void testExclusionsOption() {
    DistCpOptions options = OptionsParser.parse(new String[] { "hdfs://localhost:9820/source/first", "hdfs://localhost:9820/target/" });
    Assert.assertNull(options.getFiltersFile());
    options = OptionsParser.parse(new String[] { "-filters", "/tmp/filters.txt", "hdfs://localhost:9820/source/first", "hdfs://localhost:9820/target/" });
    Assert.assertEquals(options.getFiltersFile(), "/tmp/filters.txt");
}
Also used : DistCpOptions(org.apache.hadoop.tools.DistCpOptions) Test(org.junit.Test)

Example 15 with DistCpOptions

use of org.apache.hadoop.tools.DistCpOptions in project hadoop by apache.

the class TestOptionsParser method testParseOverwrite.

@Test
public void testParseOverwrite() {
    DistCpOptions options = OptionsParser.parse(new String[] { "hdfs://localhost:9820/source/first", "hdfs://localhost:9820/target/" });
    Assert.assertFalse(options.shouldOverwrite());
    options = OptionsParser.parse(new String[] { "-overwrite", "hdfs://localhost:9820/source/first", "hdfs://localhost:9820/target/" });
    Assert.assertTrue(options.shouldOverwrite());
    try {
        OptionsParser.parse(new String[] { "-update", "-overwrite", "hdfs://localhost:9820/source/first", "hdfs://localhost:9820/target/" });
        Assert.fail("Update and overwrite aren't allowed together");
    } catch (IllegalArgumentException ignore) {
    }
}
Also used : DistCpOptions(org.apache.hadoop.tools.DistCpOptions) Test(org.junit.Test)

Aggregations

DistCpOptions (org.apache.hadoop.tools.DistCpOptions)34 Test (org.junit.Test)22 Path (org.apache.hadoop.fs.Path)13 Configuration (org.apache.hadoop.conf.Configuration)10 IOException (java.io.IOException)6 FileSystem (org.apache.hadoop.fs.FileSystem)5 JobContextImpl (org.apache.hadoop.mapreduce.task.JobContextImpl)5 CopyListing (org.apache.hadoop.tools.CopyListing)4 DistCp (org.apache.hadoop.tools.DistCp)4 GlobbedCopyListing (org.apache.hadoop.tools.GlobbedCopyListing)4 FileNotFoundException (java.io.FileNotFoundException)3 ArrayList (java.util.ArrayList)3 Text (org.apache.hadoop.io.Text)3 CopyListingFileStatus (org.apache.hadoop.tools.CopyListingFileStatus)3 InvocationTargetException (java.lang.reflect.InvocationTargetException)2 MalformedURLException (java.net.MalformedURLException)2 AccessControlException (java.security.AccessControlException)2 NoSuchAlgorithmException (java.security.NoSuchAlgorithmException)2 StubContext (org.apache.hadoop.tools.StubContext)2 NoSuchElementException (java.util.NoSuchElementException)1