Search in sources :

Example 61 with URI

use of java.net.URI in project hadoop by apache.

the class TestFileSystemCaching method testDefaultFsUris.

@Test
public void testDefaultFsUris() throws Exception {
    final Configuration conf = new Configuration();
    conf.set("fs.defaultfs.impl", DefaultFs.class.getName());
    final URI defaultUri = URI.create("defaultfs://host");
    FileSystem.setDefaultUri(conf, defaultUri);
    FileSystem fs = null;
    // sanity check default fs
    final FileSystem defaultFs = FileSystem.get(conf);
    assertEquals(defaultUri, defaultFs.getUri());
    // has scheme, no auth
    fs = FileSystem.get(URI.create("defaultfs:/"), conf);
    assertSame(defaultFs, fs);
    fs = FileSystem.get(URI.create("defaultfs:///"), conf);
    assertSame(defaultFs, fs);
    // has scheme, same auth
    fs = FileSystem.get(URI.create("defaultfs://host"), conf);
    assertSame(defaultFs, fs);
    // has scheme, different auth
    fs = FileSystem.get(URI.create("defaultfs://host2"), conf);
    assertNotSame(defaultFs, fs);
    // no scheme, no auth
    fs = FileSystem.get(URI.create("/"), conf);
    assertSame(defaultFs, fs);
    // no scheme, same auth
    try {
        fs = FileSystem.get(URI.create("//host"), conf);
        fail("got fs with auth but no scheme");
    } catch (UnsupportedFileSystemException e) {
    }
    // no scheme, different auth
    try {
        fs = FileSystem.get(URI.create("//host2"), conf);
        fail("got fs with auth but no scheme");
    } catch (UnsupportedFileSystemException e) {
    }
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) FileSystem(org.apache.hadoop.fs.FileSystem) URI(java.net.URI) Test(org.junit.Test)

Example 62 with URI

use of java.net.URI in project hadoop by apache.

the class TestHarFileSystemBasics method testNegativeInitWithoutIndex.

// ========== Negative:
@Test
public void testNegativeInitWithoutIndex() throws Exception {
    // delete the index file:
    final Path indexPath = new Path(harPath, "_index");
    localFileSystem.delete(indexPath, false);
    // now init the HarFs:
    final HarFileSystem hfs = new HarFileSystem(localFileSystem);
    final URI uri = new URI("har://" + harPath.toString());
    try {
        hfs.initialize(uri, new Configuration());
        Assert.fail("Exception expected.");
    } catch (IOException ioe) {
    // ok, expected.
    }
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) IOException(java.io.IOException) URI(java.net.URI) Test(org.junit.Test)

Example 63 with URI

use of java.net.URI in project hadoop by apache.

the class TestHarFileSystemBasics method testHarFsWithoutAuthority.

@Test
public void testHarFsWithoutAuthority() throws Exception {
    final URI uri = harFileSystem.getUri();
    Assert.assertNull("har uri authority not null: " + uri, uri.getAuthority());
    FileContext.getFileContext(uri, conf);
}
Also used : URI(java.net.URI) Test(org.junit.Test)

Example 64 with URI

use of java.net.URI in project hadoop by apache.

the class TestHarFileSystemBasics method testPositiveNewHarFsOnTheSameUnderlyingFs.

@Test
public void testPositiveNewHarFsOnTheSameUnderlyingFs() throws Exception {
    // Init 2nd har file system on the same underlying FS, so the
    // metadata gets reused:
    final HarFileSystem hfs = new HarFileSystem(localFileSystem);
    final URI uri = new URI("har://" + harPath.toString());
    hfs.initialize(uri, new Configuration());
    // the metadata should be reused from cache:
    assertTrue(hfs.getMetadata() == harFileSystem.getMetadata());
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) URI(java.net.URI) Test(org.junit.Test)

Example 65 with URI

use of java.net.URI in project hadoop by apache.

the class TestHarFileSystemBasics method testPositiveInitWithoutUnderlyingFS.

@Test
public void testPositiveInitWithoutUnderlyingFS() throws Exception {
    // Init HarFS with no constructor arg, so that the underlying FS object
    // is created on demand or got from cache in #initialize() method.
    final HarFileSystem hfs = new HarFileSystem();
    final URI uri = new URI("har://" + harPath.toString());
    hfs.initialize(uri, new Configuration());
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) URI(java.net.URI) Test(org.junit.Test)

Aggregations

URI (java.net.URI)5680 Test (org.junit.Test)1852 URISyntaxException (java.net.URISyntaxException)1016 IOException (java.io.IOException)749 File (java.io.File)531 HashMap (java.util.HashMap)458 ArrayList (java.util.ArrayList)452 Test (org.testng.annotations.Test)394 Configuration (org.apache.hadoop.conf.Configuration)321 Path (org.apache.hadoop.fs.Path)267 URL (java.net.URL)266 Map (java.util.Map)262 Response (javax.ws.rs.core.Response)218 List (java.util.List)184 InputStream (java.io.InputStream)154 HashSet (java.util.HashSet)136 FileSystem (org.apache.hadoop.fs.FileSystem)135 RequestContext (com.linkedin.r2.message.RequestContext)129 RestRequestBuilder (com.linkedin.r2.message.rest.RestRequestBuilder)128 RestRequest (com.linkedin.r2.message.rest.RestRequest)112