Search in sources :

Example 91 with URI

use of java.net.URI in project hadoop by apache.

the class AbstractFSContractTestBase method setup.

/**
   * Setup: create the contract then init it.
   * @throws Exception on any failure
   */
@Before
public void setup() throws Exception {
    LOG.debug("== Setup ==");
    contract = createContract(createConfiguration());
    contract.init();
    //skip tests if they aren't enabled
    assumeEnabled();
    //extract the test FS
    fileSystem = contract.getTestFileSystem();
    assertNotNull("null filesystem", fileSystem);
    URI fsURI = fileSystem.getUri();
    LOG.info("Test filesystem = {} implemented by {}", fsURI, fileSystem);
    //sanity check to make sure that the test FS picked up really matches
    //the scheme chosen. This is to avoid defaulting back to the localFS
    //which would be drastic for root FS tests
    assertEquals("wrong filesystem of " + fsURI, contract.getScheme(), fsURI.getScheme());
    //create the test path
    testPath = getContract().getTestPath();
    mkdirs(testPath);
    LOG.debug("== Setup complete ==");
}
Also used : URI(java.net.URI) Before(org.junit.Before)

Example 92 with URI

use of java.net.URI in project hadoop by apache.

the class TestChRootedFileSystem method testDeleteOnExitPathHandling.

@Test
public void testDeleteOnExitPathHandling() throws IOException {
    Configuration conf = new Configuration();
    conf.setClass("fs.mockfs.impl", MockFileSystem.class, FileSystem.class);
    URI chrootUri = URI.create("mockfs://foo/a/b");
    ChRootedFileSystem chrootFs = new ChRootedFileSystem(chrootUri, conf);
    FileSystem mockFs = ((FilterFileSystem) chrootFs.getRawFileSystem()).getRawFileSystem();
    // ensure delete propagates the correct path
    Path chrootPath = new Path("/c");
    Path rawPath = new Path("/a/b/c");
    chrootFs.delete(chrootPath, false);
    verify(mockFs).delete(eq(rawPath), eq(false));
    reset(mockFs);
    // fake that the path exists for deleteOnExit
    FileStatus stat = mock(FileStatus.class);
    when(mockFs.getFileStatus(eq(rawPath))).thenReturn(stat);
    // ensure deleteOnExit propagates the correct path
    chrootFs.deleteOnExit(chrootPath);
    chrootFs.close();
    verify(mockFs).delete(eq(rawPath), eq(true));
}
Also used : Path(org.apache.hadoop.fs.Path) FileStatus(org.apache.hadoop.fs.FileStatus) Configuration(org.apache.hadoop.conf.Configuration) FileSystem(org.apache.hadoop.fs.FileSystem) ChRootedFileSystem(org.apache.hadoop.fs.viewfs.ChRootedFileSystem) FilterFileSystem(org.apache.hadoop.fs.FilterFileSystem) FilterFileSystem(org.apache.hadoop.fs.FilterFileSystem) URI(java.net.URI) ChRootedFileSystem(org.apache.hadoop.fs.viewfs.ChRootedFileSystem) Test(org.junit.Test)

Example 93 with URI

use of java.net.URI in project hadoop by apache.

the class TestChRootedFileSystem method testGetAllStoragePolicy.

@Test(timeout = 30000)
public void testGetAllStoragePolicy() throws Exception {
    Configuration conf = new Configuration();
    conf.setClass("fs.mockfs.impl", MockFileSystem.class, FileSystem.class);
    URI chrootUri = URI.create("mockfs://foo/a/b");
    ChRootedFileSystem chrootFs = new ChRootedFileSystem(chrootUri, conf);
    FileSystem mockFs = ((FilterFileSystem) chrootFs.getRawFileSystem()).getRawFileSystem();
    chrootFs.getAllStoragePolicies();
    verify(mockFs).getAllStoragePolicies();
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) FileSystem(org.apache.hadoop.fs.FileSystem) ChRootedFileSystem(org.apache.hadoop.fs.viewfs.ChRootedFileSystem) FilterFileSystem(org.apache.hadoop.fs.FilterFileSystem) FilterFileSystem(org.apache.hadoop.fs.FilterFileSystem) URI(java.net.URI) ChRootedFileSystem(org.apache.hadoop.fs.viewfs.ChRootedFileSystem) Test(org.junit.Test)

Example 94 with URI

use of java.net.URI in project hadoop by apache.

the class TestChRootedFileSystem method testAclMethodsPathTranslation.

/**
   * Tests that ChRootedFileSystem delegates calls for every ACL method to the
   * underlying FileSystem with all Path arguments translated as required to
   * enforce chroot.
   */
@Test
public void testAclMethodsPathTranslation() throws IOException {
    Configuration conf = new Configuration();
    conf.setClass("fs.mockfs.impl", MockFileSystem.class, FileSystem.class);
    URI chrootUri = URI.create("mockfs://foo/a/b");
    ChRootedFileSystem chrootFs = new ChRootedFileSystem(chrootUri, conf);
    FileSystem mockFs = ((FilterFileSystem) chrootFs.getRawFileSystem()).getRawFileSystem();
    Path chrootPath = new Path("/c");
    Path rawPath = new Path("/a/b/c");
    List<AclEntry> entries = Collections.emptyList();
    chrootFs.modifyAclEntries(chrootPath, entries);
    verify(mockFs).modifyAclEntries(rawPath, entries);
    chrootFs.removeAclEntries(chrootPath, entries);
    verify(mockFs).removeAclEntries(rawPath, entries);
    chrootFs.removeDefaultAcl(chrootPath);
    verify(mockFs).removeDefaultAcl(rawPath);
    chrootFs.removeAcl(chrootPath);
    verify(mockFs).removeAcl(rawPath);
    chrootFs.setAcl(chrootPath, entries);
    verify(mockFs).setAcl(rawPath, entries);
    chrootFs.getAclStatus(chrootPath);
    verify(mockFs).getAclStatus(rawPath);
}
Also used : Path(org.apache.hadoop.fs.Path) Configuration(org.apache.hadoop.conf.Configuration) FileSystem(org.apache.hadoop.fs.FileSystem) ChRootedFileSystem(org.apache.hadoop.fs.viewfs.ChRootedFileSystem) FilterFileSystem(org.apache.hadoop.fs.FilterFileSystem) FilterFileSystem(org.apache.hadoop.fs.FilterFileSystem) AclEntry(org.apache.hadoop.fs.permission.AclEntry) URI(java.net.URI) ChRootedFileSystem(org.apache.hadoop.fs.viewfs.ChRootedFileSystem) Test(org.junit.Test)

Example 95 with URI

use of java.net.URI in project hadoop by apache.

the class AbstractBondedFSContract method init.

@Override
public void init() throws IOException {
    super.init();
    //this test is only enabled if the test FS is present
    fsName = loadFilesystemName(getScheme());
    setEnabled(!fsName.isEmpty());
    if (isEnabled()) {
        try {
            fsURI = new URI(fsName);
            filesystem = FileSystem.get(fsURI, getConf());
        } catch (URISyntaxException e) {
            throw new IOException("Invalid URI " + fsName);
        } catch (IllegalArgumentException e) {
            throw new IOException("Unable to initialize filesystem " + fsName + ": " + e, e);
        }
    } else {
        LOG.info("skipping tests as FS name is not defined in " + getFilesystemConfKey());
    }
}
Also used : URISyntaxException(java.net.URISyntaxException) IOException(java.io.IOException) URI(java.net.URI)

Aggregations

URI (java.net.URI)5680 Test (org.junit.Test)1852 URISyntaxException (java.net.URISyntaxException)1016 IOException (java.io.IOException)749 File (java.io.File)531 HashMap (java.util.HashMap)458 ArrayList (java.util.ArrayList)452 Test (org.testng.annotations.Test)394 Configuration (org.apache.hadoop.conf.Configuration)321 Path (org.apache.hadoop.fs.Path)267 URL (java.net.URL)266 Map (java.util.Map)262 Response (javax.ws.rs.core.Response)218 List (java.util.List)184 InputStream (java.io.InputStream)154 HashSet (java.util.HashSet)136 FileSystem (org.apache.hadoop.fs.FileSystem)135 RequestContext (com.linkedin.r2.message.RequestContext)129 RestRequestBuilder (com.linkedin.r2.message.rest.RestRequestBuilder)128 RestRequest (com.linkedin.r2.message.rest.RestRequest)112