Examples with IgniteHadoopFileSystem org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem used on opensource projects

Example 1 with IgniteHadoopFileSystem

use of org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem in project ignite by apache.

the class IgniteHadoopFileSystemLoggerStateSelfTest method fileSystem.

/**
     * Instantiate new file system.
     *
     * @return New file system.
     * @throws Exception If failed.
     */
private IgniteHadoopFileSystem fileSystem() throws Exception {
    Configuration fsCfg = new Configuration();
    fsCfg.addResource(U.resolveIgniteUrl("modules/core/src/test/config/hadoop/core-site-loopback.xml"));
    fsCfg.setBoolean("fs.igfs.impl.disable.cache", true);
    if (logging)
        fsCfg.setBoolean(String.format(PARAM_IGFS_LOG_ENABLED, "igfs@"), logging);
    fsCfg.setStrings(String.format(PARAM_IGFS_LOG_DIR, "igfs@"), U.getIgniteHome());
    return (IgniteHadoopFileSystem) FileSystem.get(new URI("igfs://igfs@/"), fsCfg);
}

Example 2 with IgniteHadoopFileSystem

use of org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem in project ignite by apache.

the class HadoopFileSystemCacheUtils method fileSystemForMrUserWithCaching.

/**
     * Common method to get the V1 file system in MapRed engine.
     * It gets the filesystem for the user specified in the
     * configuration with {@link MRJobConfig#USER_NAME} property.
     * The file systems are created and cached in the given map upon first request.
     *
     * @param uri The file system uri.
     * @param cfg The configuration.
     * @param map The caching map.
     * @return The file system.
     * @throws IOException On error.
     */
public static FileSystem fileSystemForMrUserWithCaching(@Nullable URI uri, Configuration cfg, HadoopLazyConcurrentMap<FsCacheKey, FileSystem> map) throws IOException {
    assert map != null;
    assert cfg != null;
    final String usr = getMrHadoopUser(cfg);
    assert usr != null;
    if (uri == null)
        uri = FileSystem.getDefaultUri(cfg);
    final FileSystem fs;
    try {
        final FsCacheKey key = new FsCacheKey(uri, usr, cfg);
        fs = map.getOrCreate(key);
    } catch (IgniteException ie) {
        throw new IOException(ie);
    }
    assert fs != null;
    assert !(fs instanceof IgniteHadoopFileSystem) || F.eq(usr, ((IgniteHadoopFileSystem) fs).user());
    return fs;
}

Example 3 with IgniteHadoopFileSystem

use of org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem in project ignite by apache.

the class IgniteHadoopFileSystemAbstractSelfTest method testModeResolver.

/**
     * @throws Exception If failed.
     */
public void testModeResolver() throws Exception {
    IgfsModeResolver mr = ((IgniteHadoopFileSystem) fs).getModeResolver();
    assertEquals(mode, mr.resolveMode(IgfsPath.ROOT));
}

Example 4 with IgniteHadoopFileSystem

use of org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem in project ignite by apache.

the class IgniteHadoopFileSystemAbstractSelfTest method testInitialize.

/** @throws Exception If failed. */
public void testInitialize() throws Exception {
    final IgniteHadoopFileSystem fs = new IgniteHadoopFileSystem();
    fs.initialize(primaryFsUri, primaryFsCfg);
    // Check repeatable initialization.
    try {
        fs.initialize(primaryFsUri, primaryFsCfg);
        fail();
    } catch (IOException e) {
        assertTrue(e.getMessage().contains("File system is already initialized"));
    }
    assertEquals(primaryFsUri, fs.getUri());
    assertEquals(0, fs.getUsed());
    fs.close();
}

Example 5 with IgniteHadoopFileSystem

use of org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem in project ignite by apache.

the class IgniteHadoopFileSystemAbstractSelfTest method testCloseIfNotInitialized.

/** @throws Exception If failed. */
public void testCloseIfNotInitialized() throws Exception {
    final FileSystem fs = new IgniteHadoopFileSystem();
    // Check close makes nothing harmful.
    fs.close();
}