Search in sources :

Example 6 with AbstractFileName

use of org.apache.commons.vfs2.provider.AbstractFileName in project pentaho-kettle by pentaho.

the class ConnectionFileSystem method createFile.

@Override
protected FileObject createFile(AbstractFileName abstractFileName) throws Exception {
    String connectionName = ((ConnectionFileName) abstractFileName).getConnection();
    VFSConnectionDetails connectionDetails = (VFSConnectionDetails) connectionManager.get().getConnectionDetails(connectionName);
    String url = getUrl(abstractFileName, connectionDetails);
    AbstractFileObject fileObject = null;
    String domain = null;
    if (url != null) {
        domain = connectionDetails.getDomain();
        Variables variables = new Variables();
        variables.setVariable(CONNECTION, connectionName);
        fileObject = (AbstractFileObject) KettleVFS.getFileObject(url, variables);
    }
    return new ConnectionFileObject(abstractFileName, this, fileObject, domain);
}
Also used : Variables(org.pentaho.di.core.variables.Variables) VFSConnectionDetails(org.pentaho.di.connections.vfs.VFSConnectionDetails) AbstractFileObject(org.apache.commons.vfs2.provider.AbstractFileObject)

Example 7 with AbstractFileName

use of org.apache.commons.vfs2.provider.AbstractFileName in project commons-vfs by apache.

the class ZipFileSystem method init.

@Override
public void init() throws FileSystemException {
    super.init();
    try {
        // Build the index
        final Enumeration<? extends ZipEntry> entries = getZipFile().entries();
        while (entries.hasMoreElements()) {
            final ZipEntry entry = entries.nextElement();
            final AbstractFileName name = (AbstractFileName) getFileSystemManager().resolveName(getRootName(), UriParser.encode(entry.getName(), ENC));
            // Create the file
            ZipFileObject fileObj;
            if (entry.isDirectory() && getFileFromCache(name) != null) {
                fileObj = (ZipFileObject) getFileFromCache(name);
                fileObj.setZipEntry(entry);
                continue;
            }
            fileObj = createZipFileObject(name, entry);
            putFileToCache(fileObj);
            // Make sure all ancestors exist
            // TODO - create these on demand
            ZipFileObject parent;
            for (AbstractFileName parentName = (AbstractFileName) name.getParent(); parentName != null; fileObj = parent, parentName = (AbstractFileName) parentName.getParent()) {
                // Locate the parent
                parent = (ZipFileObject) getFileFromCache(parentName);
                if (parent == null) {
                    parent = createZipFileObject(parentName, null);
                    putFileToCache(parent);
                }
                // Attach child to parent
                parent.attachChild(fileObj.getName());
            }
        }
    } finally {
        closeCommunicationLink();
    }
}
Also used : ZipEntry(java.util.zip.ZipEntry) AbstractFileName(org.apache.commons.vfs2.provider.AbstractFileName)

Example 8 with AbstractFileName

use of org.apache.commons.vfs2.provider.AbstractFileName in project commons-vfs by apache.

the class HdfsFileSystem method resolveFile.

/**
 * Resolve FileName into FileObject.
 *
 * @param name The name of a file on the HdfsFileSystem.
 * @return resolved FileObject.
 * @throws FileSystemException if an error occurred.
 */
@Override
public FileObject resolveFile(final FileName name) throws FileSystemException {
    synchronized (this) {
        if (this.fs == null) {
            final String hdfsUri = name.getRootURI();
            final HdfsFileSystemConfigBuilder builder = HdfsFileSystemConfigBuilder.getInstance();
            final FileSystemOptions options = getFileSystemOptions();
            final String[] configNames = builder.getConfigNames(options);
            final Path[] configPaths = builder.getConfigPaths(options);
            final URL[] configURLs = builder.getConfigURLs(options);
            final InputStream configStream = builder.getConfigInputStream(options);
            final Configuration configConfiguration = builder.getConfigConfiguration(options);
            final Configuration conf = new Configuration(true);
            conf.set(FileSystem.FS_DEFAULT_NAME_KEY, hdfsUri);
            // no matter where they might come from
            if (configNames != null) {
                for (final String configName : configNames) {
                    log.debug("Adding HDFS configuration resource: " + configName);
                    conf.addResource(configName);
                }
            }
            if (configPaths != null) {
                for (final Path path : configPaths) {
                    log.debug("Adding HDFS configuration path: " + path);
                    conf.addResource(path);
                }
            }
            if (configURLs != null) {
                for (final URL url : configURLs) {
                    log.debug("Adding HDFS configuration URL: " + url);
                    conf.addResource(url);
                }
            }
            if (configStream != null) {
                log.debug("Adding HDFS configuration stream");
                conf.addResource(configStream);
            }
            if (configConfiguration != null) {
                log.debug("Adding HDFS configuration object");
                conf.addResource(configConfiguration);
            }
            try {
                fs = FileSystem.get(conf);
            } catch (final IOException e) {
                log.error("Error connecting to filesystem " + hdfsUri, e);
                throw new FileSystemException("Error connecting to filesystem " + hdfsUri, e);
            }
        }
    }
    final boolean useCache = null != getFileSystemManager().getFilesCache();
    FileObject fileObject = useCache ? getFileFromCache(name) : null;
    if (null == fileObject) {
        String path;
        try {
            path = URLDecoder.decode(name.getPath(), "UTF-8");
        } catch (final UnsupportedEncodingException e) {
            path = name.getPath();
        }
        final Path filePath = new Path(path);
        fileObject = new HdfsFileObject((AbstractFileName) name, this, fs, filePath);
        fileObject = decorateFileObject(fileObject);
        if (useCache) {
            this.putFileToCache(fileObject);
        }
    }
    /*
          resync the file information if requested
         */
    if (getFileSystemManager().getCacheStrategy().equals(CacheStrategy.ON_RESOLVE)) {
        fileObject.refresh();
    }
    return fileObject;
}
Also used : Path(org.apache.hadoop.fs.Path) Configuration(org.apache.hadoop.conf.Configuration) InputStream(java.io.InputStream) UnsupportedEncodingException(java.io.UnsupportedEncodingException) IOException(java.io.IOException) URL(java.net.URL) FileSystemException(org.apache.commons.vfs2.FileSystemException) FileObject(org.apache.commons.vfs2.FileObject) AbstractFileName(org.apache.commons.vfs2.provider.AbstractFileName) FileSystemOptions(org.apache.commons.vfs2.FileSystemOptions)

Example 9 with AbstractFileName

use of org.apache.commons.vfs2.provider.AbstractFileName in project commons-vfs by apache.

the class DefaultFileSystemManager method resolveName.

/**
 * Resolves a name, relative to the root.
 *
 * @param base the base file name
 * @param name the name
 * @param scope the {@link NameScope}
 * @return The FileName of the file.
 * @throws FileSystemException if an error occurs.
 */
@Override
public FileName resolveName(final FileName base, final String name, final NameScope scope) throws FileSystemException {
    FileSystemException.requireNonNull(base, "Invalid base FileName.");
    FileSystemException.requireNonNull(name, "Invalid name FileName.");
    final FileName realBase;
    if (VFS.isUriStyle() && base.isFile()) {
        realBase = base.getParent();
    } else {
        realBase = base;
    }
    final StringBuilder buffer = new StringBuilder(name);
    // Adjust separators
    UriParser.fixSeparators(buffer);
    String scheme = UriParser.extractScheme(getSchemes(), buffer.toString());
    // Determine whether to prepend the base path
    if (name.isEmpty() || scheme == null && buffer.charAt(0) != FileName.SEPARATOR_CHAR) {
        // Supplied path is not absolute
        if (!VFS.isUriStyle()) {
            // when using URIs the parent already do have the trailing "/"
            buffer.insert(0, FileName.SEPARATOR_CHAR);
        }
        buffer.insert(0, realBase.getPath());
    }
    // Normalise the path
    final FileType fileType = UriParser.normalisePath(buffer);
    // Check the name is ok
    final String resolvedPath = buffer.toString();
    if (!AbstractFileName.checkName(realBase.getPath(), resolvedPath, scope)) {
        throw new FileSystemException("vfs.provider/invalid-descendent-name.error", name);
    }
    final String fullPath;
    if (scheme != null) {
        fullPath = resolvedPath;
    } else {
        scheme = realBase.getScheme();
        fullPath = realBase.getRootURI() + resolvedPath;
    }
    final FileProvider provider = providers.get(scheme);
    if (provider != null) {
        return provider.parseUri(realBase, fullPath);
    }
    // An unknown scheme - hand it to the default provider - if possible
    if (scheme != null && defaultProvider != null) {
        return defaultProvider.parseUri(realBase, fullPath);
    }
    // this happens if we have a virtual filesystem (no provider for scheme)
    return ((AbstractFileName) realBase).createName(resolvedPath, fileType);
}
Also used : FileSystemException(org.apache.commons.vfs2.FileSystemException) FileType(org.apache.commons.vfs2.FileType) AbstractFileName(org.apache.commons.vfs2.provider.AbstractFileName) FileName(org.apache.commons.vfs2.FileName) LocalFileProvider(org.apache.commons.vfs2.provider.LocalFileProvider) AbstractFileProvider(org.apache.commons.vfs2.provider.AbstractFileProvider) FileProvider(org.apache.commons.vfs2.provider.FileProvider) AbstractFileName(org.apache.commons.vfs2.provider.AbstractFileName)

Example 10 with AbstractFileName

use of org.apache.commons.vfs2.provider.AbstractFileName in project commons-vfs by apache.

the class VirtualFileProvider method createFileSystem.

/**
 * Creates an empty virtual file system.
 *
 * @param rootUri The root of the file system.
 * @return A FileObject in the FileSystem.
 * @throws FileSystemException if an error occurs.
 */
public FileObject createFileSystem(final String rootUri) throws FileSystemException {
    final AbstractFileName rootName = new VirtualFileName(rootUri, FileName.ROOT_PATH, FileType.FOLDER);
    final VirtualFileSystem fs = new VirtualFileSystem(rootName, null);
    addComponent(fs);
    return fs.getRoot();
}
Also used : AbstractFileName(org.apache.commons.vfs2.provider.AbstractFileName)

Aggregations

AbstractFileName (org.apache.commons.vfs2.provider.AbstractFileName)8 FileSystemException (org.apache.commons.vfs2.FileSystemException)4 FileName (org.apache.commons.vfs2.FileName)3 IOException (java.io.IOException)2 FileObject (org.apache.commons.vfs2.FileObject)2 FileType (org.apache.commons.vfs2.FileType)2 DelegateFileObject (org.apache.commons.vfs2.provider.DelegateFileObject)2 InputStream (java.io.InputStream)1 UnsupportedEncodingException (java.io.UnsupportedEncodingException)1 URL (java.net.URL)1 ZipEntry (java.util.zip.ZipEntry)1 TarArchiveEntry (org.apache.commons.compress.archivers.tar.TarArchiveEntry)1 FileSystemOptions (org.apache.commons.vfs2.FileSystemOptions)1 AbstractFileObject (org.apache.commons.vfs2.provider.AbstractFileObject)1 AbstractFileProvider (org.apache.commons.vfs2.provider.AbstractFileProvider)1 FileProvider (org.apache.commons.vfs2.provider.FileProvider)1 LocalFileProvider (org.apache.commons.vfs2.provider.LocalFileProvider)1 Configuration (org.apache.hadoop.conf.Configuration)1 Path (org.apache.hadoop.fs.Path)1 VFSConnectionDetails (org.pentaho.di.connections.vfs.VFSConnectionDetails)1