Search in sources :

Example 1 with IgfsPathNotFoundException

use of org.apache.ignite.igfs.IgfsPathNotFoundException in project ignite by apache.

the class LocalIgfsSecondaryFileSystem method affinity.

/** {@inheritDoc} */
@Override
public Collection<IgfsBlockLocation> affinity(IgfsPath path, long start, long len, long maxLen) throws IgniteException {
    File f = fileForPath(path);
    if (!f.exists())
        throw new IgfsPathNotFoundException("File not found: " + path);
    // Create fake block & fake affinity for blocks
    long blockSize = igfs.configuration().getBlockSize();
    if (maxLen <= 0)
        maxLen = Long.MAX_VALUE;
    assert maxLen > 0 : "maxLen : " + maxLen;
    long end = start + len;
    Collection<IgfsBlockLocation> blocks = new ArrayList<>((int) (len / maxLen));
    IgfsDataManager data = igfs.context().data();
    Collection<ClusterNode> lastNodes = null;
    long lastBlockIdx = -1;
    IgfsBlockLocationImpl lastBlock = null;
    for (long offset = start; offset < end; ) {
        long blockIdx = offset / blockSize;
        // Each step is min of maxLen and end of block.
        long lenStep = Math.min(maxLen - (lastBlock != null ? lastBlock.length() : 0), (blockIdx + 1) * blockSize - offset);
        lenStep = Math.min(lenStep, end - offset);
        // Create fake affinity key to map blocks of secondary filesystem to nodes.
        LocalFileSystemBlockKey affKey = new LocalFileSystemBlockKey(path, blockIdx);
        if (blockIdx != lastBlockIdx) {
            Collection<ClusterNode> nodes = data.affinityNodes(affKey);
            if (!nodes.equals(lastNodes) && lastNodes != null && lastBlock != null) {
                blocks.add(lastBlock);
                lastBlock = null;
            }
            lastNodes = nodes;
            lastBlockIdx = blockIdx;
        }
        if (lastBlock == null)
            lastBlock = new IgfsBlockLocationImpl(offset, lenStep, lastNodes);
        else
            lastBlock.increaseLength(lenStep);
        if (lastBlock.length() == maxLen || lastBlock.start() + lastBlock.length() == end) {
            blocks.add(lastBlock);
            lastBlock = null;
        }
        offset += lenStep;
    }
    return blocks;
}
Also used : IgfsDataManager(org.apache.ignite.internal.processors.igfs.IgfsDataManager) ClusterNode(org.apache.ignite.cluster.ClusterNode) LocalFileSystemBlockKey(org.apache.ignite.internal.processors.igfs.secondary.local.LocalFileSystemBlockKey) IgfsBlockLocationImpl(org.apache.ignite.internal.processors.igfs.IgfsBlockLocationImpl) ArrayList(java.util.ArrayList) IgfsBlockLocation(org.apache.ignite.igfs.IgfsBlockLocation) IgfsPathNotFoundException(org.apache.ignite.igfs.IgfsPathNotFoundException) IgfsFile(org.apache.ignite.igfs.IgfsFile) LocalFileSystemIgfsFile(org.apache.ignite.internal.processors.igfs.secondary.local.LocalFileSystemIgfsFile) File(java.io.File)

Example 2 with IgfsPathNotFoundException

use of org.apache.ignite.igfs.IgfsPathNotFoundException in project ignite by apache.

the class LocalIgfsSecondaryFileSystem method rename.

/** {@inheritDoc} */
@Override
public void rename(IgfsPath src, IgfsPath dest) {
    File srcFile = fileForPath(src);
    File destFile = fileForPath(dest);
    if (!srcFile.exists())
        throw new IgfsPathNotFoundException("Failed to perform rename because source path not found: " + src);
    if (srcFile.isDirectory() && destFile.isFile())
        throw new IgfsPathIsNotDirectoryException("Failed to perform rename because destination path is " + "directory and source path is file [src=" + src + ", dest=" + dest + ']');
    try {
        if (destFile.isDirectory())
            Files.move(srcFile.toPath(), destFile.toPath().resolve(srcFile.getName()));
        else if (!srcFile.renameTo(destFile))
            throw new IgfsException("Failed to perform rename (underlying file system returned false) " + "[src=" + src + ", dest=" + dest + ']');
    } catch (IOException e) {
        throw handleSecondaryFsError(e, "Failed to rename [src=" + src + ", dest=" + dest + ']');
    }
}
Also used : IgfsException(org.apache.ignite.igfs.IgfsException) IOException(java.io.IOException) IgfsPathNotFoundException(org.apache.ignite.igfs.IgfsPathNotFoundException) IgfsPathIsNotDirectoryException(org.apache.ignite.igfs.IgfsPathIsNotDirectoryException) IgfsFile(org.apache.ignite.igfs.IgfsFile) LocalFileSystemIgfsFile(org.apache.ignite.internal.processors.igfs.secondary.local.LocalFileSystemIgfsFile) File(java.io.File)

Example 3 with IgfsPathNotFoundException

use of org.apache.ignite.igfs.IgfsPathNotFoundException in project ignite by apache.

the class LocalIgfsSecondaryFileSystem method updatePropertiesIfNeeded.

/**
     * Update path properties if needed.
     *
     * @param path IGFS path
     * @param props Properties map.
     */
private void updatePropertiesIfNeeded(IgfsPath path, Map<String, String> props) {
    if (props == null || props.isEmpty())
        return;
    File file = fileForPath(path);
    if (!file.exists())
        throw new IgfsPathNotFoundException("Failed to update properties for path: " + path);
    LocalFileSystemUtils.updateProperties(file, props.get(IgfsUtils.PROP_GROUP_NAME), props.get(IgfsUtils.PROP_PERMISSION));
}
Also used : IgfsPathNotFoundException(org.apache.ignite.igfs.IgfsPathNotFoundException) IgfsFile(org.apache.ignite.igfs.IgfsFile) LocalFileSystemIgfsFile(org.apache.ignite.internal.processors.igfs.secondary.local.LocalFileSystemIgfsFile) File(java.io.File)

Example 4 with IgfsPathNotFoundException

use of org.apache.ignite.igfs.IgfsPathNotFoundException in project ignite by apache.

the class IgfsMetaManager method renameDual.

/**
     * Rename path in DUAL mode.
     *
     * @param fs Secondary file system.
     * @param src Source path.
     * @param dest Destination path.
     * @return Operation result.
     * @throws IgniteCheckedException If failed.
     */
public boolean renameDual(final IgfsSecondaryFileSystem fs, final IgfsPath src, final IgfsPath dest) throws IgniteCheckedException {
    if (busyLock.enterBusy()) {
        try {
            assert fs != null;
            assert src != null;
            assert dest != null;
            if (src.parent() == null)
                // Root directory cannot be renamed.
                return false;
            // Events to fire (can be done outside of a transaction).
            final Collection<IgfsEvent> pendingEvts = new LinkedList<>();
            SynchronizationTask<Boolean> task = new SynchronizationTask<Boolean>() {

                @Override
                public Boolean onSuccess(Map<IgfsPath, IgfsEntryInfo> infos) throws Exception {
                    IgfsEntryInfo srcInfo = infos.get(src);
                    IgfsEntryInfo srcParentInfo = infos.get(src.parent());
                    IgfsEntryInfo destInfo = infos.get(dest);
                    IgfsEntryInfo destParentInfo = dest.parent() != null ? infos.get(dest.parent()) : null;
                    // Source path and destination (or destination parent) must exist.
                    if (srcInfo == null)
                        throw fsException(new IgfsPathNotFoundException("Failed to rename " + "(source path not found): " + src));
                    if (destInfo == null && destParentInfo == null)
                        throw fsException(new IgfsPathNotFoundException("Failed to rename " + "(destination path not found): " + dest));
                    // Delegate to the secondary file system.
                    fs.rename(src, dest);
                    // Rename was successful, perform compensation in the local file system.
                    if (destInfo == null)
                        moveNonTx(srcInfo.id(), src.name(), srcParentInfo.id(), dest.name(), destParentInfo.id());
                    else {
                        // Move.
                        if (destInfo.isFile())
                            throw fsException("Failed to rename the path in the local file system " + "because destination path already exists and it is a file: " + dest);
                        else
                            moveNonTx(srcInfo.id(), src.name(), srcParentInfo.id(), src.name(), destInfo.id());
                    }
                    // Record event if needed.
                    if (srcInfo.isFile()) {
                        if (evts.isRecordable(EventType.EVT_IGFS_FILE_RENAMED))
                            pendingEvts.add(new IgfsEvent(src, destInfo == null ? dest : new IgfsPath(dest, src.name()), locNode, EventType.EVT_IGFS_FILE_RENAMED));
                    } else if (evts.isRecordable(EventType.EVT_IGFS_DIR_RENAMED))
                        pendingEvts.add(new IgfsEvent(src, dest, locNode, EventType.EVT_IGFS_DIR_RENAMED));
                    return true;
                }

                @Override
                public Boolean onFailure(@Nullable Exception err) throws IgniteCheckedException {
                    U.error(log, "Path rename in DUAL mode failed [source=" + src + ", destination=" + dest + ']', err);
                    throw new IgniteCheckedException("Failed to rename the path due to secondary file system " + "exception: " + src, err);
                }
            };
            try {
                return synchronizeAndExecute(task, fs, false, src, dest);
            } finally {
                for (IgfsEvent evt : pendingEvts) evts.record(evt);
            }
        } finally {
            busyLock.leaveBusy();
        }
    } else
        throw new IllegalStateException("Failed to rename in DUAL mode because Grid is stopping [src=" + src + ", dest=" + dest + ']');
}
Also used : IgfsPathNotFoundException(org.apache.ignite.igfs.IgfsPathNotFoundException) LinkedList(java.util.LinkedList) IgfsPathAlreadyExistsException(org.apache.ignite.igfs.IgfsPathAlreadyExistsException) IgfsParentNotDirectoryException(org.apache.ignite.igfs.IgfsParentNotDirectoryException) IgfsPathIsDirectoryException(org.apache.ignite.igfs.IgfsPathIsDirectoryException) IgniteCheckedException(org.apache.ignite.IgniteCheckedException) IgniteException(org.apache.ignite.IgniteException) ClusterTopologyException(org.apache.ignite.cluster.ClusterTopologyException) IgniteInterruptedException(org.apache.ignite.IgniteInterruptedException) IgniteInterruptedCheckedException(org.apache.ignite.internal.IgniteInterruptedCheckedException) IgfsDirectoryNotEmptyException(org.apache.ignite.igfs.IgfsDirectoryNotEmptyException) IgfsException(org.apache.ignite.igfs.IgfsException) IgfsConcurrentModificationException(org.apache.ignite.igfs.IgfsConcurrentModificationException) IgfsPathIsNotDirectoryException(org.apache.ignite.igfs.IgfsPathIsNotDirectoryException) IgfsPathNotFoundException(org.apache.ignite.igfs.IgfsPathNotFoundException) GridClosureException(org.apache.ignite.internal.util.lang.GridClosureException) IgfsPath(org.apache.ignite.igfs.IgfsPath) IgniteCheckedException(org.apache.ignite.IgniteCheckedException) IgfsEvent(org.apache.ignite.events.IgfsEvent) Map(java.util.Map) HashMap(java.util.HashMap) GridLeanMap(org.apache.ignite.internal.util.GridLeanMap) TreeMap(java.util.TreeMap) Nullable(org.jetbrains.annotations.Nullable)

Example 5 with IgfsPathNotFoundException

use of org.apache.ignite.igfs.IgfsPathNotFoundException in project ignite by apache.

the class IgfsMetaManager method updateTimes.

/**
     * Update times.
     *
     * @param path Path.
     * @param accessTime Access time.
     * @param modificationTime Modification time.
     * @param secondaryFs Secondary file system.
     * @throws IgniteCheckedException If failed.
     */
public void updateTimes(IgfsPath path, long modificationTime, long accessTime, IgfsSecondaryFileSystem secondaryFs) throws IgniteCheckedException {
    while (true) {
        if (busyLock.enterBusy()) {
            try {
                validTxState(false);
                // Prepare path IDs.
                IgfsPathIds pathIds = pathIds(path);
                // Prepare lock IDs.
                Set<IgniteUuid> lockIds = new TreeSet<>(PATH_ID_SORTING_COMPARATOR);
                pathIds.addExistingIds(lockIds, relaxed);
                // Start TX.
                try (GridNearTxLocal tx = startTx()) {
                    Map<IgniteUuid, IgfsEntryInfo> lockInfos = lockIds(lockIds);
                    if (secondaryFs != null && isRetryForSecondary(pathIds, lockInfos))
                        continue;
                    if (!pathIds.verifyIntegrity(lockInfos, relaxed))
                        // Directory structure changed concurrently. So we re-try.
                        continue;
                    if (pathIds.allExists()) {
                        // All files are in place. Update both primary and secondary file systems.
                        if (secondaryFs != null)
                            secondaryFs.setTimes(path, modificationTime, accessTime);
                        IgniteUuid targetId = pathIds.lastExistingId();
                        IgfsEntryInfo targetInfo = lockInfos.get(targetId);
                        id2InfoPrj.invoke(targetId, new IgfsMetaUpdateTimesProcessor(accessTime == -1 ? targetInfo.accessTime() : accessTime, modificationTime == -1 ? targetInfo.modificationTime() : modificationTime));
                        tx.commit();
                        return;
                    } else {
                        // Propagate call to the secondary FS, as we might haven't cache this part yet.
                        if (secondaryFs != null) {
                            secondaryFs.setTimes(path, modificationTime, accessTime);
                            return;
                        } else
                            throw new IgfsPathNotFoundException("Failed to update times (path not found): " + path);
                    }
                }
            } catch (IgniteException | IgniteCheckedException e) {
                throw e;
            } catch (Exception e) {
                throw new IgniteCheckedException("setTimes failed due to unexpected exception: " + path, e);
            } finally {
                busyLock.leaveBusy();
            }
        } else
            throw new IllegalStateException("Failed to update times because Grid is stopping: " + path);
    }
}
Also used : GridNearTxLocal(org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal) IgfsPathNotFoundException(org.apache.ignite.igfs.IgfsPathNotFoundException) IgfsPathAlreadyExistsException(org.apache.ignite.igfs.IgfsPathAlreadyExistsException) IgfsParentNotDirectoryException(org.apache.ignite.igfs.IgfsParentNotDirectoryException) IgfsPathIsDirectoryException(org.apache.ignite.igfs.IgfsPathIsDirectoryException) IgniteCheckedException(org.apache.ignite.IgniteCheckedException) IgniteException(org.apache.ignite.IgniteException) ClusterTopologyException(org.apache.ignite.cluster.ClusterTopologyException) IgniteInterruptedException(org.apache.ignite.IgniteInterruptedException) IgniteInterruptedCheckedException(org.apache.ignite.internal.IgniteInterruptedCheckedException) IgfsDirectoryNotEmptyException(org.apache.ignite.igfs.IgfsDirectoryNotEmptyException) IgfsException(org.apache.ignite.igfs.IgfsException) IgfsConcurrentModificationException(org.apache.ignite.igfs.IgfsConcurrentModificationException) IgfsPathIsNotDirectoryException(org.apache.ignite.igfs.IgfsPathIsNotDirectoryException) IgfsPathNotFoundException(org.apache.ignite.igfs.IgfsPathNotFoundException) GridClosureException(org.apache.ignite.internal.util.lang.GridClosureException) IgfsMetaUpdateTimesProcessor(org.apache.ignite.internal.processors.igfs.meta.IgfsMetaUpdateTimesProcessor) IgniteCheckedException(org.apache.ignite.IgniteCheckedException) TreeSet(java.util.TreeSet) IgniteUuid(org.apache.ignite.lang.IgniteUuid) IgniteException(org.apache.ignite.IgniteException)

Aggregations

IgfsPathNotFoundException (org.apache.ignite.igfs.IgfsPathNotFoundException)13 IgfsFile (org.apache.ignite.igfs.IgfsFile)7 IgfsException (org.apache.ignite.igfs.IgfsException)6 IOException (java.io.IOException)5 IgniteException (org.apache.ignite.IgniteException)5 IgfsPath (org.apache.ignite.igfs.IgfsPath)5 File (java.io.File)4 IgfsPathAlreadyExistsException (org.apache.ignite.igfs.IgfsPathAlreadyExistsException)4 IgfsPathIsDirectoryException (org.apache.ignite.igfs.IgfsPathIsDirectoryException)4 IgfsPathIsNotDirectoryException (org.apache.ignite.igfs.IgfsPathIsNotDirectoryException)4 LocalFileSystemIgfsFile (org.apache.ignite.internal.processors.igfs.secondary.local.LocalFileSystemIgfsFile)4 FileNotFoundException (java.io.FileNotFoundException)3 ArrayList (java.util.ArrayList)3 TreeSet (java.util.TreeSet)3 IgniteCheckedException (org.apache.ignite.IgniteCheckedException)3 GridNearTxLocal (org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal)3 IgniteUuid (org.apache.ignite.lang.IgniteUuid)3 FileStatus (org.apache.hadoop.fs.FileStatus)2 IgniteInterruptedException (org.apache.ignite.IgniteInterruptedException)2 ClusterTopologyException (org.apache.ignite.cluster.ClusterTopologyException)2