Search in sources :

Example 1 with FileCorruptedCacheException

use of diskCacheV111.util.FileCorruptedCacheException in project dcache by dCache.

the class HttpPoolRequestHandler method userEventTriggered.

@Override
public void userEventTriggered(ChannelHandlerContext ctx, Object event) throws Exception {
    if (event instanceof IdleStateEvent) {
        IdleStateEvent idleStateEvent = (IdleStateEvent) event;
        if (idleStateEvent.state() == IdleState.ALL_IDLE) {
            if (LOGGER.isInfoEnabled()) {
                LOGGER.info("Connection from {} id idle; disconnecting.", ctx.channel().remoteAddress());
            }
            releaseAllFiles(uploadsSeeError(new FileCorruptedCacheException("Channel idle for too long during upload.")));
            ctx.close();
        }
    }
}
Also used : IdleStateEvent(io.netty.handler.timeout.IdleStateEvent) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException)

Example 2 with FileCorruptedCacheException

use of diskCacheV111.util.FileCorruptedCacheException in project dcache by dCache.

the class ChimeraNameSpaceProvider method commitUpload.

@Override
public FileAttributes commitUpload(Subject subject, FsPath temporaryPath, FsPath finalPath, Set<CreateOption> options, Set<FileAttribute> attributesToFetch) throws CacheException {
    try {
        FsPath temporaryDir = getParentOfFile(temporaryPath);
        FsPath finalDir = getParentOfFile(finalPath);
        checkIsTemporaryDirectory(temporaryPath, temporaryDir);
        /* File must have been created...
             */
        ExtendedInode uploadDirInode;
        ExtendedInode temporaryDirInode;
        ExtendedInode inodeOfFile;
        try {
            uploadDirInode = new ExtendedInode(_fs, _fs.path2inode(temporaryDir.parent().toString()));
            temporaryDirInode = uploadDirInode.inodeOf(temporaryDir.name(), STAT);
            inodeOfFile = temporaryDirInode.inodeOf(temporaryPath.name(), STAT);
        } catch (FileNotFoundChimeraFsException e) {
            throw new FileNotFoundCacheException("No such file or directory: " + temporaryPath, e);
        }
        /* ...and upload must have completed...
             */
        ImmutableList<StorageLocatable> locations = inodeOfFile.getLocations();
        if (locations.isEmpty()) {
            throw new FileIsNewCacheException("Upload has not completed.");
        }
        /* ...and it must have the correct size.
             */
        ImmutableList<String> size = inodeOfFile.getTag(TAG_EXPECTED_SIZE);
        if (!size.isEmpty()) {
            long expectedSize = Long.parseLong(size.get(0));
            long actualSize = inodeOfFile.statCache().getSize();
            if (expectedSize != actualSize) {
                throw new FileCorruptedCacheException(expectedSize, actualSize);
            }
        }
        /* Target directory must exist.
             */
        ExtendedInode finalDirInode;
        try {
            finalDirInode = new ExtendedInode(_fs, _fs.path2inode(finalDir.toString()));
        } catch (FileNotFoundChimeraFsException e) {
            throw new FileNotFoundCacheException("No such file or directory: " + finalDir, e);
        }
        /* File must not exist unless overwrite is enabled.
             */
        try {
            ExtendedInode inodeOfExistingFile = finalDirInode.inodeOf(finalPath.name(), STAT);
            if (!options.contains(CreateOption.OVERWRITE_EXISTING)) {
                throw new FileExistsCacheException("File exists: " + finalPath);
            }
            /* User must be authorized to delete existing file.
                 */
            if (!Subjects.isExemptFromNamespaceChecks(subject)) {
                FileAttributes attributesOfParent = getFileAttributesForPermissionHandler(finalDirInode);
                FileAttributes attributesOfFile = getFileAttributesForPermissionHandler(inodeOfExistingFile);
                if (_permissionHandler.canDeleteFile(subject, attributesOfParent, attributesOfFile) != ACCESS_ALLOWED) {
                    throw new PermissionDeniedCacheException("Overwrite denied: " + finalPath);
                }
            }
        } catch (FileNotFoundChimeraFsException ignored) {
        }
        /* Read file attributes before moving the file. Otherwise the cached parent will
             * be gone.
             */
        FileAttributes attributes = getFileAttributes(inodeOfFile, attributesToFetch);
        /* File is moved to correct directory.
             */
        _fs.rename(inodeOfFile, temporaryDirInode, temporaryPath.name(), finalDirInode, finalPath.name());
        /* Delete temporary upload directory and any files in it.
             */
        removeRecursively(uploadDirInode, temporaryDir.name(), temporaryDirInode, i -> {
        });
        return attributes;
    } catch (ChimeraFsException e) {
        throw new CacheException(CacheException.UNEXPECTED_SYSTEM_EXCEPTION, e.getMessage());
    } catch (NumberFormatException e) {
        throw new FileCorruptedCacheException("Failed to commit file: " + e.getMessage());
    }
}
Also used : FileIsNewCacheException(diskCacheV111.util.FileIsNewCacheException) FileExistsCacheException(diskCacheV111.util.FileExistsCacheException) LockedCacheException(diskCacheV111.util.LockedCacheException) AttributeExistsCacheException(diskCacheV111.util.AttributeExistsCacheException) NotDirCacheException(diskCacheV111.util.NotDirCacheException) InvalidMessageCacheException(diskCacheV111.util.InvalidMessageCacheException) FileNotFoundCacheException(diskCacheV111.util.FileNotFoundCacheException) NotFileCacheException(diskCacheV111.util.NotFileCacheException) CacheException(diskCacheV111.util.CacheException) NoAttributeCacheException(diskCacheV111.util.NoAttributeCacheException) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException) PermissionDeniedCacheException(diskCacheV111.util.PermissionDeniedCacheException) StorageLocatable(org.dcache.chimera.StorageLocatable) FileNotFoundChimeraFsException(org.dcache.chimera.FileNotFoundChimeraFsException) PermissionDeniedCacheException(diskCacheV111.util.PermissionDeniedCacheException) ChimeraFsException(org.dcache.chimera.ChimeraFsException) DirNotEmptyChimeraFsException(org.dcache.chimera.DirNotEmptyChimeraFsException) FileExistsChimeraFsException(org.dcache.chimera.FileExistsChimeraFsException) FileNotFoundChimeraFsException(org.dcache.chimera.FileNotFoundChimeraFsException) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException) FileNotFoundCacheException(diskCacheV111.util.FileNotFoundCacheException) FileIsNewCacheException(diskCacheV111.util.FileIsNewCacheException) FileAttributes(org.dcache.vehicles.FileAttributes) FileExistsCacheException(diskCacheV111.util.FileExistsCacheException) FsPath(diskCacheV111.util.FsPath)

Example 3 with FileCorruptedCacheException

use of diskCacheV111.util.FileCorruptedCacheException in project dcache by dCache.

the class XrootdPoolRequestHandler method channelInactive.

@Override
public void channelInactive(ChannelHandlerContext ctx) {
    writeLock.lock();
    try {
        /* close leftover descriptors */
        for (FileDescriptor descriptor : _descriptors) {
            if (descriptor != null) {
                if (descriptor.isPersistOnSuccessfulClose()) {
                    removeDescriptorAtomically(descriptor);
                    descriptor.getChannel().release(new FileCorruptedCacheException("File was opened with Persist On Successful Close and not closed."));
                } else if (descriptor.getChannel().getIoMode().contains(StandardOpenOption.WRITE)) {
                    removeDescriptorAtomically(descriptor);
                    descriptor.getChannel().release(new CacheException("Client disconnected without closing file."));
                } else if (!descriptor.getChannel().getIoMode().contains(StandardOpenOption.READ)) {
                    descriptor.getChannel().release();
                } else {
                    /*
                         *  Because IO stall during a read may trigger the xrootd client
                         *  to attempt, after a timeout, to reconnect by opening another socket,
                         *  we would like not to reject it on the basis of a missing mover.  Thus in the
                         *  case that the file descriptor maps to a READ mover channel, we leave the
                         *  mover in the map held by the transfer service.  We start a timer
                         *  in case there is no reconnect, in which case the channel is then released.
                         */
                    _server.scheduleReconnectTimerForMover(descriptor);
                    _log.debug("{} channeInactive, starting timer for reconnect with mover {}.", ctx.channel(), descriptor.getChannel().getMoverUuid());
                }
            }
        }
    } finally {
        writeLock.unlock();
        ;
    }
}
Also used : CacheException(diskCacheV111.util.CacheException) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException)

Example 4 with FileCorruptedCacheException

use of diskCacheV111.util.FileCorruptedCacheException in project dcache by dCache.

the class Storage method putDone.

@Override
public void putDone(SRMUser user, String localTransferPath, URI surl, boolean overwrite) throws SRMException {
    try {
        Subject subject = asDcacheUser(user).getSubject();
        Restriction restriction = asDcacheUser(user).getRestriction();
        FsPath fullPath = getPath(surl);
        checkNonBrokenUpload(localTransferPath);
        EnumSet<CreateOption> options = EnumSet.noneOf(CreateOption.class);
        if (overwrite) {
            options.add(CreateOption.OVERWRITE_EXISTING);
        }
        PnfsCommitUpload msg = new PnfsCommitUpload(subject, restriction, FsPath.create(localTransferPath), fullPath, options, EnumSet.of(PNFSID, SIZE, STORAGEINFO));
        msg = _pnfsStub.sendAndWait(msg);
        DoorRequestInfoMessage infoMsg = new DoorRequestInfoMessage(getCellAddress());
        infoMsg.setSubject(subject);
        infoMsg.setBillingPath(fullPath.toString());
        infoMsg.setTransferPath(localTransferPath);
        infoMsg.setTransaction(CDC.getSession());
        infoMsg.setPnfsId(msg.getFileAttributes().getPnfsId());
        infoMsg.setResult(0, "");
        infoMsg.setFileSize(msg.getFileAttributes().getSizeIfPresent().orElse(0L));
        infoMsg.setStorageInfo(msg.getFileAttributes().getStorageInfo());
        Origin origin = Subjects.getOrigin(subject);
        if (origin != null) {
            infoMsg.setClient(origin.getAddress().getHostAddress());
        }
        _billingStub.notify(infoMsg);
    } catch (FileNotFoundCacheException e) {
        throw new SRMInvalidPathException(e.getMessage(), e);
    } catch (FileIsNewCacheException | FileCorruptedCacheException e) {
        throw new SRMException(e.getMessage(), e);
    } catch (PermissionDeniedCacheException e) {
        throw new SRMAuthorizationException("Permission denied.", e);
    } catch (FileExistsCacheException e) {
        throw new SRMDuplicationException(surl + " exists.", e);
    } catch (CacheException e) {
        throw new SRMInternalErrorException(e.getMessage(), e);
    } catch (InterruptedException e) {
        throw new SRMInternalErrorException("Operation interrupted", e);
    } catch (NoRouteToCellException e) {
        throw new SRMInternalErrorException("Internal communication failure", e);
    }
}
Also used : DoorRequestInfoMessage(diskCacheV111.vehicles.DoorRequestInfoMessage) Origin(org.dcache.auth.Origin) SRMAuthorizationException(org.dcache.srm.SRMAuthorizationException) FileIsNewCacheException(diskCacheV111.util.FileIsNewCacheException) FileExistsCacheException(diskCacheV111.util.FileExistsCacheException) NotDirCacheException(diskCacheV111.util.NotDirCacheException) FileNotFoundCacheException(diskCacheV111.util.FileNotFoundCacheException) TimeoutCacheException(diskCacheV111.util.TimeoutCacheException) CacheException(diskCacheV111.util.CacheException) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException) PermissionDeniedCacheException(diskCacheV111.util.PermissionDeniedCacheException) SRMInvalidPathException(org.dcache.srm.SRMInvalidPathException) Subject(javax.security.auth.Subject) SRMInternalErrorException(org.dcache.srm.SRMInternalErrorException) Restriction(org.dcache.auth.attributes.Restriction) PermissionDeniedCacheException(diskCacheV111.util.PermissionDeniedCacheException) SRMException(org.dcache.srm.SRMException) NoRouteToCellException(dmg.cells.nucleus.NoRouteToCellException) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException) FileNotFoundCacheException(diskCacheV111.util.FileNotFoundCacheException) FileIsNewCacheException(diskCacheV111.util.FileIsNewCacheException) CreateOption(org.dcache.namespace.CreateOption) PnfsCommitUpload(diskCacheV111.vehicles.PnfsCommitUpload) FileExistsCacheException(diskCacheV111.util.FileExistsCacheException) SRMDuplicationException(org.dcache.srm.SRMDuplicationException) FsPath(diskCacheV111.util.FsPath)

Example 5 with FileCorruptedCacheException

use of diskCacheV111.util.FileCorruptedCacheException in project dcache by dCache.

the class ReplicaRepository method openEntry.

@Override
public ReplicaDescriptor openEntry(PnfsId id, Set<? extends OpenOption> flags) throws CacheException {
    _stateLock.readLock().lock();
    try {
        checkInitialized();
        FileAttributes fileAttributes;
        ReplicaRecord entry = getReplicaRecord(id);
        synchronized (entry) {
            switch(entry.getState()) {
                case NEW:
                case FROM_CLIENT:
                case FROM_STORE:
                case FROM_POOL:
                    throw new LockedCacheException("File is incomplete");
                case BROKEN:
                    throw new FileCorruptedCacheException("File is broken");
                case REMOVED:
                case DESTROYED:
                    throw new LockedCacheException("File has been removed");
                case PRECIOUS:
                case CACHED:
                    break;
            }
            fileAttributes = entry.getFileAttributes();
            if (!flags.contains(OpenFlags.NOATIME)) {
                entry.setLastAccessTime(System.currentTimeMillis());
            }
            entry.incrementLinkCount();
        }
        // Here we assume that all client-drive activity can (potentially)
        // update the file's atime (hence does not have NOATIME flag), while
        // all dCache-internal activity cannot (hence has NOATIME flag).
        boolean isInternalActivity = flags.contains(OpenFlags.NOATIME);
        return new ReadHandleImpl(_pnfs, entry, fileAttributes, isInternalActivity);
    } catch (FileNotInCacheException e) {
        /* Somebody got the idea that we have the file, so we make
             * sure to remove any stray pointers.
             */
        try {
            ReplicaRecord entry = _store.create(id, EnumSet.noneOf(OpenFlags.class));
            entry.update("Removing replica created to recover from unknown open request", r -> r.setState(REMOVED));
        } catch (DuplicateEntryException concurrentCreation) {
            return openEntry(id, flags);
        } catch (CacheException | RuntimeException f) {
            e.addSuppressed(f);
        }
        throw e;
    } finally {
        _stateLock.readLock().unlock();
    }
}
Also used : ReplicaRecord(org.dcache.pool.repository.ReplicaRecord) SpaceSweeperPolicy(org.dcache.pool.repository.SpaceSweeperPolicy) ScheduledFuture(java.util.concurrent.ScheduledFuture) ImmediateAllocator(org.dcache.pool.classic.ImmediateAllocator) LoggerFactory(org.slf4j.LoggerFactory) NEW(org.dcache.pool.repository.ReplicaState.NEW) Allocator(org.dcache.pool.repository.Allocator) DiskErrorCacheException(diskCacheV111.util.DiskErrorCacheException) CompletionService(java.util.concurrent.CompletionService) PoolDataBeanProvider(org.dcache.pool.PoolDataBeanProvider) IllegalTransitionException(org.dcache.pool.repository.IllegalTransitionException) Command(dmg.util.command.Command) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) PnfsHandler(diskCacheV111.util.PnfsHandler) Future(java.util.concurrent.Future) REMOVED(org.dcache.pool.repository.ReplicaState.REMOVED) DiskSpace(diskCacheV111.util.DiskSpace) FaultListener(org.dcache.pool.FaultListener) Map(java.util.Map) ReplicaState(org.dcache.pool.repository.ReplicaState) CellSetupProvider(dmg.cells.nucleus.CellSetupProvider) EnumSet(java.util.EnumSet) ReadWriteLock(java.util.concurrent.locks.ReadWriteLock) PrintWriter(java.io.PrintWriter) FaultAction(org.dcache.pool.FaultAction) FaultEvent(org.dcache.pool.FaultEvent) FileAttributes(org.dcache.vehicles.FileAttributes) PNFSID(org.dcache.namespace.FileAttribute.PNFSID) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) LockedCacheException(diskCacheV111.util.LockedCacheException) Set(java.util.Set) BlockingQueue(java.util.concurrent.BlockingQueue) ReplicaStore(org.dcache.pool.repository.ReplicaStore) CellCommandListener(dmg.cells.nucleus.CellCommandListener) GuardedBy(javax.annotation.concurrent.GuardedBy) CacheEntry(org.dcache.pool.repository.CacheEntry) CacheExceptionFactory(org.dcache.util.CacheExceptionFactory) ReplicaStoreCache(org.dcache.pool.repository.ReplicaStoreCache) Preconditions.checkState(com.google.common.base.Preconditions.checkState) ArrayBlockingQueue(java.util.concurrent.ArrayBlockingQueue) List(java.util.List) FairQueueAllocator(org.dcache.pool.classic.FairQueueAllocator) GiB(org.dcache.util.ByteUnit.GiB) EntryChangeEvent(org.dcache.pool.repository.EntryChangeEvent) FileInCacheException(diskCacheV111.util.FileInCacheException) ExecutorCompletionService(java.util.concurrent.ExecutorCompletionService) Argument(dmg.util.command.Argument) CopyOnWriteArrayList(java.util.concurrent.CopyOnWriteArrayList) CellInfoProvider(dmg.cells.nucleus.CellInfoProvider) StickyChangeEvent(org.dcache.pool.repository.StickyChangeEvent) ThreadPoolExecutor(java.util.concurrent.ThreadPoolExecutor) Stopwatch(com.google.common.base.Stopwatch) CellAddressCore(dmg.cells.nucleus.CellAddressCore) DuplicateEntryException(org.dcache.pool.repository.DuplicateEntryException) Callable(java.util.concurrent.Callable) CompletableFuture(java.util.concurrent.CompletableFuture) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) ArrayList(java.util.ArrayList) SpaceRecord(org.dcache.pool.repository.SpaceRecord) HashSet(java.util.HashSet) StickyRecord(org.dcache.pool.repository.StickyRecord) OptionalLong(java.util.OptionalLong) RejectedExecutionException(java.util.concurrent.RejectedExecutionException) CacheException(diskCacheV111.util.CacheException) StateChangeListener(org.dcache.pool.repository.StateChangeListener) Objects.requireNonNull(java.util.Objects.requireNonNull) CellIdentityAware(dmg.cells.nucleus.CellIdentityAware) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException) PRECIOUS(org.dcache.pool.repository.ReplicaState.PRECIOUS) RepositoryData(org.dcache.pool.repository.json.RepositoryData) STORAGEINFO(org.dcache.namespace.FileAttribute.STORAGEINFO) PnfsId(diskCacheV111.util.PnfsId) Repository(org.dcache.pool.repository.Repository) Logger(org.slf4j.Logger) Iterator(java.util.Iterator) OpenOption(java.nio.file.OpenOption) StateChangeEvent(org.dcache.pool.repository.StateChangeEvent) FileNotInCacheException(diskCacheV111.util.FileNotInCacheException) BlockingAllocator(org.dcache.pool.classic.BlockingAllocator) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) Account(org.dcache.pool.repository.Account) PnfsAddCacheLocationMessage(diskCacheV111.vehicles.PnfsAddCacheLocationMessage) LimitedAllocator(org.dcache.pool.repository.LimitedAllocator) ReplicaDescriptor(org.dcache.pool.repository.ReplicaDescriptor) Collections(java.util.Collections) LockedCacheException(diskCacheV111.util.LockedCacheException) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException) DuplicateEntryException(org.dcache.pool.repository.DuplicateEntryException) ReplicaRecord(org.dcache.pool.repository.ReplicaRecord) FileAttributes(org.dcache.vehicles.FileAttributes) FileNotInCacheException(diskCacheV111.util.FileNotInCacheException)

Aggregations

FileCorruptedCacheException (diskCacheV111.util.FileCorruptedCacheException)8 CacheException (diskCacheV111.util.CacheException)6 FsPath (diskCacheV111.util.FsPath)4 FileAttributes (org.dcache.vehicles.FileAttributes)4 FileExistsCacheException (diskCacheV111.util.FileExistsCacheException)3 FileIsNewCacheException (diskCacheV111.util.FileIsNewCacheException)3 FileNotFoundCacheException (diskCacheV111.util.FileNotFoundCacheException)3 Collection (java.util.Collection)3 Checksum (org.dcache.util.Checksum)3 ChecksumType (org.dcache.util.ChecksumType)3 Preconditions.checkArgument (com.google.common.base.Preconditions.checkArgument)2 Preconditions.checkState (com.google.common.base.Preconditions.checkState)2 ImmutableList (com.google.common.collect.ImmutableList)2 AttributeExistsCacheException (diskCacheV111.util.AttributeExistsCacheException)2 LockedCacheException (diskCacheV111.util.LockedCacheException)2 NotDirCacheException (diskCacheV111.util.NotDirCacheException)2 PermissionDeniedCacheException (diskCacheV111.util.PermissionDeniedCacheException)2 IOException (java.io.IOException)2 StandardCharsets (java.nio.charset.StandardCharsets)2 Collections (java.util.Collections)2