Search in sources :

Example 1 with ReplicaRecord

use of org.dcache.pool.repository.ReplicaRecord in project dcache by dCache.

the class ReplicaRepository method createEntry.

@Override
public ReplicaDescriptor createEntry(FileAttributes fileAttributes, ReplicaState transferState, ReplicaState targetState, List<StickyRecord> stickyRecords, Set<? extends OpenOption> flags, OptionalLong maximumSize) throws CacheException {
    if (!fileAttributes.isDefined(EnumSet.of(PNFSID, STORAGEINFO))) {
        throw new IllegalArgumentException("PNFSID and STORAGEINFO are required, only got " + fileAttributes.getDefinedAttributes());
    }
    if (stickyRecords == null) {
        throw new IllegalArgumentException("List of sticky records must not be null");
    }
    PnfsId id = fileAttributes.getPnfsId();
    _stateLock.readLock().lock();
    try {
        checkOpen();
        switch(transferState) {
            case FROM_CLIENT:
            case FROM_STORE:
            case FROM_POOL:
                break;
            default:
                throw new IllegalArgumentException("Invalid initial state");
        }
        switch(targetState) {
            case PRECIOUS:
            case CACHED:
                break;
            default:
                throw new IllegalArgumentException("Invalid target state");
        }
        LOGGER.info("Creating new entry for {}", id);
        ReplicaRecord entry = _store.create(id, flags);
        return entry.update("Creating new replica", r -> {
            r.setFileAttributes(fileAttributes);
            r.setState(transferState);
            return new WriteHandleImpl(this, buildAllocator(flags, maximumSize), _pnfs, entry, fileAttributes, targetState, stickyRecords);
        });
    } catch (DuplicateEntryException e) {
        /* Somebody got the idea that we don't have the file, so we make
             * sure to register it.
             */
        _pnfs.notify(new PnfsAddCacheLocationMessage(id, getPoolName()));
        throw new FileInCacheException("Entry already exists: " + id);
    } finally {
        _stateLock.readLock().unlock();
    }
}
Also used : FileInCacheException(diskCacheV111.util.FileInCacheException) PnfsId(diskCacheV111.util.PnfsId) DuplicateEntryException(org.dcache.pool.repository.DuplicateEntryException) ReplicaRecord(org.dcache.pool.repository.ReplicaRecord) PnfsAddCacheLocationMessage(diskCacheV111.vehicles.PnfsAddCacheLocationMessage)

Example 2 with ReplicaRecord

use of org.dcache.pool.repository.ReplicaRecord in project dcache by dCache.

the class ReplicaRepository method loadRecord.

private PnfsId loadRecord(PnfsId id) throws CacheException, IllegalStateException, InterruptedException {
    ReplicaRecord entry = readReplicaRecord(id);
    if (entry != null) {
        ReplicaState state = entry.getState();
        LOGGER.debug("{} {}", id, state);
    }
    // Lazily check if repository was closed
    if (_state != State.LOADING) {
        throw new IllegalStateException("Repository was closed during loading.");
    }
    return id;
}
Also used : ReplicaRecord(org.dcache.pool.repository.ReplicaRecord) ReplicaState(org.dcache.pool.repository.ReplicaState)

Example 3 with ReplicaRecord

use of org.dcache.pool.repository.ReplicaRecord in project dcache by dCache.

the class ReplicaRepository method openEntry.

@Override
public ReplicaDescriptor openEntry(PnfsId id, Set<? extends OpenOption> flags) throws CacheException {
    _stateLock.readLock().lock();
    try {
        checkInitialized();
        FileAttributes fileAttributes;
        ReplicaRecord entry = getReplicaRecord(id);
        synchronized (entry) {
            switch(entry.getState()) {
                case NEW:
                case FROM_CLIENT:
                case FROM_STORE:
                case FROM_POOL:
                    throw new LockedCacheException("File is incomplete");
                case BROKEN:
                    throw new FileCorruptedCacheException("File is broken");
                case REMOVED:
                case DESTROYED:
                    throw new LockedCacheException("File has been removed");
                case PRECIOUS:
                case CACHED:
                    break;
            }
            fileAttributes = entry.getFileAttributes();
            if (!flags.contains(OpenFlags.NOATIME)) {
                entry.setLastAccessTime(System.currentTimeMillis());
            }
            entry.incrementLinkCount();
        }
        // Here we assume that all client-drive activity can (potentially)
        // update the file's atime (hence does not have NOATIME flag), while
        // all dCache-internal activity cannot (hence has NOATIME flag).
        boolean isInternalActivity = flags.contains(OpenFlags.NOATIME);
        return new ReadHandleImpl(_pnfs, entry, fileAttributes, isInternalActivity);
    } catch (FileNotInCacheException e) {
        /* Somebody got the idea that we have the file, so we make
             * sure to remove any stray pointers.
             */
        try {
            ReplicaRecord entry = _store.create(id, EnumSet.noneOf(OpenFlags.class));
            entry.update("Removing replica created to recover from unknown open request", r -> r.setState(REMOVED));
        } catch (DuplicateEntryException concurrentCreation) {
            return openEntry(id, flags);
        } catch (CacheException | RuntimeException f) {
            e.addSuppressed(f);
        }
        throw e;
    } finally {
        _stateLock.readLock().unlock();
    }
}
Also used : ReplicaRecord(org.dcache.pool.repository.ReplicaRecord) SpaceSweeperPolicy(org.dcache.pool.repository.SpaceSweeperPolicy) ScheduledFuture(java.util.concurrent.ScheduledFuture) ImmediateAllocator(org.dcache.pool.classic.ImmediateAllocator) LoggerFactory(org.slf4j.LoggerFactory) NEW(org.dcache.pool.repository.ReplicaState.NEW) Allocator(org.dcache.pool.repository.Allocator) DiskErrorCacheException(diskCacheV111.util.DiskErrorCacheException) CompletionService(java.util.concurrent.CompletionService) PoolDataBeanProvider(org.dcache.pool.PoolDataBeanProvider) IllegalTransitionException(org.dcache.pool.repository.IllegalTransitionException) Command(dmg.util.command.Command) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) PnfsHandler(diskCacheV111.util.PnfsHandler) Future(java.util.concurrent.Future) REMOVED(org.dcache.pool.repository.ReplicaState.REMOVED) DiskSpace(diskCacheV111.util.DiskSpace) FaultListener(org.dcache.pool.FaultListener) Map(java.util.Map) ReplicaState(org.dcache.pool.repository.ReplicaState) CellSetupProvider(dmg.cells.nucleus.CellSetupProvider) EnumSet(java.util.EnumSet) ReadWriteLock(java.util.concurrent.locks.ReadWriteLock) PrintWriter(java.io.PrintWriter) FaultAction(org.dcache.pool.FaultAction) FaultEvent(org.dcache.pool.FaultEvent) FileAttributes(org.dcache.vehicles.FileAttributes) PNFSID(org.dcache.namespace.FileAttribute.PNFSID) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) LockedCacheException(diskCacheV111.util.LockedCacheException) Set(java.util.Set) BlockingQueue(java.util.concurrent.BlockingQueue) ReplicaStore(org.dcache.pool.repository.ReplicaStore) CellCommandListener(dmg.cells.nucleus.CellCommandListener) GuardedBy(javax.annotation.concurrent.GuardedBy) CacheEntry(org.dcache.pool.repository.CacheEntry) CacheExceptionFactory(org.dcache.util.CacheExceptionFactory) ReplicaStoreCache(org.dcache.pool.repository.ReplicaStoreCache) Preconditions.checkState(com.google.common.base.Preconditions.checkState) ArrayBlockingQueue(java.util.concurrent.ArrayBlockingQueue) List(java.util.List) FairQueueAllocator(org.dcache.pool.classic.FairQueueAllocator) GiB(org.dcache.util.ByteUnit.GiB) EntryChangeEvent(org.dcache.pool.repository.EntryChangeEvent) FileInCacheException(diskCacheV111.util.FileInCacheException) ExecutorCompletionService(java.util.concurrent.ExecutorCompletionService) Argument(dmg.util.command.Argument) CopyOnWriteArrayList(java.util.concurrent.CopyOnWriteArrayList) CellInfoProvider(dmg.cells.nucleus.CellInfoProvider) StickyChangeEvent(org.dcache.pool.repository.StickyChangeEvent) ThreadPoolExecutor(java.util.concurrent.ThreadPoolExecutor) Stopwatch(com.google.common.base.Stopwatch) CellAddressCore(dmg.cells.nucleus.CellAddressCore) DuplicateEntryException(org.dcache.pool.repository.DuplicateEntryException) Callable(java.util.concurrent.Callable) CompletableFuture(java.util.concurrent.CompletableFuture) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) ArrayList(java.util.ArrayList) SpaceRecord(org.dcache.pool.repository.SpaceRecord) HashSet(java.util.HashSet) StickyRecord(org.dcache.pool.repository.StickyRecord) OptionalLong(java.util.OptionalLong) RejectedExecutionException(java.util.concurrent.RejectedExecutionException) CacheException(diskCacheV111.util.CacheException) StateChangeListener(org.dcache.pool.repository.StateChangeListener) Objects.requireNonNull(java.util.Objects.requireNonNull) CellIdentityAware(dmg.cells.nucleus.CellIdentityAware) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException) PRECIOUS(org.dcache.pool.repository.ReplicaState.PRECIOUS) RepositoryData(org.dcache.pool.repository.json.RepositoryData) STORAGEINFO(org.dcache.namespace.FileAttribute.STORAGEINFO) PnfsId(diskCacheV111.util.PnfsId) Repository(org.dcache.pool.repository.Repository) Logger(org.slf4j.Logger) Iterator(java.util.Iterator) OpenOption(java.nio.file.OpenOption) StateChangeEvent(org.dcache.pool.repository.StateChangeEvent) FileNotInCacheException(diskCacheV111.util.FileNotInCacheException) BlockingAllocator(org.dcache.pool.classic.BlockingAllocator) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) Account(org.dcache.pool.repository.Account) PnfsAddCacheLocationMessage(diskCacheV111.vehicles.PnfsAddCacheLocationMessage) LimitedAllocator(org.dcache.pool.repository.LimitedAllocator) ReplicaDescriptor(org.dcache.pool.repository.ReplicaDescriptor) Collections(java.util.Collections) LockedCacheException(diskCacheV111.util.LockedCacheException) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException) DuplicateEntryException(org.dcache.pool.repository.DuplicateEntryException) ReplicaRecord(org.dcache.pool.repository.ReplicaRecord) FileAttributes(org.dcache.vehicles.FileAttributes) FileNotInCacheException(diskCacheV111.util.FileNotInCacheException)

Example 4 with ReplicaRecord

use of org.dcache.pool.repository.ReplicaRecord in project dcache by dCache.

the class ReplicaRepository method setState.

@Override
public void setState(PnfsId id, ReplicaState state, String why) throws IllegalArgumentException, InterruptedException, CacheException {
    if (id == null) {
        throw new IllegalArgumentException("id is null");
    }
    _stateLock.readLock().lock();
    try {
        checkOpen();
        try {
            ReplicaRecord entry = getReplicaRecord(id);
            entry.update(why, r -> {
                ReplicaState source = r.getState();
                switch(source) {
                    case NEW:
                    case REMOVED:
                    case DESTROYED:
                        if (state == ReplicaState.REMOVED) {
                            /* File doesn't exist or is already
                                 * deleted. That's all we care about.
                                 */
                            return null;
                        }
                        break;
                    case PRECIOUS:
                    case CACHED:
                    case BROKEN:
                        switch(state) {
                            case REMOVED:
                            case CACHED:
                            case PRECIOUS:
                            case BROKEN:
                                return r.setState(state);
                            default:
                                break;
                        }
                    default:
                        break;
                }
                throw new IllegalTransitionException(id, source, state);
            });
        } catch (FileNotInCacheException e) {
            /* File disappeared before we could change the
                 * state. That's okay if we wanted to remove it, otherwise
                 * not.
                 */
            if (state != REMOVED) {
                throw new IllegalTransitionException(id, NEW, state);
            }
        }
    } finally {
        _stateLock.readLock().unlock();
    }
}
Also used : IllegalTransitionException(org.dcache.pool.repository.IllegalTransitionException) ReplicaRecord(org.dcache.pool.repository.ReplicaRecord) ReplicaState(org.dcache.pool.repository.ReplicaState) FileNotInCacheException(diskCacheV111.util.FileNotInCacheException)

Example 5 with ReplicaRecord

use of org.dcache.pool.repository.ReplicaRecord in project dcache by dCache.

the class ReplicaRepository method setSticky.

@Override
public void setSticky(PnfsId id, String owner, long expire, boolean overwrite) throws IllegalArgumentException, CacheException, InterruptedException {
    requireNonNull(id);
    requireNonNull(owner);
    checkArgument(expire >= -1, "Expiration time must be -1 or non-negative");
    _stateLock.readLock().lock();
    try {
        checkInitialized();
        ReplicaRecord entry;
        try {
            entry = getReplicaRecord(id);
        } catch (FileNotInCacheException e) {
            /* Attempts to set a sticky bit on a missing file may
                 * indicate a stale registration in the name space.
                 */
            try {
                entry = _store.create(id, EnumSet.noneOf(OpenFlags.class));
                entry.update("Removing replica created to recover from unknown setSticky request", r -> r.setState(REMOVED));
            } catch (DuplicateEntryException concurrentCreation) {
                setSticky(id, owner, expire, overwrite);
                return;
            } catch (CacheException | RuntimeException f) {
                e.addSuppressed(f);
            }
            throw e;
        }
        entry.update("Setting " + owner + " sticky", r -> {
            switch(r.getState()) {
                case NEW:
                case FROM_CLIENT:
                case FROM_STORE:
                case FROM_POOL:
                    throw new LockedCacheException("File is incomplete");
                case REMOVED:
                case DESTROYED:
                    throw new LockedCacheException("File has been removed");
                case BROKEN:
                case PRECIOUS:
                case CACHED:
                    break;
            }
            return r.setSticky(owner, expire, overwrite);
        });
    } finally {
        _stateLock.readLock().unlock();
    }
}
Also used : ReplicaRecord(org.dcache.pool.repository.ReplicaRecord) SpaceSweeperPolicy(org.dcache.pool.repository.SpaceSweeperPolicy) ScheduledFuture(java.util.concurrent.ScheduledFuture) ImmediateAllocator(org.dcache.pool.classic.ImmediateAllocator) LoggerFactory(org.slf4j.LoggerFactory) NEW(org.dcache.pool.repository.ReplicaState.NEW) Allocator(org.dcache.pool.repository.Allocator) DiskErrorCacheException(diskCacheV111.util.DiskErrorCacheException) CompletionService(java.util.concurrent.CompletionService) PoolDataBeanProvider(org.dcache.pool.PoolDataBeanProvider) IllegalTransitionException(org.dcache.pool.repository.IllegalTransitionException) Command(dmg.util.command.Command) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) PnfsHandler(diskCacheV111.util.PnfsHandler) Future(java.util.concurrent.Future) REMOVED(org.dcache.pool.repository.ReplicaState.REMOVED) DiskSpace(diskCacheV111.util.DiskSpace) FaultListener(org.dcache.pool.FaultListener) Map(java.util.Map) ReplicaState(org.dcache.pool.repository.ReplicaState) CellSetupProvider(dmg.cells.nucleus.CellSetupProvider) EnumSet(java.util.EnumSet) ReadWriteLock(java.util.concurrent.locks.ReadWriteLock) PrintWriter(java.io.PrintWriter) FaultAction(org.dcache.pool.FaultAction) FaultEvent(org.dcache.pool.FaultEvent) FileAttributes(org.dcache.vehicles.FileAttributes) PNFSID(org.dcache.namespace.FileAttribute.PNFSID) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) LockedCacheException(diskCacheV111.util.LockedCacheException) Set(java.util.Set) BlockingQueue(java.util.concurrent.BlockingQueue) ReplicaStore(org.dcache.pool.repository.ReplicaStore) CellCommandListener(dmg.cells.nucleus.CellCommandListener) GuardedBy(javax.annotation.concurrent.GuardedBy) CacheEntry(org.dcache.pool.repository.CacheEntry) CacheExceptionFactory(org.dcache.util.CacheExceptionFactory) ReplicaStoreCache(org.dcache.pool.repository.ReplicaStoreCache) Preconditions.checkState(com.google.common.base.Preconditions.checkState) ArrayBlockingQueue(java.util.concurrent.ArrayBlockingQueue) List(java.util.List) FairQueueAllocator(org.dcache.pool.classic.FairQueueAllocator) GiB(org.dcache.util.ByteUnit.GiB) EntryChangeEvent(org.dcache.pool.repository.EntryChangeEvent) FileInCacheException(diskCacheV111.util.FileInCacheException) ExecutorCompletionService(java.util.concurrent.ExecutorCompletionService) Argument(dmg.util.command.Argument) CopyOnWriteArrayList(java.util.concurrent.CopyOnWriteArrayList) CellInfoProvider(dmg.cells.nucleus.CellInfoProvider) StickyChangeEvent(org.dcache.pool.repository.StickyChangeEvent) ThreadPoolExecutor(java.util.concurrent.ThreadPoolExecutor) Stopwatch(com.google.common.base.Stopwatch) CellAddressCore(dmg.cells.nucleus.CellAddressCore) DuplicateEntryException(org.dcache.pool.repository.DuplicateEntryException) Callable(java.util.concurrent.Callable) CompletableFuture(java.util.concurrent.CompletableFuture) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) ArrayList(java.util.ArrayList) SpaceRecord(org.dcache.pool.repository.SpaceRecord) HashSet(java.util.HashSet) StickyRecord(org.dcache.pool.repository.StickyRecord) OptionalLong(java.util.OptionalLong) RejectedExecutionException(java.util.concurrent.RejectedExecutionException) CacheException(diskCacheV111.util.CacheException) StateChangeListener(org.dcache.pool.repository.StateChangeListener) Objects.requireNonNull(java.util.Objects.requireNonNull) CellIdentityAware(dmg.cells.nucleus.CellIdentityAware) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) FileCorruptedCacheException(diskCacheV111.util.FileCorruptedCacheException) PRECIOUS(org.dcache.pool.repository.ReplicaState.PRECIOUS) RepositoryData(org.dcache.pool.repository.json.RepositoryData) STORAGEINFO(org.dcache.namespace.FileAttribute.STORAGEINFO) PnfsId(diskCacheV111.util.PnfsId) Repository(org.dcache.pool.repository.Repository) Logger(org.slf4j.Logger) Iterator(java.util.Iterator) OpenOption(java.nio.file.OpenOption) StateChangeEvent(org.dcache.pool.repository.StateChangeEvent) FileNotInCacheException(diskCacheV111.util.FileNotInCacheException) BlockingAllocator(org.dcache.pool.classic.BlockingAllocator) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) Account(org.dcache.pool.repository.Account) PnfsAddCacheLocationMessage(diskCacheV111.vehicles.PnfsAddCacheLocationMessage) LimitedAllocator(org.dcache.pool.repository.LimitedAllocator) ReplicaDescriptor(org.dcache.pool.repository.ReplicaDescriptor) Collections(java.util.Collections) LockedCacheException(diskCacheV111.util.LockedCacheException) DuplicateEntryException(org.dcache.pool.repository.DuplicateEntryException) ReplicaRecord(org.dcache.pool.repository.ReplicaRecord) FileNotInCacheException(diskCacheV111.util.FileNotInCacheException)

Aggregations

ReplicaRecord (org.dcache.pool.repository.ReplicaRecord)7 FileNotInCacheException (diskCacheV111.util.FileNotInCacheException)4 ReplicaState (org.dcache.pool.repository.ReplicaState)4 FileInCacheException (diskCacheV111.util.FileInCacheException)3 PnfsId (diskCacheV111.util.PnfsId)3 PnfsAddCacheLocationMessage (diskCacheV111.vehicles.PnfsAddCacheLocationMessage)3 Preconditions.checkArgument (com.google.common.base.Preconditions.checkArgument)2 Preconditions.checkState (com.google.common.base.Preconditions.checkState)2 Stopwatch (com.google.common.base.Stopwatch)2 CacheException (diskCacheV111.util.CacheException)2 DiskErrorCacheException (diskCacheV111.util.DiskErrorCacheException)2 DiskSpace (diskCacheV111.util.DiskSpace)2 FileCorruptedCacheException (diskCacheV111.util.FileCorruptedCacheException)2 LockedCacheException (diskCacheV111.util.LockedCacheException)2 PnfsHandler (diskCacheV111.util.PnfsHandler)2 CellAddressCore (dmg.cells.nucleus.CellAddressCore)2 CellCommandListener (dmg.cells.nucleus.CellCommandListener)2 CellIdentityAware (dmg.cells.nucleus.CellIdentityAware)2 CellInfoProvider (dmg.cells.nucleus.CellInfoProvider)2 CellSetupProvider (dmg.cells.nucleus.CellSetupProvider)2