Search in sources :

Example 11 with ReleasableLock

use of org.elasticsearch.common.util.concurrent.ReleasableLock in project elasticsearch by elastic.

the class Cache method invalidateAll.

/**
     * Invalidate all cache entries. A removal notification will be issued for invalidated entries with
     * {@link org.elasticsearch.common.cache.RemovalNotification.RemovalReason} INVALIDATED.
     */
public void invalidateAll() {
    Entry<K, V> h;
    boolean[] haveSegmentLock = new boolean[NUMBER_OF_SEGMENTS];
    try {
        for (int i = 0; i < NUMBER_OF_SEGMENTS; i++) {
            segments[i].segmentLock.writeLock().lock();
            haveSegmentLock[i] = true;
        }
        try (ReleasableLock ignored = lruLock.acquire()) {
            h = head;
            Arrays.stream(segments).forEach(segment -> segment.map = new HashMap<>());
            Entry<K, V> current = head;
            while (current != null) {
                current.state = State.DELETED;
                current = current.after;
            }
            head = tail = null;
            count = 0;
            weight = 0;
        }
    } finally {
        for (int i = NUMBER_OF_SEGMENTS - 1; i >= 0; i--) {
            if (haveSegmentLock[i]) {
                segments[i].segmentLock.writeLock().unlock();
            }
        }
    }
    while (h != null) {
        removalListener.onRemoval(new RemovalNotification<>(h.key, h.value, RemovalNotification.RemovalReason.INVALIDATED));
        h = h.after;
    }
}
Also used : HashMap(java.util.HashMap) ReleasableLock(org.elasticsearch.common.util.concurrent.ReleasableLock)

Example 12 with ReleasableLock

use of org.elasticsearch.common.util.concurrent.ReleasableLock in project elasticsearch by elastic.

the class Cache method invalidate.

/**
     * Invalidate the association for the specified key. A removal notification will be issued for invalidated
     * entries with {@link org.elasticsearch.common.cache.RemovalNotification.RemovalReason} INVALIDATED.
     *
     * @param key the key whose mapping is to be invalidated from the cache
     */
public void invalidate(K key) {
    CacheSegment<K, V> segment = getCacheSegment(key);
    Entry<K, V> entry = segment.remove(key);
    if (entry != null) {
        try (ReleasableLock ignored = lruLock.acquire()) {
            delete(entry, RemovalNotification.RemovalReason.INVALIDATED);
        }
    }
}
Also used : ReleasableLock(org.elasticsearch.common.util.concurrent.ReleasableLock)

Example 13 with ReleasableLock

use of org.elasticsearch.common.util.concurrent.ReleasableLock in project elasticsearch by elastic.

the class Cache method computeIfAbsent.

/**
     * If the specified key is not already associated with a value (or is mapped to null), attempts to compute its
     * value using the given mapping function and enters it into this map unless null. The load method for a given key
     * will be invoked at most once.
     *
     * Use of different {@link CacheLoader} implementations on the same key concurrently may result in only the first
     * loader function being called and the second will be returned the result provided by the first including any exceptions
     * thrown during the execution of the first.
     *
     * @param key    the key whose associated value is to be returned or computed for if non-existent
     * @param loader the function to compute a value given a key
     * @return the current (existing or computed) non-null value associated with the specified key
     * @throws ExecutionException thrown if loader throws an exception or returns a null value
     */
public V computeIfAbsent(K key, CacheLoader<K, V> loader) throws ExecutionException {
    long now = now();
    V value = get(key, now);
    if (value == null) {
        // we need to synchronize loading of a value for a given key; however, holding the segment lock while
        // invoking load can lead to deadlock against another thread due to dependent key loading; therefore, we
        // need a mechanism to ensure that load is invoked at most once, but we are not invoking load while holding
        // the segment lock; to do this, we atomically put a future in the map that can load the value, and then
        // get the value from this future on the thread that won the race to place the future into the segment map
        CacheSegment<K, V> segment = getCacheSegment(key);
        CompletableFuture<Entry<K, V>> future;
        CompletableFuture<Entry<K, V>> completableFuture = new CompletableFuture<>();
        try (ReleasableLock ignored = segment.writeLock.acquire()) {
            future = segment.map.putIfAbsent(key, completableFuture);
        }
        BiFunction<? super Entry<K, V>, Throwable, ? extends V> handler = (ok, ex) -> {
            if (ok != null) {
                try (ReleasableLock ignored = lruLock.acquire()) {
                    promote(ok, now);
                }
                return ok.value;
            } else {
                try (ReleasableLock ignored = segment.writeLock.acquire()) {
                    CompletableFuture<Entry<K, V>> sanity = segment.map.get(key);
                    if (sanity != null && sanity.isCompletedExceptionally()) {
                        segment.map.remove(key);
                    }
                }
                return null;
            }
        };
        CompletableFuture<V> completableValue;
        if (future == null) {
            future = completableFuture;
            completableValue = future.handle(handler);
            V loaded;
            try {
                loaded = loader.load(key);
            } catch (Exception e) {
                future.completeExceptionally(e);
                throw new ExecutionException(e);
            }
            if (loaded == null) {
                NullPointerException npe = new NullPointerException("loader returned a null value");
                future.completeExceptionally(npe);
                throw new ExecutionException(npe);
            } else {
                future.complete(new Entry<>(key, loaded, now));
            }
        } else {
            completableValue = future.handle(handler);
        }
        try {
            value = completableValue.get();
            // check to ensure the future hasn't been completed with an exception
            if (future.isCompletedExceptionally()) {
                // call get to force the exception to be thrown for other concurrent callers
                future.get();
                throw new IllegalStateException("the future was completed exceptionally but no exception was thrown");
            }
        } catch (InterruptedException e) {
            throw new IllegalStateException(e);
        }
    }
    return value;
}
Also used : LongAdder(java.util.concurrent.atomic.LongAdder) ReleasableLock(org.elasticsearch.common.util.concurrent.ReleasableLock) Arrays(java.util.Arrays) Iterator(java.util.Iterator) ReentrantLock(java.util.concurrent.locks.ReentrantLock) BiFunction(java.util.function.BiFunction) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) Objects(java.util.Objects) ExecutionException(java.util.concurrent.ExecutionException) ToLongBiFunction(java.util.function.ToLongBiFunction) Map(java.util.Map) Tuple(org.elasticsearch.common.collect.Tuple) ReadWriteLock(java.util.concurrent.locks.ReadWriteLock) ReleasableLock(org.elasticsearch.common.util.concurrent.ReleasableLock) ExecutionException(java.util.concurrent.ExecutionException) CompletableFuture(java.util.concurrent.CompletableFuture) ExecutionException(java.util.concurrent.ExecutionException)

Example 14 with ReleasableLock

use of org.elasticsearch.common.util.concurrent.ReleasableLock in project elasticsearch by elastic.

the class Translog method trimUnreferencedReaders.

void trimUnreferencedReaders() {
    try (ReleasableLock ignored = writeLock.acquire()) {
        if (closed.get()) {
            // we're shutdown potentially on some tragic event - don't delete anything
            return;
        }
        long minReferencedGen = outstandingViews.stream().mapToLong(View::minTranslogGeneration).min().orElse(Long.MAX_VALUE);
        minReferencedGen = Math.min(lastCommittedTranslogFileGeneration, minReferencedGen);
        final long finalMinReferencedGen = minReferencedGen;
        List<TranslogReader> unreferenced = readers.stream().filter(r -> r.getGeneration() < finalMinReferencedGen).collect(Collectors.toList());
        for (final TranslogReader unreferencedReader : unreferenced) {
            Path translogPath = unreferencedReader.path();
            logger.trace("delete translog file - not referenced and not current anymore {}", translogPath);
            IOUtils.closeWhileHandlingException(unreferencedReader);
            IOUtils.deleteFilesIgnoringExceptions(translogPath, translogPath.resolveSibling(getCommitCheckpointFileName(unreferencedReader.getGeneration())));
        }
        readers.removeAll(unreferenced);
    }
}
Also used : StreamOutput(org.elasticsearch.common.io.stream.StreamOutput) Versions(org.elasticsearch.common.lucene.uid.Versions) LongSupplier(java.util.function.LongSupplier) Releasables(org.elasticsearch.common.lease.Releasables) BigArrays(org.elasticsearch.common.util.BigArrays) ConcurrentCollections(org.elasticsearch.common.util.concurrent.ConcurrentCollections) Term(org.apache.lucene.index.Term) AlreadyClosedException(org.apache.lucene.store.AlreadyClosedException) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) ReentrantReadWriteLock(java.util.concurrent.locks.ReentrantReadWriteLock) ParameterizedMessage(org.apache.logging.log4j.message.ParameterizedMessage) VersionType(org.elasticsearch.index.VersionType) StandardCopyOption(java.nio.file.StandardCopyOption) ArrayList(java.util.ArrayList) BytesArray(org.elasticsearch.common.bytes.BytesArray) IndexShardComponent(org.elasticsearch.index.shard.IndexShardComponent) IndexRequest(org.elasticsearch.action.index.IndexRequest) Matcher(java.util.regex.Matcher) SequenceNumbersService(org.elasticsearch.index.seqno.SequenceNumbersService) ReleasableBytesStreamOutput(org.elasticsearch.common.io.stream.ReleasableBytesStreamOutput) Path(java.nio.file.Path) ReadWriteLock(java.util.concurrent.locks.ReadWriteLock) TwoPhaseCommit(org.apache.lucene.index.TwoPhaseCommit) ReleasableLock(org.elasticsearch.common.util.concurrent.ReleasableLock) Files(java.nio.file.Files) UUIDs(org.elasticsearch.common.UUIDs) StandardOpenOption(java.nio.file.StandardOpenOption) IOUtils(org.apache.lucene.util.IOUtils) Set(java.util.Set) IOException(java.io.IOException) BytesReference(org.elasticsearch.common.bytes.BytesReference) EOFException(java.io.EOFException) Collectors(java.util.stream.Collectors) ReleasablePagedBytesReference(org.elasticsearch.common.bytes.ReleasablePagedBytesReference) Engine(org.elasticsearch.index.engine.Engine) AbstractIndexShardComponent(org.elasticsearch.index.shard.AbstractIndexShardComponent) List(java.util.List) Stream(java.util.stream.Stream) Supplier(org.apache.logging.log4j.util.Supplier) StreamInput(org.elasticsearch.common.io.stream.StreamInput) Closeable(java.io.Closeable) Optional(java.util.Optional) Pattern(java.util.regex.Pattern) Writeable(org.elasticsearch.common.io.stream.Writeable) FileChannel(java.nio.channels.FileChannel) Path(java.nio.file.Path) ReleasableLock(org.elasticsearch.common.util.concurrent.ReleasableLock)

Example 15 with ReleasableLock

use of org.elasticsearch.common.util.concurrent.ReleasableLock in project elasticsearch by elastic.

the class Translog method createSnapshot.

private Snapshot createSnapshot(long minGeneration) {
    try (ReleasableLock ignored = readLock.acquire()) {
        ensureOpen();
        Snapshot[] snapshots = Stream.concat(readers.stream(), Stream.of(current)).filter(reader -> reader.getGeneration() >= minGeneration).map(BaseTranslogReader::newSnapshot).toArray(Snapshot[]::new);
        return new MultiSnapshot(snapshots);
    }
}
Also used : ReleasableLock(org.elasticsearch.common.util.concurrent.ReleasableLock)

Aggregations

ReleasableLock (org.elasticsearch.common.util.concurrent.ReleasableLock)48 IOException (java.io.IOException)30 AlreadyClosedException (org.apache.lucene.store.AlreadyClosedException)21 LockObtainFailedException (org.apache.lucene.store.LockObtainFailedException)12 TranslogCorruptedException (org.elasticsearch.index.translog.TranslogCorruptedException)12 ArrayList (java.util.ArrayList)7 EOFException (java.io.EOFException)6 Path (java.nio.file.Path)6 Translog (org.elasticsearch.index.translog.Translog)6 IndexFormatTooOldException (org.apache.lucene.index.IndexFormatTooOldException)5 Releasable (org.elasticsearch.common.lease.Releasable)5 Closeable (java.io.Closeable)4 ReadWriteLock (java.util.concurrent.locks.ReadWriteLock)3 ReentrantReadWriteLock (java.util.concurrent.locks.ReentrantReadWriteLock)3 LongSupplier (java.util.function.LongSupplier)3 ParameterizedMessage (org.apache.logging.log4j.message.ParameterizedMessage)3 ReleasableBytesReference (org.elasticsearch.common.bytes.ReleasableBytesReference)3 ReleasableBytesStreamOutput (org.elasticsearch.common.io.stream.ReleasableBytesStreamOutput)3 FileChannel (java.nio.channels.FileChannel)2 Files (java.nio.file.Files)2