Search in sources :

Example 36 with LongSupplier

use of java.util.function.LongSupplier in project crate by crate.

the class InboundPipelineTests method testDecodeExceptionIsPropagated.

public void testDecodeExceptionIsPropagated() throws IOException {
    BiConsumer<TcpChannel, InboundMessage> messageHandler = (c, m) -> {
    };
    final StatsTracker statsTracker = new StatsTracker();
    final LongSupplier millisSupplier = () -> TimeValue.nsecToMSec(System.nanoTime());
    final InboundDecoder decoder = new InboundDecoder(Version.CURRENT, PageCacheRecycler.NON_RECYCLING_INSTANCE);
    final Supplier<CircuitBreaker> breaker = () -> new NoopCircuitBreaker("test");
    final InboundAggregator aggregator = new InboundAggregator(breaker, (Predicate<String>) action -> true);
    final InboundPipeline pipeline = new InboundPipeline(statsTracker, millisSupplier, decoder, aggregator, messageHandler);
    try (BytesStreamOutput streamOutput = new BytesStreamOutput()) {
        String actionName = "actionName";
        final Version invalidVersion = Version.V_3_2_0;
        final String value = randomAlphaOfLength(1000);
        final boolean isRequest = randomBoolean();
        final long requestId = randomNonNegativeLong();
        OutboundMessage message;
        if (isRequest) {
            message = new OutboundMessage.Request(new TestRequest(value), invalidVersion, actionName, requestId, false, false);
        } else {
            message = new OutboundMessage.Response(new TestResponse(value), invalidVersion, requestId, false, false);
        }
        final BytesReference reference = message.serialize(streamOutput);
        try (ReleasableBytesReference releasable = ReleasableBytesReference.wrap(reference)) {
            expectThrows(IllegalStateException.class, () -> pipeline.handleBytes(new FakeTcpChannel(), releasable));
        }
        // Pipeline cannot be reused after uncaught exception
        final IllegalStateException ise = expectThrows(IllegalStateException.class, () -> pipeline.handleBytes(new FakeTcpChannel(), ReleasableBytesReference.wrap(BytesArray.EMPTY)));
        assertEquals("Pipeline state corrupted by uncaught exception", ise.getMessage());
    }
}
Also used : Tuple(io.crate.common.collections.Tuple) LongSupplier(java.util.function.LongSupplier) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) BytesStreamOutput(org.elasticsearch.common.io.stream.BytesStreamOutput) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) BytesArray(org.elasticsearch.common.bytes.BytesArray) CircuitBreaker(org.elasticsearch.common.breaker.CircuitBreaker) BiConsumer(java.util.function.BiConsumer) NoopCircuitBreaker(org.elasticsearch.common.breaker.NoopCircuitBreaker) ESTestCase(org.elasticsearch.test.ESTestCase) Streams(io.crate.common.io.Streams) Releasable(org.elasticsearch.common.lease.Releasable) Predicate(java.util.function.Predicate) PageCacheRecycler(org.elasticsearch.common.util.PageCacheRecycler) IOException(java.io.IOException) BytesReference(org.elasticsearch.common.bytes.BytesReference) TestCircuitBreaker(org.elasticsearch.common.breaker.TestCircuitBreaker) Objects(java.util.Objects) Matchers.instanceOf(org.hamcrest.Matchers.instanceOf) List(java.util.List) Version(org.elasticsearch.Version) CircuitBreakingException(org.elasticsearch.common.breaker.CircuitBreakingException) TimeValue(io.crate.common.unit.TimeValue) ReleasableBytesReference(org.elasticsearch.common.bytes.ReleasableBytesReference) CircuitBreaker(org.elasticsearch.common.breaker.CircuitBreaker) NoopCircuitBreaker(org.elasticsearch.common.breaker.NoopCircuitBreaker) TestCircuitBreaker(org.elasticsearch.common.breaker.TestCircuitBreaker) BytesStreamOutput(org.elasticsearch.common.io.stream.BytesStreamOutput) Version(org.elasticsearch.Version) NoopCircuitBreaker(org.elasticsearch.common.breaker.NoopCircuitBreaker) BytesReference(org.elasticsearch.common.bytes.BytesReference) ReleasableBytesReference(org.elasticsearch.common.bytes.ReleasableBytesReference) ReleasableBytesReference(org.elasticsearch.common.bytes.ReleasableBytesReference) LongSupplier(java.util.function.LongSupplier)

Example 37 with LongSupplier

use of java.util.function.LongSupplier in project crate by crate.

the class InternalEngineTests method testTranslogReplay.

@Test
public void testTranslogReplay() throws IOException {
    final LongSupplier inSyncGlobalCheckpointSupplier = () -> this.engine.getProcessedLocalCheckpoint();
    final int numDocs = randomIntBetween(1, 10);
    for (int i = 0; i < numDocs; i++) {
        ParsedDocument doc = testParsedDocument(Integer.toString(i), null, testDocument(), new BytesArray("{}"), null);
        Engine.Index firstIndexRequest = new Engine.Index(newUid(doc), doc, UNASSIGNED_SEQ_NO, 0, Versions.MATCH_DELETED, VersionType.INTERNAL, PRIMARY, System.nanoTime(), -1, false, UNASSIGNED_SEQ_NO, 0);
        Engine.IndexResult indexResult = engine.index(firstIndexRequest);
        assertThat(indexResult.getVersion(), equalTo(1L));
    }
    assertVisibleCount(engine, numDocs);
    translogHandler = createTranslogHandler(engine.engineConfig.getIndexSettings());
    engine.close();
    // we need to reuse the engine config unless the parser.mappingModified won't work
    engine = new InternalEngine(copy(engine.config(), inSyncGlobalCheckpointSupplier));
    engine.recoverFromTranslog(translogHandler, Long.MAX_VALUE);
    engine.refresh("warm_up");
    assertVisibleCount(engine, numDocs, false);
    engine.close();
    translogHandler = createTranslogHandler(engine.engineConfig.getIndexSettings());
    engine = createEngine(store, primaryTranslogDir, inSyncGlobalCheckpointSupplier);
    engine.refresh("warm_up");
    assertVisibleCount(engine, numDocs, false);
    final boolean flush = randomBoolean();
    int randomId = randomIntBetween(numDocs + 1, numDocs + 10);
    ParsedDocument doc = testParsedDocument(Integer.toString(randomId), null, testDocument(), new BytesArray("{}"), null);
    Engine.Index firstIndexRequest = new Engine.Index(newUid(doc), doc, UNASSIGNED_SEQ_NO, 0, 1, VersionType.EXTERNAL, PRIMARY, System.nanoTime(), -1, false, UNASSIGNED_SEQ_NO, 0);
    Engine.IndexResult indexResult = engine.index(firstIndexRequest);
    assertThat(indexResult.getVersion(), equalTo(1L));
    if (flush) {
        engine.flush();
        engine.refresh("test");
    }
    doc = testParsedDocument(Integer.toString(randomId), null, testDocument(), new BytesArray("{}"), null);
    Engine.Index idxRequest = new Engine.Index(newUid(doc), doc, UNASSIGNED_SEQ_NO, 0, 2, VersionType.EXTERNAL, PRIMARY, System.nanoTime(), -1, false, UNASSIGNED_SEQ_NO, 0);
    Engine.IndexResult result = engine.index(idxRequest);
    engine.refresh("test");
    assertThat(result.getVersion(), equalTo(2L));
    try (Engine.Searcher searcher = engine.acquireSearcher("test")) {
        TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), numDocs + 1);
        assertThat(topDocs.totalHits.value, equalTo(numDocs + 1L));
    }
    engine.close();
    translogHandler = createTranslogHandler(engine.engineConfig.getIndexSettings());
    engine = createEngine(store, primaryTranslogDir, inSyncGlobalCheckpointSupplier);
    engine.refresh("warm_up");
    try (Engine.Searcher searcher = engine.acquireSearcher("test")) {
        TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), numDocs + 1);
        assertThat(topDocs.totalHits.value, equalTo(numDocs + 1L));
    }
    engine.delete(new Engine.Delete(Integer.toString(randomId), newUid(doc), UNASSIGNED_SEQ_NO, primaryTerm.get(), Versions.MATCH_ANY, VersionType.INTERNAL, Engine.Operation.Origin.PRIMARY, System.nanoTime(), UNASSIGNED_SEQ_NO, 0));
    if (randomBoolean()) {
        engine.close();
        engine = createEngine(store, primaryTranslogDir, inSyncGlobalCheckpointSupplier);
    }
    engine.refresh("test");
    try (Engine.Searcher searcher = engine.acquireSearcher("test")) {
        TopDocs topDocs = searcher.search(new MatchAllDocsQuery(), numDocs);
        assertThat(topDocs.totalHits.value, equalTo((long) numDocs));
    }
}
Also used : Searcher(org.elasticsearch.index.engine.Engine.Searcher) BytesArray(org.elasticsearch.common.bytes.BytesArray) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery) LongPoint(org.apache.lucene.document.LongPoint) TopDocs(org.apache.lucene.search.TopDocs) ParsedDocument(org.elasticsearch.index.mapper.ParsedDocument) LongSupplier(java.util.function.LongSupplier) Test(org.junit.Test)

Example 38 with LongSupplier

use of java.util.function.LongSupplier in project elasticsearch by elastic.

the class InternalEngineTests method testOutOfOrderSequenceNumbersWithVersionConflict.

public void testOutOfOrderSequenceNumbersWithVersionConflict() throws IOException {
    final List<Engine.Operation> operations = new ArrayList<>();
    final int numberOfOperations = randomIntBetween(16, 32);
    final Document document = testDocumentWithTextField();
    final AtomicLong sequenceNumber = new AtomicLong();
    final Engine.Operation.Origin origin = randomFrom(LOCAL_TRANSLOG_RECOVERY, PEER_RECOVERY, PRIMARY, REPLICA);
    final LongSupplier sequenceNumberSupplier = origin == PRIMARY ? () -> SequenceNumbersService.UNASSIGNED_SEQ_NO : sequenceNumber::getAndIncrement;
    document.add(new Field(SourceFieldMapper.NAME, BytesReference.toBytes(B_1), SourceFieldMapper.Defaults.FIELD_TYPE));
    final ParsedDocument doc = testParsedDocument("1", "test", null, document, B_1, null);
    final Term uid = newUid(doc);
    for (int i = 0; i < numberOfOperations; i++) {
        if (randomBoolean()) {
            final Engine.Index index = new Engine.Index(uid, doc, sequenceNumberSupplier.getAsLong(), 1, i, VersionType.EXTERNAL, origin, System.nanoTime(), IndexRequest.UNSET_AUTO_GENERATED_TIMESTAMP, false);
            operations.add(index);
        } else {
            final Engine.Delete delete = new Engine.Delete("test", "1", uid, sequenceNumberSupplier.getAsLong(), 1, i, VersionType.EXTERNAL, origin, System.nanoTime());
            operations.add(delete);
        }
    }
    final boolean exists = operations.get(operations.size() - 1) instanceof Engine.Index;
    Randomness.shuffle(operations);
    for (final Engine.Operation operation : operations) {
        if (operation instanceof Engine.Index) {
            engine.index((Engine.Index) operation);
        } else {
            engine.delete((Engine.Delete) operation);
        }
    }
    final long expectedLocalCheckpoint;
    if (origin == PRIMARY) {
        // we can only advance as far as the number of operations that did not conflict
        int count = 0;
        // each time the version increments as we walk the list, that counts as a successful operation
        long version = -1;
        for (int i = 0; i < numberOfOperations; i++) {
            if (operations.get(i).version() >= version) {
                count++;
                version = operations.get(i).version();
            }
        }
        // sequence numbers start at zero, so the expected local checkpoint is the number of successful operations minus one
        expectedLocalCheckpoint = count - 1;
    } else {
        expectedLocalCheckpoint = numberOfOperations - 1;
    }
    assertThat(engine.seqNoService().getLocalCheckpoint(), equalTo(expectedLocalCheckpoint));
    try (Engine.GetResult result = engine.get(new Engine.Get(true, uid))) {
        assertThat(result.exists(), equalTo(exists));
    }
}
Also used : ArrayList(java.util.ArrayList) Index(org.elasticsearch.index.Index) Term(org.apache.lucene.index.Term) ParsedDocument(org.elasticsearch.index.mapper.ParsedDocument) Document(org.elasticsearch.index.mapper.ParseContext.Document) LongPoint(org.apache.lucene.document.LongPoint) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) TextField(org.apache.lucene.document.TextField) IndexableField(org.apache.lucene.index.IndexableField) StoredField(org.apache.lucene.document.StoredField) Field(org.apache.lucene.document.Field) AtomicLong(java.util.concurrent.atomic.AtomicLong) ParsedDocument(org.elasticsearch.index.mapper.ParsedDocument) LongSupplier(java.util.function.LongSupplier)

Example 39 with LongSupplier

use of java.util.function.LongSupplier in project elasticsearch by elastic.

the class DateMathParserTests method testOnlyCallsNowIfNecessary.

public void testOnlyCallsNowIfNecessary() {
    final AtomicBoolean called = new AtomicBoolean();
    final LongSupplier now = () -> {
        called.set(true);
        return 42L;
    };
    parser.parse("2014-11-18T14:27:32", now, false, null);
    assertFalse(called.get());
    parser.parse("now/d", now, false, null);
    assertTrue(called.get());
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) LongSupplier(java.util.function.LongSupplier)

Example 40 with LongSupplier

use of java.util.function.LongSupplier in project intellij-community by JetBrains.

the class Main method test.

private static void test(List<String> list) {
    long count = // and filter!
    list.stream().filter(s -> !s.isEmpty()).co < caret > unt();
    if (count > 10) {
        LongSupplier sup = () -> count * 2;
        long result = sup.get();
        System.out.println(result);
    }
}
Also used : List(java.util.List) LongSupplier(java.util.function.LongSupplier) LongSupplier(java.util.function.LongSupplier)

Aggregations

LongSupplier (java.util.function.LongSupplier)59 Random (java.util.Random)21 Test (org.junit.Test)9 IOException (java.io.IOException)7 Path (java.nio.file.Path)7 ArrayList (java.util.ArrayList)7 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)7 AtomicLong (java.util.concurrent.atomic.AtomicLong)5 CircuitBreaker (org.elasticsearch.common.breaker.CircuitBreaker)5 BytesArray (org.elasticsearch.common.bytes.BytesArray)5 Tuple (io.crate.common.collections.Tuple)4 Streams (io.crate.common.io.Streams)4 TimeValue (io.crate.common.unit.TimeValue)4 List (java.util.List)4 Predicate (java.util.function.Predicate)4 Supplier (java.util.function.Supplier)4 StoredField (org.apache.lucene.document.StoredField)4 Version (org.elasticsearch.Version)4 NoopCircuitBreaker (org.elasticsearch.common.breaker.NoopCircuitBreaker)4 BytesReference (org.elasticsearch.common.bytes.BytesReference)4