Search in sources :

Example 86 with BiConsumer

use of java.util.function.BiConsumer in project hazelcast by hazelcast.

the class SplitBrainTest method when_quorumIsLostOnMinority_then_jobDoesNotRestartOnMinorityAndCancelledAfterMerge.

@Test
public void when_quorumIsLostOnMinority_then_jobDoesNotRestartOnMinorityAndCancelledAfterMerge() {
    int firstSubClusterSize = 3;
    int secondSubClusterSize = 2;
    int clusterSize = firstSubClusterSize + secondSubClusterSize;
    NoOutputSourceP.executionStarted = new CountDownLatch(clusterSize * PARALLELISM);
    Job[] jobRef = new Job[1];
    Consumer<HazelcastInstance[]> beforeSplit = instances -> {
        MockPS processorSupplier = new MockPS(NoOutputSourceP::new, clusterSize);
        DAG dag = new DAG().vertex(new Vertex("test", processorSupplier));
        jobRef[0] = instances[0].getJet().newJob(dag, new JobConfig().setSplitBrainProtection(true));
        assertOpenEventually(NoOutputSourceP.executionStarted);
    };
    Future[] minorityJobFutureRef = new Future[1];
    BiConsumer<HazelcastInstance[], HazelcastInstance[]> onSplit = (firstSubCluster, secondSubCluster) -> {
        NoOutputSourceP.proceedLatch.countDown();
        assertTrueEventually(() -> assertEquals(clusterSize + firstSubClusterSize, MockPS.initCount.get()));
        long jobId = jobRef[0].getId();
        assertTrueEventually(() -> {
            JetServiceBackend service = getJetServiceBackend(firstSubCluster[0]);
            assertEquals(COMPLETED, service.getJobCoordinationService().getJobStatus(jobId).get());
        });
        JetServiceBackend service2 = getJetServiceBackend(secondSubCluster[0]);
        assertTrueEventually(() -> {
            MasterContext masterContext = service2.getJobCoordinationService().getMasterContext(jobId);
            assertNotNull(masterContext);
            minorityJobFutureRef[0] = masterContext.jobContext().jobCompletionFuture();
        });
        assertTrueAllTheTime(() -> {
            assertStatusNotRunningOrStarting(service2.getJobCoordinationService().getJobStatus(jobId).get());
        }, 20);
    };
    Consumer<HazelcastInstance[]> afterMerge = instances -> {
        assertTrueEventually(() -> {
            assertEquals(clusterSize + firstSubClusterSize, MockPS.initCount.get());
            assertEquals(clusterSize + firstSubClusterSize, MockPS.closeCount.get());
        });
        assertEquals(clusterSize, MockPS.receivedCloseErrors.size());
        MockPS.receivedCloseErrors.forEach(t -> assertTrue("received " + t, t instanceof CancellationException));
        try {
            minorityJobFutureRef[0].get();
            fail();
        } catch (CancellationException expected) {
        } catch (Exception e) {
            throw new AssertionError(e);
        }
    };
    testSplitBrain(firstSubClusterSize, secondSubClusterSize, beforeSplit, onSplit, afterMerge);
}
Also used : MasterContext(com.hazelcast.jet.impl.MasterContext) NOT_RUNNING(com.hazelcast.jet.core.JobStatus.NOT_RUNNING) RunWith(org.junit.runner.RunWith) HazelcastSerialClassRunner(com.hazelcast.test.HazelcastSerialClassRunner) ClusterService(com.hazelcast.internal.cluster.ClusterService) Future(java.util.concurrent.Future) STARTING(com.hazelcast.jet.core.JobStatus.STARTING) BiConsumer(java.util.function.BiConsumer) Assert.fail(org.junit.Assert.fail) ExpectedException(org.junit.rules.ExpectedException) Job(com.hazelcast.jet.Job) JobRepository(com.hazelcast.jet.impl.JobRepository) Config(com.hazelcast.config.Config) HazelcastInstance(com.hazelcast.core.HazelcastInstance) MockPS(com.hazelcast.jet.core.TestProcessors.MockPS) NightlyTest(com.hazelcast.test.annotation.NightlyTest) CancellationException(java.util.concurrent.CancellationException) Assert.assertNotNull(org.junit.Assert.assertNotNull) JobConfig(com.hazelcast.jet.config.JobConfig) MAX_BACKUP_COUNT(com.hazelcast.internal.partition.IPartition.MAX_BACKUP_COUNT) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) Category(org.junit.experimental.categories.Category) NoOutputSourceP(com.hazelcast.jet.core.TestProcessors.NoOutputSourceP) TimeUnit(java.util.concurrent.TimeUnit) Consumer(java.util.function.Consumer) CountDownLatch(java.util.concurrent.CountDownLatch) Rule(org.junit.Rule) RUNNING(com.hazelcast.jet.core.JobStatus.RUNNING) JobExecutionRecord(com.hazelcast.jet.impl.JobExecutionRecord) COMPLETED(com.hazelcast.jet.core.JobStatus.COMPLETED) Lists.newArrayList(org.assertj.core.util.Lists.newArrayList) Assert.assertEquals(org.junit.Assert.assertEquals) JetServiceBackend(com.hazelcast.jet.impl.JetServiceBackend) MockPS(com.hazelcast.jet.core.TestProcessors.MockPS) CountDownLatch(java.util.concurrent.CountDownLatch) JobConfig(com.hazelcast.jet.config.JobConfig) ExpectedException(org.junit.rules.ExpectedException) CancellationException(java.util.concurrent.CancellationException) CancellationException(java.util.concurrent.CancellationException) Future(java.util.concurrent.Future) Job(com.hazelcast.jet.Job) MasterContext(com.hazelcast.jet.impl.MasterContext) JetServiceBackend(com.hazelcast.jet.impl.JetServiceBackend) NightlyTest(com.hazelcast.test.annotation.NightlyTest) Test(org.junit.Test)

Example 87 with BiConsumer

use of java.util.function.BiConsumer in project hazelcast by hazelcast.

the class AbstractEventJournalBasicTest method allowReadingWithFutureSeq.

@Test
public void allowReadingWithFutureSeq() throws Exception {
    final EventJournalTestContext<String, Integer, EJ_TYPE> context = createContext();
    final EventJournalInitialSubscriberState state = subscribeToEventJournal(context.dataAdapter, partitionId);
    assertEquals(0, state.getOldestSequence());
    assertEquals(-1, state.getNewestSequence());
    assertEventJournalSize(context.dataAdapter, 0);
    final Integer value = RANDOM.nextInt();
    final CountDownLatch latch = new CountDownLatch(1);
    final int startSequence = 1;
    final BiConsumer<ReadResultSet<EJ_TYPE>, Throwable> callback = (response, t) -> {
        if (t == null) {
            latch.countDown();
            assertEquals(1, response.size());
            final EventJournalEventAdapter<String, Integer, EJ_TYPE> journalAdapter = context.eventJournalAdapter;
            final EJ_TYPE e = response.get(0);
            assertEquals(ADDED, journalAdapter.getType(e));
            assertEquals(value, journalAdapter.getNewValue(e));
        } else {
            rethrow(t);
        }
    };
    CompletionStage<ReadResultSet<EJ_TYPE>> callbackStage = readFromEventJournal(context.dataAdapter, startSequence, 1, partitionId, TRUE_PREDICATE, IDENTITY_FUNCTION).whenCompleteAsync(callback);
    assertTrueEventually(() -> {
        context.dataAdapter.put(randomPartitionKey(), value);
        assertTrue(latch.await(200, TimeUnit.MILLISECONDS));
    }, 30);
    // ensure no exception thrown from callback
    callbackStage.toCompletableFuture().join();
}
Also used : ADDED(com.hazelcast.journal.EventJournalEventAdapter.EventType.ADDED) EventType(com.hazelcast.journal.EventJournalEventAdapter.EventType) EventJournalConfig(com.hazelcast.config.EventJournalConfig) HashMap(java.util.HashMap) Random(java.util.Random) Function(java.util.function.Function) ExceptionUtil.rethrow(com.hazelcast.internal.util.ExceptionUtil.rethrow) MapUtil.createHashMap(com.hazelcast.internal.util.MapUtil.createHashMap) LOADED(com.hazelcast.journal.EventJournalEventAdapter.EventType.LOADED) ArrayList(java.util.ArrayList) ConcurrentMap(java.util.concurrent.ConcurrentMap) Map(java.util.Map) Accessors.getNode(com.hazelcast.test.Accessors.getNode) BiConsumer(java.util.function.BiConsumer) RingbufferService(com.hazelcast.ringbuffer.impl.RingbufferService) Before(org.junit.Before) Config(com.hazelcast.config.Config) HazelcastInstance(com.hazelcast.core.HazelcastInstance) NodeEngineImpl(com.hazelcast.spi.impl.NodeEngineImpl) RingbufferContainer(com.hazelcast.ringbuffer.impl.RingbufferContainer) ObjectNamespace(com.hazelcast.internal.services.ObjectNamespace) Predicate(java.util.function.Predicate) Assert.assertNotNull(org.junit.Assert.assertNotNull) SetUtil(com.hazelcast.internal.util.SetUtil) HazelcastTestSupport(com.hazelcast.test.HazelcastTestSupport) Set(java.util.Set) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) Node(com.hazelcast.instance.impl.Node) Assert.assertNotEquals(org.junit.Assert.assertNotEquals) ClusterProperty(com.hazelcast.spi.properties.ClusterProperty) TimeUnit(java.util.concurrent.TimeUnit) CountDownLatch(java.util.concurrent.CountDownLatch) CompletionStage(java.util.concurrent.CompletionStage) Assert.assertNull(org.junit.Assert.assertNull) Entry(java.util.Map.Entry) AssertTask(com.hazelcast.test.AssertTask) ReadResultSet(com.hazelcast.ringbuffer.ReadResultSet) EventJournalInitialSubscriberState(com.hazelcast.internal.journal.EventJournalInitialSubscriberState) EVICTED(com.hazelcast.journal.EventJournalEventAdapter.EventType.EVICTED) Assert.assertEquals(org.junit.Assert.assertEquals) ReadResultSet(com.hazelcast.ringbuffer.ReadResultSet) CountDownLatch(java.util.concurrent.CountDownLatch) EventJournalInitialSubscriberState(com.hazelcast.internal.journal.EventJournalInitialSubscriberState) Test(org.junit.Test)

Example 88 with BiConsumer

use of java.util.function.BiConsumer in project hazelcast by hazelcast.

the class AbstractCompletableFutureTest method andThen_whenResultAvailable.

@Test
public void andThen_whenResultAvailable() throws Exception {
    TestFutureImpl future = new TestFutureImpl();
    final Object result = "result";
    final BiConsumer callback = mock(BiConsumer.class);
    future.complete(result);
    future.whenCompleteAsync(callback, executor);
    assertSame(result, future.get());
    assertTrueEventually(new AssertTask() {

        @Override
        public void run() {
            verify(callback).accept(result, null);
        }
    });
}
Also used : AssertTask(com.hazelcast.test.AssertTask) BiConsumer(java.util.function.BiConsumer) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 89 with BiConsumer

use of java.util.function.BiConsumer in project hazelcast by hazelcast.

the class AbstractCompletableFutureTest method andThen_whenCancelled.

@Test
public void andThen_whenCancelled() {
    TestFutureImpl future = new TestFutureImpl();
    BiConsumer callback = mock(BiConsumer.class);
    future.cancel(false);
    future.whenCompleteAsync(callback, executor);
    verify(callback).accept(isNull(), isA(CancellationException.class));
}
Also used : CancellationException(java.util.concurrent.CancellationException) BiConsumer(java.util.function.BiConsumer) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 90 with BiConsumer

use of java.util.function.BiConsumer in project DataX by alibaba.

the class DefaultGdbMapper method forStrColumn.

private static Function<Record, String> forStrColumn(final boolean numPattern, final String rule) {
    final List<BiConsumer<StringBuilder, Record>> list = new ArrayList<>();
    final Pattern pattern = numPattern ? STR_NUM_PATTERN : STR_DOLLAR_PATTERN;
    final Matcher m = pattern.matcher(rule);
    int last = 0;
    while (m.find()) {
        final String index = m.group(1);
        // as simple integer index.
        final int i = Integer.parseInt(index);
        final int tmp = last;
        final int start = m.start();
        list.add((sb, record) -> {
            sb.append(rule.subSequence(tmp, start));
            if (record.getColumn(i) != null && record.getColumn(i).getByteSize() > 0) {
                sb.append(record.getColumn(i).asString());
            }
        });
        last = m.end();
    }
    final int tmp = last;
    list.add((sb, record) -> {
        sb.append(rule.subSequence(tmp, rule.length()));
    });
    return r -> {
        final StringBuilder sb = new StringBuilder();
        list.forEach(c -> c.accept(sb, r));
        final String res = sb.toString();
        return res.isEmpty() ? null : res;
    };
}
Also used : VERTEX(com.alibaba.datax.plugin.writer.gdbwriter.Key.ImportType.VERTEX) UUID(java.util.UUID) Function(java.util.function.Function) ArrayList(java.util.ArrayList) Configuration(com.alibaba.datax.common.util.Configuration) Key(com.alibaba.datax.plugin.writer.gdbwriter.Key) JSONArray(com.alibaba.fastjson.JSONArray) List(java.util.List) Slf4j(lombok.extern.slf4j.Slf4j) Matcher(java.util.regex.Matcher) Record(com.alibaba.datax.common.element.Record) GdbElement(com.alibaba.datax.plugin.writer.gdbwriter.model.GdbElement) GdbEdge(com.alibaba.datax.plugin.writer.gdbwriter.model.GdbEdge) BiConsumer(java.util.function.BiConsumer) GdbVertex(com.alibaba.datax.plugin.writer.gdbwriter.model.GdbVertex) JSONObject(com.alibaba.fastjson.JSONObject) Pattern(java.util.regex.Pattern) Pattern(java.util.regex.Pattern) Matcher(java.util.regex.Matcher) ArrayList(java.util.ArrayList) BiConsumer(java.util.function.BiConsumer)

Aggregations

BiConsumer (java.util.function.BiConsumer)255 Test (org.junit.Test)110 List (java.util.List)106 Map (java.util.Map)77 IOException (java.io.IOException)75 Consumer (java.util.function.Consumer)69 ArrayList (java.util.ArrayList)68 HashMap (java.util.HashMap)64 Collectors (java.util.stream.Collectors)53 CountDownLatch (java.util.concurrent.CountDownLatch)52 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)50 Collections (java.util.Collections)46 Set (java.util.Set)46 Collection (java.util.Collection)45 Arrays (java.util.Arrays)44 TimeUnit (java.util.concurrent.TimeUnit)43 Assert (org.junit.Assert)43 Function (java.util.function.Function)41 Optional (java.util.Optional)40 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)35