Search in sources :

Example 1 with PaxosContainer

use of com.palantir.atlasdb.timelock.paxos.PaxosQuorumCheckingCoalescingFunction.PaxosContainer in project atlasdb by palantir.

the class GetSuspectedLeaderWithUuid method accept.

@Override
public void accept(List<BatchElement<UUID, Optional<ClientAwareLeaderPinger>>> batchElements) {
    Multimap<UUID, DisruptorFuture<Optional<ClientAwareLeaderPinger>>> uuidsToRequests = batchElements.stream().collect(ImmutableListMultimap.toImmutableListMultimap(BatchElement::argument, BatchElement::result));
    KeyedStream.of(uuidsToRequests.keySet()).filterKeys(cache::containsKey).map(cache::get).forEach((cachedUuid, pingable) -> completeRequest(uuidsToRequests.get(cachedUuid), Optional.of(clientAwareLeaders.get(pingable))));
    Set<UUID> uncachedUuids = uuidsToRequests.keySet().stream().filter(uuid -> !cache.containsKey(uuid)).collect(toSet());
    if (uncachedUuids.isEmpty()) {
        return;
    }
    PaxosResponsesWithRemote<LeaderPingerContext<BatchPingableLeader>, PaxosContainer<UUID>> results = PaxosQuorumChecker.collectUntil(ImmutableList.copyOf(executors.keySet()), pingable -> PaxosContainer.of(pingable.pinger().uuid()), executors, leaderPingResponseWait, state -> state.responses().values().stream().map(PaxosContainer::get).collect(toSet()).containsAll(uncachedUuids), PaxosConstants.CANCEL_REMAINING_CALLS);
    for (Map.Entry<LeaderPingerContext<BatchPingableLeader>, PaxosContainer<UUID>> resultEntries : results.responses().entrySet()) {
        LeaderPingerContext<BatchPingableLeader> pingable = resultEntries.getKey();
        UUID uuid = resultEntries.getValue().get();
        LeaderPingerContext<BatchPingableLeader> oldCachedEntry = cache.putIfAbsent(uuid, pingable);
        throwIfInvalidSetup(oldCachedEntry, pingable, uuid);
        completeRequest(uuidsToRequests.get(uuid), Optional.of(clientAwareLeaders.get(pingable)));
    }
    Set<UUID> missingUuids = Sets.difference(uncachedUuids, results.withoutRemotes().stream().map(PaxosContainer::get).collect(toSet()));
    missingUuids.forEach(missingUuid -> completeRequest(uuidsToRequests.get(missingUuid), Optional.empty()));
}
Also used : DisruptorFuture(com.palantir.atlasdb.autobatch.DisruptorAutobatcher.DisruptorFuture) SafeLoggerFactory(com.palantir.logsafe.logger.SafeLoggerFactory) Throwables(com.palantir.common.base.Throwables) SafeIllegalStateException(com.palantir.logsafe.exceptions.SafeIllegalStateException) PaxosContainer(com.palantir.atlasdb.timelock.paxos.PaxosQuorumCheckingCoalescingFunction.PaxosContainer) HashMap(java.util.HashMap) Multimap(com.google.common.collect.Multimap) LeaderPingerContext(com.palantir.paxos.LeaderPingerContext) BatchElement(com.palantir.atlasdb.autobatch.BatchElement) SafeLogger(com.palantir.logsafe.logger.SafeLogger) ImmutableList(com.google.common.collect.ImmutableList) Duration(java.time.Duration) Map(java.util.Map) PaxosQuorumChecker(com.palantir.paxos.PaxosQuorumChecker) Collectors.toSet(java.util.stream.Collectors.toSet) BiMap(com.google.common.collect.BiMap) KeyedStream(com.palantir.common.streams.KeyedStream) CheckedRejectionExecutorService(com.palantir.common.concurrent.CheckedRejectionExecutorService) Collection(java.util.Collection) Set(java.util.Set) UUID(java.util.UUID) Sets(com.google.common.collect.Sets) Consumer(java.util.function.Consumer) HashBiMap(com.google.common.collect.HashBiMap) List(java.util.List) PaxosResponsesWithRemote(com.palantir.paxos.PaxosResponsesWithRemote) ImmutableListMultimap(com.google.common.collect.ImmutableListMultimap) Optional(java.util.Optional) PaxosConstants(com.palantir.paxos.PaxosConstants) NotThreadSafe(javax.annotation.concurrent.NotThreadSafe) LeaderPingerContext(com.palantir.paxos.LeaderPingerContext) PaxosContainer(com.palantir.atlasdb.timelock.paxos.PaxosQuorumCheckingCoalescingFunction.PaxosContainer) UUID(java.util.UUID) HashMap(java.util.HashMap) Map(java.util.Map) BiMap(com.google.common.collect.BiMap) HashBiMap(com.google.common.collect.HashBiMap) DisruptorFuture(com.palantir.atlasdb.autobatch.DisruptorAutobatcher.DisruptorFuture)

Example 2 with PaxosContainer

use of com.palantir.atlasdb.timelock.paxos.PaxosQuorumCheckingCoalescingFunction.PaxosContainer in project atlasdb by palantir.

the class LearnedValuesCoalescingFunctionTests method canProcessBatch.

@Test
public void canProcessBatch() {
    Set<WithSeq<Client>> remoteRequest = ImmutableSet.of(WithSeq.of(CLIENT_1, 10), WithSeq.of(CLIENT_1, 12), WithSeq.of(CLIENT_1, 15), WithSeq.of(CLIENT_2, 10), WithSeq.of(CLIENT_2, 15));
    PaxosValue paxosValue1 = paxosValue(10);
    PaxosValue paxosValue2 = paxosValue(12);
    SetMultimap<Client, PaxosValue> remoteResponse = ImmutableSetMultimap.<Client, PaxosValue>builder().putAll(CLIENT_1, paxosValue1, paxosValue2).put(CLIENT_2, paxosValue1).build();
    when(remote.getLearnedValues(remoteRequest)).thenReturn(remoteResponse);
    LearnedValuesCoalescingFunction function = new LearnedValuesCoalescingFunction(remote);
    Map<WithSeq<Client>, PaxosContainer<Optional<PaxosValue>>> results = function.apply(remoteRequest);
    assertThat(results).containsOnly(entry(WithSeq.of(CLIENT_2, 10), asResult(paxosValue1)), entry(WithSeq.of(CLIENT_1, 12), asResult(paxosValue2)), entry(WithSeq.of(CLIENT_1, 10), asResult(paxosValue1)), entry(WithSeq.of(CLIENT_1, 15), EMPTY_RESULT), entry(WithSeq.of(CLIENT_2, 15), EMPTY_RESULT));
}
Also used : PaxosContainer(com.palantir.atlasdb.timelock.paxos.PaxosQuorumCheckingCoalescingFunction.PaxosContainer) Client(com.palantir.paxos.Client) PaxosValue(com.palantir.paxos.PaxosValue) Test(org.junit.Test)

Example 3 with PaxosContainer

use of com.palantir.atlasdb.timelock.paxos.PaxosQuorumCheckingCoalescingFunction.PaxosContainer in project atlasdb by palantir.

the class TestableTimelockCluster method currentLeaders.

SetMultimap<String, TestableTimelockServer> currentLeaders(Iterable<String> namespaces) {
    Set<String> namespacesIterable = ImmutableSet.copyOf(namespaces);
    KeyedStream<TestableTimelockServer, PaxosContainer<Set<String>>> responses = PaxosQuorumChecker.collectUntil(ImmutableList.copyOf(servers), server -> PaxosContainer.of(server.pinger().ping(namespaces)), Maps.toMap(servers, unused -> new CheckedRejectionExecutorService(executorService)), Duration.ofSeconds(2), untilAllNamespacesAreSeen(namespacesIterable), true).stream();
    return responses.filter(PaxosContainer::isSuccessful).map(PaxosContainer::get).flatMap(Collection::stream).mapEntries((server, namespace) -> Maps.immutableEntry(namespace, server)).collectToSetMultimap();
}
Also used : Statement(org.junit.runners.model.Statement) Iterables(com.google.common.collect.Iterables) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) TestRule(org.junit.rules.TestRule) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) PaxosContainer(com.palantir.atlasdb.timelock.paxos.PaxosQuorumCheckingCoalescingFunction.PaxosContainer) Hashing(com.google.common.hash.Hashing) Random(java.util.Random) Multimap(com.google.common.collect.Multimap) ProxyMode(com.palantir.atlasdb.timelock.util.TestProxies.ProxyMode) ImmutableList(com.google.common.collect.ImmutableList) Duration(java.time.Duration) Map(java.util.Map) StreamSupport(java.util.stream.StreamSupport) PaxosQuorumChecker(com.palantir.paxos.PaxosQuorumChecker) NoSuchElementException(java.util.NoSuchElementException) ExecutorService(java.util.concurrent.ExecutorService) ImmutableSet(com.google.common.collect.ImmutableSet) KeyedStream(com.palantir.common.streams.KeyedStream) Predicate(java.util.function.Predicate) CheckedRejectionExecutorService(com.palantir.common.concurrent.CheckedRejectionExecutorService) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) TestProxies(com.palantir.atlasdb.timelock.util.TestProxies) Set(java.util.Set) Description(org.junit.runner.Description) UUID(java.util.UUID) Maps(com.google.common.collect.Maps) Collectors(java.util.stream.Collectors) SetMultimap(com.google.common.collect.SetMultimap) Sets(com.google.common.collect.Sets) Executors(java.util.concurrent.Executors) InProgressResponseState(com.palantir.paxos.InProgressResponseState) ExecutionException(java.util.concurrent.ExecutionException) RuleChain(org.junit.rules.RuleChain) Futures(com.google.common.util.concurrent.Futures) List(java.util.List) Awaitility(org.awaitility.Awaitility) TemporaryFolder(org.junit.rules.TemporaryFolder) PaxosContainer(com.palantir.atlasdb.timelock.paxos.PaxosQuorumCheckingCoalescingFunction.PaxosContainer) CheckedRejectionExecutorService(com.palantir.common.concurrent.CheckedRejectionExecutorService) Collection(java.util.Collection)

Aggregations

PaxosContainer (com.palantir.atlasdb.timelock.paxos.PaxosQuorumCheckingCoalescingFunction.PaxosContainer)3 ImmutableList (com.google.common.collect.ImmutableList)2 Multimap (com.google.common.collect.Multimap)2 Sets (com.google.common.collect.Sets)2 CheckedRejectionExecutorService (com.palantir.common.concurrent.CheckedRejectionExecutorService)2 KeyedStream (com.palantir.common.streams.KeyedStream)2 PaxosQuorumChecker (com.palantir.paxos.PaxosQuorumChecker)2 Duration (java.time.Duration)2 Collection (java.util.Collection)2 List (java.util.List)2 Map (java.util.Map)2 Set (java.util.Set)2 UUID (java.util.UUID)2 BiMap (com.google.common.collect.BiMap)1 HashBiMap (com.google.common.collect.HashBiMap)1 ImmutableListMultimap (com.google.common.collect.ImmutableListMultimap)1 ImmutableSet (com.google.common.collect.ImmutableSet)1 Iterables (com.google.common.collect.Iterables)1 Maps (com.google.common.collect.Maps)1 SetMultimap (com.google.common.collect.SetMultimap)1