Search in sources :

Example 1 with LeadershipCoordinator

use of com.palantir.leader.proxy.LeadershipCoordinator in project atlasdb by palantir.

the class DefaultLockAndTimestampServiceFactory method createRawLeaderServices.

private static LockAndTimestampServices createRawLeaderServices(MetricsManager metricsManager, LeaderConfig leaderConfig, Consumer<Object> env, Supplier<LockService> lock, Supplier<ManagedTimestampService> time, UserAgent userAgent) {
    // Create local services, that may or may not end up being registered in an Consumer<Object>.
    LocalPaxosServices localPaxosServices = Leaders.createAndRegisterLocalServices(metricsManager, env, leaderConfig, userAgent);
    LeadershipCoordinator leadershipCoordinator = localPaxosServices.leadershipCoordinator();
    LockService localLock = AwaitingLeadershipProxy.newProxyInstance(LockService.class, lock, leadershipCoordinator);
    ManagedTimestampService managedTimestampProxy = AwaitingLeadershipProxy.newProxyInstance(ManagedTimestampService.class, time, leadershipCoordinator);
    // These facades are necessary because of the semantics of the JAX-RS algorithm (in particular, accepting
    // just the managed timestamp service will *not* work).
    TimestampService localTime = getTimestampFacade(managedTimestampProxy);
    TimestampManagementService localManagement = getTimestampManagementFacade(managedTimestampProxy);
    env.accept(localLock);
    env.accept(localTime);
    env.accept(localManagement);
    // Create remote services, that may end up calling our own local services.
    ImmutableServerListConfig serverListConfig = ImmutableServerListConfig.builder().servers(leaderConfig.leaders()).sslConfiguration(leaderConfig.sslConfiguration()).build();
    ServiceCreator creator = ServiceCreator.noPayloadLimiter(metricsManager, Refreshable.only(serverListConfig), userAgent, () -> RemotingClientConfigs.DEFAULT);
    LockService remoteLock = new RemoteLockServiceAdapter(creator.createService(NamespaceAgnosticLockRpcClient.class));
    TimestampService remoteTime = creator.createService(TimestampService.class);
    TimestampManagementService remoteManagement = creator.createService(TimestampManagementService.class);
    if (leaderConfig.leaders().size() == 1) {
        // Attempting to connect to ourself while processing a request can lead to deadlock if incoming request
        // volume is high, as all Jetty threads end up waiting for the timestamp server, and no threads remain to
        // actually handle the timestamp server requests. If we are the only single leader, we can avoid the
        // deadlock entirely; so use PingableLeader's getUUID() to detect this situation and eliminate the redundant
        // call.
        PingableLeader localPingableLeader = localPaxosServices.localPingableLeader();
        String localServerId = localPingableLeader.getUUID();
        PingableLeader remotePingableLeader = AtlasDbHttpClients.createProxy(ServiceCreator.createTrustContext(leaderConfig.sslConfiguration()), Iterables.getOnlyElement(leaderConfig.leaders()), PingableLeader.class, AuxiliaryRemotingParameters.builder().userAgent(userAgent).shouldRetry(true).shouldLimitPayload(true).shouldUseExtendedTimeout(false).build());
        // Determine asynchronously whether the remote services are talking to our local services.
        CompletableFuture<Boolean> useLocalServicesFuture = new CompletableFuture<>();
        TransactionManagers.runAsync.accept(() -> {
            int attemptsLeftBeforeLog = ATTEMPTS_BEFORE_LOGGING_FAILURE_TO_READ_REMOTE_TIMESTAMP_SERVER_ID;
            while (true) {
                try {
                    String remoteServerId = remotePingableLeader.getUUID();
                    useLocalServicesFuture.complete(localServerId.equals(remoteServerId));
                    return;
                } catch (ClientErrorException e) {
                    useLocalServicesFuture.complete(false);
                    return;
                } catch (UnknownRemoteException e) {
                    // manifest as ClientErrorExceptions.
                    if (400 <= e.getStatus() && e.getStatus() <= 499) {
                        useLocalServicesFuture.complete(false);
                        return;
                    }
                    attemptsLeftBeforeLog = logFailureToReadRemoteTimestampServerId(attemptsLeftBeforeLog, e);
                } catch (Throwable e) {
                    attemptsLeftBeforeLog = logFailureToReadRemoteTimestampServerId(attemptsLeftBeforeLog, e);
                }
                Uninterruptibles.sleepUninterruptibly(Duration.ofSeconds(1));
            }
        });
        // Create dynamic service proxies, that switch to talking directly to our local services if it turns out our
        // remote services are pointed at them anyway.
        LockService dynamicLockService = LocalOrRemoteProxy.newProxyInstance(LockService.class, localLock, remoteLock, useLocalServicesFuture);
        // Use managedTimestampProxy here to avoid local calls going through indirection.
        TimestampService dynamicTimeService = LocalOrRemoteProxy.newProxyInstance(TimestampService.class, managedTimestampProxy, remoteTime, useLocalServicesFuture);
        TimestampManagementService dynamicManagementService = LocalOrRemoteProxy.newProxyInstance(TimestampManagementService.class, managedTimestampProxy, remoteManagement, useLocalServicesFuture);
        return ImmutableLockAndTimestampServices.builder().lock(dynamicLockService).timestamp(dynamicTimeService).timestampManagement(dynamicManagementService).timelock(new LegacyTimelockService(dynamicTimeService, dynamicLockService, TransactionManagers.LOCK_CLIENT)).build();
    } else {
        return ImmutableLockAndTimestampServices.builder().lock(remoteLock).timestamp(remoteTime).timestampManagement(remoteManagement).timelock(new LegacyTimelockService(remoteTime, remoteLock, TransactionManagers.LOCK_CLIENT)).build();
    }
}
Also used : LegacyTimelockService(com.palantir.lock.impl.LegacyTimelockService) ImmutableServerListConfig(com.palantir.atlasdb.config.ImmutableServerListConfig) RemoteLockServiceAdapter(com.palantir.lock.client.RemoteLockServiceAdapter) LockRefreshingLockService(com.palantir.lock.client.LockRefreshingLockService) LockService(com.palantir.lock.LockService) ManagedTimestampService(com.palantir.timestamp.ManagedTimestampService) UnknownRemoteException(com.palantir.conjure.java.api.errors.UnknownRemoteException) LeadershipCoordinator(com.palantir.leader.proxy.LeadershipCoordinator) LocalPaxosServices(com.palantir.atlasdb.factory.Leaders.LocalPaxosServices) TimestampManagementService(com.palantir.timestamp.TimestampManagementService) CompletableFuture(java.util.concurrent.CompletableFuture) NamespaceAgnosticLockRpcClient(com.palantir.lock.NamespaceAgnosticLockRpcClient) PingableLeader(com.palantir.leader.PingableLeader) ClientErrorException(javax.ws.rs.ClientErrorException) TimestampService(com.palantir.timestamp.TimestampService) ManagedTimestampService(com.palantir.timestamp.ManagedTimestampService)

Example 2 with LeadershipCoordinator

use of com.palantir.leader.proxy.LeadershipCoordinator in project atlasdb by palantir.

the class Leaders method createInstrumentedLocalServices.

public static LocalPaxosServices createInstrumentedLocalServices(MetricsManager metricsManager, LeaderConfig config, RemotePaxosServerSpec remotePaxosServerSpec, Supplier<RemotingClientConfig> remotingClientConfig, UserAgent userAgent, LeadershipObserver leadershipObserver) {
    UUID leaderUuid = UUID.randomUUID();
    PaxosLeadershipEventRecorder leadershipEventRecorder = PaxosLeadershipEventRecorder.create(metricsManager.getTaggedRegistry(), leaderUuid.toString(), leadershipObserver, ImmutableList.of());
    PaxosAcceptor ourAcceptor = AtlasDbMetrics.instrumentTimed(metricsManager.getRegistry(), PaxosAcceptor.class, PaxosAcceptorImpl.newAcceptor(config.acceptorLogDir().getPath()));
    PaxosLearner ourLearner = AtlasDbMetrics.instrumentTimed(metricsManager.getRegistry(), PaxosLearner.class, PaxosLearnerImpl.newLearner(config.learnerLogDir().getPath(), leadershipEventRecorder));
    Optional<TrustContext> trustContext = ServiceCreator.createTrustContext(config.sslConfiguration());
    List<PaxosLearner> learners = createProxyAndLocalList(ourLearner, remotePaxosServerSpec.remoteLearnerUris(), remotingClientConfig, trustContext, PaxosLearner.class, userAgent);
    List<PaxosLearner> remoteLearners = learners.stream().filter(learner -> !learner.equals(ourLearner)).collect(ImmutableList.toImmutableList());
    PaxosLearnerNetworkClient learnerNetworkClient = SingleLeaderLearnerNetworkClient.createLegacy(ourLearner, remoteLearners, config.quorumSize(), createExecutorsForService(metricsManager, learners, "knowledge-update"), PaxosConstants.CANCEL_REMAINING_CALLS);
    List<PaxosAcceptor> acceptors = createProxyAndLocalList(ourAcceptor, remotePaxosServerSpec.remoteAcceptorUris(), remotingClientConfig, trustContext, PaxosAcceptor.class, userAgent);
    PaxosAcceptorNetworkClient acceptorNetworkClient = SingleLeaderAcceptorNetworkClient.createLegacy(acceptors, config.quorumSize(), createExecutorsForService(metricsManager, acceptors, "latest-round-verifier"), PaxosConstants.CANCEL_REMAINING_CALLS);
    List<LeaderPingerContext<PingableLeader>> otherLeaders = generatePingables(remotePaxosServerSpec.remoteLeaderUris(), remotingClientConfig, trustContext, userAgent);
    LeaderPinger leaderPinger = SingleLeaderPinger.createLegacy(createExecutorsForService(metricsManager, otherLeaders, "leader-ping"), config.leaderPingResponseWait(), leaderUuid, PaxosConstants.CANCEL_REMAINING_CALLS);
    LeaderElectionService uninstrumentedLeaderElectionService = new LeaderElectionServiceBuilder().leaderUuid(leaderUuid).knowledge(ourLearner).eventRecorder(leadershipEventRecorder).randomWaitBeforeProposingLeadership(config.randomWaitBeforeProposingLeadership()).pingRate(config.pingRate()).leaderPinger(leaderPinger).acceptorClient(acceptorNetworkClient).learnerClient(learnerNetworkClient).decorateProposer(proposer -> AtlasDbMetrics.instrumentTimed(metricsManager.getRegistry(), PaxosProposer.class, proposer)).leaderAddressCacheTtl(config.leaderAddressCacheTtl()).build();
    LeaderElectionService leaderElectionService = AtlasDbMetrics.instrumentTimed(metricsManager.getRegistry(), LeaderElectionService.class, uninstrumentedLeaderElectionService);
    PingableLeader pingableLeader = AtlasDbMetrics.instrumentTimed(metricsManager.getRegistry(), PingableLeader.class, new LocalPingableLeader(ourLearner, leaderUuid));
    List<PingableLeader> remotePingableLeaders = otherLeaders.stream().map(LeaderPingerContext::pinger).collect(Collectors.toList());
    BatchingLeaderElectionService batchingLeaderElectionService = new BatchingLeaderElectionService(leaderElectionService);
    return ImmutableLocalPaxosServices.builder().ourAcceptor(ourAcceptor).ourLearner(ourLearner).leaderElectionService(batchingLeaderElectionService).leadershipCoordinator(LeadershipCoordinator.create(batchingLeaderElectionService)).localPingableLeader(pingableLeader).remotePingableLeaders(remotePingableLeaders).build();
}
Also used : PaxosLearnerImpl(com.palantir.paxos.PaxosLearnerImpl) LeaderConfig(com.palantir.atlasdb.config.LeaderConfig) PaxosLearner(com.palantir.paxos.PaxosLearner) BatchingLeaderElectionService(com.palantir.leader.BatchingLeaderElectionService) TrustContext(com.palantir.conjure.java.config.ssl.TrustContext) MetricsManager(com.palantir.atlasdb.util.MetricsManager) PTExecutors(com.palantir.common.concurrent.PTExecutors) UserAgent(com.palantir.conjure.java.api.config.service.UserAgent) Map(java.util.Map) NotCurrentLeaderExceptionMapper(com.palantir.atlasdb.http.NotCurrentLeaderExceptionMapper) PaxosAcceptor(com.palantir.paxos.PaxosAcceptor) PaxosLeadershipEventRecorder(com.palantir.leader.PaxosLeadershipEventRecorder) URI(java.net.URI) SingleLeaderLearnerNetworkClient(com.palantir.paxos.SingleLeaderLearnerNetworkClient) LeaderElectionService(com.palantir.leader.LeaderElectionService) LeaderElectionServiceBuilder(com.palantir.leader.LeaderElectionServiceBuilder) LeaderPinger(com.palantir.paxos.LeaderPinger) ImmutableSet(com.google.common.collect.ImmutableSet) SingleLeaderPinger(com.palantir.paxos.SingleLeaderPinger) KeyedStream(com.palantir.common.streams.KeyedStream) Collection(java.util.Collection) AtlasDbHttpClients(com.palantir.atlasdb.http.AtlasDbHttpClients) ImmutableLeaderPingerContext(com.palantir.paxos.ImmutableLeaderPingerContext) Set(java.util.Set) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) Sets(com.google.common.collect.Sets) List(java.util.List) Optional(java.util.Optional) RemotingClientConfigs(com.palantir.atlasdb.config.RemotingClientConfigs) LeadershipCoordinator(com.palantir.leader.proxy.LeadershipCoordinator) Iterables(com.google.common.collect.Iterables) AtlasDbMetrics(com.palantir.atlasdb.util.AtlasDbMetrics) LocalPingableLeader(com.palantir.leader.LocalPingableLeader) HashMap(java.util.HashMap) LeaderPingerContext(com.palantir.paxos.LeaderPingerContext) PaxosLearnerNetworkClient(com.palantir.paxos.PaxosLearnerNetworkClient) Supplier(java.util.function.Supplier) HashSet(java.util.HashSet) ImmutableList(com.google.common.collect.ImmutableList) Value(org.immutables.value.Value) AuxiliaryRemotingParameters(com.palantir.atlasdb.config.AuxiliaryRemotingParameters) ExecutorService(java.util.concurrent.ExecutorService) HostAndPort(com.google.common.net.HostAndPort) SingleLeaderAcceptorNetworkClient(com.palantir.paxos.SingleLeaderAcceptorNetworkClient) Consumer(java.util.function.Consumer) PaxosAcceptorImpl(com.palantir.paxos.PaxosAcceptorImpl) PaxosProposer(com.palantir.paxos.PaxosProposer) PaxosAcceptorNetworkClient(com.palantir.paxos.PaxosAcceptorNetworkClient) RemotingClientConfig(com.palantir.atlasdb.config.RemotingClientConfig) LeadershipObserver(com.palantir.leader.LeadershipObserver) PaxosConstants(com.palantir.paxos.PaxosConstants) PingableLeader(com.palantir.leader.PingableLeader) PaxosAcceptor(com.palantir.paxos.PaxosAcceptor) ImmutableLeaderPingerContext(com.palantir.paxos.ImmutableLeaderPingerContext) LeaderPingerContext(com.palantir.paxos.LeaderPingerContext) PaxosAcceptorNetworkClient(com.palantir.paxos.PaxosAcceptorNetworkClient) PaxosProposer(com.palantir.paxos.PaxosProposer) LeaderPinger(com.palantir.paxos.LeaderPinger) SingleLeaderPinger(com.palantir.paxos.SingleLeaderPinger) LocalPingableLeader(com.palantir.leader.LocalPingableLeader) PaxosLearner(com.palantir.paxos.PaxosLearner) PaxosLearnerNetworkClient(com.palantir.paxos.PaxosLearnerNetworkClient) LeaderElectionServiceBuilder(com.palantir.leader.LeaderElectionServiceBuilder) BatchingLeaderElectionService(com.palantir.leader.BatchingLeaderElectionService) LocalPingableLeader(com.palantir.leader.LocalPingableLeader) PingableLeader(com.palantir.leader.PingableLeader) BatchingLeaderElectionService(com.palantir.leader.BatchingLeaderElectionService) LeaderElectionService(com.palantir.leader.LeaderElectionService) PaxosLeadershipEventRecorder(com.palantir.leader.PaxosLeadershipEventRecorder) UUID(java.util.UUID) TrustContext(com.palantir.conjure.java.config.ssl.TrustContext)

Example 3 with LeadershipCoordinator

use of com.palantir.leader.proxy.LeadershipCoordinator in project atlasdb by palantir.

the class LeadershipContextFactory method create.

@Override
public LeadershipContext create(Client client) {
    ClientAwareComponents clientAwareComponents = ImmutableClientAwareComponents.builder().from(this).proxyClient(client).build();
    BatchingLeaderElectionService leaderElectionService = leaderElectionServiceFactory().create(clientAwareComponents);
    LeadershipCoordinator leadershipCoordinator = leadershipCoordinatorFactory().create(leaderElectionService);
    return ImmutableLeadershipContext.builder().leadershipMetrics(clientAwareComponents.leadershipMetrics()).leaderElectionService(leaderElectionService).addCloseables(leaderElectionService).leadershipCoordinator(leadershipCoordinator).addCloseables(leadershipCoordinatorFactory()).addAllCloseables(leaderPingerFactory().closeables()).build();
}
Also used : BatchingLeaderElectionService(com.palantir.leader.BatchingLeaderElectionService) LeadershipCoordinator(com.palantir.leader.proxy.LeadershipCoordinator)

Aggregations

LeadershipCoordinator (com.palantir.leader.proxy.LeadershipCoordinator)3 BatchingLeaderElectionService (com.palantir.leader.BatchingLeaderElectionService)2 PingableLeader (com.palantir.leader.PingableLeader)2 ImmutableList (com.google.common.collect.ImmutableList)1 ImmutableSet (com.google.common.collect.ImmutableSet)1 Iterables (com.google.common.collect.Iterables)1 Sets (com.google.common.collect.Sets)1 HostAndPort (com.google.common.net.HostAndPort)1 AuxiliaryRemotingParameters (com.palantir.atlasdb.config.AuxiliaryRemotingParameters)1 ImmutableServerListConfig (com.palantir.atlasdb.config.ImmutableServerListConfig)1 LeaderConfig (com.palantir.atlasdb.config.LeaderConfig)1 RemotingClientConfig (com.palantir.atlasdb.config.RemotingClientConfig)1 RemotingClientConfigs (com.palantir.atlasdb.config.RemotingClientConfigs)1 LocalPaxosServices (com.palantir.atlasdb.factory.Leaders.LocalPaxosServices)1 AtlasDbHttpClients (com.palantir.atlasdb.http.AtlasDbHttpClients)1 NotCurrentLeaderExceptionMapper (com.palantir.atlasdb.http.NotCurrentLeaderExceptionMapper)1 AtlasDbMetrics (com.palantir.atlasdb.util.AtlasDbMetrics)1 MetricsManager (com.palantir.atlasdb.util.MetricsManager)1 PTExecutors (com.palantir.common.concurrent.PTExecutors)1 KeyedStream (com.palantir.common.streams.KeyedStream)1