Search in sources :

Example 6 with CassandraClientPoolingContainer

use of com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer in project atlasdb by palantir.

the class CassandraService method addPool.

public void addPool(InetSocketAddress server) {
    int currentPoolNumber = cassandraHosts.indexOf(server) + 1;
    currentPools.put(server, new CassandraClientPoolingContainer(metricsManager, server, config, currentPoolNumber, poolMetrics));
}
Also used : CassandraClientPoolingContainer(com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer)

Example 7 with CassandraClientPoolingContainer

use of com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer in project atlasdb by palantir.

the class WeightedHostsTest method testWeightedHostsWithMaxActivityPool.

@Test
public void testWeightedHostsWithMaxActivityPool() {
    InetSocketAddress highActivityHost = new InetSocketAddress(2);
    Map<InetSocketAddress, CassandraClientPoolingContainer> pools = ImmutableMap.of(new InetSocketAddress(0), createMockClientPoolingContainerWithUtilization(5), new InetSocketAddress(1), createMockClientPoolingContainerWithUtilization(5), highActivityHost, createMockClientPoolingContainerWithUtilization(20));
    NavigableMap<Integer, InetSocketAddress> result = WeightedHosts.create(pools).hosts;
    int smallestWeight = result.firstEntry().getKey();
    InetSocketAddress hostWithSmallestWeight = result.firstEntry().getValue();
    int prevKey = 0;
    for (Map.Entry<Integer, InetSocketAddress> entry : result.entrySet()) {
        int currWeight = entry.getKey() - prevKey;
        prevKey = entry.getKey();
        if (currWeight < smallestWeight) {
            smallestWeight = currWeight;
            hostWithSmallestWeight = entry.getValue();
        }
    }
    assertThat(hostWithSmallestWeight).isEqualTo(highActivityHost);
}
Also used : CassandraClientPoolingContainer(com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer) InetSocketAddress(java.net.InetSocketAddress) ImmutableMap(com.google.common.collect.ImmutableMap) Map(java.util.Map) HashMap(java.util.HashMap) NavigableMap(java.util.NavigableMap) Test(org.junit.Test)

Example 8 with CassandraClientPoolingContainer

use of com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer in project atlasdb by palantir.

the class WeightedHostsTest method testWeightedHostsWithLowActivityPool.

@Test
public void testWeightedHostsWithLowActivityPool() {
    InetSocketAddress lowActivityHost = new InetSocketAddress(2);
    Map<InetSocketAddress, CassandraClientPoolingContainer> pools = ImmutableMap.of(new InetSocketAddress(0), createMockClientPoolingContainerWithUtilization(10), new InetSocketAddress(1), createMockClientPoolingContainerWithUtilization(10), lowActivityHost, createMockClientPoolingContainerWithUtilization(0));
    NavigableMap<Integer, InetSocketAddress> result = WeightedHosts.create(pools).hosts;
    int largestWeight = result.firstEntry().getKey();
    InetSocketAddress hostWithLargestWeight = result.firstEntry().getValue();
    int prevKey = 0;
    for (Map.Entry<Integer, InetSocketAddress> entry : result.entrySet()) {
        int currWeight = entry.getKey() - prevKey;
        prevKey = entry.getKey();
        if (currWeight > largestWeight) {
            largestWeight = currWeight;
            hostWithLargestWeight = entry.getValue();
        }
    }
    assertThat(hostWithLargestWeight).isEqualTo(lowActivityHost);
}
Also used : CassandraClientPoolingContainer(com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer) InetSocketAddress(java.net.InetSocketAddress) ImmutableMap(com.google.common.collect.ImmutableMap) Map(java.util.Map) HashMap(java.util.HashMap) NavigableMap(java.util.NavigableMap) Test(org.junit.Test)

Example 9 with CassandraClientPoolingContainer

use of com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer in project atlasdb by palantir.

the class WeightedHostsTest method testWeightedHostsWithNonZeroWeights.

@Test
public void testWeightedHostsWithNonZeroWeights() {
    Map<InetSocketAddress, CassandraClientPoolingContainer> pools = ImmutableMap.of(new InetSocketAddress(0), createMockClientPoolingContainerWithUtilization(5), new InetSocketAddress(1), createMockClientPoolingContainerWithUtilization(10), new InetSocketAddress(2), createMockClientPoolingContainerWithUtilization(15));
    NavigableMap<Integer, InetSocketAddress> result = WeightedHosts.create(pools).hosts;
    int prevKey = 0;
    for (Map.Entry<Integer, InetSocketAddress> entry : result.entrySet()) {
        assertThat(entry.getKey()).isGreaterThan(prevKey);
        prevKey = entry.getKey();
    }
}
Also used : CassandraClientPoolingContainer(com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer) InetSocketAddress(java.net.InetSocketAddress) ImmutableMap(com.google.common.collect.ImmutableMap) Map(java.util.Map) HashMap(java.util.HashMap) NavigableMap(java.util.NavigableMap) Test(org.junit.Test)

Example 10 with CassandraClientPoolingContainer

use of com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer in project atlasdb by palantir.

the class CassandraService method getRandomGoodHostForPredicate.

public Optional<CassandraClientPoolingContainer> getRandomGoodHostForPredicate(Predicate<CassandraServer> predicate, Set<CassandraServer> triedNodes) {
    Map<CassandraServer, CassandraClientPoolingContainer> pools = currentPools;
    Set<CassandraServer> hostsMatchingPredicate = pools.keySet().stream().filter(predicate).collect(Collectors.toSet());
    Map<String, Long> triedDatacenters = triedNodes.stream().map(hostToDatacenter::get).filter(Objects::nonNull).collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));
    Optional<Long> maximumAttemptsPerDatacenter = triedDatacenters.values().stream().max(Long::compareTo);
    Set<String> maximallyAttemptedDatacenters = KeyedStream.stream(triedDatacenters).filter(attempts -> Objects.equals(attempts, maximumAttemptsPerDatacenter.orElseThrow(() -> new SafeIllegalStateException("Unexpectedly could not find the max attempts per datacenter")))).keys().collect(Collectors.toSet());
    Set<CassandraServer> hostsInPermittedDatacenters = hostsMatchingPredicate.stream().filter(pool -> {
        String datacenter = hostToDatacenter.get(pool);
        return datacenter == null || !maximallyAttemptedDatacenters.contains(datacenter);
    }).collect(Collectors.toSet());
    Set<CassandraServer> filteredHosts = hostsInPermittedDatacenters.isEmpty() ? hostsMatchingPredicate : hostsInPermittedDatacenters;
    if (filteredHosts.isEmpty()) {
        log.info("No hosts match the provided predicate.");
        return Optional.empty();
    }
    Set<CassandraServer> livingHosts = blacklist.filterBlacklistedHostsFrom(filteredHosts);
    if (livingHosts.isEmpty()) {
        log.info("There are no known live hosts in the connection pool matching the predicate. We're choosing" + " one at random in a last-ditch attempt at forward progress.");
        livingHosts = filteredHosts;
    }
    Optional<CassandraServer> randomLivingHost = getRandomHostByActiveConnections(livingHosts);
    return randomLivingHost.map(pools::get);
}
Also used : Throwables(com.palantir.common.base.Throwables) SafeIllegalStateException(com.palantir.logsafe.exceptions.SafeIllegalStateException) Random(java.util.Random) Blacklist(com.palantir.atlasdb.keyvalue.cassandra.Blacklist) CassandraUtils(com.palantir.atlasdb.keyvalue.cassandra.CassandraUtils) LightweightOppToken(com.palantir.atlasdb.keyvalue.cassandra.LightweightOppToken) InetAddress(java.net.InetAddress) ImmutableRangeMap(com.google.common.collect.ImmutableRangeMap) MetricsManager(com.palantir.atlasdb.util.MetricsManager) ThriftHostsExtractingVisitor(com.palantir.atlasdb.cassandra.CassandraServersConfigs.ThriftHostsExtractingVisitor) Map(java.util.Map) TokenRange(org.apache.cassandra.thrift.TokenRange) PoolingContainer(com.palantir.common.pooling.PoolingContainer) EndpointDetails(org.apache.cassandra.thrift.EndpointDetails) Refreshable(com.palantir.refreshable.Refreshable) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) KeyedStream(com.palantir.common.streams.KeyedStream) Predicate(java.util.function.Predicate) Collection(java.util.Collection) Range(com.google.common.collect.Range) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Set(java.util.Set) Interner(com.google.common.collect.Interner) InetSocketAddress(java.net.InetSocketAddress) Collectors(java.util.stream.Collectors) Sets(com.google.common.collect.Sets) Objects(java.util.Objects) List(java.util.List) Stream(java.util.stream.Stream) UnsafeArg(com.palantir.logsafe.UnsafeArg) CassandraLogHelper(com.palantir.atlasdb.keyvalue.cassandra.CassandraLogHelper) Optional(java.util.Optional) Iterables(com.google.common.collect.Iterables) CassandraKeyValueServiceConfig(com.palantir.atlasdb.cassandra.CassandraKeyValueServiceConfig) CassandraKeyValueServiceRuntimeConfig(com.palantir.atlasdb.cassandra.CassandraKeyValueServiceRuntimeConfig) SafeLoggerFactory(com.palantir.logsafe.logger.SafeLoggerFactory) HashMap(java.util.HashMap) Function(java.util.function.Function) Supplier(java.util.function.Supplier) SafeLogger(com.palantir.logsafe.logger.SafeLogger) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) SafeArg(com.palantir.logsafe.SafeArg) ImmutableList(com.google.common.collect.ImmutableList) Suppliers(com.google.common.base.Suppliers) BaseEncoding(com.google.common.io.BaseEncoding) FunctionCheckedException(com.palantir.common.base.FunctionCheckedException) CassandraClientPoolingContainer(com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer) Interners(com.google.common.collect.Interners) UnknownHostException(java.net.UnknownHostException) TimeUnit(java.util.concurrent.TimeUnit) RangeMap(com.google.common.collect.RangeMap) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Comparator(java.util.Comparator) Collections(java.util.Collections) CassandraClient(com.palantir.atlasdb.keyvalue.cassandra.CassandraClient) CassandraClientPoolingContainer(com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer) SafeIllegalStateException(com.palantir.logsafe.exceptions.SafeIllegalStateException)

Aggregations

CassandraClientPoolingContainer (com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer)12 ImmutableMap (com.google.common.collect.ImmutableMap)7 InetSocketAddress (java.net.InetSocketAddress)7 HashMap (java.util.HashMap)7 Map (java.util.Map)7 NavigableMap (java.util.NavigableMap)5 Test (org.junit.Test)5 FunctionCheckedException (com.palantir.common.base.FunctionCheckedException)3 SafeIllegalStateException (com.palantir.logsafe.exceptions.SafeIllegalStateException)3 UnknownHostException (java.net.UnknownHostException)3 VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 Suppliers (com.google.common.base.Suppliers)2 ImmutableList (com.google.common.collect.ImmutableList)2 ImmutableRangeMap (com.google.common.collect.ImmutableRangeMap)2 ImmutableSet (com.google.common.collect.ImmutableSet)2 Interner (com.google.common.collect.Interner)2 Interners (com.google.common.collect.Interners)2 Iterables (com.google.common.collect.Iterables)2 Range (com.google.common.collect.Range)2 RangeMap (com.google.common.collect.RangeMap)2