Search in sources :

Example 6 with MapKeyResult

use of com.linkedin.d2.balancer.util.MapKeyResult in project rest.li by linkedin.

the class ConsistentHashKeyMapper method mapKeysV2.

@Override
public <K> MapKeyResult<URI, K> mapKeysV2(URI serviceUri, Iterable<K> keys) throws ServiceUnavailableException {
    // distribute keys to rings for partitions
    // Note that we assume the keys are partitioning keys
    MapKeyResult<Ring<URI>, K> keyToPartitionResult = _ringProvider.getRings(serviceUri, keys);
    Map<Ring<URI>, Collection<K>> ringToKeys = keyToPartitionResult.getMapResult();
    Map<URI, Collection<K>> result = new HashMap<>();
    Collection<MapKeyResult.UnmappedKey<K>> unmappedKeys = new ArrayList<>();
    // first collect unmappedkeys in ditributing keys to partitions
    unmappedKeys.addAll(keyToPartitionResult.getUnmappedKeys());
    // for each partition, distribute keys to different server uris
    for (Map.Entry<Ring<URI>, Collection<K>> entry : ringToKeys.entrySet()) {
        MapKeyResult<URI, K> keyToHostResult = doMapKeys(entry.getKey(), entry.getValue());
        // collect map key to host result
        Map<URI, Collection<K>> hostToKeys = keyToHostResult.getMapResult();
        for (Map.Entry<URI, Collection<K>> hostEntry : hostToKeys.entrySet()) {
            URI uri = hostEntry.getKey();
            Collection<K> collection = result.get(uri);
            if (collection == null) {
                collection = new ArrayList<>();
                result.put(uri, collection);
            }
            collection.addAll(hostEntry.getValue());
        }
        // collect unmapped keys
        unmappedKeys.addAll(keyToHostResult.getUnmappedKeys());
    }
    return new MapKeyResult<>(result, unmappedKeys);
}
Also used : HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) URI(java.net.URI) Collection(java.util.Collection) HashMap(java.util.HashMap) Map(java.util.Map) MapKeyResult(com.linkedin.d2.balancer.util.MapKeyResult)

Aggregations

URI (java.net.URI)6 Collection (java.util.Collection)6 HashMap (java.util.HashMap)6 ArrayList (java.util.ArrayList)5 Map (java.util.Map)5 ClusterProperties (com.linkedin.d2.balancer.properties.ClusterProperties)4 ServiceProperties (com.linkedin.d2.balancer.properties.ServiceProperties)4 UriProperties (com.linkedin.d2.balancer.properties.UriProperties)4 MapKeyResult (com.linkedin.d2.balancer.util.MapKeyResult)4 HashSet (java.util.HashSet)4 FutureCallback (com.linkedin.common.callback.FutureCallback)3 None (com.linkedin.common.util.None)3 DarkClusterConfigMap (com.linkedin.d2.DarkClusterConfigMap)3 ServiceUnavailableException (com.linkedin.d2.balancer.ServiceUnavailableException)3 PartitionData (com.linkedin.d2.balancer.properties.PartitionData)3 Ring (com.linkedin.d2.balancer.util.hashing.Ring)3 Test (org.testng.annotations.Test)3 KeyMapper (com.linkedin.d2.balancer.KeyMapper)2 LoadBalancerState (com.linkedin.d2.balancer.LoadBalancerState)2 RewriteLoadBalancerClient (com.linkedin.d2.balancer.clients.RewriteLoadBalancerClient)2