Search in sources :

Example 76 with DrillbitEndpoint

use of org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint in project drill by apache.

the class PhysicalOperatorUtil method getIndexOrderedEndpoints.

/**
   * Helper method to create a list of MinorFragmentEndpoint instances from a given endpoint assignment list.
   *
   * @param endpoints Assigned endpoint list. Index of each endpoint in list indicates the MinorFragmentId of the
   *                  fragment that is assigned to the endpoint.
   * @return
   */
public static List<MinorFragmentEndpoint> getIndexOrderedEndpoints(List<DrillbitEndpoint> endpoints) {
    List<MinorFragmentEndpoint> destinations = Lists.newArrayList();
    int minorFragmentId = 0;
    for (DrillbitEndpoint endpoint : endpoints) {
        destinations.add(new MinorFragmentEndpoint(minorFragmentId, endpoint));
        minorFragmentId++;
    }
    return destinations;
}
Also used : MinorFragmentEndpoint(org.apache.drill.exec.physical.MinorFragmentEndpoint) DrillbitEndpoint(org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint) DrillbitEndpoint(org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint) MinorFragmentEndpoint(org.apache.drill.exec.physical.MinorFragmentEndpoint)

Example 77 with DrillbitEndpoint

use of org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint in project drill by apache.

the class AbstractMuxExchange method getReceiverParallelizationInfo.

@Override
public ParallelizationInfo getReceiverParallelizationInfo(List<DrillbitEndpoint> senderFragmentEndpoints) {
    Preconditions.checkArgument(senderFragmentEndpoints != null && senderFragmentEndpoints.size() > 0, "Sender fragment endpoint list should not be empty");
    // We want to run one mux receiver per Drillbit endpoint.
    // Identify the number of unique Drillbit endpoints in sender fragment endpoints.
    List<DrillbitEndpoint> drillbitEndpoints = ImmutableSet.copyOf(senderFragmentEndpoints).asList();
    List<EndpointAffinity> affinities = Lists.newArrayList();
    for (DrillbitEndpoint ep : drillbitEndpoints) {
        affinities.add(new EndpointAffinity(ep, Double.POSITIVE_INFINITY));
    }
    return ParallelizationInfo.create(affinities.size(), affinities.size(), affinities);
}
Also used : DrillbitEndpoint(org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint) EndpointAffinity(org.apache.drill.exec.physical.EndpointAffinity)

Aggregations

DrillbitEndpoint (org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint)77 Test (org.junit.Test)23 EndpointAffinity (org.apache.drill.exec.physical.EndpointAffinity)14 IOException (java.io.IOException)9 Stopwatch (com.google.common.base.Stopwatch)7 ArrayList (java.util.ArrayList)7 PlanFragment (org.apache.drill.exec.proto.BitControl.PlanFragment)7 ServerName (org.apache.hadoop.hbase.ServerName)7 HRegionInfo (org.apache.hadoop.hbase.HRegionInfo)6 Entry (java.util.Map.Entry)5 DrillConfig (org.apache.drill.common.config.DrillConfig)5 FragmentHandle (org.apache.drill.exec.proto.ExecProtos.FragmentHandle)5 DrillbitContext (org.apache.drill.exec.server.DrillbitContext)5 HBaseGroupScan (org.apache.drill.exec.store.hbase.HBaseGroupScan)5 HBaseScanSpec (org.apache.drill.exec.store.hbase.HBaseScanSpec)5 QueryWorkUnit (org.apache.drill.exec.work.QueryWorkUnit)5 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)4 HashMap (java.util.HashMap)4 AtomicLong (java.util.concurrent.atomic.AtomicLong)4 NonStrictExpectations (mockit.NonStrictExpectations)4