Search in sources :

Example 36 with CompositeInputAttemptIdentifier

use of org.apache.tez.runtime.library.common.CompositeInputAttemptIdentifier in project tez by apache.

the class TestShuffleInputEventHandlerOrderedGrouped method basicTest.

@Test(timeout = 5000)
public void basicTest() throws IOException {
    List<Event> events = new LinkedList<Event>();
    int srcIdx = 0;
    int targetIdx = 1;
    Event dme = createDataMovementEvent(srcIdx, targetIdx, null, false);
    events.add(dme);
    handler.handleEvents(events);
    CompositeInputAttemptIdentifier expectedIdentifier = new CompositeInputAttemptIdentifier(targetIdx, 0, PATH_COMPONENT, 1);
    int partitionId = srcIdx;
    verify(scheduler).addKnownMapOutput(eq(HOST), eq(PORT), eq(partitionId), eq(expectedIdentifier));
    assertTrue("Shuffle info events should be empty for regular shuffle codepath", scheduler.pipelinedShuffleInfoEventsMap.isEmpty());
}
Also used : CompositeInputAttemptIdentifier(org.apache.tez.runtime.library.common.CompositeInputAttemptIdentifier) InputFailedEvent(org.apache.tez.runtime.api.events.InputFailedEvent) Event(org.apache.tez.runtime.api.Event) DataMovementEvent(org.apache.tez.runtime.api.events.DataMovementEvent) LinkedList(java.util.LinkedList) Test(org.junit.Test)

Aggregations

CompositeInputAttemptIdentifier (org.apache.tez.runtime.library.common.CompositeInputAttemptIdentifier)36 InputAttemptIdentifier (org.apache.tez.runtime.library.common.InputAttemptIdentifier)24 Test (org.junit.Test)24 InputContext (org.apache.tez.runtime.api.InputContext)12 IOException (java.io.IOException)10 Event (org.apache.tez.runtime.api.Event)10 DataMovementEvent (org.apache.tez.runtime.api.events.DataMovementEvent)10 TezConfiguration (org.apache.tez.dag.api.TezConfiguration)8 Configuration (org.apache.hadoop.conf.Configuration)7 TezRuntimeConfiguration (org.apache.tez.runtime.library.api.TezRuntimeConfiguration)7 Matchers.anyString (org.mockito.Matchers.anyString)7 LinkedList (java.util.LinkedList)6 Path (org.apache.hadoop.fs.Path)5 InputFailedEvent (org.apache.tez.runtime.api.events.InputFailedEvent)5 FetchedInputAllocator (org.apache.tez.runtime.library.common.shuffle.FetchedInputAllocator)5 ByteString (com.google.protobuf.ByteString)4 ExecutorService (java.util.concurrent.ExecutorService)4 TezIndexRecord (org.apache.tez.runtime.library.common.sort.impl.TezIndexRecord)4 InvocationOnMock (org.mockito.invocation.InvocationOnMock)4 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)3