Search in sources :

Example 16 with TimelineEntityGroupId

use of org.apache.hadoop.yarn.api.records.timeline.TimelineEntityGroupId in project tez by apache.

the class TimelineCachePluginImpl method createTimelineEntityGroupIds.

private Set<TimelineEntityGroupId> createTimelineEntityGroupIds(TezDAGID dagId) {
    ApplicationId appId = dagId.getApplicationId();
    HashSet<TimelineEntityGroupId> groupIds = Sets.newHashSet(TimelineEntityGroupId.newInstance(appId, dagId.toString()));
    for (int numGroupsPerDag : allNumGroupsPerDag) {
        groupIds.add(TimelineEntityGroupId.newInstance(appId, dagId.getGroupId(numGroupsPerDag)));
    }
    return groupIds;
}
Also used : TimelineEntityGroupId(org.apache.hadoop.yarn.api.records.timeline.TimelineEntityGroupId) ApplicationId(org.apache.hadoop.yarn.api.records.ApplicationId)

Example 17 with TimelineEntityGroupId

use of org.apache.hadoop.yarn.api.records.timeline.TimelineEntityGroupId in project tez by apache.

the class TestATSHistoryV15 method testGetGroupId.

@Test
public void testGetGroupId() throws Exception {
    ApplicationId appId = ApplicationId.newInstance(1000l, 1);
    TezDAGID dagid = TezDAGID.getInstance(appId, 1);
    for (final HistoryEventType eventType : HistoryEventType.values()) {
        HistoryEvent historyEvent = new HistoryEvent() {

            @Override
            public HistoryEventType getEventType() {
                return eventType;
            }

            @Override
            public boolean isRecoveryEvent() {
                return false;
            }

            @Override
            public boolean isHistoryEvent() {
                return false;
            }

            @Override
            public void toProtoStream(OutputStream outputStream) throws IOException {
            }

            @Override
            public void fromProtoStream(InputStream inputStream) throws IOException {
            }
        };
        DAGHistoryEvent event = new DAGHistoryEvent(dagid, historyEvent);
        ATSV15HistoryLoggingService service = new ATSV15HistoryLoggingService();
        AppContext appContext = mock(AppContext.class);
        when(appContext.getApplicationID()).thenReturn(appId);
        when(appContext.getHadoopShim()).thenReturn(new HadoopShim() {
        });
        service.setAppContext(appContext);
        TimelineEntityGroupId grpId = service.getGroupId(event);
        Assert.assertNotNull(grpId);
        Assert.assertEquals(appId, grpId.getApplicationId());
        switch(eventType) {
            case AM_LAUNCHED:
            case APP_LAUNCHED:
            case AM_STARTED:
            case CONTAINER_LAUNCHED:
            case CONTAINER_STOPPED:
                Assert.assertEquals(appId.toString(), grpId.getTimelineEntityGroupId());
                break;
            default:
                Assert.assertEquals(dagid.toString(), grpId.getTimelineEntityGroupId());
        }
        service.close();
    }
}
Also used : HadoopShim(org.apache.tez.hadoop.shim.HadoopShim) InputStream(java.io.InputStream) OutputStream(java.io.OutputStream) AppContext(org.apache.tez.dag.app.AppContext) TezDAGID(org.apache.tez.dag.records.TezDAGID) DAGHistoryEvent(org.apache.tez.dag.history.DAGHistoryEvent) HistoryEventType(org.apache.tez.dag.history.HistoryEventType) TimelineEntityGroupId(org.apache.hadoop.yarn.api.records.timeline.TimelineEntityGroupId) ApplicationId(org.apache.hadoop.yarn.api.records.ApplicationId) DAGHistoryEvent(org.apache.tez.dag.history.DAGHistoryEvent) HistoryEvent(org.apache.tez.dag.history.HistoryEvent) ATSV15HistoryLoggingService(org.apache.tez.dag.history.logging.ats.ATSV15HistoryLoggingService) Test(org.junit.Test)

Aggregations

TimelineEntityGroupId (org.apache.hadoop.yarn.api.records.timeline.TimelineEntityGroupId)17 Test (org.junit.Test)12 ApplicationId (org.apache.hadoop.yarn.api.records.ApplicationId)5 TreeSet (java.util.TreeSet)2 Path (org.apache.hadoop.fs.Path)2 TimelineEntity (org.apache.hadoop.yarn.api.records.timeline.TimelineEntity)2 ThreadFactoryBuilder (com.google.common.util.concurrent.ThreadFactoryBuilder)1 IOException (java.io.IOException)1 InputStream (java.io.InputStream)1 OutputStream (java.io.OutputStream)1 HashSet (java.util.HashSet)1 LinkedHashMap (java.util.LinkedHashMap)1 LinkedList (java.util.LinkedList)1 ApplicationAttemptId (org.apache.hadoop.yarn.api.records.ApplicationAttemptId)1 NameValuePair (org.apache.hadoop.yarn.server.timeline.NameValuePair)1 AppContext (org.apache.tez.dag.app.AppContext)1 DAGHistoryEvent (org.apache.tez.dag.history.DAGHistoryEvent)1 HistoryEvent (org.apache.tez.dag.history.HistoryEvent)1 HistoryEventType (org.apache.tez.dag.history.HistoryEventType)1 ATSV15HistoryLoggingService (org.apache.tez.dag.history.logging.ats.ATSV15HistoryLoggingService)1