Search in sources :

Example 96 with TimelineEntity

use of org.apache.hadoop.yarn.api.records.timeline.TimelineEntity in project hadoop by apache.

the class ApplicationMaster method publishContainerStartEvent.

private void publishContainerStartEvent(final TimelineClient timelineClient, final Container container, String domainId, UserGroupInformation ugi) {
    final TimelineEntity entity = new TimelineEntity();
    entity.setEntityId(container.getId().toString());
    entity.setEntityType(DSEntity.DS_CONTAINER.toString());
    entity.setDomainId(domainId);
    entity.addPrimaryFilter(USER_TIMELINE_FILTER_NAME, ugi.getShortUserName());
    entity.addPrimaryFilter(APPID_TIMELINE_FILTER_NAME, container.getId().getApplicationAttemptId().getApplicationId().toString());
    TimelineEvent event = new TimelineEvent();
    event.setTimestamp(System.currentTimeMillis());
    event.setEventType(DSEvent.DS_CONTAINER_START.toString());
    event.addEventInfo("Node", container.getNodeId().toString());
    event.addEventInfo("Resources", container.getResource().toString());
    entity.addEvent(event);
    try {
        processTimelineResponseErrors(putContainerEntity(timelineClient, container.getId().getApplicationAttemptId(), entity));
    } catch (YarnException | IOException | ClientHandlerException e) {
        LOG.error("Container start event could not be published for " + container.getId().toString(), e);
    }
}
Also used : TimelineEvent(org.apache.hadoop.yarn.api.records.timeline.TimelineEvent) ClientHandlerException(com.sun.jersey.api.client.ClientHandlerException) IOException(java.io.IOException) TimelineEntity(org.apache.hadoop.yarn.api.records.timeline.TimelineEntity) YarnException(org.apache.hadoop.yarn.exceptions.YarnException)

Example 97 with TimelineEntity

use of org.apache.hadoop.yarn.api.records.timeline.TimelineEntity in project hadoop by apache.

the class TimelineWriter method putEntities.

public TimelinePutResponse putEntities(TimelineEntity... entities) throws IOException, YarnException {
    TimelineEntities entitiesContainer = new TimelineEntities();
    for (TimelineEntity entity : entities) {
        if (entity.getEntityId() == null || entity.getEntityType() == null) {
            throw new YarnException("Incomplete entity without entity id/type");
        }
        entitiesContainer.addEntity(entity);
    }
    ClientResponse resp = doPosting(entitiesContainer, null);
    return resp.getEntity(TimelinePutResponse.class);
}
Also used : ClientResponse(com.sun.jersey.api.client.ClientResponse) TimelineEntities(org.apache.hadoop.yarn.api.records.timeline.TimelineEntities) TimelineEntity(org.apache.hadoop.yarn.api.records.timeline.TimelineEntity) YarnException(org.apache.hadoop.yarn.exceptions.YarnException)

Example 98 with TimelineEntity

use of org.apache.hadoop.yarn.api.records.timeline.TimelineEntity in project hive by apache.

the class TestATSHook method testCreatePreHookEventJsonShhouldMatch.

@Test
public void testCreatePreHookEventJsonShhouldMatch() throws Exception {
    TimelineEntity timelineEntity = uut.createPreHookEvent("test-query-id", "test-query", new org.json.JSONObject(), 0L, "test-user", "test-request-user", 0, 0, "test-opid", "client-ip-address", "hive-instance-address", "hive-instance-type", "session-id", "log-id", "thread-id", "execution-mode", Collections.<String>emptyList(), Collections.<String>emptyList(), new HiveConf(), null, "domain-id");
    String resultStr = (String) timelineEntity.getOtherInfo().get(ATSHook.OtherInfoTypes.QUERY.name());
    JsonNode result = objectMapper.readTree(resultStr);
    JsonNode expected = objectMapper.readTree("{\"queryText\":\"test-query\"," + "\"queryPlan\":{}}");
    assertEquals(expected, result);
}
Also used : HiveConf(org.apache.hadoop.hive.conf.HiveConf) JsonNode(com.fasterxml.jackson.databind.JsonNode) TimelineEntity(org.apache.hadoop.yarn.api.records.timeline.TimelineEntity) Test(org.junit.Test)

Aggregations

TimelineEntity (org.apache.hadoop.yarn.api.records.timeline.TimelineEntity)98 Test (org.junit.Test)37 TimelineEvent (org.apache.hadoop.yarn.api.records.timeline.TimelineEvent)32 TimelineEntities (org.apache.hadoop.yarn.api.records.timeline.TimelineEntities)30 TimelinePutResponse (org.apache.hadoop.yarn.api.records.timeline.TimelinePutResponse)20 HashMap (java.util.HashMap)16 YarnException (org.apache.hadoop.yarn.exceptions.YarnException)16 IOException (java.io.IOException)15 ClientResponse (com.sun.jersey.api.client.ClientResponse)12 WebResource (com.sun.jersey.api.client.WebResource)11 Configuration (org.apache.hadoop.conf.Configuration)9 HashSet (java.util.HashSet)8 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)7 Field (org.apache.hadoop.yarn.server.timeline.TimelineReader.Field)6 LinkedHashMap (java.util.LinkedHashMap)5 Set (java.util.Set)5 Path (org.apache.hadoop.fs.Path)5 AdminACLsManager (org.apache.hadoop.yarn.security.AdminACLsManager)5 ClientHandlerException (com.sun.jersey.api.client.ClientHandlerException)4 ArrayList (java.util.ArrayList)4