Search in sources :

Example 6 with HiveHookEventProto

use of org.apache.hadoop.hive.ql.hooks.proto.HiveHookEvents.HiveHookEventProto in project hive by apache.

the class TestHiveProtoLoggingHook method testPreEventLog.

@Test
public void testPreEventLog() throws Exception {
    context.setHookType(HookType.PRE_EXEC_HOOK);
    EventLogger evtLogger = new EventLogger(conf, SystemClock.getInstance());
    evtLogger.handle(context);
    evtLogger.shutdown();
    HiveHookEventProto event = loadEvent(conf, tmpFolder);
    Assert.assertEquals(EventType.QUERY_SUBMITTED.name(), event.getEventType());
    Assert.assertEquals(1234L, event.getTimestamp());
    Assert.assertEquals(System.getProperty("user.name"), event.getUser());
    Assert.assertEquals("test_user", event.getRequestUser());
    Assert.assertEquals("test_queryId", event.getHiveQueryId());
    Assert.assertEquals("test_op_id", event.getOperationId());
    Assert.assertEquals(ExecutionMode.NONE.name(), event.getExecutionMode());
    Assert.assertFalse(event.hasQueue());
    assertOtherInfo(event, OtherInfoType.TEZ, Boolean.FALSE.toString());
    assertOtherInfo(event, OtherInfoType.MAPRED, Boolean.FALSE.toString());
    assertOtherInfo(event, OtherInfoType.CLIENT_IP_ADDRESS, "192.168.10.10");
    assertOtherInfo(event, OtherInfoType.SESSION_ID, "test_session_id");
    assertOtherInfo(event, OtherInfoType.THREAD_NAME, "test_thread_id");
    assertOtherInfo(event, OtherInfoType.HIVE_INSTANCE_TYPE, "HS2");
    assertOtherInfo(event, OtherInfoType.HIVE_ADDRESS, "hive_addr");
    assertOtherInfo(event, OtherInfoType.CONF, null);
    assertOtherInfo(event, OtherInfoType.QUERY, null);
}
Also used : EventLogger(org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook.EventLogger) HiveHookEventProto(org.apache.hadoop.hive.ql.hooks.proto.HiveHookEvents.HiveHookEventProto) Test(org.junit.Test)

Example 7 with HiveHookEventProto

use of org.apache.hadoop.hive.ql.hooks.proto.HiveHookEvents.HiveHookEventProto in project hive by apache.

the class TestHiveProtoLoggingHook method testPostEventLog.

@Test
public void testPostEventLog() throws Exception {
    context.setHookType(HookType.POST_EXEC_HOOK);
    context.getPerfLogger().perfLogBegin("test", "LogTest");
    context.getPerfLogger().perfLogEnd("test", "LogTest");
    EventLogger evtLogger = new EventLogger(conf, SystemClock.getInstance());
    evtLogger.handle(context);
    evtLogger.shutdown();
    HiveHookEventProto event = loadEvent(conf, tmpFolder);
    Assert.assertEquals(EventType.QUERY_COMPLETED.name(), event.getEventType());
    Assert.assertEquals(System.getProperty("user.name"), event.getUser());
    Assert.assertEquals("test_user", event.getRequestUser());
    Assert.assertEquals("test_queryId", event.getHiveQueryId());
    Assert.assertEquals("test_op_id", event.getOperationId());
    assertOtherInfo(event, OtherInfoType.STATUS, Boolean.TRUE.toString());
    String val = findOtherInfo(event, OtherInfoType.PERF);
    Map<String, Long> map = new ObjectMapper().readValue(val, new TypeReference<Map<String, Long>>() {
    });
    // This should be really close to zero.
    Assert.assertTrue("Expected LogTest in PERF", map.get("LogTest") < 100);
}
Also used : EventLogger(org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook.EventLogger) AtomicLong(java.util.concurrent.atomic.AtomicLong) HiveHookEventProto(org.apache.hadoop.hive.ql.hooks.proto.HiveHookEvents.HiveHookEventProto) Map(java.util.Map) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Test(org.junit.Test)

Example 8 with HiveHookEventProto

use of org.apache.hadoop.hive.ql.hooks.proto.HiveHookEvents.HiveHookEventProto in project hive by apache.

the class TestHiveProtoLoggingHook method loadEvent.

private HiveHookEventProto loadEvent(HiveConf conf, String tmpFolder) throws IOException {
    ProtoMessageReader<HiveHookEventProto> reader = getTestReader(conf, tmpFolder);
    HiveHookEventProto event = reader.readEvent();
    Assert.assertNotNull(event);
    return event;
}
Also used : HiveHookEventProto(org.apache.hadoop.hive.ql.hooks.proto.HiveHookEvents.HiveHookEventProto)

Aggregations

HiveHookEventProto (org.apache.hadoop.hive.ql.hooks.proto.HiveHookEvents.HiveHookEventProto)8 EventLogger (org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook.EventLogger)6 Test (org.junit.Test)6 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 EOFException (java.io.EOFException)1 Map (java.util.Map)1 AtomicLong (java.util.concurrent.atomic.AtomicLong)1 FileStatus (org.apache.hadoop.fs.FileStatus)1 FileSystem (org.apache.hadoop.fs.FileSystem)1 Path (org.apache.hadoop.fs.Path)1 ExecDriver (org.apache.hadoop.hive.ql.exec.mr.ExecDriver)1 TezTask (org.apache.hadoop.hive.ql.exec.tez.TezTask)1 MapWork (org.apache.hadoop.hive.ql.plan.MapWork)1 TezWork (org.apache.hadoop.hive.ql.plan.TezWork)1 DatePartitionedLogger (org.apache.tez.dag.history.logging.proto.DatePartitionedLogger)1