Search in sources :

Example 1 with Log4jConfig

use of org.apache.hive.testutils.junit.extensions.Log4jConfig in project hive by apache.

the class TestLlapDaemonLogging method testQueryRoutingNoLeakFileDescriptors.

@Test
@Log4jConfig("llap-daemon-routing-log4j2.properties")
@ExtendWith(LlapDaemonExtension.class)
@ExtendWith(DoNothingTCPServerExtension.class)
void testQueryRoutingNoLeakFileDescriptors(LlapDaemon daemon, DoNothingTCPServer amMockServer) throws IOException, InterruptedException {
    final int amPort = amMockServer.port();
    Credentials credentials = validSessionCredentials();
    String appId = "application_1540489363818_0021";
    for (int i = 0; i < 10; i++) {
        String queryId = "query" + i;
        int dagId = 1000 + i;
        daemon.registerDag(LlapDaemonTestUtils.buildRegisterDagRequest(appId, dagId, credentials));
        daemon.submitWork(LlapDaemonTestUtils.buildSubmitProtoRequest(appId, dagId, queryId, amPort, credentials));
        daemon.queryComplete(LlapDaemonTestUtils.buildQueryCompleteRequest(appId, dagId));
    }
    // Busy wait till all daemon tasks are treated
    while (!daemon.getExecutorsStatus().isEmpty()) {
        Thread.sleep(100);
    }
    // The IdlePurgePolicy used should close appenders after 5 seconds of inactivity.
    // Wait for 8 sec to give some margin.
    Thread.sleep(8000);
    Pattern pn = Pattern.compile("query\\d++-dag_1540489363818_0021_\\d{4}\\.log");
    assertEquals(0, findOpenFileDescriptors(pn).count());
}
Also used : Pattern(java.util.regex.Pattern) Credentials(org.apache.hadoop.security.Credentials) Test(org.junit.jupiter.api.Test) Log4jConfig(org.apache.hive.testutils.junit.extensions.Log4jConfig) ExtendWith(org.junit.jupiter.api.extension.ExtendWith)

Example 2 with Log4jConfig

use of org.apache.hive.testutils.junit.extensions.Log4jConfig in project hive by apache.

the class TestLlapDaemonLogging method testQueryRoutingLogFileNameOnCompleteQuery.

@Test
@Log4jConfig("llap-daemon-routing-log4j2.properties")
@ExtendWith(LlapDaemonExtension.class)
@ExtendWith(DoNothingTCPServerExtension.class)
void testQueryRoutingLogFileNameOnCompleteQuery(LlapDaemon daemon, DoNothingTCPServer amMockServer) throws IOException, InterruptedException {
    final int amPort = amMockServer.port();
    Credentials credentials = validSessionCredentials();
    String appId = "application_3500489363818_0021";
    String queryId = "query0";
    int dagId = 3000;
    daemon.registerDag(LlapDaemonTestUtils.buildRegisterDagRequest(appId, dagId, credentials));
    daemon.submitWork(LlapDaemonTestUtils.buildSubmitProtoRequest(appId, dagId, queryId, amPort, credentials));
    daemon.queryComplete(LlapDaemonTestUtils.buildQueryCompleteRequest(appId, dagId));
    // Busy wait till all daemon tasks are treated
    while (!daemon.getExecutorsStatus().isEmpty()) {
        Thread.sleep(100);
    }
    // The IdlePurgePolicy used should close appenders after 5 seconds of inactivity.
    // Wait for 8 sec to give some margin.
    Thread.sleep(8000);
    // A complete query should always have a corresponding log file with ".done" suffix
    assertFileExists("query0-dag_3500489363818_0021_3000.log.done");
}
Also used : Credentials(org.apache.hadoop.security.Credentials) Test(org.junit.jupiter.api.Test) Log4jConfig(org.apache.hive.testutils.junit.extensions.Log4jConfig) ExtendWith(org.junit.jupiter.api.extension.ExtendWith)

Example 3 with Log4jConfig

use of org.apache.hive.testutils.junit.extensions.Log4jConfig in project hive by apache.

the class TestLlapDaemonLogging method testQueryRoutingLogFileNameOnIncompleteQuery.

@Test
@Log4jConfig("llap-daemon-routing-log4j2.properties")
@ExtendWith(LlapDaemonExtension.class)
@ExtendWith(DoNothingTCPServerExtension.class)
void testQueryRoutingLogFileNameOnIncompleteQuery(LlapDaemon daemon, DoNothingTCPServer amMockServer) throws IOException, InterruptedException {
    final int amPort = amMockServer.port();
    Credentials credentials = validSessionCredentials();
    String appId = "application_2500489363818_0021";
    String queryId = "query0";
    int dagId = 2000;
    daemon.registerDag(LlapDaemonTestUtils.buildRegisterDagRequest(appId, dagId, credentials));
    daemon.submitWork(LlapDaemonTestUtils.buildSubmitProtoRequest(appId, dagId, queryId, amPort, credentials));
    // Busy wait till all daemon tasks are treated
    while (!daemon.getExecutorsStatus().isEmpty()) {
        Thread.sleep(100);
    }
    // The IdlePurgePolicy used should close appenders after 5 seconds of inactivity.
    // Wait for 8 sec to give some margin.
    Thread.sleep(8000);
    assertFileExists("query0-dag_2500489363818_0021_2000.log");
}
Also used : Credentials(org.apache.hadoop.security.Credentials) Test(org.junit.jupiter.api.Test) Log4jConfig(org.apache.hive.testutils.junit.extensions.Log4jConfig) ExtendWith(org.junit.jupiter.api.extension.ExtendWith)

Aggregations

Credentials (org.apache.hadoop.security.Credentials)3 Log4jConfig (org.apache.hive.testutils.junit.extensions.Log4jConfig)3 Test (org.junit.jupiter.api.Test)3 ExtendWith (org.junit.jupiter.api.extension.ExtendWith)3 Pattern (java.util.regex.Pattern)1