Search in sources :

Example 1 with SessionsGet

use of com.thinkbiganalytics.kylo.spark.model.SessionsGet in project kylo by Teradata.

the class TestSparkLivyRestClient method testGetLivySessions.

/**
 * This test assumes: 1) you have configured a Livy Server for Kerberos and that it is running at 8998 2) that server has the library 'kylo-spark-shell-client-v1-0.10.0-SNAPSHOT.jar' installed
 */
@Test
// ignore, for now, because this is an integration test that requires livy configured on the test system
@Ignore
public void testGetLivySessions() throws JsonProcessingException, InterruptedException {
    System.setProperty("sun.security.krb5.debug", "true");
    System.setProperty("sun.security.jgss.debug", "true");
    SessionsGet sg = new SessionsGet.Builder().from(1).size(2).build();
    String sgJson = new ObjectMapper().writeValueAsString(sg);
    logger.debug("{}", sgJson);
    assertThat(sgJson).isEqualToIgnoringCase("{\"from\":1,\"size\":2}");
    logger.debug("kerberosSparkProperties={}", kerberosSparkProperties);
    SparkShellProcess sparkProcess = sparkLivyProcessManager.getProcessForUser("kylo");
    // ((SparkLivyProcess) sparkProcess).setPort(8999);
    JerseyRestClient client = sparkLivyProcessManager.getClient(sparkProcess);
    sparkLivyProcessManager.start("kylo");
    Integer stmtId = 0;
    Statement statement = livyRestProvider.pollStatement(client, sparkProcess, stmtId);
    logger.debug("statement={}", statement);
    assertThat(statement.getState()).isEqualTo(StatementState.available);
    assertThat(statement.getOutput().getStatus()).isEqualTo(StatementOutputStatus.ok);
}
Also used : SparkShellProcess(com.thinkbiganalytics.spark.shell.SparkShellProcess) Statement(com.thinkbiganalytics.kylo.spark.model.Statement) SessionsGet(com.thinkbiganalytics.kylo.spark.model.SessionsGet) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) JerseyRestClient(com.thinkbiganalytics.rest.JerseyRestClient) Ignore(org.junit.Ignore) Test(org.junit.Test)

Aggregations

ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 SessionsGet (com.thinkbiganalytics.kylo.spark.model.SessionsGet)1 Statement (com.thinkbiganalytics.kylo.spark.model.Statement)1 JerseyRestClient (com.thinkbiganalytics.rest.JerseyRestClient)1 SparkShellProcess (com.thinkbiganalytics.spark.shell.SparkShellProcess)1 Ignore (org.junit.Ignore)1 Test (org.junit.Test)1