Search in sources :

Example 16 with TransformResponse

use of com.thinkbiganalytics.spark.rest.model.TransformResponse in project kylo by Teradata.

the class AbstractTransformControllerTest method getTable.

/**
 * Verify requesting a transformation status.
 */
@Test
public void getTable() throws Exception {
    // Mock transform objects
    TransformJob pendingJob = Mockito.mock(TransformJob.class);
    Mockito.when(pendingJob.getGroupId()).thenReturn("PendingJob");
    Mockito.when(pendingJob.progress()).thenReturn(0.5);
    TransformJob successJob = Mockito.mock(TransformJob.class);
    TransformResponse successResponse = new TransformResponse();
    Mockito.when(successJob.get()).thenReturn(successResponse);
    Mockito.when(successJob.isDone()).thenReturn(true);
    TransformService transformService = Mockito.mock(TransformService.class);
    Mockito.when(transformService.getTransformJob("PendingJob")).thenReturn(pendingJob);
    Mockito.when(transformService.getTransformJob("SuccessJob")).thenReturn(successJob);
    // Test with pending job
    AbstractTransformController controller = new AbstractTransformController() {
    };
    controller.transformService = transformService;
    Response response = controller.getTable("PendingJob");
    Assert.assertEquals(Response.Status.OK, response.getStatusInfo());
    TransformResponse transformResponse = (TransformResponse) response.getEntity();
    Assert.assertEquals(0.5, transformResponse.getProgress(), 0.001);
    Assert.assertEquals(TransformResponse.Status.PENDING, transformResponse.getStatus());
    Assert.assertEquals("PendingJob", transformResponse.getTable());
    // Test with success job
    response = controller.getTable("SuccessJob");
    Assert.assertEquals(successResponse, response.getEntity());
    Assert.assertEquals(Response.Status.OK, response.getStatusInfo());
}
Also used : TransformResponse(com.thinkbiganalytics.spark.rest.model.TransformResponse) Response(javax.ws.rs.core.Response) TransformJob(com.thinkbiganalytics.spark.metadata.TransformJob) TransformResponse(com.thinkbiganalytics.spark.rest.model.TransformResponse) TransformService(com.thinkbiganalytics.spark.service.TransformService) Test(org.junit.Test)

Example 17 with TransformResponse

use of com.thinkbiganalytics.spark.rest.model.TransformResponse in project kylo by Teradata.

the class SparkShellTransformControllerTest method createWithMissingScript.

/**
 * Verify response if missing script.
 */
@Test
public void createWithMissingScript() {
    SparkShellTransformController controller = new SparkShellTransformController();
    Response response = controller.create(new TransformRequest());
    Assert.assertEquals(Response.Status.BAD_REQUEST, response.getStatusInfo());
    TransformResponse entity = (TransformResponse) response.getEntity();
    Assert.assertEquals("The request must include a script with the transformations to perform.", entity.getMessage());
    Assert.assertEquals(TransformResponse.Status.ERROR, entity.getStatus());
}
Also used : TransformResponse(com.thinkbiganalytics.spark.rest.model.TransformResponse) Response(javax.ws.rs.core.Response) TransformResponse(com.thinkbiganalytics.spark.rest.model.TransformResponse) TransformRequest(com.thinkbiganalytics.spark.rest.model.TransformRequest) Test(org.junit.Test)

Example 18 with TransformResponse

use of com.thinkbiganalytics.spark.rest.model.TransformResponse in project kylo by Teradata.

the class SparkShellTransformControllerTest method createWithMissingParentTable.

/**
 * Verify response if missing parent table.
 */
@Test
public void createWithMissingParentTable() {
    // Create transform request
    TransformRequest request = new TransformRequest();
    request.setScript("parent");
    TransformRequest.Parent parent = new TransformRequest.Parent();
    parent.setScript("sqlContext.sql(\"SELECT * FROM invalid\")");
    request.setParent(parent);
    // Test missing parent table
    SparkShellTransformController controller = new SparkShellTransformController();
    Response response = controller.create(request);
    Assert.assertEquals(Response.Status.BAD_REQUEST, response.getStatusInfo());
    TransformResponse entity = (TransformResponse) response.getEntity();
    Assert.assertEquals("The parent must include the table containing the results.", entity.getMessage());
    Assert.assertEquals(TransformResponse.Status.ERROR, entity.getStatus());
}
Also used : TransformResponse(com.thinkbiganalytics.spark.rest.model.TransformResponse) Response(javax.ws.rs.core.Response) TransformResponse(com.thinkbiganalytics.spark.rest.model.TransformResponse) TransformRequest(com.thinkbiganalytics.spark.rest.model.TransformRequest) Test(org.junit.Test)

Example 19 with TransformResponse

use of com.thinkbiganalytics.spark.rest.model.TransformResponse in project kylo by Teradata.

the class SparkShellTransformControllerTest method create.

/**
 * Verify requesting a Spark transformation.
 */
@Test
public void create() throws Exception {
    // Mock transform objects
    TransformRequest transformRequest = new TransformRequest();
    transformRequest.setScript("sqlContext.sql(\"SELECT * FROM invalid\")");
    TransformResponse transformResponse = new TransformResponse();
    transformResponse.setProgress(0.0);
    transformResponse.setStatus(TransformResponse.Status.PENDING);
    transformResponse.setTable("results");
    TransformService transformService = Mockito.mock(TransformService.class);
    Mockito.when(transformService.execute(transformRequest)).thenReturn(transformResponse);
    // Test transforming
    SparkShellTransformController controller = new SparkShellTransformController();
    controller.transformService = transformService;
    Response response = controller.create(transformRequest);
    Assert.assertEquals(Response.Status.OK, response.getStatusInfo());
    Assert.assertEquals(transformResponse, response.getEntity());
}
Also used : TransformResponse(com.thinkbiganalytics.spark.rest.model.TransformResponse) Response(javax.ws.rs.core.Response) TransformResponse(com.thinkbiganalytics.spark.rest.model.TransformResponse) TransformService(com.thinkbiganalytics.spark.service.TransformService) TransformRequest(com.thinkbiganalytics.spark.rest.model.TransformRequest) Test(org.junit.Test)

Example 20 with TransformResponse

use of com.thinkbiganalytics.spark.rest.model.TransformResponse in project kylo by Teradata.

the class SparkShellTransformControllerTest method createWithMissingParentScript.

/**
 * Verify response if missing parent script.
 */
@Test
public void createWithMissingParentScript() {
    // Create transform request
    TransformRequest request = new TransformRequest();
    request.setScript("parent");
    request.setParent(new TransformRequest.Parent());
    // Test missing parent script
    SparkShellTransformController controller = new SparkShellTransformController();
    Response response = controller.create(request);
    Assert.assertEquals(Response.Status.BAD_REQUEST, response.getStatusInfo());
    TransformResponse entity = (TransformResponse) response.getEntity();
    Assert.assertEquals("The parent must include a script with the transformations performed.", entity.getMessage());
    Assert.assertEquals(TransformResponse.Status.ERROR, entity.getStatus());
}
Also used : TransformResponse(com.thinkbiganalytics.spark.rest.model.TransformResponse) Response(javax.ws.rs.core.Response) TransformResponse(com.thinkbiganalytics.spark.rest.model.TransformResponse) TransformRequest(com.thinkbiganalytics.spark.rest.model.TransformRequest) Test(org.junit.Test)

Aggregations

TransformResponse (com.thinkbiganalytics.spark.rest.model.TransformResponse)23 Test (org.junit.Test)12 TransformRequest (com.thinkbiganalytics.spark.rest.model.TransformRequest)8 Nonnull (javax.annotation.Nonnull)7 Response (javax.ws.rs.core.Response)7 SaveResponse (com.thinkbiganalytics.spark.rest.model.SaveResponse)4 DataSet (com.thinkbiganalytics.spark.DataSet)3 TransformJob (com.thinkbiganalytics.spark.metadata.TransformJob)3 TransformQueryResult (com.thinkbiganalytics.spark.rest.model.TransformQueryResult)3 TransformService (com.thinkbiganalytics.spark.service.TransformService)3 InputStream (java.io.InputStream)3 List (java.util.List)3 StructType (org.apache.spark.sql.types.StructType)3 SparkContextService (com.thinkbiganalytics.spark.SparkContextService)2 SparkScriptEngine (com.thinkbiganalytics.spark.repl.SparkScriptEngine)2 MissingResourceException (java.util.MissingResourceException)2 ScriptException (javax.script.ScriptException)2 SparkContext (org.apache.spark.SparkContext)2 StorageLevel (org.apache.spark.storage.StorageLevel)2 NamedParam (scala.tools.nsc.interpreter.NamedParam)2