Search in sources :

Example 1 with SparkJobResult

use of com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResult in project kylo by Teradata.

the class SparkJobCacheService method getResult.

/**
 * Gets the specified Spark job result and removes it from this cache.
 */
@Nullable
public SparkJobResult getResult(@Nonnull final String id) {
    final SparkJobResult result = results.get(id);
    clearResult(id);
    return result;
}
Also used : SparkJobResult(com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResult) Nullable(javax.annotation.Nullable)

Example 2 with SparkJobResult

use of com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResult in project kylo by Teradata.

the class DefaultSparkJobContextTest method testSuccess.

/**
 * Verify handling of a <i>SUCCESS</i> response.
 */
@Test
public void testSuccess() {
    // Mock response
    final SparkJobResponse response = new SparkJobResponse();
    response.setId("mock-job");
    response.setResult(new SparkJobResult());
    response.setStatus(SparkJobResponse.Status.SUCCESS);
    // Test first subscriber
    final DefaultSparkJobContext context = DefaultSparkJobContext.create(() -> response, cache, executor);
    Assert.assertNotNull(take(context));
    // Test second subscriber
    Assert.assertEquals(response.getResult(), take(context).getResult());
}
Also used : SparkJobResponse(com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResponse) SparkJobResult(com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResult) Test(org.junit.Test)

Example 3 with SparkJobResult

use of com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResult in project kylo by Teradata.

the class TransformService method submit.

/**
 * Executes the specified transformation and returns the name of the Hive table containing the results.
 *
 * @param request the transformation request
 * @return the Hive table containing the results
 * @throws IllegalStateException if this service is not running
 * @throws ScriptException       if the script cannot be executed
 */
@Nonnull
public SparkJobResponse submit(@Nonnull final SparkJobRequest request) throws ScriptException {
    log.entry(request);
    final Supplier<SparkJobResult> jobTask = createJobTask(request);
    final SparkJobResponse response = submitSparkJob(jobTask);
    return log.exit(response);
}
Also used : SparkJobResponse(com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResponse) SparkJobResult(com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResult) Nonnull(javax.annotation.Nonnull)

Example 4 with SparkJobResult

use of com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResult in project kylo by Teradata.

the class SparkJobController method getJobResult.

@GET
@Path("/{job}")
@Produces(MediaType.APPLICATION_JSON)
@ApiOperation("Fetches the status of a job")
@ApiResponses({ @ApiResponse(code = 200, message = "Returns the status of the job.", response = SparkJobResponse.class), @ApiResponse(code = 404, message = "The job does not exist.", response = RestResponseStatus.class), @ApiResponse(code = 500, message = "There was a problem accessing the data.", response = SparkJobResponse.class) })
public Response getJobResult(@PathParam("job") final String id) {
    try {
        final SparkJob job = transformService.getSparkJob(id);
        final SparkJobResponse response = new SparkJobResponse();
        response.setId(job.getGroupId());
        if (job.isDone()) {
            final SparkJobResult result = job.get();
            response.setResult(result);
            response.setStatus(SparkJobResponse.Status.SUCCESS);
        } else {
            response.setStatus(SparkJobResponse.Status.PENDING);
        }
        return Response.ok(response).build();
    } catch (final IllegalArgumentException e) {
        throw new NotFoundException(getMessage("job.not-found"));
    } catch (final Exception e) {
        throw new InternalServerErrorException(e);
    }
}
Also used : SparkJobResponse(com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResponse) SparkJob(com.thinkbiganalytics.spark.metadata.SparkJob) SparkJobResult(com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResult) NotFoundException(javax.ws.rs.NotFoundException) InternalServerErrorException(javax.ws.rs.InternalServerErrorException) NoSuchMessageException(org.springframework.context.NoSuchMessageException) BadRequestException(javax.ws.rs.BadRequestException) ScriptException(javax.script.ScriptException) InternalServerErrorException(javax.ws.rs.InternalServerErrorException) NotFoundException(javax.ws.rs.NotFoundException) Path(javax.ws.rs.Path) Produces(javax.ws.rs.Produces) GET(javax.ws.rs.GET) ApiOperation(io.swagger.annotations.ApiOperation) ApiResponses(io.swagger.annotations.ApiResponses)

Aggregations

SparkJobResult (com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResult)4 SparkJobResponse (com.thinkbiganalytics.kylo.spark.rest.model.job.SparkJobResponse)3 SparkJob (com.thinkbiganalytics.spark.metadata.SparkJob)1 ApiOperation (io.swagger.annotations.ApiOperation)1 ApiResponses (io.swagger.annotations.ApiResponses)1 Nonnull (javax.annotation.Nonnull)1 Nullable (javax.annotation.Nullable)1 ScriptException (javax.script.ScriptException)1 BadRequestException (javax.ws.rs.BadRequestException)1 GET (javax.ws.rs.GET)1 InternalServerErrorException (javax.ws.rs.InternalServerErrorException)1 NotFoundException (javax.ws.rs.NotFoundException)1 Path (javax.ws.rs.Path)1 Produces (javax.ws.rs.Produces)1 Test (org.junit.Test)1 NoSuchMessageException (org.springframework.context.NoSuchMessageException)1