Search in sources :

Example 1 with BatchProgramStatus

use of io.cdap.cdap.proto.BatchProgramStatus in project cdap by caskdata.

the class ProgramClientTestRun method testBatchProgramCalls.

@Test
public void testBatchProgramCalls() throws Exception {
    final NamespaceId namespace = NamespaceId.DEFAULT;
    final ApplicationId appId = namespace.app(FakeApp.NAME);
    BatchProgram pingService = new BatchProgram(FakeApp.NAME, ProgramType.SERVICE, PingService.NAME);
    BatchProgram writerService = new BatchProgram(FakeApp.NAME, ProgramType.SERVICE, DatasetWriterService.NAME);
    BatchProgram missing = new BatchProgram(FakeApp.NAME, ProgramType.SERVICE, "not" + PingService.NAME);
    appClient.deploy(namespace, createAppJarFile(FakeApp.class));
    try {
        // make a batch call to start multiple programs, one of which does not exist
        List<BatchProgramStart> programStarts = ImmutableList.of(new BatchProgramStart(pingService), new BatchProgramStart(writerService), new BatchProgramStart(missing));
        List<BatchProgramResult> results = programClient.start(namespace, programStarts);
        // check that we got a 200 for programs that exist, and a 404 for the one that doesn't
        for (BatchProgramResult result : results) {
            if (missing.getProgramId().equals(result.getProgramId())) {
                Assert.assertEquals(404, result.getStatusCode());
            } else {
                Assert.assertEquals(200, result.getStatusCode());
            }
        }
        // wait for all programs to be in RUNNING status
        assertProgramRuns(programClient, namespace.app(pingService.getAppId()).service(pingService.getProgramId()), ProgramRunStatus.RUNNING, 1, 10);
        assertProgramRuns(programClient, namespace.app(writerService.getAppId()).service(writerService.getProgramId()), ProgramRunStatus.RUNNING, 1, 10);
        // make a batch call for status of programs, one of which does not exist
        List<BatchProgram> programs = ImmutableList.of(pingService, writerService, missing);
        List<BatchProgramStatus> statusList = programClient.getStatus(namespace, programs);
        // check status is running for programs that exist, and that we get a 404 for the one that doesn't
        for (BatchProgramStatus status : statusList) {
            if (missing.getProgramId().equals(status.getProgramId())) {
                Assert.assertEquals(404, status.getStatusCode());
            } else {
                Assert.assertEquals(200, status.getStatusCode());
                Assert.assertEquals("RUNNING", status.getStatus());
            }
        }
        // make a batch call to stop programs, one of which does not exist
        results = programClient.stop(namespace, programs);
        // check that we got a 200 for programs that exist, and a 404 for the one that doesn't
        for (BatchProgramResult result : results) {
            if (missing.getProgramId().equals(result.getProgramId())) {
                Assert.assertEquals(404, result.getStatusCode());
            } else {
                Assert.assertEquals(200, result.getStatusCode());
            }
        }
        // check programs are in stopped state
        final List<BatchProgram> stoppedPrograms = ImmutableList.of(pingService, writerService);
        Tasks.waitFor(true, new Callable<Boolean>() {

            @Override
            public Boolean call() throws Exception {
                List<BatchProgramStatus> statusList = programClient.getStatus(namespace, stoppedPrograms);
                for (BatchProgramStatus status : statusList) {
                    if (status.getStatusCode() != 200) {
                        return false;
                    }
                    if (status.getStatus().equals("RUNNING")) {
                        return false;
                    }
                }
                return true;
            }
        }, 10, TimeUnit.SECONDS);
    } finally {
        try {
            appClient.delete(appId);
        } catch (Exception e) {
            LOG.error("Error deleting app {} during test cleanup.", appId, e);
        }
    }
}
Also used : FakeApp(io.cdap.cdap.client.app.FakeApp) BatchProgramResult(io.cdap.cdap.proto.BatchProgramResult) BatchProgramStatus(io.cdap.cdap.proto.BatchProgramStatus) BatchProgramStart(io.cdap.cdap.proto.BatchProgramStart) List(java.util.List) ImmutableList(com.google.common.collect.ImmutableList) NamespaceId(io.cdap.cdap.proto.id.NamespaceId) ApplicationId(io.cdap.cdap.proto.id.ApplicationId) BatchProgram(io.cdap.cdap.proto.BatchProgram) Test(org.junit.Test)

Example 2 with BatchProgramStatus

use of io.cdap.cdap.proto.BatchProgramStatus in project cdap by caskdata.

the class ProgramLifecycleHttpHandler method getStatuses.

/**
 * Returns the status for all programs that are passed into the data. The data is an array of JSON objects
 * where each object must contain the following three elements: appId, programType, and programId
 * (flow name, service name, etc.).
 * <p>
 * Example input:
 * <pre><code>
 * [{"appId": "App1", "programType": "Service", "programId": "Service1"},
 * {"appId": "App1", "programType": "Mapreduce", "programId": "MapReduce2"}]
 * </code></pre>
 * </p><p>
 * The response will be an array of JsonObjects each of which will contain the three input parameters
 * as well as 2 fields, "status" which maps to the status of the program and "statusCode" which maps to the
 * status code for the data in that JsonObjects.
 * </p><p>
 * If an error occurs in the input (for the example above, App2 does not exist), then all JsonObjects for which the
 * parameters have a valid status will have the status field but all JsonObjects for which the parameters do not have
 * a valid status will have an error message and statusCode.
 * </p><p>
 * For example, if there is no App2 in the data above, then the response would be 200 OK with following possible data:
 * </p>
 * <pre><code>
 * [{"appId": "App1", "programType": "Service", "programId": "Service1", "statusCode": 200, "status": "RUNNING"},
 * {"appId": "App1", "programType": "Mapreduce", "programId": "Mapreduce2", "statusCode": 200, "status": "STOPPED"}]
 * </code></pre>
 */
@POST
@Path("/status")
@AuditPolicy(AuditDetail.REQUEST_BODY)
public void getStatuses(FullHttpRequest request, HttpResponder responder, @PathParam("namespace-id") String namespaceId) throws Exception {
    List<BatchProgram> batchPrograms = validateAndGetBatchInput(request, BATCH_PROGRAMS_TYPE);
    List<ProgramId> programs = batchPrograms.stream().map(p -> new ProgramId(namespaceId, p.getAppId(), p.getProgramType(), p.getProgramId())).collect(Collectors.toList());
    Map<ProgramId, ProgramStatus> statuses = lifecycleService.getProgramStatuses(programs);
    List<BatchProgramStatus> result = new ArrayList<>(programs.size());
    for (BatchProgram program : batchPrograms) {
        ProgramId programId = new ProgramId(namespaceId, program.getAppId(), program.getProgramType(), program.getProgramId());
        ProgramStatus status = statuses.get(programId);
        if (status == null) {
            result.add(new BatchProgramStatus(program, HttpResponseStatus.NOT_FOUND.code(), new NotFoundException(programId).getMessage(), null));
        } else {
            result.add(new BatchProgramStatus(program, HttpResponseStatus.OK.code(), null, status.name()));
        }
    }
    responder.sendJson(HttpResponseStatus.OK, GSON.toJson(result));
}
Also used : BatchProgramSchedule(io.cdap.cdap.proto.BatchProgramSchedule) AuditDetail(io.cdap.cdap.common.security.AuditDetail) RunRecordDetail(io.cdap.cdap.internal.app.store.RunRecordDetail) BatchProgramResult(io.cdap.cdap.proto.BatchProgramResult) TypeToken(com.google.gson.reflect.TypeToken) MRJobInfoFetcher(io.cdap.cdap.app.mapreduce.MRJobInfoFetcher) MRJobInfo(io.cdap.cdap.proto.MRJobInfo) GsonBuilder(com.google.gson.GsonBuilder) ScheduledRuntime(io.cdap.cdap.proto.ScheduledRuntime) ProgramScheduleStatus(io.cdap.cdap.internal.app.runtime.schedule.ProgramScheduleStatus) ScheduleId(io.cdap.cdap.proto.id.ScheduleId) Map(java.util.Map) ProgramStatus(io.cdap.cdap.proto.ProgramStatus) ScheduleDetail(io.cdap.cdap.proto.ScheduleDetail) EnumSet(java.util.EnumSet) HttpRequest(io.netty.handler.codec.http.HttpRequest) Set(java.util.Set) Reader(java.io.Reader) Constraint(io.cdap.cdap.internal.schedule.constraint.Constraint) ProgramRunStatus(io.cdap.cdap.proto.ProgramRunStatus) ProgramScheduleRecord(io.cdap.cdap.internal.app.runtime.schedule.ProgramScheduleRecord) FullHttpRequest(io.netty.handler.codec.http.FullHttpRequest) StandardCharsets(java.nio.charset.StandardCharsets) Id(io.cdap.cdap.common.id.Id) ApplicationSpecificationAdapter(io.cdap.cdap.internal.app.ApplicationSpecificationAdapter) TriggerCodec(io.cdap.cdap.internal.app.runtime.schedule.trigger.TriggerCodec) ApplicationId(io.cdap.cdap.proto.id.ApplicationId) Joiner(com.google.common.base.Joiner) Singleton(com.google.inject.Singleton) RunRecord(io.cdap.cdap.proto.RunRecord) GET(javax.ws.rs.GET) SatisfiableTrigger(io.cdap.cdap.internal.app.runtime.schedule.trigger.SatisfiableTrigger) UnauthorizedException(io.cdap.cdap.security.spi.authorization.UnauthorizedException) ArrayList(java.util.ArrayList) NamespaceNotFoundException(io.cdap.cdap.common.NamespaceNotFoundException) ProgramRunId(io.cdap.cdap.proto.id.ProgramRunId) DiscoveryServiceClient(org.apache.twill.discovery.DiscoveryServiceClient) Nullable(javax.annotation.Nullable) Charsets(com.google.common.base.Charsets) AuditPolicy(io.cdap.cdap.common.security.AuditPolicy) BatchRunnableInstances(io.cdap.cdap.proto.BatchRunnableInstances) ProgramLiveInfo(io.cdap.cdap.proto.ProgramLiveInfo) ProgramLifecycleService(io.cdap.cdap.internal.app.services.ProgramLifecycleService) Throwables(com.google.common.base.Throwables) IOException(java.io.IOException) ConflictException(io.cdap.cdap.common.ConflictException) NotImplementedException(io.cdap.cdap.common.NotImplementedException) ServiceInstances(io.cdap.cdap.proto.ServiceInstances) InputStreamReader(java.io.InputStreamReader) ProgramRuntimeService(io.cdap.cdap.app.runtime.ProgramRuntimeService) Futures(com.google.common.util.concurrent.Futures) ProgramSpecification(io.cdap.cdap.api.ProgramSpecification) Schedulers(io.cdap.cdap.internal.app.runtime.schedule.store.Schedulers) RunCountResult(io.cdap.cdap.proto.RunCountResult) BatchProgramStatus(io.cdap.cdap.proto.BatchProgramStatus) JsonObject(com.google.gson.JsonObject) NamespaceQueryAdmin(io.cdap.cdap.common.namespace.NamespaceQueryAdmin) RandomEndpointStrategy(io.cdap.cdap.common.discovery.RandomEndpointStrategy) NamespaceId(io.cdap.cdap.proto.id.NamespaceId) Inject(com.google.inject.Inject) ProgramScheduleService(io.cdap.cdap.scheduler.ProgramScheduleService) LoggerFactory(org.slf4j.LoggerFactory) Path(javax.ws.rs.Path) ServiceUnavailableException(io.cdap.cdap.common.ServiceUnavailableException) EndpointStrategy(io.cdap.cdap.common.discovery.EndpointStrategy) QueryParam(javax.ws.rs.QueryParam) Gson(com.google.gson.Gson) DefaultValue(javax.ws.rs.DefaultValue) Objects(com.google.common.base.Objects) ProgramHistory(io.cdap.cdap.proto.ProgramHistory) ConstraintCodec(io.cdap.cdap.internal.app.runtime.schedule.constraint.ConstraintCodec) DELETE(javax.ws.rs.DELETE) Containers(io.cdap.cdap.proto.Containers) Function(com.google.common.base.Function) ImmutableMap(com.google.common.collect.ImmutableMap) Predicate(java.util.function.Predicate) Collection(java.util.Collection) ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) BatchProgramStart(io.cdap.cdap.proto.BatchProgramStart) BatchRunnable(io.cdap.cdap.proto.BatchRunnable) HttpResponseStatus(io.netty.handler.codec.http.HttpResponseStatus) Collectors(java.util.stream.Collectors) ProgramStatusTrigger(io.cdap.cdap.internal.app.runtime.schedule.trigger.ProgramStatusTrigger) List(java.util.List) Type(java.lang.reflect.Type) CaseInsensitiveEnumTypeAdapterFactory(io.cdap.cdap.common.io.CaseInsensitiveEnumTypeAdapterFactory) Constants(io.cdap.cdap.common.conf.Constants) NotFoundException(io.cdap.cdap.common.NotFoundException) PathParam(javax.ws.rs.PathParam) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) BatchProgramHistory(io.cdap.cdap.proto.BatchProgramHistory) BatchProgramCount(io.cdap.cdap.proto.BatchProgramCount) HashMap(java.util.HashMap) ProgramType(io.cdap.cdap.proto.ProgramType) JsonElement(com.google.gson.JsonElement) NotRunningProgramLiveInfo(io.cdap.cdap.proto.NotRunningProgramLiveInfo) HashSet(java.util.HashSet) Trigger(io.cdap.cdap.api.schedule.Trigger) BatchProgram(io.cdap.cdap.proto.BatchProgram) Instances(io.cdap.cdap.proto.Instances) ByteBufInputStream(io.netty.buffer.ByteBufInputStream) AbstractAppFabricHttpHandler(io.cdap.cdap.gateway.handlers.util.AbstractAppFabricHttpHandler) ProtoTrigger(io.cdap.cdap.proto.ProtoTrigger) Logger(org.slf4j.Logger) POST(javax.ws.rs.POST) HttpResponder(io.cdap.http.HttpResponder) JsonSyntaxException(com.google.gson.JsonSyntaxException) SchedulerException(io.cdap.cdap.internal.app.runtime.schedule.SchedulerException) ProgramId(io.cdap.cdap.proto.id.ProgramId) BadRequestException(io.cdap.cdap.common.BadRequestException) ProgramSchedule(io.cdap.cdap.internal.app.runtime.schedule.ProgramSchedule) Store(io.cdap.cdap.app.store.Store) TimeUnit(java.util.concurrent.TimeUnit) ServiceDiscoverable(io.cdap.cdap.common.service.ServiceDiscoverable) PUT(javax.ws.rs.PUT) Collections(java.util.Collections) BatchProgramStatus(io.cdap.cdap.proto.BatchProgramStatus) ArrayList(java.util.ArrayList) NamespaceNotFoundException(io.cdap.cdap.common.NamespaceNotFoundException) NotFoundException(io.cdap.cdap.common.NotFoundException) ProgramId(io.cdap.cdap.proto.id.ProgramId) BatchProgram(io.cdap.cdap.proto.BatchProgram) ProgramStatus(io.cdap.cdap.proto.ProgramStatus) BatchProgramStatus(io.cdap.cdap.proto.BatchProgramStatus) Path(javax.ws.rs.Path) AuditPolicy(io.cdap.cdap.common.security.AuditPolicy) POST(javax.ws.rs.POST)

Example 3 with BatchProgramStatus

use of io.cdap.cdap.proto.BatchProgramStatus in project cdap by caskdata.

the class StatusProgramsCommand method runBatchCommand.

@Override
protected void runBatchCommand(PrintStream printStream, Args<BatchProgram> args) throws Exception {
    List<BatchProgramStatus> results = programClient.getStatus(args.appId.getParent(), args.programs);
    Table table = Table.builder().setHeader("name", "type", "status", "error").setRows(results, new RowMaker<BatchProgramStatus>() {

        @Override
        public List<?> makeRow(BatchProgramStatus result) {
            return Lists.newArrayList(result.getProgramId(), result.getProgramType(), result.getStatus(), result.getError());
        }
    }).build();
    cliConfig.getTableRenderer().render(cliConfig, printStream, table);
}
Also used : BatchProgramStatus(io.cdap.cdap.proto.BatchProgramStatus) Table(io.cdap.cdap.cli.util.table.Table) RowMaker(io.cdap.cdap.cli.util.RowMaker)

Example 4 with BatchProgramStatus

use of io.cdap.cdap.proto.BatchProgramStatus in project cdap by caskdata.

the class ProgramClient method getStatus.

/**
 * Gets the status of multiple programs.
 *
 * @param namespace the namespace of the programs
 * @param programs the list of programs to get status for
 * @return the status of each program
 */
public List<BatchProgramStatus> getStatus(NamespaceId namespace, List<BatchProgram> programs) throws IOException, UnauthenticatedException, UnauthorizedException {
    URL url = config.resolveNamespacedURLV3(namespace, "status");
    HttpRequest request = HttpRequest.post(url).withBody(GSON.toJson(programs)).build();
    HttpResponse response = restClient.execute(request, config.getAccessToken());
    return ObjectResponse.<List<BatchProgramStatus>>fromJsonBody(response, BATCH_STATUS_RESPONSE_TYPE, GSON).getResponseObject();
}
Also used : HttpRequest(io.cdap.common.http.HttpRequest) BatchProgramStatus(io.cdap.cdap.proto.BatchProgramStatus) HttpResponse(io.cdap.common.http.HttpResponse) URL(java.net.URL)

Aggregations

BatchProgramStatus (io.cdap.cdap.proto.BatchProgramStatus)4 BatchProgram (io.cdap.cdap.proto.BatchProgram)2 BatchProgramResult (io.cdap.cdap.proto.BatchProgramResult)2 BatchProgramStart (io.cdap.cdap.proto.BatchProgramStart)2 ApplicationId (io.cdap.cdap.proto.id.ApplicationId)2 NamespaceId (io.cdap.cdap.proto.id.NamespaceId)2 List (java.util.List)2 Charsets (com.google.common.base.Charsets)1 Function (com.google.common.base.Function)1 Joiner (com.google.common.base.Joiner)1 Objects (com.google.common.base.Objects)1 Throwables (com.google.common.base.Throwables)1 ImmutableList (com.google.common.collect.ImmutableList)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 Futures (com.google.common.util.concurrent.Futures)1 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)1 Gson (com.google.gson.Gson)1 GsonBuilder (com.google.gson.GsonBuilder)1 JsonElement (com.google.gson.JsonElement)1 JsonObject (com.google.gson.JsonObject)1