Search in sources :

Example 16 with ProgramNotFoundException

use of io.cdap.cdap.common.ProgramNotFoundException in project cdap by cdapio.

the class DefaultStore method getRuns.

@Override
public List<ProgramHistory> getRuns(Collection<ProgramId> programs, ProgramRunStatus status, long startTime, long endTime, int limitPerProgram) {
    return TransactionRunners.run(transactionRunner, context -> {
        List<ProgramHistory> result = new ArrayList<>(programs.size());
        AppMetadataStore appMetadataStore = getAppMetadataStore(context);
        Set<ProgramId> existingPrograms = appMetadataStore.filterProgramsExistence(programs);
        for (ProgramId programId : programs) {
            if (!existingPrograms.contains(programId)) {
                result.add(new ProgramHistory(programId, Collections.emptyList(), new ProgramNotFoundException(programId)));
                continue;
            }
            List<RunRecord> runs = appMetadataStore.getRuns(programId, status, startTime, endTime, limitPerProgram, null).values().stream().map(record -> RunRecord.builder(record).build()).collect(Collectors.toList());
            result.add(new ProgramHistory(programId, runs, null));
        }
        return result;
    });
}
Also used : TransactionRunners(io.cdap.cdap.spi.data.transaction.TransactionRunners) NamespaceId(io.cdap.cdap.proto.id.NamespaceId) Inject(com.google.inject.Inject) LoggerFactory(org.slf4j.LoggerFactory) WorkflowNode(io.cdap.cdap.api.workflow.WorkflowNode) ScanApplicationsRequest(io.cdap.cdap.app.store.ScanApplicationsRequest) DatasetId(io.cdap.cdap.proto.id.DatasetId) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Map(java.util.Map) RunId(org.apache.twill.api.RunId) ApplicationNotFoundException(io.cdap.cdap.common.ApplicationNotFoundException) ProgramHistory(io.cdap.cdap.proto.ProgramHistory) ForwardingApplicationSpecification(io.cdap.cdap.internal.app.ForwardingApplicationSpecification) SortOrder(io.cdap.cdap.spi.data.SortOrder) BasicThrowable(io.cdap.cdap.proto.BasicThrowable) StoreDefinition(io.cdap.cdap.store.StoreDefinition) ImmutableMap(com.google.common.collect.ImmutableMap) Predicate(java.util.function.Predicate) Collection(java.util.Collection) ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) Table(io.cdap.cdap.api.dataset.table.Table) Set(java.util.Set) ProgramRunStatus(io.cdap.cdap.proto.ProgramRunStatus) StructuredTableContext(io.cdap.cdap.spi.data.StructuredTableContext) Collectors(java.util.stream.Collectors) List(java.util.List) TransactionRunner(io.cdap.cdap.spi.data.transaction.TransactionRunner) ApplicationId(io.cdap.cdap.proto.id.ApplicationId) ServiceSpecification(io.cdap.cdap.api.service.ServiceSpecification) NotFoundException(io.cdap.cdap.common.NotFoundException) RunRecord(io.cdap.cdap.proto.RunRecord) WorkflowId(io.cdap.cdap.proto.id.WorkflowId) WorkflowSpecification(io.cdap.cdap.api.workflow.WorkflowSpecification) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) UnauthorizedException(io.cdap.cdap.security.spi.authorization.UnauthorizedException) Deque(java.util.Deque) ProgramType(io.cdap.cdap.proto.ProgramType) AtomicReference(java.util.concurrent.atomic.AtomicReference) ArrayList(java.util.ArrayList) DatasetProperties(io.cdap.cdap.api.dataset.DatasetProperties) HashSet(java.util.HashSet) WorkflowToken(io.cdap.cdap.api.workflow.WorkflowToken) ProgramRunId(io.cdap.cdap.proto.id.ProgramRunId) Lists(com.google.common.collect.Lists) BiConsumer(java.util.function.BiConsumer) WorkflowActionNode(io.cdap.cdap.api.workflow.WorkflowActionNode) NoSuchElementException(java.util.NoSuchElementException) Nullable(javax.annotation.Nullable) WorkflowNodeStateDetail(io.cdap.cdap.proto.WorkflowNodeStateDetail) Logger(org.slf4j.Logger) WorkflowStatistics(io.cdap.cdap.proto.WorkflowStatistics) DatasetFramework(io.cdap.cdap.data2.dataset2.DatasetFramework) ProgramId(io.cdap.cdap.proto.id.ProgramId) ProgramDescriptor(io.cdap.cdap.app.program.ProgramDescriptor) DatasetManagementException(io.cdap.cdap.api.dataset.DatasetManagementException) IOException(java.io.IOException) WorkerSpecification(io.cdap.cdap.api.worker.WorkerSpecification) Maps(com.google.common.collect.Maps) TableNotFoundException(io.cdap.cdap.spi.data.TableNotFoundException) Store(io.cdap.cdap.app.store.Store) Consumer(java.util.function.Consumer) MapDifference(com.google.common.collect.MapDifference) ProgramNotFoundException(io.cdap.cdap.common.ProgramNotFoundException) ProgramSpecification(io.cdap.cdap.api.ProgramSpecification) RunCountResult(io.cdap.cdap.proto.RunCountResult) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ArrayDeque(java.util.ArrayDeque) Collections(java.util.Collections) ArtifactId(io.cdap.cdap.api.artifact.ArtifactId) RunRecord(io.cdap.cdap.proto.RunRecord) ArrayList(java.util.ArrayList) ProgramId(io.cdap.cdap.proto.id.ProgramId) ProgramNotFoundException(io.cdap.cdap.common.ProgramNotFoundException) ProgramHistory(io.cdap.cdap.proto.ProgramHistory)

Example 17 with ProgramNotFoundException

use of io.cdap.cdap.common.ProgramNotFoundException in project cdap by cdapio.

the class LocalScheduleFetcher method list.

@Override
public List<ScheduleDetail> list(ProgramId programId) throws IOException, ProgramNotFoundException {
    Predicate<ProgramScheduleRecord> predicate = (record) -> true;
    Collection<ProgramScheduleRecord> schedules = null;
    try {
        schedules = programScheduleService.list(programId, predicate);
    } catch (Exception e) {
        Throwables.propagateIfPossible(e.getCause(), IOException.class, ProgramNotFoundException.class);
        throw new IOException(e);
    }
    return schedules.stream().map(ProgramScheduleRecord::toScheduleDetail).collect(Collectors.toList());
}
Also used : Predicate(java.util.function.Predicate) Inject(com.google.inject.Inject) ProgramScheduleService(io.cdap.cdap.scheduler.ProgramScheduleService) Collection(java.util.Collection) ScheduleNotFoundException(io.cdap.cdap.internal.app.runtime.schedule.ScheduleNotFoundException) ProgramId(io.cdap.cdap.proto.id.ProgramId) Throwables(com.google.common.base.Throwables) IOException(java.io.IOException) ProgramScheduleRecord(io.cdap.cdap.internal.app.runtime.schedule.ProgramScheduleRecord) Collectors(java.util.stream.Collectors) ProgramNotFoundException(io.cdap.cdap.common.ProgramNotFoundException) List(java.util.List) ScheduleId(io.cdap.cdap.proto.id.ScheduleId) NotFoundException(io.cdap.cdap.common.NotFoundException) ScheduleDetail(io.cdap.cdap.proto.ScheduleDetail) ProgramScheduleRecord(io.cdap.cdap.internal.app.runtime.schedule.ProgramScheduleRecord) IOException(java.io.IOException) ProgramNotFoundException(io.cdap.cdap.common.ProgramNotFoundException) ScheduleNotFoundException(io.cdap.cdap.internal.app.runtime.schedule.ScheduleNotFoundException) IOException(java.io.IOException) ProgramNotFoundException(io.cdap.cdap.common.ProgramNotFoundException) NotFoundException(io.cdap.cdap.common.NotFoundException)

Example 18 with ProgramNotFoundException

use of io.cdap.cdap.common.ProgramNotFoundException in project cdap by cdapio.

the class ProgramExistenceVerifier method ensureExists.

@Override
public void ensureExists(ProgramId programId) throws ApplicationNotFoundException, ProgramNotFoundException {
    ApplicationId appId = programId.getParent();
    ApplicationSpecification appSpec = store.getApplication(appId);
    if (appSpec == null) {
        throw new ApplicationNotFoundException(appId);
    }
    ProgramType programType = programId.getType();
    Set<String> programNames = null;
    if (programType == ProgramType.MAPREDUCE && appSpec.getMapReduce() != null) {
        programNames = appSpec.getMapReduce().keySet();
    } else if (programType == ProgramType.WORKFLOW && appSpec.getWorkflows() != null) {
        programNames = appSpec.getWorkflows().keySet();
    } else if (programType == ProgramType.SERVICE && appSpec.getServices() != null) {
        programNames = appSpec.getServices().keySet();
    } else if (programType == ProgramType.SPARK && appSpec.getSpark() != null) {
        programNames = appSpec.getSpark().keySet();
    } else if (programType == ProgramType.WORKER && appSpec.getWorkers() != null) {
        programNames = appSpec.getWorkers().keySet();
    }
    if (programNames != null) {
        if (programNames.contains(programId.getProgram())) {
            // is valid.
            return;
        }
    }
    throw new ProgramNotFoundException(programId);
}
Also used : ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) ApplicationNotFoundException(io.cdap.cdap.common.ApplicationNotFoundException) ProgramType(io.cdap.cdap.proto.ProgramType) ApplicationId(io.cdap.cdap.proto.id.ApplicationId) ProgramNotFoundException(io.cdap.cdap.common.ProgramNotFoundException)

Example 19 with ProgramNotFoundException

use of io.cdap.cdap.common.ProgramNotFoundException in project cdap by caskdata.

the class ProgramClient method stopAll.

/**
 * Stops all currently running programs.
 */
public void stopAll(NamespaceId namespace) throws IOException, UnauthenticatedException, InterruptedException, TimeoutException, UnauthorizedException, ApplicationNotFoundException, BadRequestException {
    List<ApplicationRecord> allApps = applicationClient.list(namespace);
    for (ApplicationRecord applicationRecord : allApps) {
        ApplicationId appId = new ApplicationId(namespace.getNamespace(), applicationRecord.getName(), applicationRecord.getAppVersion());
        List<ProgramRecord> programRecords = applicationClient.listPrograms(appId);
        for (ProgramRecord programRecord : programRecords) {
            try {
                ProgramId program = appId.program(programRecord.getType(), programRecord.getName());
                String status = this.getStatus(program);
                if (!status.equals("STOPPED")) {
                    try {
                        this.stop(program);
                    } catch (IOException ioe) {
                        // ProgramClient#stop calls RestClient, which throws an IOException if the HTTP response code is 400,
                        // which can be due to the program already being stopped when calling stop on it.
                        // Most likely, there was a race condition that the program stopped between the time we checked its
                        // status and calling the stop method.
                        LOG.warn("Program {} is already stopped, proceeding even though the following exception is raised.", program, ioe);
                    }
                    this.waitForStatus(program, ProgramStatus.STOPPED, 60, TimeUnit.SECONDS);
                }
            } catch (ProgramNotFoundException e) {
            // IGNORE
            }
        }
    }
}
Also used : ProgramRecord(io.cdap.cdap.proto.ProgramRecord) IOException(java.io.IOException) ApplicationId(io.cdap.cdap.proto.id.ApplicationId) ProgramId(io.cdap.cdap.proto.id.ProgramId) ProgramNotFoundException(io.cdap.cdap.common.ProgramNotFoundException) ApplicationRecord(io.cdap.cdap.proto.ApplicationRecord)

Example 20 with ProgramNotFoundException

use of io.cdap.cdap.common.ProgramNotFoundException in project cdap by caskdata.

the class ProgramClient method getProgramLogs.

/**
 * Gets the logs of a program.
 *
 * @param program the program
 * @param start start time of the time range of desired logs
 * @param stop end time of the time range of desired logs
 * @return the logs of the program
 * @throws IOException if a network error occurred
 * @throws NotFoundException if the application or program could not be found
 * @throws UnauthenticatedException if the request is not authorized successfully in the gateway server
 */
public String getProgramLogs(ProgramId program, long start, long stop) throws IOException, NotFoundException, UnauthenticatedException, UnauthorizedException {
    String path = String.format("apps/%s/%s/%s/logs?start=%d&stop=%d&escape=false", program.getApplication(), program.getType().getCategoryName(), program.getProgram(), start, stop);
    URL url = config.resolveNamespacedURLV3(program.getNamespaceId(), path);
    HttpResponse response = restClient.execute(HttpMethod.GET, url, config.getAccessToken());
    if (response.getResponseCode() == HttpURLConnection.HTTP_NOT_FOUND) {
        throw new ProgramNotFoundException(program);
    }
    return new String(response.getResponseBody(), Charsets.UTF_8);
}
Also used : HttpResponse(io.cdap.common.http.HttpResponse) ProgramNotFoundException(io.cdap.cdap.common.ProgramNotFoundException) URL(java.net.URL)

Aggregations

ProgramNotFoundException (io.cdap.cdap.common.ProgramNotFoundException)34 HttpResponse (io.cdap.common.http.HttpResponse)22 URL (java.net.URL)20 NotFoundException (io.cdap.cdap.common.NotFoundException)10 ApplicationId (io.cdap.cdap.proto.id.ApplicationId)10 ProgramId (io.cdap.cdap.proto.id.ProgramId)10 ApplicationSpecification (io.cdap.cdap.api.app.ApplicationSpecification)8 ApplicationNotFoundException (io.cdap.cdap.common.ApplicationNotFoundException)8 HttpRequest (io.cdap.common.http.HttpRequest)8 WorkflowSpecification (io.cdap.cdap.api.workflow.WorkflowSpecification)6 ProgramRunId (io.cdap.cdap.proto.id.ProgramRunId)6 IOException (java.io.IOException)6 List (java.util.List)6 Inject (com.google.inject.Inject)4 InstanceNotFoundException (io.cdap.cdap.api.dataset.InstanceNotFoundException)4 ProgramType (io.cdap.cdap.proto.ProgramType)4 WorkflowNodeStateDetail (io.cdap.cdap.proto.WorkflowNodeStateDetail)4 Collection (java.util.Collection)4 Predicate (java.util.function.Predicate)4 Collectors (java.util.stream.Collectors)4