Search in sources :

Example 1 with StairwayException

use of bio.terra.stairway.exception.StairwayException in project jade-data-repo by DataBiosphere.

the class JobService method initialize.

/**
 * This method is called from StartupInitializer as part of the sequence of migrating databases
 * and recovering any jobs; i.e., Stairway flights. It lives in this class so that JobService encapsulates
 * all Stairway interaction.
 */
public void initialize() {
    try {
        List<String> recordedStairways;
        boolean weMigrated = false;
        try {
            weMigrated = migrate.migrateDatabase();
            // Initialize stairway - only do the stairway migration if we did the data repo migration
            recordedStairways = stairway.initialize(stairwayJdbcConfiguration.getDataSource(), (weMigrated && migrateConfiguration.getDropAllOnStart()), (weMigrated && migrateConfiguration.getUpdateAllOnStart()));
        } finally {
            if (weMigrated) {
                migrate.releaseMigrateLock();
            }
        }
        // Order is important here. There are two concerns we need to handle:
        // 1. We need to avoid a window where a running pod could get onto the Stairway list, but not be
        // on the pod list. That is why we get the recorded list from Stairway *before* we read the Kubernetes
        // pod list.
        // 2. We want to clean up pods that fail, so we start the kubePodListener as early as possible so we
        // detect pods that get deleted. The Stairway recovery method called by kubePodListener works once
        // Stairway initialization is done, so it is safe to start the listener before we have called
        // Stairway recoveryAndStart
        kubeService.startPodListener(stairway);
        // Lookup all of the stairway instances we know about
        Set<String> existingStairways = kubeService.getApiPodList();
        List<String> obsoleteStairways = new LinkedList<>();
        // Any instances that stairway knows about, but we cannot see are obsolete.
        for (String recordedStairway : recordedStairways) {
            if (!existingStairways.contains(recordedStairway)) {
                obsoleteStairways.add(recordedStairway);
            }
        }
        // Recover and start stairway - step 3 of the stairway startup sequence
        stairway.recoverAndStart(obsoleteStairways);
    } catch (StairwayException stairwayEx) {
        throw new InternalStairwayException("Stairway initialization failed", stairwayEx);
    } catch (InterruptedException ex) {
        throw new JobServiceStartupException("Stairway startup process interrupted", ex);
    }
}
Also used : JobServiceStartupException(bio.terra.service.job.exception.JobServiceStartupException) StairwayException(bio.terra.stairway.exception.StairwayException) InternalStairwayException(bio.terra.service.job.exception.InternalStairwayException) LinkedList(java.util.LinkedList) InternalStairwayException(bio.terra.service.job.exception.InternalStairwayException)

Example 2 with StairwayException

use of bio.terra.stairway.exception.StairwayException in project jade-data-repo by DataBiosphere.

the class JobService method retrieveJobResultWorker.

private <T> JobResultWithStatus<T> retrieveJobResultWorker(String jobId, Class<T> resultClass) throws StairwayException, InterruptedException {
    FlightState flightState = stairway.getFlightState(jobId);
    FlightMap resultMap = flightState.getResultMap().orElse(null);
    if (resultMap == null) {
        throw new InvalidResultStateException("No result map returned from flight");
    }
    switch(flightState.getFlightStatus()) {
        case FATAL:
        case ERROR:
            if (flightState.getException().isPresent()) {
                Exception exception = flightState.getException().get();
                if (exception instanceof RuntimeException) {
                    throw (RuntimeException) exception;
                } else {
                    throw new JobResponseException("wrap non-runtime exception", exception);
                }
            }
            throw new InvalidResultStateException("Failed operation with no exception reported");
        case SUCCESS:
            HttpStatus statusCode = resultMap.get(JobMapKeys.STATUS_CODE.getKeyName(), HttpStatus.class);
            if (statusCode == null) {
                statusCode = HttpStatus.OK;
            }
            return new JobResultWithStatus<T>().statusCode(statusCode).result(resultMap.get(JobMapKeys.RESPONSE.getKeyName(), resultClass));
        case RUNNING:
            throw new JobNotCompleteException("Attempt to retrieve job result before job is complete; job id: " + flightState.getFlightId());
        default:
            throw new InvalidResultStateException("Impossible case reached");
    }
}
Also used : FlightState(bio.terra.stairway.FlightState) InvalidResultStateException(bio.terra.service.job.exception.InvalidResultStateException) HttpStatus(org.springframework.http.HttpStatus) FlightMap(bio.terra.stairway.FlightMap) JobNotCompleteException(bio.terra.service.job.exception.JobNotCompleteException) JobResponseException(bio.terra.service.job.exception.JobResponseException) JobServiceStartupException(bio.terra.service.job.exception.JobServiceStartupException) JobServiceShutdownException(bio.terra.service.job.exception.JobServiceShutdownException) JobNotCompleteException(bio.terra.service.job.exception.JobNotCompleteException) JobUnauthorizedException(bio.terra.service.job.exception.JobUnauthorizedException) InvalidResultStateException(bio.terra.service.job.exception.InvalidResultStateException) StairwayException(bio.terra.stairway.exception.StairwayException) FlightNotFoundException(bio.terra.stairway.exception.FlightNotFoundException) DatabaseOperationException(bio.terra.stairway.exception.DatabaseOperationException) JobNotFoundException(bio.terra.service.job.exception.JobNotFoundException) JobResponseException(bio.terra.service.job.exception.JobResponseException) InternalStairwayException(bio.terra.service.job.exception.InternalStairwayException)

Example 3 with StairwayException

use of bio.terra.stairway.exception.StairwayException in project terra-workspace-manager by DataBiosphere.

the class JobService method enumerateJobs.

public List<ApiJobReport> enumerateJobs(int offset, int limit, AuthenticatedUserRequest userRequest) {
    List<FlightState> flightStateList;
    try {
        FlightFilter filter = new FlightFilter();
        filter.addFilterInputParameter(JobMapKeys.SUBJECT_ID.getKeyName(), FlightFilterOp.EQUAL, userRequest.getSubjectId());
        flightStateList = stairwayComponent.get().getFlights(offset, limit, filter);
    } catch (StairwayException | InterruptedException stairwayEx) {
        throw new InternalStairwayException(stairwayEx);
    }
    List<ApiJobReport> jobReportList = new ArrayList<>();
    for (FlightState flightState : flightStateList) {
        ApiJobReport jobReport = mapFlightStateToApiJobReport(flightState);
        jobReportList.add(jobReport);
    }
    return jobReportList;
}
Also used : FlightState(bio.terra.stairway.FlightState) ApiJobReport(bio.terra.workspace.generated.model.ApiJobReport) ArrayList(java.util.ArrayList) FlightFilter(bio.terra.stairway.FlightFilter) StairwayException(bio.terra.stairway.exception.StairwayException) InternalStairwayException(bio.terra.workspace.service.job.exception.InternalStairwayException) InternalStairwayException(bio.terra.workspace.service.job.exception.InternalStairwayException)

Example 4 with StairwayException

use of bio.terra.stairway.exception.StairwayException in project terra-workspace-manager by DataBiosphere.

the class Alpha1Service method enumerateJobs.

/**
 * List Stairway flights related to a workspace. These inputs are translated into inputs to
 * Stairway's getFlights calls. The resulting flights are translated into enumerated jobs. The
 * jobs are ordered by submit time.
 *
 * @param workspaceId workspace we are listing in
 * @param userRequest authenticated user
 * @param limit max number of jobs to return
 * @param pageToken optional starting place in the result set; start at beginning if missing
 * @param cloudResourceType optional filter by cloud resource type
 * @param stewardshipType optional filter by stewardship type
 * @param resourceName optional filter by resource name
 * @param jobStateFilter optional filter by job state
 * @return POJO containing the results
 */
public EnumeratedJobs enumerateJobs(UUID workspaceId, AuthenticatedUserRequest userRequest, int limit, @Nullable String pageToken, @Nullable WsmResourceFamily cloudResourceType, @Nullable StewardshipType stewardshipType, @Nullable String resourceName, @Nullable JobStateFilter jobStateFilter) {
    features.alpha1EnabledCheck();
    // Readers can see the workspace jobs list
    workspaceService.validateWorkspaceAndAction(userRequest, workspaceId, SamConstants.SamWorkspaceAction.READ);
    FlightEnumeration flightEnumeration;
    try {
        FlightFilter filter = buildFlightFilter(workspaceId, cloudResourceType, stewardshipType, resourceName, jobStateFilter);
        flightEnumeration = stairwayComponent.get().getFlights(pageToken, limit, filter);
    } catch (StairwayException | InterruptedException stairwayEx) {
        throw new InternalStairwayException(stairwayEx);
    }
    List<EnumeratedJob> jobList = new ArrayList<>();
    for (FlightState state : flightEnumeration.getFlightStateList()) {
        FlightMap inputMap = state.getInputParameters();
        OperationType operationType = (inputMap.containsKey(WorkspaceFlightMapKeys.OPERATION_TYPE)) ? inputMap.get(WorkspaceFlightMapKeys.OPERATION_TYPE, OperationType.class) : OperationType.UNKNOWN;
        WsmResource wsmResource = (inputMap.containsKey(ResourceKeys.RESOURCE)) ? inputMap.get(ResourceKeys.RESOURCE, new TypeReference<>() {
        }) : null;
        String jobDescription = (inputMap.containsKey(JobMapKeys.DESCRIPTION.getKeyName())) ? inputMap.get(JobMapKeys.DESCRIPTION.getKeyName(), String.class) : StringUtils.EMPTY;
        EnumeratedJob enumeratedJob = new EnumeratedJob().flightState(state).jobDescription(jobDescription).operationType(operationType).resource(wsmResource);
        jobList.add(enumeratedJob);
    }
    return new EnumeratedJobs().pageToken(flightEnumeration.getNextPageToken()).totalResults(flightEnumeration.getTotalFlights()).results(jobList);
}
Also used : WsmResource(bio.terra.workspace.service.resource.model.WsmResource) FlightEnumeration(bio.terra.stairway.FlightEnumeration) ArrayList(java.util.ArrayList) EnumeratedJobs(bio.terra.workspace.service.workspace.model.EnumeratedJobs) InternalStairwayException(bio.terra.workspace.service.job.exception.InternalStairwayException) FlightState(bio.terra.stairway.FlightState) FlightFilter(bio.terra.stairway.FlightFilter) InternalStairwayException(bio.terra.workspace.service.job.exception.InternalStairwayException) StairwayException(bio.terra.stairway.exception.StairwayException) FlightMap(bio.terra.stairway.FlightMap) OperationType(bio.terra.workspace.service.workspace.model.OperationType) TypeReference(com.fasterxml.jackson.core.type.TypeReference) EnumeratedJob(bio.terra.workspace.service.workspace.model.EnumeratedJob)

Example 5 with StairwayException

use of bio.terra.stairway.exception.StairwayException in project terra-workspace-manager by DataBiosphere.

the class JobService method retrieveJob.

@Traced
public ApiJobReport retrieveJob(String jobId, AuthenticatedUserRequest userRequest) {
    try {
        // jobId=flightId
        verifyUserAccess(jobId, userRequest);
        FlightState flightState = stairwayComponent.get().getFlightState(jobId);
        return mapFlightStateToApiJobReport(flightState);
    } catch (StairwayException | InterruptedException stairwayEx) {
        throw new InternalStairwayException(stairwayEx);
    }
}
Also used : FlightState(bio.terra.stairway.FlightState) StairwayException(bio.terra.stairway.exception.StairwayException) InternalStairwayException(bio.terra.workspace.service.job.exception.InternalStairwayException) InternalStairwayException(bio.terra.workspace.service.job.exception.InternalStairwayException) Traced(io.opencensus.contrib.spring.aop.Traced)

Aggregations

StairwayException (bio.terra.stairway.exception.StairwayException)9 FlightState (bio.terra.stairway.FlightState)6 InternalStairwayException (bio.terra.workspace.service.job.exception.InternalStairwayException)5 FlightMap (bio.terra.stairway.FlightMap)4 InternalStairwayException (bio.terra.service.job.exception.InternalStairwayException)3 FlightFilter (bio.terra.stairway.FlightFilter)3 ArrayList (java.util.ArrayList)3 JobServiceShutdownException (bio.terra.service.job.exception.JobServiceShutdownException)2 JobServiceStartupException (bio.terra.service.job.exception.JobServiceStartupException)2 DatabaseOperationException (bio.terra.stairway.exception.DatabaseOperationException)2 DuplicateFlightIdException (bio.terra.stairway.exception.DuplicateFlightIdException)2 FlightNotFoundException (bio.terra.stairway.exception.FlightNotFoundException)2 ApiJobReport (bio.terra.workspace.generated.model.ApiJobReport)2 JobModel (bio.terra.model.JobModel)1 InvalidResultStateException (bio.terra.service.job.exception.InvalidResultStateException)1 JobNotCompleteException (bio.terra.service.job.exception.JobNotCompleteException)1 JobNotFoundException (bio.terra.service.job.exception.JobNotFoundException)1 JobResponseException (bio.terra.service.job.exception.JobResponseException)1 JobUnauthorizedException (bio.terra.service.job.exception.JobUnauthorizedException)1 FlightEnumeration (bio.terra.stairway.FlightEnumeration)1