Search in sources :

Example 1 with CustomWebApplicationException

use of io.dockstore.webservice.CustomWebApplicationException in project dockstore by dockstore.

the class LimitedCRUDClientIT method generateSourceFiles.

private List<SourceFile> generateSourceFiles(DescriptorLanguage descriptorLanguage) throws IOException {
    String resourceFilePath;
    String dockstorePath;
    SourceFile.TypeEnum type;
    if (descriptorLanguage == CWL) {
        resourceFilePath = "tar-param.cwl";
        dockstorePath = "/Dockstore.cwl";
        type = SourceFile.TypeEnum.DOCKSTORE_CWL;
    } else if (descriptorLanguage == WDL) {
        resourceFilePath = "hello.wdl";
        dockstorePath = "/Dockstore.wdl";
        type = SourceFile.TypeEnum.DOCKSTORE_WDL;
    } else {
        throw new CustomWebApplicationException("Only WDL and CWL are an option", HttpStatus.SC_BAD_REQUEST);
    }
    SourceFile descriptorFile = new SourceFile();
    descriptorFile.setContent(FileUtils.readFileToString(new File(ResourceHelpers.resourceFilePath(resourceFilePath)), StandardCharsets.UTF_8));
    descriptorFile.setType(type);
    descriptorFile.setPath(dockstorePath);
    descriptorFile.setAbsolutePath(dockstorePath);
    SourceFile dockerfile = new SourceFile();
    dockerfile.setContent("FROM ubuntu:latest");
    dockerfile.setType(SourceFile.TypeEnum.DOCKERFILE);
    dockerfile.setPath("/Dockerfile");
    dockerfile.setAbsolutePath("/Dockerfile");
    return Lists.newArrayList(descriptorFile, dockerfile);
}
Also used : CustomWebApplicationException(io.dockstore.webservice.CustomWebApplicationException) SourceFile(io.swagger.client.model.SourceFile) SourceFile(io.swagger.client.model.SourceFile) File(java.io.File)

Example 2 with CustomWebApplicationException

use of io.dockstore.webservice.CustomWebApplicationException in project dockstore by dockstore.

the class NextflowHandler method getContent.

@Override
public Optional<String> getContent(String mainDescName, String mainDescriptor, Set<SourceFile> secondarySourceFiles, Type type, ToolDAO dao) {
    // This may change later (ex. tool, workflow)
    String callType = "call";
    String toolType = "tool";
    // Write main descriptor to file
    // The use of temporary files is not needed here and might cause new problems
    // Iterate over each call, grab docker containers
    // nextflow uses the main script from the manifest as the main descriptor
    // add the Nextflow scripts
    Configuration configuration = null;
    try {
        configuration = NextflowUtilities.grabConfig(mainDescriptor);
    } catch (NextflowUtilities.NextflowParsingException e) {
        throw new CustomWebApplicationException(e.getMessage(), HttpStatus.SC_UNPROCESSABLE_ENTITY);
    }
    String mainScriptPath = "main.nf";
    if (configuration.containsKey("manifest.mainScript")) {
        mainScriptPath = configuration.getString("manifest.mainScript");
    }
    final String finalMainScriptPath = mainScriptPath;
    mainDescriptor = secondarySourceFiles.stream().filter(sf -> sf.getPath().equals(finalMainScriptPath)).findFirst().map(sf -> sf.getContent()).orElse(null);
    // Get default container (process.container takes precedence over params.container)
    String defaultContainer = null;
    if (configuration.containsKey("params.container")) {
        defaultContainer = configuration.getString("params.container");
    }
    if (configuration.containsKey("process.container")) {
        defaultContainer = configuration.getString("process.container");
    }
    Map<String, String> callToDockerMap = new HashMap<>();
    String finalDefaultContainer = defaultContainer;
    // Add all DockerMap from each secondary sourcefile
    if (!secondarySourceFiles.isEmpty()) {
        secondarySourceFiles.forEach(sourceFile -> callToDockerMap.putAll(this.getCallsToDockerMap(sourceFile.getContent(), finalDefaultContainer)));
    }
    callToDockerMap.putAll(this.getCallsToDockerMap(mainDescriptor, defaultContainer));
    // Iterate over each call, determine dependencies
    // Mapping of stepId -> array of dependencies for the step
    Map<String, List<String>> callToDependencies = this.getCallsToDependencies(mainDescriptor);
    // Get import files
    Map<String, String> namespaceToPath = this.getImportMap(mainDescriptor);
    Map<String, ToolInfo> toolInfoMap = WDLHandler.mapConverterToToolInfo(WDLHandler.convertToDockerParameter(callToDockerMap), callToDependencies);
    return convertMapsToContent(mainScriptPath, type, dao, callType, toolType, toolInfoMap, namespaceToPath);
}
Also used : Configuration(org.apache.commons.configuration2.Configuration) HashMap(java.util.HashMap) NextflowUtilities(io.dockstore.common.NextflowUtilities) CustomWebApplicationException(io.dockstore.webservice.CustomWebApplicationException) ArrayList(java.util.ArrayList) List(java.util.List)

Example 3 with CustomWebApplicationException

use of io.dockstore.webservice.CustomWebApplicationException in project dockstore by dockstore.

the class AbstractWorkflowResource method getSourceCodeRepoInterface.

protected SourceCodeRepoInterface getSourceCodeRepoInterface(String gitUrl, User user) {
    List<Token> tokens = getAndRefreshTokens(user, tokenDAO, client, bitbucketClientID, bitbucketClientSecret);
    final String bitbucketTokenContent = getToken(tokens, TokenType.BITBUCKET_ORG);
    Token gitHubToken = Token.extractToken(tokens, TokenType.GITHUB_COM);
    final String gitlabTokenContent = getToken(tokens, TokenType.GITLAB_COM);
    final SourceCodeRepoInterface sourceCodeRepo = SourceCodeRepoFactory.createSourceCodeRepo(gitUrl, bitbucketTokenContent, gitlabTokenContent, gitHubToken);
    if (sourceCodeRepo == null) {
        throw new CustomWebApplicationException("Git tokens invalid, please re-link your git accounts.", HttpStatus.SC_BAD_REQUEST);
    }
    return sourceCodeRepo;
}
Also used : Token(io.dockstore.webservice.core.Token) CustomWebApplicationException(io.dockstore.webservice.CustomWebApplicationException) SourceCodeRepoInterface(io.dockstore.webservice.helpers.SourceCodeRepoInterface)

Example 4 with CustomWebApplicationException

use of io.dockstore.webservice.CustomWebApplicationException in project dockstore by dockstore.

the class AbstractWorkflowResource method createServicesAndVersionsFromDockstoreYml.

/**
 * Create or retrieve services based on Dockstore.yml, add or update tag version
 * ONLY WORKS FOR v1.1
 * @param repository Repository path (ex. dockstore/dockstore-ui2)
 * @param gitReference Git reference from GitHub (ex. refs/tags/1.0)
 * @param installationId installation id needed to set up GitHub Apps
 * @param user User that triggered action
 * @param dockstoreYml
 * @return List of new and updated services
 */
private List<Workflow> createServicesAndVersionsFromDockstoreYml(Service12 service, String repository, String gitReference, String installationId, User user, final SourceFile dockstoreYml) {
    GitHubSourceCodeRepo gitHubSourceCodeRepo = (GitHubSourceCodeRepo) SourceCodeRepoFactory.createGitHubAppRepo(gitHubAppSetup(installationId));
    final List<Workflow> updatedServices = new ArrayList<>();
    if (service != null) {
        if (!DockstoreYamlHelper.filterGitReference(Path.of(gitReference), service.getFilters())) {
            return updatedServices;
        }
        final DescriptorLanguageSubclass subclass = service.getSubclass();
        final Boolean publish = service.getPublish();
        final var defaultVersion = service.getLatestTagAsDefault();
        final List<YamlAuthor> yamlAuthors = service.getAuthors();
        Workflow workflow = createOrGetWorkflow(Service.class, repository, user, "", subclass.getShortName(), gitHubSourceCodeRepo);
        addDockstoreYmlVersionToWorkflow(repository, gitReference, dockstoreYml, gitHubSourceCodeRepo, workflow, defaultVersion, yamlAuthors);
        if (publish != null && workflow.getIsPublished() != publish) {
            LambdaEvent lambdaEvent = createBasicEvent(repository, gitReference, user.getUsername(), LambdaEvent.LambdaEventType.PUBLISH);
            try {
                workflow = publishWorkflow(workflow, publish);
            } catch (CustomWebApplicationException ex) {
                LOG.warn("Could not set publish state from YML.", ex);
                lambdaEvent.setSuccess(false);
                lambdaEvent.setMessage(ex.getMessage());
            }
            lambdaEventDAO.create(lambdaEvent);
        }
        updatedServices.add(workflow);
    }
    return updatedServices;
}
Also used : ArrayList(java.util.ArrayList) YamlWorkflow(io.dockstore.common.yaml.YamlWorkflow) BioWorkflow(io.dockstore.webservice.core.BioWorkflow) Workflow(io.dockstore.webservice.core.Workflow) CustomWebApplicationException(io.dockstore.webservice.CustomWebApplicationException) YamlAuthor(io.dockstore.common.yaml.YamlAuthor) LambdaEvent(io.dockstore.webservice.core.LambdaEvent) GitHubSourceCodeRepo(io.dockstore.webservice.helpers.GitHubSourceCodeRepo) DescriptorLanguageSubclass(io.dockstore.common.DescriptorLanguageSubclass)

Example 5 with CustomWebApplicationException

use of io.dockstore.webservice.CustomWebApplicationException in project dockstore by dockstore.

the class AbstractWorkflowResource method githubWebhookDelete.

/**
 * Handle webhooks from GitHub apps after branch deletion (redirected from AWS Lambda)
 * - Delete version for corresponding service and workflow
 * @param repository Repository path (ex. dockstore/dockstore-ui2)
 * @param gitReference Git reference from GitHub (ex. refs/tags/1.0)
 * @param username Git user who triggered the event
 * @param installationId GitHub App installation ID
 * @return List of updated workflows
 */
protected List<Workflow> githubWebhookDelete(String repository, String gitReference, String username, String installationId) {
    // Retrieve name from gitReference
    Optional<String> gitReferenceName = GitHelper.parseGitHubReference(gitReference);
    if (gitReferenceName.isEmpty()) {
        String msg = "Reference " + gitReference + " is not of the valid form";
        LOG.error(msg);
        sessionFactory.getCurrentSession().clear();
        LambdaEvent lambdaEvent = createBasicEvent(repository, gitReference, username, LambdaEvent.LambdaEventType.DELETE);
        lambdaEvent.setMessage(msg);
        lambdaEvent.setSuccess(false);
        lambdaEventDAO.create(lambdaEvent);
        sessionFactory.getCurrentSession().getTransaction().commit();
        throw new CustomWebApplicationException(msg, LAMBDA_FAILURE);
    }
    // Find all workflows and services that are github apps and use the given repo
    List<Workflow> workflows = workflowDAO.findAllByPath("github.com/" + repository, false).stream().filter(workflow -> Objects.equals(workflow.getMode(), DOCKSTORE_YML)).collect(Collectors.toList());
    // When the git reference to delete is the default version, set it to the next latest version
    workflows.forEach(workflow -> {
        if (workflow.getActualDefaultVersion() != null && workflow.getActualDefaultVersion().getName().equals(gitReferenceName.get())) {
            Optional<WorkflowVersion> max = workflow.getWorkflowVersions().stream().filter(v -> !Objects.equals(v.getName(), gitReferenceName.get())).max(Comparator.comparingLong(ver -> ver.getDate().getTime()));
            workflow.setActualDefaultVersion(max.orElse(null));
        }
    });
    // Delete all non-frozen versions that have the same git reference name and then update the file formats of the entry.
    workflows.forEach(workflow -> {
        workflow.getWorkflowVersions().removeIf(workflowVersion -> Objects.equals(workflowVersion.getName(), gitReferenceName.get()) && !workflowVersion.isFrozen());
        FileFormatHelper.updateEntryLevelFileFormats(workflow);
    });
    LambdaEvent lambdaEvent = createBasicEvent(repository, gitReference, username, LambdaEvent.LambdaEventType.DELETE);
    lambdaEventDAO.create(lambdaEvent);
    return workflows;
}
Also used : WorkflowVersion(io.dockstore.webservice.core.WorkflowVersion) WorkflowDAO(io.dockstore.webservice.jdbi.WorkflowDAO) LoggerFactory(org.slf4j.LoggerFactory) CustomWebApplicationException(io.dockstore.webservice.CustomWebApplicationException) HttpStatus(org.apache.http.HttpStatus) DockstoreWebserviceConfiguration(io.dockstore.webservice.DockstoreWebserviceConfiguration) SourceFile(io.dockstore.webservice.core.SourceFile) STUB(io.dockstore.webservice.core.WorkflowMode.STUB) Transaction(org.hibernate.Transaction) SourceCodeRepoFactory(io.dockstore.webservice.helpers.SourceCodeRepoFactory) YamlWorkflow(io.dockstore.common.yaml.YamlWorkflow) Duration(java.time.Duration) Map(java.util.Map) TokenType(io.dockstore.webservice.core.TokenType) CacheConfigManager(io.dockstore.webservice.helpers.CacheConfigManager) User(io.dockstore.webservice.core.User) Path(java.nio.file.Path) WorkflowVersionDAO(io.dockstore.webservice.jdbi.WorkflowVersionDAO) SKIP_COMMIT_ID(io.dockstore.webservice.Constants.SKIP_COMMIT_ID) Service(io.dockstore.webservice.core.Service) GitHubHelper(io.dockstore.webservice.helpers.GitHubHelper) LAMBDA_FAILURE(io.dockstore.webservice.Constants.LAMBDA_FAILURE) Collection(java.util.Collection) YamlAuthor(io.dockstore.common.yaml.YamlAuthor) SessionFactory(org.hibernate.SessionFactory) FULL(io.dockstore.webservice.core.WorkflowMode.FULL) Set(java.util.Set) Instant(java.time.Instant) Collectors(java.util.stream.Collectors) Sets(com.google.common.collect.Sets) DOCKSTORE_YML_PATH(io.dockstore.webservice.Constants.DOCKSTORE_YML_PATH) Objects(java.util.Objects) List(java.util.List) BioWorkflow(io.dockstore.webservice.core.BioWorkflow) DOCKSTORE_YML(io.dockstore.webservice.core.WorkflowMode.DOCKSTORE_YML) Optional(java.util.Optional) GHRateLimit(org.kohsuke.github.GHRateLimit) LambdaEvent(io.dockstore.webservice.core.LambdaEvent) Validation(io.dockstore.webservice.core.Validation) WorkflowMode(io.dockstore.webservice.core.WorkflowMode) HashMap(java.util.HashMap) PublicStateManager(io.dockstore.webservice.helpers.PublicStateManager) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) Author(io.dockstore.webservice.core.Author) OrcidAuthor(io.dockstore.webservice.core.OrcidAuthor) HttpClient(org.apache.http.client.HttpClient) FileFormatDAO(io.dockstore.webservice.jdbi.FileFormatDAO) Checksum(io.dockstore.webservice.core.Checksum) EventDAO(io.dockstore.webservice.jdbi.EventDAO) Api(io.swagger.annotations.Api) Token(io.dockstore.webservice.core.Token) Workflow(io.dockstore.webservice.core.Workflow) UserDAO(io.dockstore.webservice.jdbi.UserDAO) FileDAO(io.dockstore.webservice.jdbi.FileDAO) DockstoreYaml12(io.dockstore.common.yaml.DockstoreYaml12) DockstoreYamlHelper(io.dockstore.common.yaml.DockstoreYamlHelper) GitHelper(io.dockstore.webservice.helpers.GitHelper) FileFormatHelper(io.dockstore.webservice.helpers.FileFormatHelper) Logger(org.slf4j.Logger) DescriptorLanguageSubclass(io.dockstore.common.DescriptorLanguageSubclass) IOException(java.io.IOException) SourceCodeRepoInterface(io.dockstore.webservice.helpers.SourceCodeRepoInterface) StateManagerMode(io.dockstore.webservice.helpers.StateManagerMode) Version(io.dockstore.webservice.core.Version) LambdaEventDAO(io.dockstore.webservice.jdbi.LambdaEventDAO) GitHubSourceCodeRepo(io.dockstore.webservice.helpers.GitHubSourceCodeRepo) TokenDAO(io.dockstore.webservice.jdbi.TokenDAO) Comparator(java.util.Comparator) Service12(io.dockstore.common.yaml.Service12) YamlWorkflow(io.dockstore.common.yaml.YamlWorkflow) BioWorkflow(io.dockstore.webservice.core.BioWorkflow) Workflow(io.dockstore.webservice.core.Workflow) CustomWebApplicationException(io.dockstore.webservice.CustomWebApplicationException) LambdaEvent(io.dockstore.webservice.core.LambdaEvent) WorkflowVersion(io.dockstore.webservice.core.WorkflowVersion)

Aggregations

CustomWebApplicationException (io.dockstore.webservice.CustomWebApplicationException)168 Timed (com.codahale.metrics.annotation.Timed)70 UnitOfWork (io.dropwizard.hibernate.UnitOfWork)70 Path (javax.ws.rs.Path)70 Operation (io.swagger.v3.oas.annotations.Operation)69 ApiOperation (io.swagger.annotations.ApiOperation)68 User (io.dockstore.webservice.core.User)48 SourceFile (io.dockstore.webservice.core.SourceFile)39 Workflow (io.dockstore.webservice.core.Workflow)37 Token (io.dockstore.webservice.core.Token)34 Tool (io.dockstore.webservice.core.Tool)33 IOException (java.io.IOException)33 HashMap (java.util.HashMap)33 POST (javax.ws.rs.POST)33 GET (javax.ws.rs.GET)32 BioWorkflow (io.dockstore.webservice.core.BioWorkflow)30 ArrayList (java.util.ArrayList)30 List (java.util.List)29 OrganizationUser (io.dockstore.webservice.core.OrganizationUser)28 WorkflowVersion (io.dockstore.webservice.core.WorkflowVersion)28