Search in sources :

Example 1 with GitRepository

use of io.hops.hopsworks.persistence.entity.git.GitRepository in project hopsworks by logicalclocks.

the class GitController method updateBranchCommits.

@TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public void updateBranchCommits(Project project, BranchCommits commits, Integer repositoryId, String branchName) throws GitOpException {
    if (Strings.isNullOrEmpty(branchName)) {
        throw new IllegalArgumentException("Branch name cannot be null");
    }
    GitRepository repository = commandConfigurationValidator.verifyRepository(project, repositoryId);
    // delete all
    gitCommitsFacade.deleteAllInBranchAndRepository(branchName, repository);
    // create new entries
    for (GitCommit commit : commits.getCommits()) {
        commit.setBranch(branchName);
        commit.setRepository(repository);
        gitCommitsFacade.create(commit);
    }
}
Also used : GitRepository(io.hops.hopsworks.persistence.entity.git.GitRepository) GitCommit(io.hops.hopsworks.persistence.entity.git.GitCommit) TransactionAttribute(javax.ejb.TransactionAttribute)

Example 2 with GitRepository

use of io.hops.hopsworks.persistence.entity.git.GitRepository in project hopsworks by logicalclocks.

the class GitController method commit.

public GitOpExecution commit(CommitCommandConfiguration commitConfigurationDTO, Project project, Users hopsworksUser, Integer repositoryId) throws IllegalArgumentException, GitOpException, HopsSecurityException {
    commandConfigurationValidator.verifyCommitOptions(commitConfigurationDTO);
    String userFullName = hopsworksUser.getFname() + " " + hopsworksUser.getLname();
    GitRepository repository = commandConfigurationValidator.verifyRepository(project, repositoryId);
    String repositoryFullPath = inodeController.getPath(repository.getInode());
    GitCommandConfiguration commandConfiguration = new GitCommandConfigurationBuilder().setCommandType(GitCommandType.COMMIT).setMessage(commitConfigurationDTO.getMessage()).setFiles(commitConfigurationDTO.getFiles()).setAll(commitConfigurationDTO.isAll()).setCommitter(new CommitterSignature(userFullName, hopsworksUser.getEmail())).setPath(repositoryFullPath).build();
    return executionController.createExecution(commandConfiguration, project, hopsworksUser, repository);
}
Also used : GitRepository(io.hops.hopsworks.persistence.entity.git.GitRepository) GitCommandConfiguration(io.hops.hopsworks.persistence.entity.git.config.GitCommandConfiguration) CommitterSignature(io.hops.hopsworks.persistence.entity.git.CommitterSignature)

Example 3 with GitRepository

use of io.hops.hopsworks.persistence.entity.git.GitRepository in project hopsworks by logicalclocks.

the class GitController method clone.

public GitOpExecution clone(CloneCommandConfiguration cloneConfigurationDTO, Project project, Users hopsworksUser) throws IllegalArgumentException, GitOpException, HopsSecurityException, DatasetException {
    commandConfigurationValidator.verifyCloneOptions(cloneConfigurationDTO);
    // create the repository dir. The go-git does not create a directory, so we need to create it before
    String fullRepoDirPath = cloneConfigurationDTO.getPath() + File.separator + commandConfigurationValidator.getRepositoryName(cloneConfigurationDTO.getUrl());
    DistributedFileSystemOps udfso = dfsService.getDfsOps(hdfsUsersController.getHdfsUserName(project, hopsworksUser));
    try {
        datasetController.createSubDirectory(project, new Path(fullRepoDirPath), udfso);
    } finally {
        // Close the udfso
        dfsService.closeDfsClient(udfso);
    }
    Inode inode = inodeController.getInodeAtPath(fullRepoDirPath);
    GitRepository repository = gitRepositoryFacade.create(inode, project, cloneConfigurationDTO.getProvider(), hopsworksUser);
    // Create the default remote
    gitRepositoryRemotesFacade.save(new GitRepositoryRemote(repository, Constants.REPOSITORY_DEFAULT_REMOTE_NAME, cloneConfigurationDTO.getUrl()));
    GitCommandConfiguration configuration = new GitCommandConfigurationBuilder().setCommandType(GitCommandType.CLONE).setUrl(cloneConfigurationDTO.getUrl()).setProvider(cloneConfigurationDTO.getProvider()).setPath(fullRepoDirPath).setBranchName(cloneConfigurationDTO.getBranch()).build();
    return executionController.createExecution(configuration, project, hopsworksUser, repository);
}
Also used : Path(org.apache.hadoop.fs.Path) GitRepository(io.hops.hopsworks.persistence.entity.git.GitRepository) Inode(io.hops.hopsworks.persistence.entity.hdfs.inode.Inode) GitRepositoryRemote(io.hops.hopsworks.persistence.entity.git.GitRepositoryRemote) GitCommandConfiguration(io.hops.hopsworks.persistence.entity.git.config.GitCommandConfiguration) DistributedFileSystemOps(io.hops.hopsworks.common.hdfs.DistributedFileSystemOps)

Example 4 with GitRepository

use of io.hops.hopsworks.persistence.entity.git.GitRepository in project hopsworks by logicalclocks.

the class GitController method push.

public GitOpExecution push(PushCommandConfiguration configurationDTO, Project project, Users hopsworksUser, Integer repositoryId) throws GitOpException, HopsSecurityException, IllegalArgumentException {
    commandConfigurationValidator.verifyRemoteNameAndBranch(configurationDTO.getRemoteName(), configurationDTO.getBranchName());
    GitRepository repository = commandConfigurationValidator.verifyRepository(project, repositoryId);
    String repositoryFullPath = inodeController.getPath(repository.getInode());
    GitCommandConfiguration pushCommandConfiguration = new GitCommandConfigurationBuilder().setCommandType(GitCommandType.PUSH).setRemoteName(configurationDTO.getRemoteName()).setBranchName(configurationDTO.getBranchName()).setForce(configurationDTO.isForce()).setPath(repositoryFullPath).build();
    return executionController.createExecution(pushCommandConfiguration, project, hopsworksUser, repository);
}
Also used : GitRepository(io.hops.hopsworks.persistence.entity.git.GitRepository) GitCommandConfiguration(io.hops.hopsworks.persistence.entity.git.config.GitCommandConfiguration)

Example 5 with GitRepository

use of io.hops.hopsworks.persistence.entity.git.GitRepository in project hopsworks by logicalclocks.

the class AsynchronousGitCommandExecutor method execute.

@Asynchronous
@TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)
public void execute(GitOpExecution gitOpExecution, GitPaths gitPaths) {
    int maxTries = 5;
    String pid = "";
    String gitCommand = gitOpExecution.getGitCommandConfiguration().getCommandType().getGitCommand();
    String prog = settings.getSudoersDir() + "/git.sh";
    String commandArgumentsFile = gitPaths.getConfDirPath() + File.separator + GitContainerLaunchScriptArgumentsTemplate.FILE_NAME;
    while (maxTries > 0 && Strings.isNullOrEmpty(pid)) {
        try {
            ProcessDescriptor processDescriptor = new ProcessDescriptor.Builder().addCommand("/usr/bin/sudo").addCommand(prog).addCommand("start").addCommand(commandArgumentsFile).redirectErrorStream(true).setCurrentWorkingDirectory(new File(gitPaths.getGitPath())).setWaitTimeout(60L, TimeUnit.SECONDS).build();
            String pidFile = gitPaths.getRunDirPath() + "/git.pid";
            ProcessResult processResult = osProcessExecutor.execute(processDescriptor);
            if (processResult.getExitCode() != 0) {
                String errorMsg = "Could not start git service to execute command " + gitCommand + " . " + "Exit code: " + processResult.getExitCode() + " Error: stdout: " + processResult.getStdout() + " stderr: " + processResult.getStderr();
                LOGGER.log(Level.SEVERE, errorMsg);
                throw new IOException(errorMsg);
            } else {
                pid = com.google.common.io.Files.readFirstLine(new File(pidFile), Charset.defaultCharset());
                // Get the updated repository
                Optional<GitRepository> optional = gitRepositoryFacade.findById(gitOpExecution.getRepository().getId());
                gitRepositoryFacade.updateRepositoryCid(optional.get(), pid);
            // gitOpExecutionFacade.updateState(gitOpExecution, GitOpExecutionState.SUBMITTED);
            }
        } catch (Exception ex) {
            LOGGER.log(Level.SEVERE, "Problem executing shell script to start git command service", ex);
            maxTries--;
        }
    }
    if (Strings.isNullOrEmpty(pid)) {
        updateExecutionStateToFail(gitOpExecution);
    }
}
Also used : GitRepository(io.hops.hopsworks.persistence.entity.git.GitRepository) ProcessResult(io.hops.hopsworks.common.util.ProcessResult) ProcessDescriptor(io.hops.hopsworks.common.util.ProcessDescriptor) IOException(java.io.IOException) File(java.io.File) IOException(java.io.IOException) Asynchronous(javax.ejb.Asynchronous) TransactionAttribute(javax.ejb.TransactionAttribute)

Aggregations

GitRepository (io.hops.hopsworks.persistence.entity.git.GitRepository)20 GitCommandConfiguration (io.hops.hopsworks.persistence.entity.git.config.GitCommandConfiguration)7 ResourceRequest (io.hops.hopsworks.common.api.ResourceRequest)6 AllowedProjectRoles (io.hops.hopsworks.api.filter.AllowedProjectRoles)5 ApiKeyRequired (io.hops.hopsworks.api.filter.apiKey.ApiKeyRequired)5 JWTRequired (io.hops.hopsworks.jwt.annotation.JWTRequired)5 ApiOperation (io.swagger.annotations.ApiOperation)5 GET (javax.ws.rs.GET)5 Path (javax.ws.rs.Path)5 Produces (javax.ws.rs.Produces)5 GitOpException (io.hops.hopsworks.exceptions.GitOpException)3 TransactionAttribute (javax.ejb.TransactionAttribute)3 GitRepositoryRemoteDTO (io.hops.hopsworks.api.git.remote.GitRepositoryRemoteDTO)2 GitCommitDTO (io.hops.hopsworks.common.git.GitCommitDTO)2 GitCommit (io.hops.hopsworks.persistence.entity.git.GitCommit)2 GitOpExecution (io.hops.hopsworks.persistence.entity.git.GitOpExecution)2 BranchDTO (io.hops.hopsworks.api.git.branch.BranchDTO)1 GitRepositoryDTO (io.hops.hopsworks.api.git.repository.GitRepositoryDTO)1 AbstractFacade (io.hops.hopsworks.common.dao.AbstractFacade)1 GitCommitsFacade (io.hops.hopsworks.common.dao.git.GitCommitsFacade)1