Search in sources :

Example 21 with DatasetSharedWith

use of io.hops.hopsworks.persistence.entity.dataset.DatasetSharedWith in project hopsworks by logicalclocks.

the class DatasetController method updateSharePermission.

public void updateSharePermission(Dataset ds, DatasetAccessPermission datasetPermissions, Project project, String targetProjectName, Users user) throws DatasetException, ProjectException {
    if (ds.isShared(project)) {
        throw new DatasetException(RESTCodes.DatasetErrorCode.DATASET_OWNER_ERROR, Level.FINE);
    }
    if (ds.isPublicDs()) {
        throw new DatasetException(RESTCodes.DatasetErrorCode.DATASET_PUBLIC_IMMUTABLE, Level.FINE);
    }
    Project targetProject = projectFacade.findByName(targetProjectName);
    if (targetProject == null) {
        throw new ProjectException(RESTCodes.ProjectErrorCode.PROJECT_NOT_FOUND, Level.FINE, "Target project not " + "found.");
    }
    DatasetSharedWith datasetSharedWith = datasetSharedWithFacade.findByProjectAndDataset(targetProject, ds);
    if (datasetSharedWith == null) {
        throw new DatasetException(RESTCodes.DatasetErrorCode.DATASET_NOT_SHARED_WITH_PROJECT, Level.FINE, "project: " + targetProject.getName());
    }
    PermissionTransition permissionTransition = PermissionTransition.valueOf(datasetSharedWith.getPermission(), datasetPermissions);
    updateSharePermission(datasetSharedWith, permissionTransition, project, user);
}
Also used : ProjectException(io.hops.hopsworks.exceptions.ProjectException) Project(io.hops.hopsworks.persistence.entity.project.Project) DatasetSharedWith(io.hops.hopsworks.persistence.entity.dataset.DatasetSharedWith) PermissionTransition(io.hops.hopsworks.persistence.entity.dataset.PermissionTransition) DatasetException(io.hops.hopsworks.exceptions.DatasetException)

Example 22 with DatasetSharedWith

use of io.hops.hopsworks.persistence.entity.dataset.DatasetSharedWith in project hopsworks by logicalclocks.

the class DatasetController method getOrCreateSharedTrainingDataset.

private DatasetSharedWith getOrCreateSharedTrainingDataset(Project project, Project parentProject, DatasetAccessPermission permission, Users sharedBy) {
    Dataset trainingDataset = getTrainingDataset(parentProject);
    DatasetSharedWith sharedTrainingDataset = datasetSharedWithFacade.findByProjectAndDataset(project, trainingDataset);
    if (sharedTrainingDataset == null) {
        sharedTrainingDataset = new DatasetSharedWith(project, trainingDataset, permission, false, sharedBy);
        datasetSharedWithFacade.save(sharedTrainingDataset);
        sharedTrainingDataset = datasetSharedWithFacade.findByProjectAndDataset(project, trainingDataset);
    }
    return sharedTrainingDataset;
}
Also used : Dataset(io.hops.hopsworks.persistence.entity.dataset.Dataset) DatasetSharedWith(io.hops.hopsworks.persistence.entity.dataset.DatasetSharedWith)

Example 23 with DatasetSharedWith

use of io.hops.hopsworks.persistence.entity.dataset.DatasetSharedWith in project hopsworks by logicalclocks.

the class DatasetController method getAllByName.

/**
 * @param project
 * @param dsName
 * @return The list of datasets that match the name, including the datasets shared with this project
 */
public List<Dataset> getAllByName(Project project, String dsName) {
    List<Dataset> result = new ArrayList<>();
    try {
        Dataset nativeDataset = getByName(project, dsName);
        result.add(nativeDataset);
    } catch (DatasetException e) {
    // not found, don't do anything
    }
    List<Dataset> sharedDatasets = project.getDatasetSharedWithCollection().stream().filter(DatasetSharedWith::getAccepted).filter((sds) -> sds.getDataset().getName().equals(dsName)).map((sds) -> sds.getDataset()).collect(Collectors.toCollection(ArrayList::new));
    result.addAll(sharedDatasets);
    return result;
}
Also used : FsPermissions(io.hops.hopsworks.common.hdfs.FsPermissions) FileStatus(org.apache.hadoop.fs.FileStatus) Base64(org.apache.commons.codec.binary.Base64) FsPermission(org.apache.hadoop.fs.permission.FsPermission) HopsFSProvenanceController(io.hops.hopsworks.common.provenance.core.HopsFSProvenanceController) Settings(io.hops.hopsworks.common.util.Settings) TransactionAttributeType(javax.ejb.TransactionAttributeType) DatasetSharedWith(io.hops.hopsworks.persistence.entity.dataset.DatasetSharedWith) Path(org.apache.hadoop.fs.Path) ProjectException(io.hops.hopsworks.exceptions.ProjectException) Utils(io.hops.hopsworks.common.hdfs.Utils) FSDataInputStream(org.apache.hadoop.fs.FSDataInputStream) Pair(io.hops.common.Pair) Stateless(javax.ejb.Stateless) FeaturestoreConstants(io.hops.hopsworks.common.featurestore.FeaturestoreConstants) PermissionTransition(io.hops.hopsworks.persistence.entity.dataset.PermissionTransition) DatasetException(io.hops.hopsworks.exceptions.DatasetException) RESTCodes(io.hops.hopsworks.restutils.RESTCodes) ActivityFacade(io.hops.hopsworks.common.dao.user.activity.ActivityFacade) DatasetType(io.hops.hopsworks.persistence.entity.dataset.DatasetType) Logger(java.util.logging.Logger) Collectors(java.util.stream.Collectors) FileNotFoundException(java.io.FileNotFoundException) ProjectTeamFacade(io.hops.hopsworks.common.dao.project.team.ProjectTeamFacade) List(java.util.List) DatasetFacade(io.hops.hopsworks.common.dao.dataset.DatasetFacade) JupyterController(io.hops.hopsworks.common.jupyter.JupyterController) DistributedFsService(io.hops.hopsworks.common.hdfs.DistributedFsService) ActivityFlag(io.hops.hopsworks.persistence.entity.user.activity.ActivityFlag) ProjectFacade(io.hops.hopsworks.common.dao.project.ProjectFacade) InodeController(io.hops.hopsworks.common.hdfs.inode.InodeController) DataInputStream(java.io.DataInputStream) HopsSecurityException(io.hops.hopsworks.exceptions.HopsSecurityException) DatasetSharedWithFacade(io.hops.hopsworks.common.dao.dataset.DatasetSharedWithFacade) DistributedFileSystemOps(io.hops.hopsworks.common.hdfs.DistributedFileSystemOps) DatasetRequest(io.hops.hopsworks.persistence.entity.dataset.DatasetRequest) Project(io.hops.hopsworks.persistence.entity.project.Project) FsAction(org.apache.hadoop.fs.permission.FsAction) Stack(java.util.Stack) ArrayList(java.util.ArrayList) Level(java.util.logging.Level) FSDataOutputStream(org.apache.hadoop.fs.FSDataOutputStream) DatasetAccessPermission(io.hops.hopsworks.persistence.entity.dataset.DatasetAccessPermission) OperationsLog(io.hops.hopsworks.persistence.entity.log.operation.OperationsLog) TransactionAttribute(javax.ejb.TransactionAttribute) HdfsUsersController(io.hops.hopsworks.common.hdfs.HdfsUsersController) SharedState(io.hops.hopsworks.persistence.entity.dataset.SharedState) Inode(io.hops.hopsworks.persistence.entity.hdfs.inode.Inode) OperationsLogFacade(io.hops.hopsworks.common.dao.log.operation.OperationsLogFacade) EJB(javax.ejb.EJB) InodeFacade(io.hops.hopsworks.common.dao.hdfs.inode.InodeFacade) OSProcessExecutor(io.hops.hopsworks.common.util.OSProcessExecutor) ProcessDescriptor(io.hops.hopsworks.common.util.ProcessDescriptor) ProjectTeam(io.hops.hopsworks.persistence.entity.project.team.ProjectTeam) OperationType(io.hops.hopsworks.persistence.entity.log.operation.OperationType) AllowedRoles(io.hops.hopsworks.common.constants.auth.AllowedRoles) IOException(java.io.IOException) ProvTypeDTO(io.hops.hopsworks.common.provenance.core.dto.ProvTypeDTO) File(java.io.File) ServiceException(io.hops.hopsworks.exceptions.ServiceException) Dataset(io.hops.hopsworks.persistence.entity.dataset.Dataset) ProcessResult(io.hops.hopsworks.common.util.ProcessResult) AccessControlException(org.apache.hadoop.security.AccessControlException) DatasetRequestFacade(io.hops.hopsworks.common.dao.dataset.DatasetRequestFacade) Users(io.hops.hopsworks.persistence.entity.user.Users) CompressionInfo(io.hops.hopsworks.common.dataset.util.CompressionInfo) HopsUtils(io.hops.hopsworks.common.util.HopsUtils) Dataset(io.hops.hopsworks.persistence.entity.dataset.Dataset) ArrayList(java.util.ArrayList) DatasetException(io.hops.hopsworks.exceptions.DatasetException)

Example 24 with DatasetSharedWith

use of io.hops.hopsworks.persistence.entity.dataset.DatasetSharedWith in project hopsworks by logicalclocks.

the class DatasetController method unshareDataset.

/**
 * Unshares a dataset from the target project.
 * @param targetProject
 * @param dataset
 * @throws IOException
 */
public void unshareDataset(Project project, Users user, Project targetProject, Dataset dataset) throws DatasetException {
    DatasetSharedWith datasetSharedWith = datasetSharedWithFacade.findByProjectAndDataset(targetProject, dataset);
    if (datasetSharedWith == null) {
        throw new DatasetException(RESTCodes.DatasetErrorCode.DATASET_NOT_SHARED_WITH_PROJECT, Level.FINE, "project: " + targetProject.getName());
    }
    unshareDs(project, user, dataset, datasetSharedWith);
}
Also used : DatasetSharedWith(io.hops.hopsworks.persistence.entity.dataset.DatasetSharedWith) DatasetException(io.hops.hopsworks.exceptions.DatasetException)

Example 25 with DatasetSharedWith

use of io.hops.hopsworks.persistence.entity.dataset.DatasetSharedWith in project hopsworks by logicalclocks.

the class DatasetController method makeImmutable.

public void makeImmutable(Dataset ds, Project project, Users user, Path path) throws DatasetException {
    PermissionTransition permissionTransition = PermissionTransition.valueOf(ds.getPermission(), DatasetAccessPermission.READ_ONLY);
    DistributedFileSystemOps dfso = null;
    try {
        dfso = dfs.getDfsOps();
        hdfsUsersController.makeImmutable(path, dfso);
        changePermissions(ds, permissionTransition, project, dfso);
        List<DatasetSharedWith> sharedWith = datasetSharedWithFacade.findByDataset(ds);
        for (DatasetSharedWith datasetSharedWith : sharedWith) {
            updateSharePermission(datasetSharedWith, PermissionTransition.valueOf(datasetSharedWith.getPermission(), DatasetAccessPermission.READ_ONLY), project, user, dfso);
        }
    } catch (Exception e) {
        // if rollback succeed remove DatasetPermissionOperation
        throw new DatasetException(RESTCodes.DatasetErrorCode.DATASET_PERMISSION_ERROR, Level.FINE, e.getMessage());
    } finally {
        dfs.closeDfsClient(dfso);
    }
}
Also used : PermissionTransition(io.hops.hopsworks.persistence.entity.dataset.PermissionTransition) DatasetSharedWith(io.hops.hopsworks.persistence.entity.dataset.DatasetSharedWith) DistributedFileSystemOps(io.hops.hopsworks.common.hdfs.DistributedFileSystemOps) ProjectException(io.hops.hopsworks.exceptions.ProjectException) DatasetException(io.hops.hopsworks.exceptions.DatasetException) FileNotFoundException(java.io.FileNotFoundException) HopsSecurityException(io.hops.hopsworks.exceptions.HopsSecurityException) IOException(java.io.IOException) ServiceException(io.hops.hopsworks.exceptions.ServiceException) AccessControlException(org.apache.hadoop.security.AccessControlException) DatasetException(io.hops.hopsworks.exceptions.DatasetException)

Aggregations

DatasetSharedWith (io.hops.hopsworks.persistence.entity.dataset.DatasetSharedWith)28 DatasetException (io.hops.hopsworks.exceptions.DatasetException)12 Dataset (io.hops.hopsworks.persistence.entity.dataset.Dataset)12 Project (io.hops.hopsworks.persistence.entity.project.Project)11 ArrayList (java.util.ArrayList)7 ProjectException (io.hops.hopsworks.exceptions.ProjectException)6 Inode (io.hops.hopsworks.persistence.entity.hdfs.inode.Inode)5 DistributedFileSystemOps (io.hops.hopsworks.common.hdfs.DistributedFileSystemOps)4 PermissionTransition (io.hops.hopsworks.persistence.entity.dataset.PermissionTransition)4 ProjectTeam (io.hops.hopsworks.persistence.entity.project.team.ProjectTeam)3 TransactionAttribute (javax.ejb.TransactionAttribute)3 Path (org.apache.hadoop.fs.Path)3 HopsSecurityException (io.hops.hopsworks.exceptions.HopsSecurityException)2 ServiceException (io.hops.hopsworks.exceptions.ServiceException)2 FileNotFoundException (java.io.FileNotFoundException)2 IOException (java.io.IOException)2 AccessControlException (org.apache.hadoop.security.AccessControlException)2 Pair (io.hops.common.Pair)1 ModelRegistryDTO (io.hops.hopsworks.api.modelregistry.dto.ModelRegistryDTO)1 ModelsBuilder (io.hops.hopsworks.api.modelregistry.models.ModelsBuilder)1