Search in sources :

Example 16 with DataStorageException

use of com.epam.pipeline.entity.datastorage.DataStorageException in project cloud-pipeline by epam.

the class DataStorageController method uploadFile.

@RequestMapping(value = "/datastorage/{id}/list/upload", method = RequestMethod.POST)
@ResponseBody
@ApiOperation(value = "Uploads a file to data storage.", notes = "Uploads a file to data storage.", produces = MediaType.APPLICATION_JSON_VALUE)
public List<UploadFileMetadata> uploadFile(@PathVariable(value = ID) Long id, @RequestParam(value = PATH, required = false) final String folder, HttpServletRequest request) throws FileUploadException {
    MultipartFile file = consumeMultipartFile(request);
    LinkedList<UploadFileMetadata> uploadedFiles = new LinkedList<>();
    UploadFileMetadata fileMeta = new UploadFileMetadata();
    fileMeta.setFileName(FilenameUtils.getName(file.getOriginalFilename()));
    fileMeta.setFileSize(file.getSize() / BYTES_IN_KB + " Kb");
    fileMeta.setFileType(file.getContentType());
    try {
        fileMeta.setBytes(file.getBytes());
        uploadedFiles.add(fileMeta);
    } catch (IOException e) {
        throw new DataStorageException("Failed to upload file to datastorage.", e);
    }
    dataStorageApiService.createDataStorageFile(id, folder, fileMeta.getFileName(), fileMeta.getBytes());
    return uploadedFiles;
}
Also used : MultipartFile(org.springframework.web.multipart.MultipartFile) DataStorageException(com.epam.pipeline.entity.datastorage.DataStorageException) UploadFileMetadata(com.epam.pipeline.controller.vo.UploadFileMetadata) IOException(java.io.IOException) LinkedList(java.util.LinkedList) ApiOperation(io.swagger.annotations.ApiOperation) RequestMapping(org.springframework.web.bind.annotation.RequestMapping) ResponseBody(org.springframework.web.bind.annotation.ResponseBody)

Example 17 with DataStorageException

use of com.epam.pipeline.entity.datastorage.DataStorageException in project cloud-pipeline by epam.

the class NFSStorageProvider method getFile.

@Override
public DataStorageItemContent getFile(NFSDataStorage dataStorage, String path, String version, Long maxDownloadSize) {
    File mntDir = mount(dataStorage);
    File file = new File(mntDir, path);
    try (FileInputStream fis = new FileInputStream(file)) {
        DataStorageItemContent content = new DataStorageItemContent();
        long bytesToRead = file.length();
        if (file.length() > maxDownloadSize) {
            content.setTruncated(true);
            bytesToRead = maxDownloadSize;
        }
        byte[] contentBytes = IOUtils.toByteArray(fis, bytesToRead);
        if (FileContentUtils.isBinaryContent(contentBytes)) {
            content.setMayBeBinary(true);
        } else {
            content.setContent(contentBytes);
        }
        return content;
    } catch (IOException e) {
        throw new DataStorageException(e);
    }
}
Also used : DataStorageException(com.epam.pipeline.entity.datastorage.DataStorageException) DataStorageItemContent(com.epam.pipeline.entity.datastorage.DataStorageItemContent) IOException(java.io.IOException) DataStorageFile(com.epam.pipeline.entity.datastorage.DataStorageFile) File(java.io.File) FileInputStream(java.io.FileInputStream)

Example 18 with DataStorageException

use of com.epam.pipeline.entity.datastorage.DataStorageException in project cloud-pipeline by epam.

the class NFSStorageProvider method getStream.

@Override
public DataStorageStreamingContent getStream(NFSDataStorage dataStorage, String path, String version) {
    File mntDir = mount(dataStorage);
    File file = new File(mntDir, path);
    try {
        return new DataStorageStreamingContent(file);
    } catch (FileNotFoundException e) {
        throw new DataStorageException(e);
    }
}
Also used : DataStorageException(com.epam.pipeline.entity.datastorage.DataStorageException) DataStorageStreamingContent(com.epam.pipeline.entity.datastorage.DataStorageStreamingContent) FileNotFoundException(java.io.FileNotFoundException) DataStorageFile(com.epam.pipeline.entity.datastorage.DataStorageFile) File(java.io.File)

Example 19 with DataStorageException

use of com.epam.pipeline.entity.datastorage.DataStorageException in project cloud-pipeline by epam.

the class NFSStorageProvider method createFile.

@Override
public DataStorageFile createFile(NFSDataStorage dataStorage, String path, InputStream dataStream) throws DataStorageException {
    File dataStorageDir = mount(dataStorage);
    File file = new File(dataStorageDir, path);
    try (BufferedOutputStream outputStream = new BufferedOutputStream(new FileOutputStream(file))) {
        IOUtils.copy(dataStream, outputStream);
        setUmask(file);
    } catch (IOException e) {
        throw new DataStorageException(e);
    }
    return new DataStorageFile(path, file);
}
Also used : DataStorageFile(com.epam.pipeline.entity.datastorage.DataStorageFile) DataStorageException(com.epam.pipeline.entity.datastorage.DataStorageException) FileOutputStream(java.io.FileOutputStream) IOException(java.io.IOException) DataStorageFile(com.epam.pipeline.entity.datastorage.DataStorageFile) File(java.io.File) BufferedOutputStream(java.io.BufferedOutputStream)

Example 20 with DataStorageException

use of com.epam.pipeline.entity.datastorage.DataStorageException in project cloud-pipeline by epam.

the class NFSStorageProvider method unmountNFSIfEmpty.

private synchronized void unmountNFSIfEmpty(AbstractDataStorage storage) {
    String storagePath = storage.getPath();
    File mntDir = Paths.get(rootMountPoint, getMountDirName(storagePath)).toFile();
    List<AbstractDataStorage> remaining = dataStorageDao.loadDataStoragesByNFSPath(getNfsRootPath(storagePath));
    LOGGER.debug("Remaining NFS: " + remaining.stream().map(AbstractDataStorage::getPath).collect(Collectors.joining(";")) + " related with current root path");
    if (mntDir.exists() && isStorageOnlyOnNFS(storage, remaining)) {
        try {
            String umountCmd = String.format(NFS_UNMOUNT_CMD_PATTERN, mntDir.getAbsolutePath());
            cmdExecutor.executeCommand(umountCmd);
            FileUtils.deleteDirectory(mntDir);
        } catch (IOException e) {
            throw new DataStorageException(e);
        }
    }
}
Also used : AbstractDataStorage(com.epam.pipeline.entity.datastorage.AbstractDataStorage) DataStorageException(com.epam.pipeline.entity.datastorage.DataStorageException) IOException(java.io.IOException) DataStorageFile(com.epam.pipeline.entity.datastorage.DataStorageFile) File(java.io.File)

Aggregations

DataStorageException (com.epam.pipeline.entity.datastorage.DataStorageException)28 IOException (java.io.IOException)14 DataStorageFile (com.epam.pipeline.entity.datastorage.DataStorageFile)12 File (java.io.File)11 AmazonS3 (com.amazonaws.services.s3.AmazonS3)8 AbstractDataStorage (com.epam.pipeline.entity.datastorage.AbstractDataStorage)5 DataStorageItemContent (com.epam.pipeline.entity.datastorage.DataStorageItemContent)5 AmazonS3Exception (com.amazonaws.services.s3.model.AmazonS3Exception)4 ListObjectsRequest (com.amazonaws.services.s3.model.ListObjectsRequest)4 ObjectListing (com.amazonaws.services.s3.model.ObjectListing)4 ObjectMetadata (com.amazonaws.services.s3.model.ObjectMetadata)4 DataStorageFolder (com.epam.pipeline.entity.datastorage.DataStorageFolder)4 DataStorageStreamingContent (com.epam.pipeline.entity.datastorage.DataStorageStreamingContent)4 ByteArrayInputStream (java.io.ByteArrayInputStream)4 SdkClientException (com.amazonaws.SdkClientException)3 S3ObjectSummary (com.amazonaws.services.s3.model.S3ObjectSummary)3 InputStream (java.io.InputStream)3 Date (java.util.Date)3 List (java.util.List)3 Test (org.junit.Test)3