Search in sources :

Example 1 with Operation

use of org.opensearch.repositories.hdfs.HdfsBlobStore.Operation in project OpenSearch by opensearch-project.

the class HdfsBlobContainer method writeBlobAtomic.

@Override
public void writeBlobAtomic(String blobName, InputStream inputStream, long blobSize, boolean failIfAlreadyExists) throws IOException {
    final String tempBlob = FsBlobContainer.tempBlobName(blobName);
    final Path tempBlobPath = new Path(path, tempBlob);
    final Path blob = new Path(path, blobName);
    store.execute((Operation<Void>) fileContext -> {
        writeToPath(inputStream, blobSize, fileContext, tempBlobPath, EnumSet.of(CreateFlag.CREATE, CreateFlag.SYNC_BLOCK));
        try {
            fileContext.rename(tempBlobPath, blob, failIfAlreadyExists ? Options.Rename.NONE : Options.Rename.OVERWRITE);
        } catch (org.apache.hadoop.fs.FileAlreadyExistsException faee) {
            throw new FileAlreadyExistsException(blob.toString(), null, faee.getMessage());
        }
        return null;
    });
}
Also used : Path(org.apache.hadoop.fs.Path) BlobPath(org.opensearch.common.blobstore.BlobPath) NoSuchFileException(java.nio.file.NoSuchFileException) Operation(org.opensearch.repositories.hdfs.HdfsBlobStore.Operation) BlobContainer(org.opensearch.common.blobstore.BlobContainer) FileStatus(org.apache.hadoop.fs.FileStatus) LinkedHashMap(java.util.LinkedHashMap) FSDataOutputStream(org.apache.hadoop.fs.FSDataOutputStream) Options(org.apache.hadoop.fs.Options) FilterInputStream(java.io.FilterInputStream) FileContext(org.apache.hadoop.fs.FileContext) Map(java.util.Map) Path(org.apache.hadoop.fs.Path) BlobMetadata(org.opensearch.common.blobstore.BlobMetadata) CreateFlag(org.apache.hadoop.fs.CreateFlag) EnumSet(java.util.EnumSet) IOException(java.io.IOException) FileAlreadyExistsException(java.nio.file.FileAlreadyExistsException) Nullable(org.opensearch.common.Nullable) FileNotFoundException(java.io.FileNotFoundException) CreateOpts(org.apache.hadoop.fs.Options.CreateOpts) List(java.util.List) DeleteResult(org.opensearch.common.blobstore.DeleteResult) BlobPath(org.opensearch.common.blobstore.BlobPath) PlainBlobMetadata(org.opensearch.common.blobstore.support.PlainBlobMetadata) FsBlobContainer(org.opensearch.common.blobstore.fs.FsBlobContainer) AbstractBlobContainer(org.opensearch.common.blobstore.support.AbstractBlobContainer) Collections(java.util.Collections) InputStream(java.io.InputStream) FileAlreadyExistsException(java.nio.file.FileAlreadyExistsException)

Example 2 with Operation

use of org.opensearch.repositories.hdfs.HdfsBlobStore.Operation in project OpenSearch by opensearch-project.

the class HdfsBlobContainer method writeBlob.

@Override
public void writeBlob(String blobName, InputStream inputStream, long blobSize, boolean failIfAlreadyExists) throws IOException {
    Path blob = new Path(path, blobName);
    // we pass CREATE, which means it fails if a blob already exists.
    final EnumSet<CreateFlag> flags = failIfAlreadyExists ? EnumSet.of(CreateFlag.CREATE, CreateFlag.SYNC_BLOCK) : EnumSet.of(CreateFlag.CREATE, CreateFlag.OVERWRITE, CreateFlag.SYNC_BLOCK);
    store.execute((Operation<Void>) fileContext -> {
        try {
            writeToPath(inputStream, blobSize, fileContext, blob, flags);
        } catch (org.apache.hadoop.fs.FileAlreadyExistsException faee) {
            throw new FileAlreadyExistsException(blob.toString(), null, faee.getMessage());
        }
        return null;
    });
}
Also used : Path(org.apache.hadoop.fs.Path) BlobPath(org.opensearch.common.blobstore.BlobPath) CreateFlag(org.apache.hadoop.fs.CreateFlag) NoSuchFileException(java.nio.file.NoSuchFileException) Operation(org.opensearch.repositories.hdfs.HdfsBlobStore.Operation) BlobContainer(org.opensearch.common.blobstore.BlobContainer) FileStatus(org.apache.hadoop.fs.FileStatus) LinkedHashMap(java.util.LinkedHashMap) FSDataOutputStream(org.apache.hadoop.fs.FSDataOutputStream) Options(org.apache.hadoop.fs.Options) FilterInputStream(java.io.FilterInputStream) FileContext(org.apache.hadoop.fs.FileContext) Map(java.util.Map) Path(org.apache.hadoop.fs.Path) BlobMetadata(org.opensearch.common.blobstore.BlobMetadata) CreateFlag(org.apache.hadoop.fs.CreateFlag) EnumSet(java.util.EnumSet) IOException(java.io.IOException) FileAlreadyExistsException(java.nio.file.FileAlreadyExistsException) Nullable(org.opensearch.common.Nullable) FileNotFoundException(java.io.FileNotFoundException) CreateOpts(org.apache.hadoop.fs.Options.CreateOpts) List(java.util.List) DeleteResult(org.opensearch.common.blobstore.DeleteResult) BlobPath(org.opensearch.common.blobstore.BlobPath) PlainBlobMetadata(org.opensearch.common.blobstore.support.PlainBlobMetadata) FsBlobContainer(org.opensearch.common.blobstore.fs.FsBlobContainer) AbstractBlobContainer(org.opensearch.common.blobstore.support.AbstractBlobContainer) Collections(java.util.Collections) InputStream(java.io.InputStream) FileAlreadyExistsException(java.nio.file.FileAlreadyExistsException)

Aggregations

FileNotFoundException (java.io.FileNotFoundException)2 FilterInputStream (java.io.FilterInputStream)2 IOException (java.io.IOException)2 InputStream (java.io.InputStream)2 FileAlreadyExistsException (java.nio.file.FileAlreadyExistsException)2 NoSuchFileException (java.nio.file.NoSuchFileException)2 Collections (java.util.Collections)2 EnumSet (java.util.EnumSet)2 LinkedHashMap (java.util.LinkedHashMap)2 List (java.util.List)2 Map (java.util.Map)2 CreateFlag (org.apache.hadoop.fs.CreateFlag)2 FSDataOutputStream (org.apache.hadoop.fs.FSDataOutputStream)2 FileContext (org.apache.hadoop.fs.FileContext)2 FileStatus (org.apache.hadoop.fs.FileStatus)2 Options (org.apache.hadoop.fs.Options)2 CreateOpts (org.apache.hadoop.fs.Options.CreateOpts)2 Path (org.apache.hadoop.fs.Path)2 Nullable (org.opensearch.common.Nullable)2 BlobContainer (org.opensearch.common.blobstore.BlobContainer)2