Search in sources :

Example 6 with BlockEntry

use of com.microsoft.azure.storage.blob.BlockEntry in project camel by apache.

the class BlobServiceProducer method commitBlobBlockList.

private void commitBlobBlockList(Exchange exchange) throws Exception {
    Object object = exchange.getIn().getMandatoryBody();
    List<BlockEntry> blockEntries = null;
    if (object instanceof List) {
        blockEntries = (List<BlockEntry>) blockEntries;
    } else if (object instanceof BlockEntry) {
        blockEntries = Collections.singletonList((BlockEntry) object);
    }
    if (blockEntries == null || blockEntries.isEmpty()) {
        throw new IllegalArgumentException("Illegal commit block list payload");
    }
    CloudBlockBlob client = BlobServiceUtil.createBlockBlobClient(getConfiguration());
    BlobServiceRequestOptions opts = BlobServiceUtil.getRequestOptions(exchange);
    LOG.trace("Putting a blob [{}] block list from exchange [{}]...", getConfiguration().getBlobName(), exchange);
    client.commitBlockList(blockEntries, opts.getAccessCond(), opts.getRequestOpts(), opts.getOpContext());
}
Also used : BlockEntry(com.microsoft.azure.storage.blob.BlockEntry) LinkedList(java.util.LinkedList) List(java.util.List) CloudBlockBlob(com.microsoft.azure.storage.blob.CloudBlockBlob)

Example 7 with BlockEntry

use of com.microsoft.azure.storage.blob.BlockEntry in project hadoop by apache.

the class BlockBlobAppendStream method commitAppendBlocks.

/**
   * Method to commit all the uncommited blocks to azure storage.
   * If the commit fails then blocks are automatically cleaned up
   * by Azure storage.
   * @throws IOException
   */
private synchronized void commitAppendBlocks() throws IOException {
    SelfRenewingLease lease = null;
    try {
        if (uncommittedBlockEntries.size() > 0) {
            //Acquiring lease on the blob.
            lease = new SelfRenewingLease(blob);
            // Downloading existing blocks
            List<BlockEntry> blockEntries = blob.downloadBlockList(BlockListingFilter.COMMITTED, new BlobRequestOptions(), opContext);
            // Adding uncommitted blocks.
            blockEntries.addAll(uncommittedBlockEntries);
            AccessCondition accessCondition = new AccessCondition();
            accessCondition.setLeaseID(lease.getLeaseID());
            blob.commitBlockList(blockEntries, accessCondition, new BlobRequestOptions(), opContext);
            uncommittedBlockEntries.clear();
        }
    } catch (StorageException ex) {
        LOG.error("Storage exception encountered during block commit phase of append for blob" + " : {} Storage Exception : {} Error Code: {}", key, ex, ex.getErrorCode());
        throw new IOException("Encountered Exception while committing append blocks", ex);
    } finally {
        if (lease != null) {
            try {
                lease.free();
            } catch (StorageException ex) {
                LOG.debug("Exception encountered while releasing lease for " + "blob : {} StorageException : {} ErrorCode : {}", key, ex, ex.getErrorCode());
            // Swallowing exception here as the lease is cleaned up by the SelfRenewingLease object.
            }
        }
    }
}
Also used : BlobRequestOptions(com.microsoft.azure.storage.blob.BlobRequestOptions) BlockEntry(com.microsoft.azure.storage.blob.BlockEntry) AccessCondition(com.microsoft.azure.storage.AccessCondition) IOException(java.io.IOException) StorageException(com.microsoft.azure.storage.StorageException)

Aggregations

BlockEntry (com.microsoft.azure.storage.blob.BlockEntry)7 CloudBlockBlob (com.microsoft.azure.storage.blob.CloudBlockBlob)4 StorageException (com.microsoft.azure.storage.StorageException)3 IOException (java.io.IOException)3 BlobRequestOptions (com.microsoft.azure.storage.blob.BlobRequestOptions)2 LinkedList (java.util.LinkedList)2 List (java.util.List)2 AccessCondition (com.microsoft.azure.storage.AccessCondition)1 BlockListingFilter (com.microsoft.azure.storage.blob.BlockListingFilter)1 ByteArrayInputStream (java.io.ByteArrayInputStream)1 InputStream (java.io.InputStream)1 OutputStream (java.io.OutputStream)1 Path (org.apache.hadoop.fs.Path)1