Search in sources :

Example 1 with StorageUri

use of com.microsoft.azure.storage.StorageUri in project nifi by apache.

the class ListAzureBlobStorage method performListing.

@Override
protected List<BlobInfo> performListing(final ProcessContext context, final Long minTimestamp) throws IOException {
    String containerName = context.getProperty(AzureStorageUtils.CONTAINER).evaluateAttributeExpressions().getValue();
    String prefix = context.getProperty(PROP_PREFIX).evaluateAttributeExpressions().getValue();
    if (prefix == null) {
        prefix = "";
    }
    final List<BlobInfo> listing = new ArrayList<>();
    try {
        CloudBlobClient blobClient = AzureStorageUtils.createCloudBlobClient(context, getLogger(), null);
        CloudBlobContainer container = blobClient.getContainerReference(containerName);
        for (ListBlobItem blob : container.listBlobs(prefix, true, EnumSet.of(BlobListingDetails.METADATA), null, null)) {
            if (blob instanceof CloudBlob) {
                CloudBlob cloudBlob = (CloudBlob) blob;
                BlobProperties properties = cloudBlob.getProperties();
                StorageUri uri = cloudBlob.getSnapshotQualifiedStorageUri();
                Builder builder = new BlobInfo.Builder().primaryUri(uri.getPrimaryUri().toString()).blobName(cloudBlob.getName()).containerName(containerName).contentType(properties.getContentType()).contentLanguage(properties.getContentLanguage()).etag(properties.getEtag()).lastModifiedTime(properties.getLastModified().getTime()).length(properties.getLength());
                if (uri.getSecondaryUri() != null) {
                    builder.secondaryUri(uri.getSecondaryUri().toString());
                }
                if (blob instanceof CloudBlockBlob) {
                    builder.blobType(AzureStorageUtils.BLOCK);
                } else {
                    builder.blobType(AzureStorageUtils.PAGE);
                }
                listing.add(builder.build());
            }
        }
    } catch (Throwable t) {
        throw new IOException(ExceptionUtils.getRootCause(t));
    }
    return listing;
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) ListBlobItem(com.microsoft.azure.storage.blob.ListBlobItem) Builder(org.apache.nifi.processors.azure.storage.utils.BlobInfo.Builder) ArrayList(java.util.ArrayList) StorageUri(com.microsoft.azure.storage.StorageUri) BlobInfo(org.apache.nifi.processors.azure.storage.utils.BlobInfo) IOException(java.io.IOException) CloudBlockBlob(com.microsoft.azure.storage.blob.CloudBlockBlob) CloudBlob(com.microsoft.azure.storage.blob.CloudBlob) BlobProperties(com.microsoft.azure.storage.blob.BlobProperties) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer)

Aggregations

StorageUri (com.microsoft.azure.storage.StorageUri)1 BlobProperties (com.microsoft.azure.storage.blob.BlobProperties)1 CloudBlob (com.microsoft.azure.storage.blob.CloudBlob)1 CloudBlobClient (com.microsoft.azure.storage.blob.CloudBlobClient)1 CloudBlobContainer (com.microsoft.azure.storage.blob.CloudBlobContainer)1 CloudBlockBlob (com.microsoft.azure.storage.blob.CloudBlockBlob)1 ListBlobItem (com.microsoft.azure.storage.blob.ListBlobItem)1 IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 BlobInfo (org.apache.nifi.processors.azure.storage.utils.BlobInfo)1 Builder (org.apache.nifi.processors.azure.storage.utils.BlobInfo.Builder)1