Examples with CloudBlob com.microsoft.azure.storage.blob.CloudBlob used on opensource projects

Example 16 with CloudBlob

use of com.microsoft.azure.storage.blob.CloudBlob in project azure-iot-sdk-java by Azure.

the class DeviceManagerExportSample method main.

public static void main(String[] args) throws Exception {
    System.out.println("Starting export sample...");
    CloudStorageAccount storageAccount = CloudStorageAccount.parse(SampleUtils.storageConnectionString);
    CloudBlobClient blobClient = storageAccount.createCloudBlobClient();
    CloudBlobContainer container = blobClient.getContainerReference(DeviceManagerExportSample.sampleContainerName);
    container.createIfNotExists();
    String containerSasUri = SampleUtils.getContainerSasUri(container);
    RegistryManager registryManager = RegistryManager.createFromConnectionString(SampleUtils.iotHubConnectionString);
    JobProperties exportJob = registryManager.exportDevices(containerSasUri, excludeKeys);
    while (true) {
        exportJob = registryManager.getJob(exportJob.getJobId());
        if (exportJob.getStatus() == JobProperties.JobStatus.COMPLETED) {
            break;
        }
        Thread.sleep(500);
    }
    for (ListBlobItem blobItem : container.listBlobs()) {
        if (blobItem instanceof CloudBlob) {
            CloudBlob blob = (CloudBlob) blobItem;
            blob.download(new FileOutputStream(SampleUtils.exportFileLocation + blob.getName()));
        }
    }
    registryManager.close();
    System.out.println("Export job completed. Results are in " + SampleUtils.exportFileLocation);
}

Example 17 with CloudBlob

use of com.microsoft.azure.storage.blob.CloudBlob in project crate by crate.

the class AzureStorageService method listBlobsByPrefix.

public Map<String, BlobMetadata> listBlobsByPrefix(String container, String keyPath, String prefix) throws URISyntaxException, StorageException {
    // NOTE: this should be here: if (prefix == null) prefix = "";
    // however, this is really inefficient since deleteBlobsByPrefix enumerates everything and
    // then does a prefix match on the result; it should just call listBlobsByPrefix with the prefix!
    final var blobsBuilder = new HashMap<String, BlobMetadata>();
    final EnumSet<BlobListingDetails> enumBlobListingDetails = EnumSet.of(BlobListingDetails.METADATA);
    final Tuple<CloudBlobClient, Supplier<OperationContext>> client = client();
    final CloudBlobContainer blobContainer = client.v1().getContainerReference(container);
    LOGGER.trace(() -> new ParameterizedMessage("listing container [{}], keyPath [{}], prefix [{}]", container, keyPath, prefix));
    for (final ListBlobItem blobItem : blobContainer.listBlobs(keyPath + (prefix == null ? "" : prefix), false, enumBlobListingDetails, null, client.v2().get())) {
        final URI uri = blobItem.getUri();
        LOGGER.trace(() -> new ParameterizedMessage("blob url [{}]", uri));
        // uri.getPath is of the form /container/keyPath.* and we want to strip off the /container/
        // this requires 1 + container.length() + 1, with each 1 corresponding to one of the /
        final String blobPath = uri.getPath().substring(1 + container.length() + 1);
        if (blobItem instanceof CloudBlob) {
            final BlobProperties properties = ((CloudBlob) blobItem).getProperties();
            final String name = blobPath.substring(keyPath.length());
            LOGGER.trace(() -> new ParameterizedMessage("blob url [{}], name [{}], size [{}]", uri, name, properties.getLength()));
            blobsBuilder.put(name, new PlainBlobMetadata(name, properties.getLength()));
        }
    }
    return Map.copyOf(blobsBuilder);
}

Example 18 with CloudBlob

use of com.microsoft.azure.storage.blob.CloudBlob in project druid by druid-io.

the class AzureStorage method emptyCloudBlobDirectory.

public List<String> emptyCloudBlobDirectory(final String containerName, final String virtualDirPath) throws StorageException, URISyntaxException {
    List<String> deletedFiles = new ArrayList<>();
    CloudBlobContainer container = getCloudBlobContainer(containerName);
    for (ListBlobItem blobItem : container.listBlobs(virtualDirPath, true, null, null, null)) {
        CloudBlob cloudBlob = (CloudBlob) blobItem;
        log.info("Removing file[%s] from Azure.", cloudBlob.getName());
        if (cloudBlob.deleteIfExists()) {
            deletedFiles.add(cloudBlob.getName());
        }
    }
    if (deletedFiles.isEmpty()) {
        log.warn("No files were deleted on the following Azure path: [%s]", virtualDirPath);
    }
    return deletedFiles;
}

Example 19 with CloudBlob

use of com.microsoft.azure.storage.blob.CloudBlob in project camel by apache.

the class BlobServiceComponent method checkCredentials.

private void checkCredentials(BlobServiceConfiguration cfg) {
    CloudBlob client = cfg.getAzureBlobClient();
    StorageCredentials creds = client == null ? cfg.getCredentials() : client.getServiceClient().getCredentials();
    if ((creds == null || creds instanceof StorageCredentialsAnonymous) && !cfg.isPublicForRead()) {
        throw new IllegalArgumentException("Credentials must be specified.");
    }
}

Example 20 with CloudBlob

use of com.microsoft.azure.storage.blob.CloudBlob in project nifi by apache.

the class DeleteAzureBlobStorage method onTrigger.

@Override
public void onTrigger(ProcessContext context, ProcessSession session) throws ProcessException {
    FlowFile flowFile = session.get();
    if (flowFile == null) {
        return;
    }
    final long startNanos = System.nanoTime();
    String containerName = context.getProperty(AzureStorageUtils.CONTAINER).evaluateAttributeExpressions(flowFile).getValue();
    String blobPath = context.getProperty(BLOB).evaluateAttributeExpressions(flowFile).getValue();
    try {
        CloudBlobClient blobClient = AzureStorageUtils.createCloudBlobClient(context, getLogger(), flowFile);
        CloudBlobContainer container = blobClient.getContainerReference(containerName);
        CloudBlob blob = container.getBlockBlobReference(blobPath);
        blob.deleteIfExists();
        session.transfer(flowFile, REL_SUCCESS);
        final long transferMillis = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startNanos);
        session.getProvenanceReporter().send(flowFile, blob.getSnapshotQualifiedUri().toString(), transferMillis);
    } catch (StorageException | URISyntaxException e) {
        getLogger().error("Failed to delete the specified blob {} from Azure Storage. Routing to failure", new Object[] { blobPath }, e);
        flowFile = session.penalize(flowFile);
        session.transfer(flowFile, REL_FAILURE);
    }
}