Search in sources :

Example 56 with CloudBlobClient

use of com.microsoft.azure.storage.blob.CloudBlobClient in project components by Talend.

the class AzureStorageBlobService method setPublicAccess.

public void setPublicAccess(String containerName) throws URISyntaxException, StorageException, InvalidKeyException {
    CloudBlobClient cloudBlobClient = connection.getCloudStorageAccount().createCloudBlobClient();
    CloudBlobContainer cloudBlobContainer = cloudBlobClient.getContainerReference(containerName);
    BlobContainerPermissions containerPermissions = new BlobContainerPermissions();
    containerPermissions.setPublicAccess(BlobContainerPublicAccessType.CONTAINER);
    cloudBlobContainer.uploadPermissions(containerPermissions);
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) BlobContainerPermissions(com.microsoft.azure.storage.blob.BlobContainerPermissions)

Example 57 with CloudBlobClient

use of com.microsoft.azure.storage.blob.CloudBlobClient in project components by Talend.

the class AzureStorageBlobService method listBlobs.

public Iterable<ListBlobItem> listBlobs(final String containerName, final String prefix, final boolean useFlatBlobListing) throws URISyntaxException, StorageException, InvalidKeyException {
    CloudBlobClient cloudBlobClient = connection.getCloudStorageAccount().createCloudBlobClient();
    CloudBlobContainer cloudBlobContainer = cloudBlobClient.getContainerReference(containerName);
    return cloudBlobContainer.listBlobs(prefix, useFlatBlobListing);
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer)

Example 58 with CloudBlobClient

use of com.microsoft.azure.storage.blob.CloudBlobClient in project components by Talend.

the class AzureStorageBlobService method deleteContainerIfExist.

/**
 * This method delete the container if exist
 */
public boolean deleteContainerIfExist(final String containerName) throws StorageException, URISyntaxException, InvalidKeyException {
    CloudBlobClient cloudBlobClient = connection.getCloudStorageAccount().createCloudBlobClient();
    CloudBlobContainer cloudBlobContainer = cloudBlobClient.getContainerReference(containerName);
    return cloudBlobContainer.deleteIfExists();
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer)

Example 59 with CloudBlobClient

use of com.microsoft.azure.storage.blob.CloudBlobClient in project components by Talend.

the class AzureStorageBlobService method containerExist.

/**
 * @return true if the a container exist with the given name, false otherwise
 */
public boolean containerExist(final String containerName) throws StorageException, URISyntaxException, InvalidKeyException {
    CloudBlobClient cloudBlobClient = connection.getCloudStorageAccount().createCloudBlobClient();
    CloudBlobContainer cloudBlobContainer = cloudBlobClient.getContainerReference(containerName);
    return cloudBlobContainer.exists();
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer)

Example 60 with CloudBlobClient

use of com.microsoft.azure.storage.blob.CloudBlobClient in project kylo by Teradata.

the class AzureNativeFileSystemProvider method listFiles.

@Nonnull
@Override
@SuppressWarnings("squid:S1075")
public List<DataSetFile> listFiles(@Nonnull final Path path, @Nonnull final Configuration conf) {
    // Create Azure Blob client
    final URI uri = path.toUri();
    final CloudBlobClient client = createBlobClient(uri, conf);
    // List containers as data set files
    final String pathPrefix = uri.getScheme() + "://";
    final String pathSuffix = "@" + uri.getHost() + (uri.getPort() > -1 ? ":" + uri.getPort() : "") + "/";
    return StreamSupport.stream(listContainers(client).spliterator(), false).map(container -> {
        final DataSetFile file = new DataSetFile();
        file.setName(container.getName());
        file.setDirectory(true);
        file.setModificationTime(container.getProperties().getLastModified().getTime());
        file.setPath(pathPrefix + container.getName() + pathSuffix);
        return file;
    }).collect(Collectors.toList());
}
Also used : StorageCredentialsAccountAndKey(com.microsoft.azure.storage.StorageCredentialsAccountAndKey) RetryExponentialRetry(com.microsoft.azure.storage.RetryExponentialRetry) DataSetFile(com.thinkbiganalytics.kylo.catalog.rest.model.DataSetFile) KeyProviderException(org.apache.hadoop.fs.azure.KeyProviderException) CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) AzureNativeFileSystemStore(org.apache.hadoop.fs.azure.AzureNativeFileSystemStore) FileSystemProvider(com.thinkbiganalytics.kylo.catalog.spi.FileSystemProvider) StringUtils(org.apache.commons.lang3.StringUtils) Collectors(java.util.stream.Collectors) StorageCredentials(com.microsoft.azure.storage.StorageCredentials) Component(org.springframework.stereotype.Component) List(java.util.List) RetryPolicyFactory(com.microsoft.azure.storage.RetryPolicyFactory) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) CatalogException(com.thinkbiganalytics.kylo.catalog.CatalogException) Configuration(org.apache.hadoop.conf.Configuration) Path(org.apache.hadoop.fs.Path) StreamSupport(java.util.stream.StreamSupport) VisibleForTesting(com.google.common.annotations.VisibleForTesting) URI(java.net.URI) Nonnull(javax.annotation.Nonnull) Nullable(javax.annotation.Nullable) CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) URI(java.net.URI) DataSetFile(com.thinkbiganalytics.kylo.catalog.rest.model.DataSetFile) Nonnull(javax.annotation.Nonnull)

Aggregations

CloudBlobClient (com.microsoft.azure.storage.blob.CloudBlobClient)74 CloudBlobContainer (com.microsoft.azure.storage.blob.CloudBlobContainer)42 CloudStorageAccount (com.microsoft.azure.storage.CloudStorageAccount)19 StorageException (com.microsoft.azure.storage.StorageException)19 URISyntaxException (java.net.URISyntaxException)17 CloudBlockBlob (com.microsoft.azure.storage.blob.CloudBlockBlob)16 ListBlobItem (com.microsoft.azure.storage.blob.ListBlobItem)10 URI (java.net.URI)10 Supplier (java.util.function.Supplier)9 CloudBlob (com.microsoft.azure.storage.blob.CloudBlob)8 InvalidKeyException (java.security.InvalidKeyException)8 Test (org.junit.Test)8 ArrayList (java.util.ArrayList)6 ParameterizedMessage (org.apache.logging.log4j.message.ParameterizedMessage)6 TypeLiteral (com.google.inject.TypeLiteral)5 RetryExponentialRetry (com.microsoft.azure.storage.RetryExponentialRetry)5 StorageCredentials (com.microsoft.azure.storage.StorageCredentials)5 BlobListingDetails (com.microsoft.azure.storage.blob.BlobListingDetails)5 IOException (java.io.IOException)5 BlobProperties (com.microsoft.azure.storage.blob.BlobProperties)4