Search in sources :

Example 6 with CloudBlobContainer

use of com.microsoft.azure.storage.blob.CloudBlobContainer in project elasticsearch by elastic.

the class AzureStorageServiceImpl method doesContainerExist.

@Override
public boolean doesContainerExist(String account, LocationMode mode, String container) {
    try {
        CloudBlobClient client = this.getSelectedClient(account, mode);
        CloudBlobContainer blobContainer = client.getContainerReference(container);
        return SocketAccess.doPrivilegedException(blobContainer::exists);
    } catch (Exception e) {
        logger.error("can not access container [{}]", container);
    }
    return false;
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) URISyntaxException(java.net.URISyntaxException) StorageException(com.microsoft.azure.storage.StorageException) RepositoryException(org.elasticsearch.repositories.RepositoryException) PrivilegedActionException(java.security.PrivilegedActionException)

Example 7 with CloudBlobContainer

use of com.microsoft.azure.storage.blob.CloudBlobContainer in project elasticsearch by elastic.

the class AzureStorageServiceImpl method deleteFiles.

@Override
public void deleteFiles(String account, LocationMode mode, String container, String path) throws URISyntaxException, StorageException {
    logger.trace("delete files container [{}], path [{}]", container, path);
    // Container name must be lower case.
    CloudBlobClient client = this.getSelectedClient(account, mode);
    CloudBlobContainer blobContainer = client.getContainerReference(container);
    SocketAccess.doPrivilegedVoidException(() -> {
        if (blobContainer.exists()) {
            // We list the blobs using a flat blob listing mode
            for (ListBlobItem blobItem : blobContainer.listBlobs(path, true)) {
                String blobName = blobNameFromUri(blobItem.getUri());
                logger.trace("removing blob [{}] full URI was [{}]", blobName, blobItem.getUri());
                deleteBlob(account, mode, container, blobName);
            }
        }
    });
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) ListBlobItem(com.microsoft.azure.storage.blob.ListBlobItem) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer)

Example 8 with CloudBlobContainer

use of com.microsoft.azure.storage.blob.CloudBlobContainer in project elasticsearch by elastic.

the class AzureStorageServiceImpl method deleteBlob.

@Override
public void deleteBlob(String account, LocationMode mode, String container, String blob) throws URISyntaxException, StorageException {
    logger.trace("delete blob for container [{}], blob [{}]", container, blob);
    // Container name must be lower case.
    CloudBlobClient client = this.getSelectedClient(account, mode);
    CloudBlobContainer blobContainer = client.getContainerReference(container);
    if (blobContainer.exists()) {
        logger.trace("container [{}]: blob [{}] found. removing.", container, blob);
        CloudBlockBlob azureBlob = blobContainer.getBlockBlobReference(blob);
        SocketAccess.doPrivilegedVoidException(azureBlob::delete);
    }
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) CloudBlockBlob(com.microsoft.azure.storage.blob.CloudBlockBlob)

Example 9 with CloudBlobContainer

use of com.microsoft.azure.storage.blob.CloudBlobContainer in project elasticsearch by elastic.

the class AzureStorageServiceImpl method listBlobsByPrefix.

@Override
public Map<String, BlobMetaData> listBlobsByPrefix(String account, LocationMode mode, String container, String keyPath, String prefix) throws URISyntaxException, StorageException {
    // NOTE: this should be here: if (prefix == null) prefix = "";
    // however, this is really inefficient since deleteBlobsByPrefix enumerates everything and
    // then does a prefix match on the result; it should just call listBlobsByPrefix with the prefix!
    logger.debug("listing container [{}], keyPath [{}], prefix [{}]", container, keyPath, prefix);
    MapBuilder<String, BlobMetaData> blobsBuilder = MapBuilder.newMapBuilder();
    CloudBlobClient client = this.getSelectedClient(account, mode);
    CloudBlobContainer blobContainer = client.getContainerReference(container);
    SocketAccess.doPrivilegedVoidException(() -> {
        if (blobContainer.exists()) {
            for (ListBlobItem blobItem : blobContainer.listBlobs(keyPath + (prefix == null ? "" : prefix))) {
                URI uri = blobItem.getUri();
                logger.trace("blob url [{}]", uri);
                // uri.getPath is of the form /container/keyPath.* and we want to strip off the /container/
                // this requires 1 + container.length() + 1, with each 1 corresponding to one of the /
                String blobPath = uri.getPath().substring(1 + container.length() + 1);
                CloudBlockBlob blob = blobContainer.getBlockBlobReference(blobPath);
                // fetch the blob attributes from Azure (getBlockBlobReference does not do this)
                // this is needed to retrieve the blob length (among other metadata) from Azure Storage
                blob.downloadAttributes();
                BlobProperties properties = blob.getProperties();
                String name = blobPath.substring(keyPath.length());
                logger.trace("blob url [{}], name [{}], size [{}]", uri, name, properties.getLength());
                blobsBuilder.put(name, new PlainBlobMetaData(name, properties.getLength()));
            }
        }
    });
    return blobsBuilder.immutableMap();
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) ListBlobItem(com.microsoft.azure.storage.blob.ListBlobItem) PlainBlobMetaData(org.elasticsearch.common.blobstore.support.PlainBlobMetaData) BlobMetaData(org.elasticsearch.common.blobstore.BlobMetaData) PlainBlobMetaData(org.elasticsearch.common.blobstore.support.PlainBlobMetaData) BlobProperties(com.microsoft.azure.storage.blob.BlobProperties) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) CloudBlockBlob(com.microsoft.azure.storage.blob.CloudBlockBlob) URI(java.net.URI)

Example 10 with CloudBlobContainer

use of com.microsoft.azure.storage.blob.CloudBlobContainer in project elasticsearch by elastic.

the class AzureStorageServiceImpl method moveBlob.

@Override
public void moveBlob(String account, LocationMode mode, String container, String sourceBlob, String targetBlob) throws URISyntaxException, StorageException {
    logger.debug("moveBlob container [{}], sourceBlob [{}], targetBlob [{}]", container, sourceBlob, targetBlob);
    CloudBlobClient client = this.getSelectedClient(account, mode);
    CloudBlobContainer blobContainer = client.getContainerReference(container);
    CloudBlockBlob blobSource = blobContainer.getBlockBlobReference(sourceBlob);
    if (blobSource.exists()) {
        CloudBlockBlob blobTarget = blobContainer.getBlockBlobReference(targetBlob);
        SocketAccess.doPrivilegedVoidException(() -> {
            blobTarget.startCopy(blobSource);
            blobSource.delete();
        });
        logger.debug("moveBlob container [{}], sourceBlob [{}], targetBlob [{}] -> done", container, sourceBlob, targetBlob);
    }
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) CloudBlockBlob(com.microsoft.azure.storage.blob.CloudBlockBlob)

Aggregations

CloudBlobContainer (com.microsoft.azure.storage.blob.CloudBlobContainer)29 CloudBlobClient (com.microsoft.azure.storage.blob.CloudBlobClient)13 CloudBlockBlob (com.microsoft.azure.storage.blob.CloudBlockBlob)9 CloudStorageAccount (com.microsoft.azure.storage.CloudStorageAccount)7 StorageException (com.microsoft.azure.storage.StorageException)7 URISyntaxException (java.net.URISyntaxException)6 Test (org.junit.Test)6 ListBlobItem (com.microsoft.azure.storage.blob.ListBlobItem)5 FileSystem (org.apache.hadoop.fs.FileSystem)5 Path (org.apache.hadoop.fs.Path)5 InvalidKeyException (java.security.InvalidKeyException)4 StorageAccount (com.microsoft.azure.management.storage.StorageAccount)3 FileNotFoundException (java.io.FileNotFoundException)3 IOException (java.io.IOException)3 URI (java.net.URI)3 WebApp (com.microsoft.azure.management.appservice.WebApp)2 BlobContainerPermissions (com.microsoft.azure.storage.blob.BlobContainerPermissions)2 CloudBlob (com.microsoft.azure.storage.blob.CloudBlob)2 ArrayList (java.util.ArrayList)2 RepositoryException (org.elasticsearch.repositories.RepositoryException)2