Search in sources :

Example 16 with CloudBlockBlob

use of com.microsoft.azure.storage.blob.CloudBlockBlob in project elasticsearch by elastic.

the class AzureStorageServiceImpl method deleteBlob.

@Override
public void deleteBlob(String account, LocationMode mode, String container, String blob) throws URISyntaxException, StorageException {
    logger.trace("delete blob for container [{}], blob [{}]", container, blob);
    // Container name must be lower case.
    CloudBlobClient client = this.getSelectedClient(account, mode);
    CloudBlobContainer blobContainer = client.getContainerReference(container);
    if (blobContainer.exists()) {
        logger.trace("container [{}]: blob [{}] found. removing.", container, blob);
        CloudBlockBlob azureBlob = blobContainer.getBlockBlobReference(blob);
        SocketAccess.doPrivilegedVoidException(azureBlob::delete);
    }
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) CloudBlockBlob(com.microsoft.azure.storage.blob.CloudBlockBlob)

Example 17 with CloudBlockBlob

use of com.microsoft.azure.storage.blob.CloudBlockBlob in project elasticsearch by elastic.

the class AzureStorageServiceImpl method listBlobsByPrefix.

@Override
public Map<String, BlobMetaData> listBlobsByPrefix(String account, LocationMode mode, String container, String keyPath, String prefix) throws URISyntaxException, StorageException {
    // NOTE: this should be here: if (prefix == null) prefix = "";
    // however, this is really inefficient since deleteBlobsByPrefix enumerates everything and
    // then does a prefix match on the result; it should just call listBlobsByPrefix with the prefix!
    logger.debug("listing container [{}], keyPath [{}], prefix [{}]", container, keyPath, prefix);
    MapBuilder<String, BlobMetaData> blobsBuilder = MapBuilder.newMapBuilder();
    CloudBlobClient client = this.getSelectedClient(account, mode);
    CloudBlobContainer blobContainer = client.getContainerReference(container);
    SocketAccess.doPrivilegedVoidException(() -> {
        if (blobContainer.exists()) {
            for (ListBlobItem blobItem : blobContainer.listBlobs(keyPath + (prefix == null ? "" : prefix))) {
                URI uri = blobItem.getUri();
                logger.trace("blob url [{}]", uri);
                // uri.getPath is of the form /container/keyPath.* and we want to strip off the /container/
                // this requires 1 + container.length() + 1, with each 1 corresponding to one of the /
                String blobPath = uri.getPath().substring(1 + container.length() + 1);
                CloudBlockBlob blob = blobContainer.getBlockBlobReference(blobPath);
                // fetch the blob attributes from Azure (getBlockBlobReference does not do this)
                // this is needed to retrieve the blob length (among other metadata) from Azure Storage
                blob.downloadAttributes();
                BlobProperties properties = blob.getProperties();
                String name = blobPath.substring(keyPath.length());
                logger.trace("blob url [{}], name [{}], size [{}]", uri, name, properties.getLength());
                blobsBuilder.put(name, new PlainBlobMetaData(name, properties.getLength()));
            }
        }
    });
    return blobsBuilder.immutableMap();
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) ListBlobItem(com.microsoft.azure.storage.blob.ListBlobItem) PlainBlobMetaData(org.elasticsearch.common.blobstore.support.PlainBlobMetaData) BlobMetaData(org.elasticsearch.common.blobstore.BlobMetaData) PlainBlobMetaData(org.elasticsearch.common.blobstore.support.PlainBlobMetaData) BlobProperties(com.microsoft.azure.storage.blob.BlobProperties) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) CloudBlockBlob(com.microsoft.azure.storage.blob.CloudBlockBlob) URI(java.net.URI)

Example 18 with CloudBlockBlob

use of com.microsoft.azure.storage.blob.CloudBlockBlob in project elasticsearch by elastic.

the class AzureStorageServiceImpl method moveBlob.

@Override
public void moveBlob(String account, LocationMode mode, String container, String sourceBlob, String targetBlob) throws URISyntaxException, StorageException {
    logger.debug("moveBlob container [{}], sourceBlob [{}], targetBlob [{}]", container, sourceBlob, targetBlob);
    CloudBlobClient client = this.getSelectedClient(account, mode);
    CloudBlobContainer blobContainer = client.getContainerReference(container);
    CloudBlockBlob blobSource = blobContainer.getBlockBlobReference(sourceBlob);
    if (blobSource.exists()) {
        CloudBlockBlob blobTarget = blobContainer.getBlockBlobReference(targetBlob);
        SocketAccess.doPrivilegedVoidException(() -> {
            blobTarget.startCopy(blobSource);
            blobSource.delete();
        });
        logger.debug("moveBlob container [{}], sourceBlob [{}], targetBlob [{}] -> done", container, sourceBlob, targetBlob);
    }
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) CloudBlockBlob(com.microsoft.azure.storage.blob.CloudBlockBlob)

Example 19 with CloudBlockBlob

use of com.microsoft.azure.storage.blob.CloudBlockBlob in project tdi-studio-se by Talend.

the class AzureFileSystem method copyFromLocal.

public void copyFromLocal(String localFile, String targetFolder) {
    try {
        File file = new File(localFile);
        CloudBlockBlob remotefile = this.container.getBlockBlobReference(targetFolder + "/" + file.getName());
        if (!file.exists())
            throw new FileNotFoundException();
        remotefile.upload(new FileInputStream(file), file.length());
    } catch (Exception e) {
        System.err.println(e.getMessage());
    }
}
Also used : FileNotFoundException(java.io.FileNotFoundException) CloudBlockBlob(com.microsoft.azure.storage.blob.CloudBlockBlob) File(java.io.File) FileInputStream(java.io.FileInputStream) URISyntaxException(java.net.URISyntaxException) FileNotFoundException(java.io.FileNotFoundException) StorageException(com.microsoft.azure.storage.StorageException) InvalidKeyException(java.security.InvalidKeyException)

Example 20 with CloudBlockBlob

use of com.microsoft.azure.storage.blob.CloudBlockBlob in project tdi-studio-se by Talend.

the class AzureFileSystem method open.

public InputStream open(String file) {
    InputStream is = null;
    try {
        CloudBlockBlob blob = this.container.getBlockBlobReference(file);
        is = blob.openInputStream();
    } catch (Exception e) {
        System.err.println(e.getMessage());
    }
    return is;
}
Also used : FileInputStream(java.io.FileInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) CloudBlockBlob(com.microsoft.azure.storage.blob.CloudBlockBlob) URISyntaxException(java.net.URISyntaxException) FileNotFoundException(java.io.FileNotFoundException) StorageException(com.microsoft.azure.storage.StorageException) InvalidKeyException(java.security.InvalidKeyException)

Aggregations

CloudBlockBlob (com.microsoft.azure.storage.blob.CloudBlockBlob)42 Test (org.junit.Test)17 StorageException (com.microsoft.azure.storage.StorageException)11 Path (org.apache.hadoop.fs.Path)11 URISyntaxException (java.net.URISyntaxException)10 BlobOutputStream (com.microsoft.azure.storage.blob.BlobOutputStream)9 CloudBlobContainer (com.microsoft.azure.storage.blob.CloudBlobContainer)9 CloudBlobClient (com.microsoft.azure.storage.blob.CloudBlobClient)8 FileInputStream (java.io.FileInputStream)6 InputStream (java.io.InputStream)6 DataStoreException (org.apache.jackrabbit.core.data.DataStoreException)6 FileNotFoundException (java.io.FileNotFoundException)5 BlockEntry (com.microsoft.azure.storage.blob.BlockEntry)4 ByteArrayInputStream (java.io.ByteArrayInputStream)4 IOException (java.io.IOException)4 JndiRegistry (org.apache.camel.impl.JndiRegistry)4 URI (java.net.URI)3 InvalidKeyException (java.security.InvalidKeyException)3 CloudStorageAccount (com.microsoft.azure.storage.CloudStorageAccount)2 CloudBlobDirectory (com.microsoft.azure.storage.blob.CloudBlobDirectory)2