Search in sources :

Example 11 with CloudBlobDirectory

use of com.microsoft.azure.storage.blob.CloudBlobDirectory in project jackrabbit-oak by apache.

the class SegmentTarFixture method setUpCluster.

@Override
public Oak[] setUpCluster(int n, StatisticsProvider statsProvider) throws Exception {
    init(n);
    Oak[] cluster = new Oak[n];
    for (int i = 0; i < cluster.length; i++) {
        BlobStore blobStore = null;
        if (useBlobStore) {
            blobStoreFixtures[i] = BlobStoreFixture.create(parentPath, true, dsCacheSize, statsProvider);
            blobStore = blobStoreFixtures[i].setUp();
        }
        FileStoreBuilder builder = fileStoreBuilder(new File(parentPath, "primary-" + i));
        if (azureConnectionString != null) {
            CloudStorageAccount cloud = CloudStorageAccount.parse(azureConnectionString);
            CloudBlobContainer container = cloud.createCloudBlobClient().getContainerReference(azureContainerName);
            container.createIfNotExists();
            CloudBlobDirectory directory = container.getDirectoryReference(azureRootPath + "/primary-" + i);
            builder.withCustomPersistence(new AzurePersistence(directory));
        }
        if (blobStore != null) {
            builder.withBlobStore(blobStore);
        }
        stores[i] = builder.withMaxFileSize(maxFileSize).withStatisticsProvider(statsProvider).withSegmentCacheSize(segmentCacheSize).withMemoryMapping(memoryMapping).withStrictVersionCheck(true).build();
        if (withColdStandby) {
            attachStandby(i, n, statsProvider, blobStore);
        }
        cluster[i] = newOak(SegmentNodeStoreBuilders.builder(stores[i]).build());
    }
    return cluster;
}
Also used : FileStoreBuilder(org.apache.jackrabbit.oak.segment.file.FileStoreBuilder) AzurePersistence(org.apache.jackrabbit.oak.segment.azure.AzurePersistence) Oak(org.apache.jackrabbit.oak.Oak) CloudStorageAccount(com.microsoft.azure.storage.CloudStorageAccount) CloudBlobDirectory(com.microsoft.azure.storage.blob.CloudBlobDirectory) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) File(java.io.File) DataStoreBlobStore(org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore) BlobStore(org.apache.jackrabbit.oak.spi.blob.BlobStore)

Example 12 with CloudBlobDirectory

use of com.microsoft.azure.storage.blob.CloudBlobDirectory in project crate by crate.

the class AzureStorageService method children.

public Set<String> children(String account, String container, BlobPath path) throws URISyntaxException, StorageException {
    final var blobsBuilder = new HashSet<String>();
    final Tuple<CloudBlobClient, Supplier<OperationContext>> client = client();
    final CloudBlobContainer blobContainer = client.v1().getContainerReference(container);
    final String keyPath = path.buildAsString();
    final EnumSet<BlobListingDetails> enumBlobListingDetails = EnumSet.of(BlobListingDetails.METADATA);
    for (ListBlobItem blobItem : blobContainer.listBlobs(keyPath, false, enumBlobListingDetails, null, client.v2().get())) {
        if (blobItem instanceof CloudBlobDirectory) {
            final URI uri = blobItem.getUri();
            LOGGER.trace(() -> new ParameterizedMessage("blob url [{}]", uri));
            // uri.getPath is of the form /container/keyPath.* and we want to strip off the /container/
            // this requires 1 + container.length() + 1, with each 1 corresponding to one of the /.
            // Lastly, we add the length of keyPath to the offset to strip this container's path.
            final String uriPath = uri.getPath();
            blobsBuilder.add(uriPath.substring(1 + container.length() + 1 + keyPath.length(), uriPath.length() - 1));
        }
    }
    return Set.copyOf(blobsBuilder);
}
Also used : CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) BlobListingDetails(com.microsoft.azure.storage.blob.BlobListingDetails) ListBlobItem(com.microsoft.azure.storage.blob.ListBlobItem) CloudBlobDirectory(com.microsoft.azure.storage.blob.CloudBlobDirectory) URI(java.net.URI) Supplier(java.util.function.Supplier) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) ParameterizedMessage(org.apache.logging.log4j.message.ParameterizedMessage) HashSet(java.util.HashSet)

Aggregations

CloudBlobDirectory (com.microsoft.azure.storage.blob.CloudBlobDirectory)12 StorageException (com.microsoft.azure.storage.StorageException)6 URISyntaxException (java.net.URISyntaxException)6 ListBlobItem (com.microsoft.azure.storage.blob.ListBlobItem)5 IOException (java.io.IOException)5 CloudBlobContainer (com.microsoft.azure.storage.blob.CloudBlobContainer)4 DataStoreException (org.apache.jackrabbit.core.data.DataStoreException)4 CloudStorageAccount (com.microsoft.azure.storage.CloudStorageAccount)3 CloudBlob (com.microsoft.azure.storage.blob.CloudBlob)3 AzurePersistence (org.apache.jackrabbit.oak.segment.azure.AzurePersistence)3 BlobListingDetails (com.microsoft.azure.storage.blob.BlobListingDetails)2 CloudBlockBlob (com.microsoft.azure.storage.blob.CloudBlockBlob)2 DataIdentifier (org.apache.jackrabbit.core.data.DataIdentifier)2 DataStoreBlobStore (org.apache.jackrabbit.oak.plugins.blob.datastore.DataStoreBlobStore)2 FileStore (org.apache.jackrabbit.oak.segment.file.FileStore)2 FileStoreBuilder (org.apache.jackrabbit.oak.segment.file.FileStoreBuilder)2 BlobStore (org.apache.jackrabbit.oak.spi.blob.BlobStore)2 CloudBlobClient (com.microsoft.azure.storage.blob.CloudBlobClient)1 File (java.io.File)1 FileNotFoundException (java.io.FileNotFoundException)1