Search in sources :

Example 1 with Timestamped

use of com.netflix.spinnaker.front50.api.model.Timestamped in project front50 by spinnaker.

the class AzureStorageService method listObjectVersions.

@Override
public <T extends Timestamped> Collection<T> listObjectVersions(ObjectType objectType, String objectKey, int maxResults) throws NotFoundException {
    Set<T> results = new HashSet<>();
    String fullKey = buildKeyPath(objectType.group, objectKey, objectType.defaultMetadataFilename);
    try {
        ResultContinuation token = null;
        EnumSet<BlobListingDetails> listDetails = EnumSet.of(BlobListingDetails.SNAPSHOTS);
        do {
            ResultSegment<ListBlobItem> result = getBlobContainer().listBlobsSegmented(fullKey, true, listDetails, NUM_OF_SNAPSHOT_LIMITATION, token, null, null);
            token = result.getContinuationToken();
            // By default, listBlobsSegmented with maxResult parameter will return oldest blob snapshots
            // But here we want the latest blob snapshots, and storage sdk doesn't provide a way to get
            // latest number of snapshots
            // So fetch all and then do sort and filter work (SnapshotID == null means the latest one)
            List<CloudBlockBlob> filteredResults = result.getResults().stream().map(item -> (CloudBlockBlob) item).sorted((a, b) -> {
                if (a.getSnapshotID() == null)
                    return -1;
                if (b.getSnapshotID() == null)
                    return 1;
                return b.getSnapshotID().compareTo(a.getSnapshotID());
            }).limit(maxResults).collect(Collectors.toList());
            for (ListBlobItem item : filteredResults) {
                CloudBlockBlob blob = (CloudBlockBlob) item;
                T blobObject = deserialize(blob, (Class<T>) objectType.clazz);
                blobObject.setLastModified(blob.getProperties().getLastModified().getTime());
                results.add(blobObject);
            }
        } while (token != null);
    } catch (StorageException se) {
        logStorageException(se, fullKey);
    } catch (Exception e) {
        log.error("Error retrieving versions for {} object: {}", value("key", fullKey), value("exception", e.getMessage()));
    }
    return results;
}
Also used : HttpURLConnection(java.net.HttpURLConnection) java.util(java.util) ResultContinuation(com.microsoft.azure.storage.ResultContinuation) Logger(org.slf4j.Logger) AuthenticatedRequest(com.netflix.spinnaker.security.AuthenticatedRequest) Pipeline(com.netflix.spinnaker.front50.api.model.pipeline.Pipeline) PipelineMixins(com.netflix.spinnaker.front50.jackson.mixins.PipelineMixins) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DateTime(org.joda.time.DateTime) LoggerFactory(org.slf4j.LoggerFactory) ResultSegment(com.microsoft.azure.storage.ResultSegment) IOException(java.io.IOException) CloudStorageAccount(com.microsoft.azure.storage.CloudStorageAccount) Collectors(java.util.stream.Collectors) StorageException(com.microsoft.azure.storage.StorageException) TimestampedMixins(com.netflix.spinnaker.front50.jackson.mixins.TimestampedMixins) StructuredArguments.value(net.logstash.logback.argument.StructuredArguments.value) Timestamped(com.netflix.spinnaker.front50.api.model.Timestamped) NotFoundException(com.netflix.spinnaker.kork.web.exceptions.NotFoundException) com.microsoft.azure.storage.blob(com.microsoft.azure.storage.blob) ResultContinuation(com.microsoft.azure.storage.ResultContinuation) IOException(java.io.IOException) StorageException(com.microsoft.azure.storage.StorageException) NotFoundException(com.netflix.spinnaker.kork.web.exceptions.NotFoundException) StorageException(com.microsoft.azure.storage.StorageException)

Example 2 with Timestamped

use of com.netflix.spinnaker.front50.api.model.Timestamped in project front50 by spinnaker.

the class S3StorageService method listObjectVersions.

@Override
public <T extends Timestamped> Collection<T> listObjectVersions(ObjectType objectType, String objectKey, int maxResults) throws NotFoundException {
    if (maxResults == 1) {
        List<T> results = new ArrayList<>();
        results.add(loadObject(objectType, objectKey));
        return results;
    }
    try {
        VersionListing versionListing = amazonS3.listVersions(new ListVersionsRequest(bucket, buildS3Key(objectType.group, objectKey, objectType.defaultMetadataFilename), null, null, null, maxResults));
        return versionListing.getVersionSummaries().stream().map(s3VersionSummary -> {
            try {
                S3Object s3Object = amazonS3.getObject(new GetObjectRequest(bucket, buildS3Key(objectType.group, objectKey, objectType.defaultMetadataFilename), s3VersionSummary.getVersionId()));
                T item = deserialize(s3Object, (Class<T>) objectType.clazz);
                item.setLastModified(s3Object.getObjectMetadata().getLastModified().getTime());
                return item;
            } catch (IOException e) {
                throw new IllegalStateException(e);
            }
        }).collect(Collectors.toList());
    } catch (AmazonS3Exception e) {
        if (e.getStatusCode() == 404) {
            throw new NotFoundException(String.format("No item found with id of %s", objectKey.toLowerCase()));
        }
        throw e;
    }
}
Also used : java.util(java.util) Logger(org.slf4j.Logger) AmazonServiceException(com.amazonaws.AmazonServiceException) Pipeline(com.netflix.spinnaker.front50.api.model.pipeline.Pipeline) PipelineMixins(com.netflix.spinnaker.front50.jackson.mixins.PipelineMixins) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) LoggerFactory(org.slf4j.LoggerFactory) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) IOException(java.io.IOException) com.amazonaws.services.s3.model(com.amazonaws.services.s3.model) Collectors(java.util.stream.Collectors) Lists(com.google.common.collect.Lists) ByteArrayInputStream(java.io.ByteArrayInputStream) StringUtils(com.amazonaws.util.StringUtils) Duration(java.time.Duration) TimestampedMixins(com.netflix.spinnaker.front50.jackson.mixins.TimestampedMixins) StructuredArguments.value(net.logstash.logback.argument.StructuredArguments.value) AmazonS3(com.amazonaws.services.s3.AmazonS3) DigestUtils(org.apache.commons.codec.digest.DigestUtils) Timestamped(com.netflix.spinnaker.front50.api.model.Timestamped) NotFoundException(com.netflix.spinnaker.kork.web.exceptions.NotFoundException) NotFoundException(com.netflix.spinnaker.kork.web.exceptions.NotFoundException) IOException(java.io.IOException)

Aggregations

ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)2 Timestamped (com.netflix.spinnaker.front50.api.model.Timestamped)2 Pipeline (com.netflix.spinnaker.front50.api.model.pipeline.Pipeline)2 PipelineMixins (com.netflix.spinnaker.front50.jackson.mixins.PipelineMixins)2 TimestampedMixins (com.netflix.spinnaker.front50.jackson.mixins.TimestampedMixins)2 NotFoundException (com.netflix.spinnaker.kork.web.exceptions.NotFoundException)2 IOException (java.io.IOException)2 java.util (java.util)2 Collectors (java.util.stream.Collectors)2 StructuredArguments.value (net.logstash.logback.argument.StructuredArguments.value)2 Logger (org.slf4j.Logger)2 LoggerFactory (org.slf4j.LoggerFactory)2 AmazonServiceException (com.amazonaws.AmazonServiceException)1 AmazonS3 (com.amazonaws.services.s3.AmazonS3)1 com.amazonaws.services.s3.model (com.amazonaws.services.s3.model)1 StringUtils (com.amazonaws.util.StringUtils)1 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)1 Lists (com.google.common.collect.Lists)1 CloudStorageAccount (com.microsoft.azure.storage.CloudStorageAccount)1 ResultContinuation (com.microsoft.azure.storage.ResultContinuation)1