Search in sources :

Example 6 with GetObjectTaggingRequest

use of com.amazonaws.services.s3.model.GetObjectTaggingRequest in project aws-doc-sdk-examples by awsdocs.

the class GetObjectTags method main.

public static void main(String[] args) throws IOException {
    if (args.length < 2) {
        System.out.println("Please specify a bucket name and key name");
        System.exit(1);
    }
    // snippet-start:[s3.java.getobjecttags.main]
    String bucketName = args[0];
    String keyName = args[1];
    System.out.println("Retrieving Object Tags for  " + keyName);
    final AmazonS3 s3 = AmazonS3ClientBuilder.standard().withRegion(Regions.DEFAULT_REGION).build();
    try {
        GetObjectTaggingRequest getTaggingRequest = new GetObjectTaggingRequest(bucketName, keyName);
        GetObjectTaggingResult tags = s3.getObjectTagging(getTaggingRequest);
        List<Tag> tagSet = tags.getTagSet();
        // Iterate through the list
        Iterator<Tag> tagIterator = tagSet.iterator();
        while (tagIterator.hasNext()) {
            Tag tag = (Tag) tagIterator.next();
            System.out.println(tag.getKey());
            System.out.println(tag.getValue());
        }
    } catch (AmazonServiceException e) {
        System.err.println(e.getErrorMessage());
        System.exit(1);
    }
// snippet-end:[s3.java.getobjecttags.main]
}
Also used : AmazonS3(com.amazonaws.services.s3.AmazonS3) AmazonServiceException(com.amazonaws.AmazonServiceException)

Example 7 with GetObjectTaggingRequest

use of com.amazonaws.services.s3.model.GetObjectTaggingRequest in project XRTB by benmfaul.

the class Configuration method processDirectory.

public void processDirectory(AmazonS3Client s3, ObjectListing listing, String bucket) throws Exception {
    for (S3ObjectSummary objectSummary : listing.getObjectSummaries()) {
        long size = objectSummary.getSize();
        logger.info("*** Processing S3 {}, size: {}", objectSummary.getKey(), size);
        S3Object object = s3.getObject(new GetObjectRequest(bucket, objectSummary.getKey()));
        String bucketName = object.getBucketName();
        String keyName = object.getKey();
        GetObjectTaggingRequest request = new GetObjectTaggingRequest(bucketName, keyName);
        GetObjectTaggingResult result = s3.getObjectTagging(request);
        List<Tag> tags = result.getTagSet();
        String type = null;
        String name = null;
        if (tags.isEmpty()) {
            System.err.println("Error: " + keyName + " has no tags");
        } else {
            for (Tag tag : tags) {
                String key = tag.getKey();
                String value = tag.getValue();
                if (key.equals("type")) {
                    type = value;
                }
                if (key.equals("name")) {
                    name = value;
                }
            }
            if (name == null)
                throw new Exception("Error: " + keyName + " is missing a name tag");
            if (name.contains(" "))
                throw new Exception("Error: " + keyName + " has a name attribute with a space in it");
            if (type == null)
                throw new Exception("Error: " + keyName + " has no type tag");
            if (!name.startsWith("$"))
                name = "$" + name;
            readData(type, name, object, size);
        }
    }
}
Also used : GetObjectTaggingRequest(com.amazonaws.services.s3.model.GetObjectTaggingRequest) S3ObjectSummary(com.amazonaws.services.s3.model.S3ObjectSummary) S3Object(com.amazonaws.services.s3.model.S3Object) GeoTag(com.xrtb.geo.GeoTag) Tag(com.amazonaws.services.s3.model.Tag) GetObjectRequest(com.amazonaws.services.s3.model.GetObjectRequest) GetObjectTaggingResult(com.amazonaws.services.s3.model.GetObjectTaggingResult)

Example 8 with GetObjectTaggingRequest

use of com.amazonaws.services.s3.model.GetObjectTaggingRequest in project herd by FINRAOS.

the class S3DaoImpl method tagObjects.

@Override
public void tagObjects(final S3FileTransferRequestParamsDto s3FileTransferRequestParamsDto, final S3FileTransferRequestParamsDto s3ObjectTaggerParamsDto, final Tag tag) {
    LOGGER.info("Tagging objects in S3... s3BucketName=\"{}\" s3KeyCount={} s3ObjectTagKey=\"{}\" s3ObjectTagValue=\"{}\"", s3FileTransferRequestParamsDto.getS3BucketName(), s3FileTransferRequestParamsDto.getFiles().size(), tag.getKey(), tag.getValue());
    if (!CollectionUtils.isEmpty(s3FileTransferRequestParamsDto.getFiles())) {
        // Initialize a key value pair for the error message in the catch block.
        String s3Key = s3FileTransferRequestParamsDto.getFiles().get(0).getPath().replaceAll("\\\\", "/");
        // Amazon S3 client to access S3 objects.
        AmazonS3Client s3Client = null;
        // Amazon S3 client for S3 object tagging.
        AmazonS3Client s3ObjectTaggerClient = null;
        try {
            // Create an S3 client to access S3 objects.
            s3Client = getAmazonS3(s3FileTransferRequestParamsDto);
            // Create an S3 client for S3 object tagging.
            s3ObjectTaggerClient = getAmazonS3(s3ObjectTaggerParamsDto);
            // Create a get object tagging request.
            GetObjectTaggingRequest getObjectTaggingRequest = new GetObjectTaggingRequest(s3FileTransferRequestParamsDto.getS3BucketName(), null);
            // Create a restore object request.
            SetObjectTaggingRequest setObjectTaggingRequest = new SetObjectTaggingRequest(s3FileTransferRequestParamsDto.getS3BucketName(), null, null);
            for (File file : s3FileTransferRequestParamsDto.getFiles()) {
                // Prepare an S3 key.
                s3Key = file.getPath().replaceAll("\\\\", "/");
                // Retrieve the current tagging information for the S3 key.
                getObjectTaggingRequest.setKey(s3Key);
                GetObjectTaggingResult getObjectTaggingResult = s3Operations.getObjectTagging(getObjectTaggingRequest, s3Client);
                // Update the list of tags to include the specified S3 object tag.
                List<Tag> updatedTags = new ArrayList<>();
                updatedTags.add(tag);
                if (CollectionUtils.isNotEmpty(getObjectTaggingResult.getTagSet())) {
                    for (Tag currentTag : getObjectTaggingResult.getTagSet()) {
                        if (!StringUtils.equals(tag.getKey(), currentTag.getKey())) {
                            updatedTags.add(currentTag);
                        }
                    }
                }
                // Update the tagging information.
                setObjectTaggingRequest.setKey(s3Key);
                setObjectTaggingRequest.setTagging(new ObjectTagging(updatedTags));
                s3Operations.setObjectTagging(setObjectTaggingRequest, s3ObjectTaggerClient);
            }
        } catch (Exception e) {
            throw new IllegalStateException(String.format("Failed to tag S3 object with \"%s\" key in \"%s\" bucket. Reason: %s", s3Key, s3FileTransferRequestParamsDto.getS3BucketName(), e.getMessage()), e);
        } finally {
            if (s3Client != null) {
                s3Client.shutdown();
            }
            if (s3ObjectTaggerClient != null) {
                s3ObjectTaggerClient.shutdown();
            }
        }
    }
}
Also used : GetObjectTaggingRequest(com.amazonaws.services.s3.model.GetObjectTaggingRequest) AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) SetObjectTaggingRequest(com.amazonaws.services.s3.model.SetObjectTaggingRequest) ArrayList(java.util.ArrayList) Tag(com.amazonaws.services.s3.model.Tag) File(java.io.File) ObjectTagging(com.amazonaws.services.s3.model.ObjectTagging) MultiObjectDeleteException(com.amazonaws.services.s3.model.MultiObjectDeleteException) ObjectNotFoundException(org.finra.herd.model.ObjectNotFoundException) AmazonServiceException(com.amazonaws.AmazonServiceException) AmazonClientException(com.amazonaws.AmazonClientException) AmazonS3Exception(com.amazonaws.services.s3.model.AmazonS3Exception) GetObjectTaggingResult(com.amazonaws.services.s3.model.GetObjectTaggingResult)

Example 9 with GetObjectTaggingRequest

use of com.amazonaws.services.s3.model.GetObjectTaggingRequest in project herd by FINRAOS.

the class S3DaoTest method testTagObjectsOtherTagKeyAlreadyExists.

@Test
public void testTagObjectsOtherTagKeyAlreadyExists() {
    // Create two S3 object tags having different tag keys.
    List<Tag> tags = Arrays.asList(new Tag(S3_OBJECT_TAG_KEY, S3_OBJECT_TAG_VALUE), new Tag(S3_OBJECT_TAG_KEY_2, S3_OBJECT_TAG_VALUE_2));
    // Put a file in S3 that is already tagged with the first S3 object tag.
    PutObjectRequest putObjectRequest = new PutObjectRequest(S3_BUCKET_NAME, TARGET_S3_KEY, new ByteArrayInputStream(new byte[1]), new ObjectMetadata());
    putObjectRequest.setTagging(new ObjectTagging(Arrays.asList(tags.get(0))));
    s3Operations.putObject(putObjectRequest, null);
    // Validate that the S3 object is tagged with the first tag only.
    GetObjectTaggingResult getObjectTaggingResult = s3Operations.getObjectTagging(new GetObjectTaggingRequest(S3_BUCKET_NAME, TARGET_S3_KEY), null);
    assertEquals(Arrays.asList(tags.get(0)), getObjectTaggingResult.getTagSet());
    // Tag the S3 file with the second S3 object tag.
    S3FileTransferRequestParamsDto params = new S3FileTransferRequestParamsDto();
    params.setS3BucketName(S3_BUCKET_NAME);
    params.setFiles(Arrays.asList(new File(TARGET_S3_KEY)));
    s3Dao.tagObjects(params, new S3FileTransferRequestParamsDto(), tags.get(1));
    // Validate that the S3 object is now tagged with both tags.
    getObjectTaggingResult = s3Operations.getObjectTagging(new GetObjectTaggingRequest(S3_BUCKET_NAME, TARGET_S3_KEY), null);
    assertEquals(tags.size(), getObjectTaggingResult.getTagSet().size());
    assertTrue(getObjectTaggingResult.getTagSet().containsAll(tags));
}
Also used : GetObjectTaggingRequest(com.amazonaws.services.s3.model.GetObjectTaggingRequest) S3FileTransferRequestParamsDto(org.finra.herd.model.dto.S3FileTransferRequestParamsDto) ByteArrayInputStream(java.io.ByteArrayInputStream) Tag(com.amazonaws.services.s3.model.Tag) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata) File(java.io.File) PutObjectRequest(com.amazonaws.services.s3.model.PutObjectRequest) ObjectTagging(com.amazonaws.services.s3.model.ObjectTagging) GetObjectTaggingResult(com.amazonaws.services.s3.model.GetObjectTaggingResult) Test(org.junit.Test)

Example 10 with GetObjectTaggingRequest

use of com.amazonaws.services.s3.model.GetObjectTaggingRequest in project herd by FINRAOS.

the class StoragePolicyProcessorHelperServiceTest method runExecuteStoragePolicyTransitionTest.

private void runExecuteStoragePolicyTransitionTest() {
    // Create S3FileTransferRequestParamsDto to access the S3 bucket location.
    // Since test S3 key prefix represents a directory, we add a trailing '/' character to it.
    S3FileTransferRequestParamsDto s3FileTransferRequestParamsDto = S3FileTransferRequestParamsDto.builder().withS3BucketName(S3_BUCKET_NAME).withS3KeyPrefix(TEST_S3_KEY_PREFIX + "/").build();
    // Create a list of storage files.
    List<StorageFile> storageFiles = new ArrayList<>();
    for (String file : LOCAL_FILES) {
        storageFiles.add(new StorageFile(String.format(String.format("%s/%s", TEST_S3_KEY_PREFIX, file)), FILE_SIZE_1_KB, ROW_COUNT));
    }
    try {
        // Put relative S3 files into the S3 bucket.
        for (StorageFile storageFile : storageFiles) {
            s3Operations.putObject(new PutObjectRequest(S3_BUCKET_NAME, storageFile.getFilePath(), new ByteArrayInputStream(new byte[(int) FILE_SIZE_1_KB]), null), null);
        }
        // Execute a storage policy transition.
        storagePolicyProcessorHelperService.executeStoragePolicyTransition(new StoragePolicyTransitionParamsDto(new BusinessObjectDataKey(BDEF_NAMESPACE, BDEF_NAME, FORMAT_USAGE_CODE, FORMAT_FILE_TYPE_CODE, FORMAT_VERSION, PARTITION_VALUE, NO_SUBPARTITION_VALUES, DATA_VERSION), STORAGE_NAME, NO_S3_ENDPOINT, S3_BUCKET_NAME, TEST_S3_KEY_PREFIX, NO_STORAGE_UNIT_STATUS, NO_STORAGE_UNIT_STATUS, storageFiles, S3_ARCHIVE_TO_GLACIER_TAG_KEY, S3_ARCHIVE_TO_GLACIER_TAG_VALUE, S3_OBJECT_TAGGER_ROLE_ARN, S3_OBJECT_TAGGER_ROLE_SESSION_NAME));
        // Validate that all S3 files are now tagged.
        for (StorageFile storageFile : storageFiles) {
            GetObjectTaggingResult getObjectTaggingResult = s3Operations.getObjectTagging(new GetObjectTaggingRequest(S3_BUCKET_NAME, storageFile.getFilePath()), null);
            assertEquals(Arrays.asList(new Tag(S3_ARCHIVE_TO_GLACIER_TAG_KEY, S3_ARCHIVE_TO_GLACIER_TAG_VALUE)), getObjectTaggingResult.getTagSet());
        }
    } finally {
        // Delete test files from S3 storage.
        if (!s3Dao.listDirectory(s3FileTransferRequestParamsDto).isEmpty()) {
            s3Dao.deleteDirectory(s3FileTransferRequestParamsDto);
        }
        s3Operations.rollback();
    }
}
Also used : GetObjectTaggingRequest(com.amazonaws.services.s3.model.GetObjectTaggingRequest) S3FileTransferRequestParamsDto(org.finra.herd.model.dto.S3FileTransferRequestParamsDto) ByteArrayInputStream(java.io.ByteArrayInputStream) StorageFile(org.finra.herd.model.api.xml.StorageFile) ArrayList(java.util.ArrayList) StoragePolicyTransitionParamsDto(org.finra.herd.model.dto.StoragePolicyTransitionParamsDto) Tag(com.amazonaws.services.s3.model.Tag) BusinessObjectDataKey(org.finra.herd.model.api.xml.BusinessObjectDataKey) PutObjectRequest(com.amazonaws.services.s3.model.PutObjectRequest) GetObjectTaggingResult(com.amazonaws.services.s3.model.GetObjectTaggingResult)

Aggregations

GetObjectTaggingRequest (com.amazonaws.services.s3.model.GetObjectTaggingRequest)8 GetObjectTaggingResult (com.amazonaws.services.s3.model.GetObjectTaggingResult)8 Tag (com.amazonaws.services.s3.model.Tag)8 PutObjectRequest (com.amazonaws.services.s3.model.PutObjectRequest)5 ByteArrayInputStream (java.io.ByteArrayInputStream)5 File (java.io.File)5 ArrayList (java.util.ArrayList)5 S3FileTransferRequestParamsDto (org.finra.herd.model.dto.S3FileTransferRequestParamsDto)5 AmazonServiceException (com.amazonaws.AmazonServiceException)4 Test (org.junit.Test)4 AmazonS3 (com.amazonaws.services.s3.AmazonS3)3 ObjectMetadata (com.amazonaws.services.s3.model.ObjectMetadata)3 ObjectTagging (com.amazonaws.services.s3.model.ObjectTagging)3 GetObjectTaggingRequest (software.amazon.awssdk.services.s3.model.GetObjectTaggingRequest)3 GetObjectTaggingResponse (software.amazon.awssdk.services.s3.model.GetObjectTaggingResponse)3 S3Exception (software.amazon.awssdk.services.s3.model.S3Exception)3 Tag (software.amazon.awssdk.services.s3.model.Tag)3 BusinessObjectDataKey (org.finra.herd.model.api.xml.BusinessObjectDataKey)2 StorageFile (org.finra.herd.model.api.xml.StorageFile)2 PutObjectTaggingRequest (software.amazon.awssdk.services.s3.model.PutObjectTaggingRequest)2