Search in sources :

Example 1 with GetObjectTaggingRequest

use of com.amazonaws.services.s3.model.GetObjectTaggingRequest in project herd by FINRAOS.

the class S3DaoTest method testTagObjectsTargetTagKeyAlreadyExists.

@Test
public void testTagObjectsTargetTagKeyAlreadyExists() {
    // Create two S3 object tags having the same tag key.
    List<Tag> tags = Arrays.asList(new Tag(S3_OBJECT_TAG_KEY, S3_OBJECT_TAG_VALUE), new Tag(S3_OBJECT_TAG_KEY, S3_OBJECT_TAG_VALUE_2));
    // Put a file in S3 that is already tagged with the first S3 object tag.
    PutObjectRequest putObjectRequest = new PutObjectRequest(S3_BUCKET_NAME, TARGET_S3_KEY, new ByteArrayInputStream(new byte[1]), new ObjectMetadata());
    putObjectRequest.setTagging(new ObjectTagging(Arrays.asList(tags.get(0))));
    s3Operations.putObject(putObjectRequest, null);
    // Validate that the S3 object is tagged with the first tag.
    GetObjectTaggingResult getObjectTaggingResult = s3Operations.getObjectTagging(new GetObjectTaggingRequest(S3_BUCKET_NAME, TARGET_S3_KEY), null);
    assertEquals(Arrays.asList(tags.get(0)), getObjectTaggingResult.getTagSet());
    // Tag the S3 file with the second S3 object tag.
    S3FileTransferRequestParamsDto params = new S3FileTransferRequestParamsDto();
    params.setS3BucketName(S3_BUCKET_NAME);
    params.setFiles(Arrays.asList(new File(TARGET_S3_KEY)));
    s3Dao.tagObjects(params, new S3FileTransferRequestParamsDto(), tags.get(1));
    // Validate that the S3 object is tagged with the second tag now.
    getObjectTaggingResult = s3Operations.getObjectTagging(new GetObjectTaggingRequest(S3_BUCKET_NAME, TARGET_S3_KEY), null);
    assertEquals(Arrays.asList(tags.get(1)), getObjectTaggingResult.getTagSet());
}
Also used : GetObjectTaggingRequest(com.amazonaws.services.s3.model.GetObjectTaggingRequest) S3FileTransferRequestParamsDto(org.finra.herd.model.dto.S3FileTransferRequestParamsDto) ByteArrayInputStream(java.io.ByteArrayInputStream) Tag(com.amazonaws.services.s3.model.Tag) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata) File(java.io.File) PutObjectRequest(com.amazonaws.services.s3.model.PutObjectRequest) ObjectTagging(com.amazonaws.services.s3.model.ObjectTagging) GetObjectTaggingResult(com.amazonaws.services.s3.model.GetObjectTaggingResult) Test(org.junit.Test)

Example 2 with GetObjectTaggingRequest

use of com.amazonaws.services.s3.model.GetObjectTaggingRequest in project herd by FINRAOS.

the class S3DaoTest method testTagObjects.

@Test
public void testTagObjects() {
    // Create an S3 object tag.
    Tag tag = new Tag(S3_OBJECT_TAG_KEY, S3_OBJECT_TAG_VALUE);
    // Put a file in S3.
    s3Operations.putObject(new PutObjectRequest(S3_BUCKET_NAME, TARGET_S3_KEY, new ByteArrayInputStream(new byte[1]), new ObjectMetadata()), null);
    // Tag the file with an S3 object tag.
    S3FileTransferRequestParamsDto params = new S3FileTransferRequestParamsDto();
    params.setS3BucketName(S3_BUCKET_NAME);
    params.setFiles(Arrays.asList(new File(TARGET_S3_KEY)));
    s3Dao.tagObjects(params, new S3FileTransferRequestParamsDto(), tag);
    // Validate that the object got tagged.
    GetObjectTaggingResult getObjectTaggingResult = s3Operations.getObjectTagging(new GetObjectTaggingRequest(S3_BUCKET_NAME, TARGET_S3_KEY), null);
    assertEquals(Arrays.asList(tag), getObjectTaggingResult.getTagSet());
}
Also used : GetObjectTaggingRequest(com.amazonaws.services.s3.model.GetObjectTaggingRequest) S3FileTransferRequestParamsDto(org.finra.herd.model.dto.S3FileTransferRequestParamsDto) ByteArrayInputStream(java.io.ByteArrayInputStream) Tag(com.amazonaws.services.s3.model.Tag) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata) File(java.io.File) PutObjectRequest(com.amazonaws.services.s3.model.PutObjectRequest) GetObjectTaggingResult(com.amazonaws.services.s3.model.GetObjectTaggingResult) Test(org.junit.Test)

Example 3 with GetObjectTaggingRequest

use of com.amazonaws.services.s3.model.GetObjectTaggingRequest in project herd by FINRAOS.

the class BusinessObjectDataServiceDestroyBusinessObjectDataTest method testDestroyBusinessObjectData.

@Test
public void testDestroyBusinessObjectData() throws Exception {
    // Create a primary partition value that satisfies the retention threshold check.
    String primaryPartitionValue = DateFormatUtils.format(DateUtils.addDays(new Date(), -1 * (RETENTION_PERIOD_DAYS + 1)), AbstractHerdDao.DEFAULT_SINGLE_DAY_DATE_MASK);
    // Build the expected S3 key prefix for test business object data.
    String s3KeyPrefix = getExpectedS3KeyPrefix(BDEF_NAMESPACE, DATA_PROVIDER_NAME, BDEF_NAME, FORMAT_USAGE_CODE, FORMAT_FILE_TYPE_CODE, FORMAT_VERSION, PARTITION_KEY, primaryPartitionValue, null, null, DATA_VERSION);
    // Create S3FileTransferRequestParamsDto to access the S3 bucket location.
    // Since test S3 key prefix represents a directory, we add a trailing '/' character to it.
    S3FileTransferRequestParamsDto s3FileTransferRequestParamsDto = S3FileTransferRequestParamsDto.builder().withS3BucketName(S3_BUCKET_NAME).withS3KeyPrefix(s3KeyPrefix + "/").build();
    // Create an S3 storage with the relative attributes.
    storageDaoTestHelper.createStorageEntity(STORAGE_NAME, StoragePlatformEntity.S3, Arrays.asList(new Attribute(configurationHelper.getProperty(ConfigurationValue.S3_ATTRIBUTE_NAME_BUCKET_NAME), AbstractServiceTest.S3_BUCKET_NAME), new Attribute(configurationHelper.getProperty(ConfigurationValue.S3_ATTRIBUTE_NAME_KEY_PREFIX_VELOCITY_TEMPLATE), AbstractServiceTest.S3_KEY_PREFIX_VELOCITY_TEMPLATE), new Attribute(configurationHelper.getProperty(ConfigurationValue.S3_ATTRIBUTE_NAME_VALIDATE_PATH_PREFIX), Boolean.TRUE.toString()), new Attribute(configurationHelper.getProperty(ConfigurationValue.S3_ATTRIBUTE_NAME_VALIDATE_FILE_EXISTENCE), Boolean.TRUE.toString())));
    // Create a business object data key.
    BusinessObjectDataKey businessObjectDataKey = new BusinessObjectDataKey(BDEF_NAMESPACE, BDEF_NAME, FORMAT_USAGE_CODE, FORMAT_FILE_TYPE_CODE, FORMAT_VERSION, primaryPartitionValue, NO_SUBPARTITION_VALUES, DATA_VERSION);
    // Create and persist a storage unit in the storage.
    StorageUnitEntity storageUnitEntity = storageUnitDaoTestHelper.createStorageUnitEntity(STORAGE_NAME, businessObjectDataKey, LATEST_VERSION_FLAG_SET, BusinessObjectDataStatusEntity.VALID, StorageUnitStatusEntity.ENABLED, NO_STORAGE_DIRECTORY_PATH);
    // Add storage files to the storage unit.
    for (String filePath : LOCAL_FILES) {
        storageFileDaoTestHelper.createStorageFileEntity(storageUnitEntity, s3KeyPrefix + "/" + filePath, FILE_SIZE_1_KB, ROW_COUNT_1000);
    }
    // Get the storage files.
    List<StorageFile> storageFiles = storageFileHelper.createStorageFilesFromEntities(storageUnitEntity.getStorageFiles());
    // Get the business object format entity.
    BusinessObjectFormatEntity businessObjectFormatEntity = storageUnitEntity.getBusinessObjectData().getBusinessObjectFormat();
    // Set the retention information for the business object format, which is the latest version business object format.
    businessObjectFormatEntity.setRetentionType(retentionTypeDao.getRetentionTypeByCode(RetentionTypeEntity.PARTITION_VALUE));
    businessObjectFormatEntity.setRetentionPeriodInDays(RETENTION_PERIOD_DAYS);
    businessObjectFormatDao.saveAndRefresh(businessObjectFormatEntity);
    // Override configuration to specify some settings required for testing.
    Map<String, Object> overrideMap = new HashMap<>();
    overrideMap.put(ConfigurationValue.S3_OBJECT_DELETE_TAG_KEY.getKey(), S3_OBJECT_TAG_KEY);
    overrideMap.put(ConfigurationValue.S3_OBJECT_DELETE_TAG_VALUE.getKey(), S3_OBJECT_TAG_VALUE);
    overrideMap.put(ConfigurationValue.S3_OBJECT_DELETE_ROLE_ARN.getKey(), S3_OBJECT_TAGGER_ROLE_ARN);
    overrideMap.put(ConfigurationValue.S3_OBJECT_DELETE_ROLE_SESSION_NAME.getKey(), S3_OBJECT_TAGGER_ROLE_SESSION_NAME);
    modifyPropertySourceInEnvironment(overrideMap);
    try {
        // Put relative S3 files into the S3 bucket.
        for (StorageFile storageFile : storageFiles) {
            s3Operations.putObject(new PutObjectRequest(S3_BUCKET_NAME, storageFile.getFilePath(), new ByteArrayInputStream(new byte[storageFile.getFileSizeBytes().intValue()]), null), null);
        }
        // Request to destroy business object data.
        BusinessObjectData result = businessObjectDataService.destroyBusinessObjectData(businessObjectDataKey);
        // Validate the result.
        assertNotNull(result);
        assertEquals(storageUnitEntity.getBusinessObjectData().getId(), Integer.valueOf(result.getId()));
        // Validate the status of the storage unit entity.
        assertEquals(StorageUnitStatusEntity.DISABLED, storageUnitEntity.getStatus().getCode());
        // Validate the status of the business object data entity.
        assertEquals(BusinessObjectDataStatusEntity.DELETED, storageUnitEntity.getBusinessObjectData().getStatus().getCode());
        // Validate that all S3 files are now tagged.
        for (StorageFile storageFile : storageFiles) {
            GetObjectTaggingResult getObjectTaggingResult = s3Operations.getObjectTagging(new GetObjectTaggingRequest(S3_BUCKET_NAME, storageFile.getFilePath()), null);
            assertEquals(Arrays.asList(new Tag(S3_OBJECT_TAG_KEY, S3_OBJECT_TAG_VALUE)), getObjectTaggingResult.getTagSet());
        }
    } finally {
        // Delete test files from S3 storage.
        if (!s3Dao.listDirectory(s3FileTransferRequestParamsDto).isEmpty()) {
            s3Dao.deleteDirectory(s3FileTransferRequestParamsDto);
        }
        s3Operations.rollback();
        // Restore the property sources so we don't affect other tests.
        restorePropertySourceInEnvironment();
    }
}
Also used : GetObjectTaggingRequest(com.amazonaws.services.s3.model.GetObjectTaggingRequest) S3FileTransferRequestParamsDto(org.finra.herd.model.dto.S3FileTransferRequestParamsDto) Attribute(org.finra.herd.model.api.xml.Attribute) StorageUnitEntity(org.finra.herd.model.jpa.StorageUnitEntity) HashMap(java.util.HashMap) BusinessObjectData(org.finra.herd.model.api.xml.BusinessObjectData) BusinessObjectDataKey(org.finra.herd.model.api.xml.BusinessObjectDataKey) BusinessObjectFormatEntity(org.finra.herd.model.jpa.BusinessObjectFormatEntity) Date(java.util.Date) GetObjectTaggingResult(com.amazonaws.services.s3.model.GetObjectTaggingResult) ByteArrayInputStream(java.io.ByteArrayInputStream) StorageFile(org.finra.herd.model.api.xml.StorageFile) Tag(com.amazonaws.services.s3.model.Tag) PutObjectRequest(com.amazonaws.services.s3.model.PutObjectRequest) Test(org.junit.Test)

Example 4 with GetObjectTaggingRequest

use of com.amazonaws.services.s3.model.GetObjectTaggingRequest in project aws-doc-sdk-examples by awsdocs.

the class ManagingObjectTags method updateObjectTags.

public static void updateObjectTags(S3Client s3, String bucketName, String objectKey) {
    try {
        // Retrieve the object's tags.
        GetObjectTaggingRequest taggingRequest = GetObjectTaggingRequest.builder().bucket(bucketName).key(objectKey).build();
        GetObjectTaggingResponse getTaggingRes = s3.getObjectTagging(taggingRequest);
        // Write out the tags.
        List<Tag> obTags = getTaggingRes.tagSet();
        for (Tag sinTag : obTags) {
            System.out.println("The tag key is: " + sinTag.key());
            System.out.println("The tag value is: " + sinTag.value());
        }
        // Replace the object's tags with two new tags.
        Tag tag3 = Tag.builder().key("Tag 3").value("This is tag 3").build();
        Tag tag4 = Tag.builder().key("Tag 4").value("This is tag 4").build();
        List<Tag> tags = new ArrayList<Tag>();
        tags.add(tag3);
        tags.add(tag4);
        Tagging updatedTags = Tagging.builder().tagSet(tags).build();
        PutObjectTaggingRequest taggingRequest1 = PutObjectTaggingRequest.builder().bucket(bucketName).key(objectKey).tagging(updatedTags).build();
        s3.putObjectTagging(taggingRequest1);
        // Write out the modified tags.
        GetObjectTaggingResponse getTaggingRes2 = s3.getObjectTagging(taggingRequest);
        List<Tag> modTags = getTaggingRes2.tagSet();
        for (Tag sinTag : modTags) {
            System.out.println("The tag key is: " + sinTag.key());
            System.out.println("The tag value is: " + sinTag.value());
        }
    } catch (S3Exception e) {
        System.err.println(e.getMessage());
        System.exit(1);
    }
}
Also used : GetObjectTaggingRequest(software.amazon.awssdk.services.s3.model.GetObjectTaggingRequest) PutObjectTaggingRequest(software.amazon.awssdk.services.s3.model.PutObjectTaggingRequest) GetObjectTaggingResponse(software.amazon.awssdk.services.s3.model.GetObjectTaggingResponse) S3Exception(software.amazon.awssdk.services.s3.model.S3Exception) ArrayList(java.util.ArrayList) Tagging(software.amazon.awssdk.services.s3.model.Tagging) Tag(software.amazon.awssdk.services.s3.model.Tag)

Example 5 with GetObjectTaggingRequest

use of com.amazonaws.services.s3.model.GetObjectTaggingRequest in project aws-doc-sdk-examples by awsdocs.

the class ManagingObjectTags method main.

public static void main(String[] args) {
    Regions clientRegion = Regions.DEFAULT_REGION;
    String bucketName = "*** Bucket name ***";
    String keyName = "*** Object key ***";
    String filePath = "*** File path ***";
    try {
        AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withCredentials(new ProfileCredentialsProvider()).withRegion(clientRegion).build();
        // Create an object, add two new tags, and upload the object to Amazon S3.
        PutObjectRequest putRequest = new PutObjectRequest(bucketName, keyName, new File(filePath));
        List<Tag> tags = new ArrayList<Tag>();
        tags.add(new Tag("Tag 1", "This is tag 1"));
        tags.add(new Tag("Tag 2", "This is tag 2"));
        putRequest.setTagging(new ObjectTagging(tags));
        PutObjectResult putResult = s3Client.putObject(putRequest);
        // Retrieve the object's tags.
        GetObjectTaggingRequest getTaggingRequest = new GetObjectTaggingRequest(bucketName, keyName);
        GetObjectTaggingResult getTagsResult = s3Client.getObjectTagging(getTaggingRequest);
        // Replace the object's tags with two new tags.
        List<Tag> newTags = new ArrayList<Tag>();
        newTags.add(new Tag("Tag 3", "This is tag 3"));
        newTags.add(new Tag("Tag 4", "This is tag 4"));
        s3Client.setObjectTagging(new SetObjectTaggingRequest(bucketName, keyName, new ObjectTagging(newTags)));
    } catch (AmazonServiceException e) {
        // The call was transmitted successfully, but Amazon S3 couldn't process
        // it, so it returned an error response.
        e.printStackTrace();
    } catch (SdkClientException e) {
        // Amazon S3 couldn't be contacted for a response, or the client
        // couldn't parse the response from Amazon S3.
        e.printStackTrace();
    }
}
Also used : AmazonS3(com.amazonaws.services.s3.AmazonS3) ArrayList(java.util.ArrayList) Regions(com.amazonaws.regions.Regions) SdkClientException(com.amazonaws.SdkClientException) AmazonServiceException(com.amazonaws.AmazonServiceException) ProfileCredentialsProvider(com.amazonaws.auth.profile.ProfileCredentialsProvider) File(java.io.File)

Aggregations

GetObjectTaggingRequest (com.amazonaws.services.s3.model.GetObjectTaggingRequest)8 GetObjectTaggingResult (com.amazonaws.services.s3.model.GetObjectTaggingResult)8 Tag (com.amazonaws.services.s3.model.Tag)8 PutObjectRequest (com.amazonaws.services.s3.model.PutObjectRequest)5 ByteArrayInputStream (java.io.ByteArrayInputStream)5 File (java.io.File)5 ArrayList (java.util.ArrayList)5 S3FileTransferRequestParamsDto (org.finra.herd.model.dto.S3FileTransferRequestParamsDto)5 AmazonServiceException (com.amazonaws.AmazonServiceException)4 Test (org.junit.Test)4 AmazonS3 (com.amazonaws.services.s3.AmazonS3)3 ObjectMetadata (com.amazonaws.services.s3.model.ObjectMetadata)3 ObjectTagging (com.amazonaws.services.s3.model.ObjectTagging)3 GetObjectTaggingRequest (software.amazon.awssdk.services.s3.model.GetObjectTaggingRequest)3 GetObjectTaggingResponse (software.amazon.awssdk.services.s3.model.GetObjectTaggingResponse)3 S3Exception (software.amazon.awssdk.services.s3.model.S3Exception)3 Tag (software.amazon.awssdk.services.s3.model.Tag)3 BusinessObjectDataKey (org.finra.herd.model.api.xml.BusinessObjectDataKey)2 StorageFile (org.finra.herd.model.api.xml.StorageFile)2 PutObjectTaggingRequest (software.amazon.awssdk.services.s3.model.PutObjectTaggingRequest)2