Search in sources :

Example 26 with S3ObjectMetadata

use of com.formkiq.aws.s3.S3ObjectMetadata in project iceberg by apache.

the class MockS3Client method putObject.

@Override
public PutObjectResult putObject(PutObjectRequest request) {
    S3ObjectMetadata metadata = request.getObjectMetadata();
    objectData.put(createId(request.getBucketName(), request.getKey()), ObjectData.create(convertContent(request.getEntity()), metadata));
    return new PutObjectResult();
}
Also used : PutObjectResult(com.emc.object.s3.bean.PutObjectResult) S3ObjectMetadata(com.emc.object.s3.S3ObjectMetadata)

Example 27 with S3ObjectMetadata

use of com.formkiq.aws.s3.S3ObjectMetadata in project iceberg by apache.

the class ObjectData method createFullMetadata.

public S3ObjectMetadata createFullMetadata() {
    S3ObjectMetadata metadata = new S3ObjectMetadata();
    metadata.setETag(DigestUtils.md5Hex(content));
    metadata.setContentLength((long) content.length);
    metadata.setUserMetadata(userMetadata);
    return metadata;
}
Also used : S3ObjectMetadata(com.emc.object.s3.S3ObjectMetadata)

Example 28 with S3ObjectMetadata

use of com.formkiq.aws.s3.S3ObjectMetadata in project formkiq-core by formkiq.

the class StagingS3Create method copyFile.

/**
 * Copies Documentid to a new file that is a {@link UUID}.
 *
 * @param s3Client {@link S3Client}
 * @param logger {@link LambdaLogger}
 * @param bucket {@link String}
 * @param originalkey {@link String}
 * @param date {@link Date}
 */
private void copyFile(final S3Client s3Client, final LambdaLogger logger, final String bucket, final String originalkey, final Date date) {
    String siteId = getSiteId(originalkey);
    String key = resetDatabaseKey(siteId, originalkey);
    boolean uuid = isUuid(key);
    String documentId = uuid ? key : UUID.randomUUID().toString();
    String destKey = createDatabaseKey(siteId, documentId);
    logger.log(String.format("Copying %s from bucket %s to %s in bucket %s.", originalkey, bucket, destKey, this.documentsBucket));
    S3ObjectMetadata metadata = this.s3.getObjectMetadata(s3Client, bucket, originalkey);
    String username = metadata.getMetadata().entrySet().stream().filter(s -> s.getKey().equalsIgnoreCase("userid")).findFirst().map(s -> s.getValue()).orElse("System");
    DynamicDocumentItem doc = new DynamicDocumentItem(Collections.emptyMap());
    doc.setDocumentId(documentId);
    doc.setContentLength(metadata.getContentLength());
    doc.setContentType(metadata.getContentType());
    doc.setUserId(username);
    doc.setChecksum(metadata.getEtag());
    doc.setInsertedDate(date);
    if (!uuid) {
        doc.setPath(key);
    }
    this.service.saveDocumentItemWithTag(siteId, doc);
    this.s3.copyObject(s3Client, bucket, originalkey, this.documentsBucket, destKey, metadata.getContentType());
}
Also used : SdkHttpUtils(software.amazon.awssdk.utils.http.SdkHttpUtils) DocumentService(com.formkiq.stacks.dynamodb.DocumentService) Date(java.util.Date) SimpleDateFormat(java.text.SimpleDateFormat) Context(com.amazonaws.services.lambda.runtime.Context) HashMap(java.util.HashMap) RequestHandler(com.amazonaws.services.lambda.runtime.RequestHandler) DynamicDocumentItem(com.formkiq.stacks.dynamodb.DynamicDocumentItem) SiteIdKeyGenerator.createDatabaseKey(com.formkiq.stacks.dynamodb.SiteIdKeyGenerator.createDatabaseKey) GsonBuilder(com.google.gson.GsonBuilder) DocumentServiceImpl(com.formkiq.stacks.dynamodb.DocumentServiceImpl) SiteIdKeyGenerator.resetDatabaseKey(com.formkiq.stacks.dynamodb.SiteIdKeyGenerator.resetDatabaseKey) LambdaLogger(com.amazonaws.services.lambda.runtime.LambdaLogger) Gson(com.google.gson.Gson) Map(java.util.Map) DocumentTag(com.formkiq.stacks.dynamodb.DocumentTag) S3ConnectionBuilder(com.formkiq.aws.s3.S3ConnectionBuilder) ParseException(java.text.ParseException) S3ObjectMetadata(com.formkiq.aws.s3.S3ObjectMetadata) Region(software.amazon.awssdk.regions.Region) S3Service(com.formkiq.aws.s3.S3Service) PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) SqsConnectionBuilder(com.formkiq.aws.sqs.SqsConnectionBuilder) DynamicObject(com.formkiq.stacks.common.objects.DynamicObject) DATE_FORMAT(com.formkiq.stacks.dynamodb.DocumentService.DATE_FORMAT) S3Client(software.amazon.awssdk.services.s3.S3Client) UUID(java.util.UUID) StandardCharsets(java.nio.charset.StandardCharsets) SqsService(com.formkiq.aws.sqs.SqsService) Reflectable(com.formkiq.graalvm.annotations.Reflectable) ReflectableImport(com.formkiq.graalvm.annotations.ReflectableImport) Base64(java.util.Base64) List(java.util.List) DocumentItemDynamoDb(com.formkiq.stacks.dynamodb.DocumentItemDynamoDb) SiteIdKeyGenerator.getSiteId(com.formkiq.stacks.dynamodb.SiteIdKeyGenerator.getSiteId) DynamoDbConnectionBuilder(com.formkiq.stacks.dynamodb.DynamoDbConnectionBuilder) Collections(java.util.Collections) DocumentTagType(com.formkiq.stacks.dynamodb.DocumentTagType) DynamicDocumentItem(com.formkiq.stacks.dynamodb.DynamicDocumentItem) S3ObjectMetadata(com.formkiq.aws.s3.S3ObjectMetadata)

Example 29 with S3ObjectMetadata

use of com.formkiq.aws.s3.S3ObjectMetadata in project formkiq-core by formkiq.

the class AbstractAwsTest method verifyFileExistsInDocumentsS3.

/**
 * Verify File exists in Documents S3 Bucket.
 *
 * @param s3 {@link S3Client}
 * @param key {@link String}
 * @param contentType {@link String}
 * @throws InterruptedException InterruptedException
 */
protected void verifyFileExistsInDocumentsS3(final S3Client s3, final String key, final String contentType) throws InterruptedException {
    while (true) {
        S3ObjectMetadata meta = getS3Service().getObjectMetadata(s3, getDocumentsbucketname(), key);
        if (meta.isObjectExists()) {
            assertEquals(contentType, meta.getContentType());
            break;
        }
        Thread.sleep(SLEEP);
    }
}
Also used : S3ObjectMetadata(com.formkiq.aws.s3.S3ObjectMetadata)

Example 30 with S3ObjectMetadata

use of com.formkiq.aws.s3.S3ObjectMetadata in project formkiq-core by formkiq.

the class AwsResourceTest method verifyFileNotExistInStagingS3.

/**
 * Verify File does NOT exist in Staging S3 Bucket.
 *
 * @param s3 {@link S3Client}
 * @param key {@link String}
 * @throws InterruptedException InterruptedException
 */
private void verifyFileNotExistInStagingS3(final S3Client s3, final String key) throws InterruptedException {
    while (true) {
        S3ObjectMetadata meta = getS3Service().getObjectMetadata(s3, getStagingdocumentsbucketname(), key);
        if (!meta.isObjectExists()) {
            assertFalse(meta.isObjectExists());
            break;
        }
        Thread.sleep(SLEEP);
    }
}
Also used : S3ObjectMetadata(com.formkiq.aws.s3.S3ObjectMetadata)

Aggregations

S3ObjectMetadata (com.emc.object.s3.S3ObjectMetadata)16 S3Exception (com.emc.object.s3.S3Exception)7 S3ObjectMetadata (com.formkiq.aws.s3.S3ObjectMetadata)7 S3ObjectMetadata (uk.nhs.digital.externalstorage.s3.S3ObjectMetadata)7 S3Client (software.amazon.awssdk.services.s3.S3Client)5 AccessControlList (com.emc.object.s3.bean.AccessControlList)4 PutObjectRequest (com.emc.object.s3.request.PutObjectRequest)4 Test (org.junit.Test)4 CopyPartResult (com.emc.object.s3.bean.CopyPartResult)3 MultipartPartETag (com.emc.object.s3.bean.MultipartPartETag)3 PutObjectResult (com.emc.object.s3.bean.PutObjectResult)3 CompleteMultipartUploadRequest (com.emc.object.s3.request.CompleteMultipartUploadRequest)3 CopyPartRequest (com.emc.object.s3.request.CopyPartRequest)3 ChunkAlreadyExistsException (io.pravega.segmentstore.storage.chunklayer.ChunkAlreadyExistsException)3 ChunkNotFoundException (io.pravega.segmentstore.storage.chunklayer.ChunkNotFoundException)3 ChunkStorageException (io.pravega.segmentstore.storage.chunklayer.ChunkStorageException)3 InvalidOffsetException (io.pravega.segmentstore.storage.chunklayer.InvalidOffsetException)3 Date (java.util.Date)3 TreeSet (java.util.TreeSet)3 CanonicalUser (com.emc.object.s3.bean.CanonicalUser)2