Search in sources :

Example 96 with S3Client

use of software.amazon.awssdk.crt.s3.S3Client in project DataSpaceConnector by eclipse-dataspaceconnector.

the class S3BucketWriter method write.

@Override
public Result<Void> write(DataAddress destination, String name, InputStream data, String secretToken) {
    var bucketName = destination.getProperty(S3BucketSchema.BUCKET_NAME);
    var region = destination.getProperty(S3BucketSchema.REGION);
    var awsSecretToken = typeManager.readValue(secretToken, AwsTemporarySecretToken.class);
    try (S3Client s3 = S3Client.builder().credentialsProvider(buildCredentialsProvider(awsSecretToken)).region(Region.of(region)).build()) {
        PutObjectRequest request = createRequest(bucketName, name);
        PutObjectRequest completionMarker = createRequest(bucketName, name + ".complete");
        monitor.debug("Data request: begin transfer...");
        Failsafe.with(retryPolicy).get(() -> s3.putObject(request, RequestBody.fromBytes(data.readAllBytes())));
        Failsafe.with(retryPolicy).get(() -> s3.putObject(completionMarker, RequestBody.empty()));
        monitor.debug("Data request done.");
        return Result.success();
    } catch (S3Exception ex) {
        monitor.severe("Data request: transfer failed!");
        return Result.failure("Data transfer failed");
    }
}
Also used : S3Exception(software.amazon.awssdk.services.s3.model.S3Exception) S3Client(software.amazon.awssdk.services.s3.S3Client) PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest)

Example 97 with S3Client

use of software.amazon.awssdk.crt.s3.S3Client in project spring-cloud-aws by awspring.

the class CrossRegionS3ClientGenerator method addOverriddenMethods.

private static void addOverriddenMethods(ClassOrInterfaceDeclaration crossRegionS3Client) {
    TypeSolver typeSolver = new ClassLoaderTypeSolver(CrossRegionS3ClientGenerator.class.getClassLoader());
    ResolvedReferenceTypeDeclaration resolvedReferenceTypeDeclaration = typeSolver.solveType(S3Client.class.getName());
    resolvedReferenceTypeDeclaration.getAllMethods().stream().sorted(Comparator.comparing(MethodUsage::getSignature)).forEach(method -> {
        if (isRegionSpecific(method) && isCanonical(method)) {
            MethodDeclaration methodDeclaration = crossRegionS3Client.addMethod(method.getName(), Modifier.Keyword.PUBLIC);
            // collect & create method parameters
            List<String> parameterStrings = new ArrayList<>();
            for (int i = 0; i < method.getParamTypes().size(); i++) {
                methodDeclaration.addParameter(new Parameter(StaticJavaParser.parseClassOrInterfaceType(method.getParamTypes().get(i).describe()), "p" + i));
                parameterStrings.add("p" + i);
            }
            // handle generic types
            if (method.getDeclaration().getReturnType().isTypeVariable()) {
                methodDeclaration.addTypeParameter(method.getDeclaration().getReturnType().describe());
            }
            methodDeclaration.setType(StaticJavaParser.parseClassOrInterfaceType(method.getDeclaration().getReturnType().describe()));
            methodDeclaration.setBody(new BlockStmt().addStatement("return executeInBucketRegion(p0.bucket(), s3Client -> s3Client." + method.getName() + "(" + String.join(",", parameterStrings) + "));"));
            methodDeclaration.addMarkerAnnotation(Override.class);
            methodDeclaration.addThrownException(AwsServiceException.class);
            methodDeclaration.addThrownException(SdkClientException.class);
        }
    });
}
Also used : ResolvedReferenceTypeDeclaration(com.github.javaparser.resolution.declarations.ResolvedReferenceTypeDeclaration) MethodDeclaration(com.github.javaparser.ast.body.MethodDeclaration) BlockStmt(com.github.javaparser.ast.stmt.BlockStmt) ArrayList(java.util.ArrayList) TypeSolver(com.github.javaparser.symbolsolver.model.resolution.TypeSolver) ClassLoaderTypeSolver(com.github.javaparser.symbolsolver.resolution.typesolvers.ClassLoaderTypeSolver) Parameter(com.github.javaparser.ast.body.Parameter) MethodUsage(com.github.javaparser.resolution.MethodUsage) S3Client(software.amazon.awssdk.services.s3.S3Client) ClassLoaderTypeSolver(com.github.javaparser.symbolsolver.resolution.typesolvers.ClassLoaderTypeSolver)

Example 98 with S3Client

use of software.amazon.awssdk.crt.s3.S3Client in project spring-cloud-aws by awspring.

the class DiskBufferingS3OutputStreamTests method setsMd5hash.

@Test
void setsMd5hash() throws IOException {
    S3Client s3Client = mock(S3Client.class);
    try (DiskBufferingS3OutputStream diskBufferingS3OutputStream = new DiskBufferingS3OutputStream(new Location("bucket", "key"), s3Client, null)) {
        diskBufferingS3OutputStream.write("hello".getBytes(StandardCharsets.UTF_8));
    }
    ArgumentCaptor<PutObjectRequest> captor = ArgumentCaptor.forClass(PutObjectRequest.class);
    verify(s3Client).putObject(captor.capture(), any(RequestBody.class));
    assertThat(captor.getValue().contentMD5()).isNotNull();
}
Also used : S3Client(software.amazon.awssdk.services.s3.S3Client) PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest) RequestBody(software.amazon.awssdk.core.sync.RequestBody) Test(org.junit.jupiter.api.Test)

Example 99 with S3Client

use of software.amazon.awssdk.crt.s3.S3Client in project sis by apache.

the class ClientFileSystem method close.

/**
 * Closes all channels and other closeable objects associated with the file system and releases resources.
 * If the file system is already closed then invoking this method has no effect.
 */
@Override
public synchronized void close() throws IOException {
    final S3Client c = client;
    client = null;
    if (c != null)
        try {
            provider.dispose(accessKey);
            c.close();
        } catch (SdkException e) {
            throw new IOException(e);
        }
}
Also used : SdkException(software.amazon.awssdk.core.exception.SdkException) IOException(java.io.IOException) S3Client(software.amazon.awssdk.services.s3.S3Client)

Example 100 with S3Client

use of software.amazon.awssdk.crt.s3.S3Client in project fluency by komamitsu.

the class AwsS3SenderTest method close.

@Test
void close() {
    S3Client s3Client = mock(S3Client.class);
    S3ClientBuilder s3ClientBuilder = mock(S3ClientBuilder.class);
    doReturn(s3Client).when(s3ClientBuilder).build();
    AwsS3Sender sender = new AwsS3Sender(s3ClientBuilder);
    sender.close();
    verify(s3Client, times(1)).close();
}
Also used : S3ClientBuilder(software.amazon.awssdk.services.s3.S3ClientBuilder) S3Client(software.amazon.awssdk.services.s3.S3Client) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Aggregations

S3Client (software.amazon.awssdk.services.s3.S3Client)265 Test (org.junit.jupiter.api.Test)86 Region (software.amazon.awssdk.regions.Region)80 Test (org.junit.Test)54 URI (java.net.URI)36 S3Exception (software.amazon.awssdk.services.s3.model.S3Exception)28 GetObjectResponse (software.amazon.awssdk.services.s3.model.GetObjectResponse)27 ArrayList (java.util.ArrayList)26 Assertions.assertThat (org.assertj.core.api.Assertions.assertThat)24 List (java.util.List)23 Collectors (java.util.stream.Collectors)22 IOException (java.io.IOException)20 InputStream (java.io.InputStream)20 S3ClientBuilder (software.amazon.awssdk.services.s3.S3ClientBuilder)20 RequestBody (software.amazon.awssdk.core.sync.RequestBody)19 GetObjectRequest (software.amazon.awssdk.services.s3.model.GetObjectRequest)19 PutObjectRequest (software.amazon.awssdk.services.s3.model.PutObjectRequest)19 Map (java.util.Map)17 S3AsyncClient (software.amazon.awssdk.services.s3.S3AsyncClient)17 S3Object (software.amazon.awssdk.services.s3.model.S3Object)17