Examples with S3Client org.jclouds.s3.S3Client used on opensource projects

Example 1 with S3Client

use of org.jclouds.s3.S3Client in project beam by apache.

the class S3WritableByteChannelTest method writeFromConfig.

private void writeFromConfig(S3FileSystemConfiguration config, boolean writeReadOnlyBuffer) throws IOException {
    S3Client mockS3Client = mock(S3Client.class, withSettings().defaultAnswer(RETURNS_SMART_NULLS));
    S3ResourceId path = S3ResourceId.fromUri("s3://bucket/dir/file");
    Supplier channel = () -> new S3WritableByteChannel(mockS3Client, path, "text/plain", config);
    write(mockS3Client, channel, path, config.getSSEAlgorithm(), toMd5(config.getSSECustomerKey()), config.getSSEKMSKeyId(), config.getS3UploadBufferSizeBytes(), config.getBucketKeyEnabled(), writeReadOnlyBuffer);
}

Example 2 with S3Client

use of org.jclouds.s3.S3Client in project beam by apache.

the class S3WritableByteChannelTest method writeFromOptions.

private void writeFromOptions(S3Options options, boolean writeReadOnlyBuffer) throws IOException {
    S3Client mockS3Client = mock(S3Client.class, withSettings().defaultAnswer(RETURNS_SMART_NULLS));
    S3ResourceId path = S3ResourceId.fromUri("s3://bucket/dir/file");
    Supplier channel = () -> new S3WritableByteChannel(mockS3Client, path, "text/plain", S3FileSystemConfiguration.fromS3Options(options));
    write(mockS3Client, channel, path, options.getSSEAlgorithm(), toMd5(options.getSSECustomerKey()), options.getSSEKMSKeyId(), options.getS3UploadBufferSizeBytes(), options.getBucketKeyEnabled(), writeReadOnlyBuffer);
}

Example 3 with S3Client

use of org.jclouds.s3.S3Client in project hazelcast by hazelcast.

the class S3TestBase method testSink.

void testSink(String bucketName, String prefix, int itemCount, String payload) {
    IMap<Integer, String> map = hz.getMap("map");
    for (int i = 0; i < itemCount; i++) {
        map.put(i, payload);
    }
    Pipeline p = Pipeline.create();
    p.readFrom(Sources.map(map, alwaysTrue(), Map.Entry::getValue)).writeTo(S3Sinks.s3(bucketName, prefix, CHARSET, clientSupplier(), Object::toString));
    hz.getJet().newJob(p).join();
    try (S3Client client = clientSupplier().get()) {
        assertTrueEventually(() -> {
            long lineCount = client.listObjectsV2(req -> req.bucket(bucketName).prefix(prefix)).contents().stream().flatMap(o -> s3ObjectToLines(o, client, bucketName)).peek(line -> assertEquals(payload, line)).count();
            assertEquals(itemCount, lineCount);
        });
    }
}

Example 4 with S3Client

use of org.jclouds.s3.S3Client in project flyway by flyway.

the class AwsS3Resource method read.

@Override
public Reader read() {
    S3Client s3 = S3ClientFactory.getClient();
    try {
        GetObjectRequest.Builder builder = GetObjectRequest.builder().bucket(bucketName).key(s3ObjectSummary.key());
        GetObjectRequest request = builder.build();
        ResponseInputStream o = s3.getObject(request);
        return Channels.newReader(Channels.newChannel(o), encoding.name());
    } catch (AwsServiceException e) {
        LOG.error(e.getMessage(), e);
        throw new FlywayException("Failed to get object from s3: " + e.getMessage(), e);
    }
}

Example 5 with S3Client

use of org.jclouds.s3.S3Client in project aws-doc-sdk-examples by awsdocs.

the class PutObject method main.

public static void main(String[] args) {
    final String USAGE = "\n" + "Usage:\n" + "  <bucketName> <objectKey> <objectPath> \n\n" + "Where:\n" + "  bucketName - the Amazon S3 bucket to upload an object into.\n" + "  objectKey - the object to upload (for example, book.pdf).\n" + "  objectPath - the path where the file is located (for example, C:/AWS/book2.pdf). \n\n";
    if (args.length != 3) {
        System.out.println(USAGE);
        System.exit(1);
    }
    String bucketName = args[0];
    String objectKey = args[1];
    String objectPath = args[2];
    System.out.println("Putting object " + objectKey + " into bucket " + bucketName);
    System.out.println("  in bucket: " + bucketName);
    Region region = Region.US_EAST_1;
    S3Client s3 = S3Client.builder().region(region).build();
    String result = putS3Object(s3, bucketName, objectKey, objectPath);
    System.out.println("Tag information: " + result);
    s3.close();
}