Examples with S3Object software.amazon.awssdk.services.s3.model.S3Object used on opensource projects

Example 1 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project elasticsearch by elastic.

the class MockAmazonS3 method getObject.

@Override
public S3Object getObject(GetObjectRequest getObjectRequest) throws AmazonClientException, AmazonServiceException {
    // in ESBlobStoreContainerTestCase.java, the prefix is empty,
    // so the key and blobName are equivalent to each other
    String blobName = getObjectRequest.getKey();
    if (!blobs.containsKey(blobName)) {
        throw new AmazonS3Exception("[" + blobName + "] does not exist.");
    }
    // the HTTP request attribute is irrelevant for reading
    S3ObjectInputStream stream = new S3ObjectInputStream(blobs.get(blobName), null, false);
    S3Object s3Object = new S3Object();
    s3Object.setObjectContent(stream);
    return s3Object;
}

Example 2 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project zeppelin by apache.

the class S3NotebookRepo method getNote.

private Note getNote(String key) throws IOException {
    GsonBuilder gsonBuilder = new GsonBuilder();
    gsonBuilder.setPrettyPrinting();
    Gson gson = gsonBuilder.registerTypeAdapter(Date.class, new NotebookImportDeserializer()).create();
    S3Object s3object;
    try {
        s3object = s3client.getObject(new GetObjectRequest(bucketName, key));
    } catch (AmazonClientException ace) {
        throw new IOException("Unable to retrieve object from S3: " + ace, ace);
    }
    Note note;
    try (InputStream ins = s3object.getObjectContent()) {
        String json = IOUtils.toString(ins, conf.getString(ConfVars.ZEPPELIN_ENCODING));
        note = gson.fromJson(json, Note.class);
    }
    for (Paragraph p : note.getParagraphs()) {
        if (p.getStatus() == Status.PENDING || p.getStatus() == Status.RUNNING) {
            p.setStatus(Status.ABORT);
        }
    }
    return note;
}

Example 3 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project crate by crate.

the class FileReadingCollectorTest method createBatchIterator.

private BatchIterator createBatchIterator(Collection<String> fileUris, String compression, final S3ObjectInputStream s3InputStream) {
    Reference raw = createReference("_raw", DataTypes.STRING);
    InputFactory.Context<LineCollectorExpression<?>> ctx = inputFactory.ctxForRefs(FileLineReferenceResolver::getImplementation);
    List<Input<?>> inputs = Collections.singletonList(ctx.add(raw));
    return FileReadingIterator.newInstance(fileUris, inputs, ctx.expressions(), compression, ImmutableMap.of(LocalFsFileInputFactory.NAME, new LocalFsFileInputFactory(), S3FileInputFactory.NAME, () -> new S3FileInput(new S3ClientHelper() {

        @Override
        protected AmazonS3 initClient(String accessKey, String secretKey) throws IOException {
            AmazonS3 client = mock(AmazonS3Client.class);
            ObjectListing objectListing = mock(ObjectListing.class);
            S3ObjectSummary summary = mock(S3ObjectSummary.class);
            S3Object s3Object = mock(S3Object.class);
            when(client.listObjects(anyString(), anyString())).thenReturn(objectListing);
            when(objectListing.getObjectSummaries()).thenReturn(Arrays.asList(summary));
            when(summary.getKey()).thenReturn("foo");
            when(client.getObject("fakebucket", "foo")).thenReturn(s3Object);
            when(s3Object.getObjectContent()).thenReturn(s3InputStream);
            when(client.listNextBatchOfObjects(any(ObjectListing.class))).thenReturn(objectListing);
            when(objectListing.isTruncated()).thenReturn(false);
            return client;
        }
    })), false, 1, 0);
}

Example 4 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project YCSB by brianfrankcooper.

the class S3Client method readFromStorage.

/**
  * Download an object from S3.
  *
  * @param bucket
  *            The name of the bucket
  * @param key
  *            The file key of the object to upload/update.
  * @param result
  *            The Hash map where data from the object are written
  *
  */
protected Status readFromStorage(String bucket, String key, HashMap<String, ByteIterator> result, SSECustomerKey ssecLocal) {
    try {
        Map.Entry<S3Object, ObjectMetadata> objectAndMetadata = getS3ObjectAndMetadata(bucket, key, ssecLocal);
        //consuming the stream
        InputStream objectData = objectAndMetadata.getKey().getObjectContent();
        // writing the stream to bytes and to results
        int sizeOfFile = (int) objectAndMetadata.getValue().getContentLength();
        byte[] inputStreamToByte = new byte[sizeOfFile];
        objectData.read(inputStreamToByte, 0, sizeOfFile);
        result.put(key, new ByteArrayByteIterator(inputStreamToByte));
        objectData.close();
        objectAndMetadata.getKey().close();
    } catch (Exception e) {
        System.err.println("Not possible to get the object " + key);
        e.printStackTrace();
        return Status.ERROR;
    }
    return Status.OK;
}

Example 5 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project gradle by gradle.

the class S3ResourceConnector method openResource.

public ExternalResourceReadResponse openResource(URI location, boolean revalidate) {
    LOGGER.debug("Attempting to get resource: {}", location);
    S3Object s3Object = s3Client.getResource(location);
    if (s3Object == null) {
        return null;
    }
    return new S3Resource(s3Object, location);
}