Examples with ObjectMetadata com.talend.shaded.com.amazonaws.services.s3.model.ObjectMetadata used on opensource projects

Example 6 with ObjectMetadata

use of com.talend.shaded.com.amazonaws.services.s3.model.ObjectMetadata in project hadoop by apache.

the class TestS3AGetFileStatus method testFakeDirectory.

@Test
public void testFakeDirectory() throws Exception {
    Path path = new Path("/dir");
    String key = path.toUri().getPath().substring(1);
    when(s3.getObjectMetadata(argThat(correctGetMetadataRequest(BUCKET, key)))).thenThrow(NOT_FOUND);
    ObjectMetadata meta = new ObjectMetadata();
    meta.setContentLength(0L);
    when(s3.getObjectMetadata(argThat(correctGetMetadataRequest(BUCKET, key + "/")))).thenReturn(meta);
    FileStatus stat = fs.getFileStatus(path);
    assertNotNull(stat);
    assertEquals(fs.makeQualified(path), stat.getPath());
    assertTrue(stat.isDirectory());
}

Example 7 with ObjectMetadata

use of com.talend.shaded.com.amazonaws.services.s3.model.ObjectMetadata in project airpal by airbnb.

the class S3FilePersistor method persist.

@Override
public URI persist(JobOutputBuilder outputBuilder, Job job) {
    File file = checkNotNull(outputBuilder.build(), "output builder resulting file was null");
    val objectMetaData = new ObjectMetadata();
    objectMetaData.setContentLength(file.length());
    objectMetaData.setContentType(MediaType.CSV_UTF_8.toString());
    if (compressedOutput) {
        objectMetaData.setContentEncoding("gzip");
    }
    val putRequest = new PutObjectRequest(outputBucket, getOutputKey(file.getName()), file).withMetadata(objectMetaData);
    try {
        s3Client.putObject(putRequest);
        return UriBuilder.fromPath("/api/s3/{filename}").build(file.getName());
    } catch (AmazonClientException e) {
        throw new ExecutionClient.ExecutionFailureException(job, "Could not upload CSV to S3", e);
    } finally {
        outputBuilder.delete();
    }
}

Example 8 with ObjectMetadata

use of com.talend.shaded.com.amazonaws.services.s3.model.ObjectMetadata in project airpal by airbnb.

the class ResultsPreviewResource method getS3Preview.

private Response getS3Preview(URI fileURI, int numLines) {
    val filename = getFilename(fileURI);
    val outputKey = getOutputKey(filename);
    // download ~100 kb (depending on your definition) of the file
    val request = new GetObjectRequest(outputBucket, outputKey).withRange(0, 100 * 1024);
    val object = s3Client.getObject(request);
    ObjectMetadata objectMetadata = object.getObjectMetadata();
    boolean gzip = "gzip".equalsIgnoreCase(objectMetadata.getContentEncoding());
    try (InputStream input = object.getObjectContent()) {
        InputStreamReader reader;
        if (gzip) {
            reader = new InputStreamReader(new GZIPInputStream(input));
        } else {
            reader = new InputStreamReader(input);
        }
        return getPreviewFromCSV(new CSVReader(reader), numLines);
    } catch (IOException e) {
        return Response.status(Response.Status.INTERNAL_SERVER_ERROR).build();
    }
}

Example 9 with ObjectMetadata

use of com.talend.shaded.com.amazonaws.services.s3.model.ObjectMetadata in project ice by Netflix.

the class BillingFileProcessor method updateLastMillis.

private void updateLastMillis(long millis, String filename) {
    AmazonS3Client s3Client = AwsUtils.getAmazonS3Client();
    s3Client.putObject(config.workS3BucketName, config.workS3BucketPrefix + filename, IOUtils.toInputStream(millis + ""), new ObjectMetadata());
}

Example 10 with ObjectMetadata

use of com.talend.shaded.com.amazonaws.services.s3.model.ObjectMetadata in project alluxio by Alluxio.

the class S3AUnderFileSystem method createEmptyObject.

@Override
protected boolean createEmptyObject(String key) {
    try {
        ObjectMetadata meta = new ObjectMetadata();
        meta.setContentLength(0);
        meta.setContentMD5(DIR_HASH);
        meta.setContentType(Mimetypes.MIMETYPE_OCTET_STREAM);
        mClient.putObject(new PutObjectRequest(mBucketName, key, new ByteArrayInputStream(new byte[0]), meta));
        return true;
    } catch (AmazonClientException e) {
        LOG.error("Failed to create object: {}", key, e);
        return false;
    }
}