Search in sources :

Example 11 with InputSerialization

use of com.amazonaws.services.s3.model.InputSerialization in project aws-sdk-java-v2 by aws.

the class SelectObjectContentIntegrationTest method executeSqlQueryWithHandler.

private static CompletableFuture<Void> executeSqlQueryWithHandler(String query, SelectObjectContentResponseHandler handler) {
    InputSerialization inputSerialization = InputSerialization.builder().csv(CSVInput.builder().build()).compressionType(CompressionType.NONE).build();
    OutputSerialization outputSerialization = OutputSerialization.builder().csv(CSVOutput.builder().build()).build();
    SelectObjectContentRequest select = SelectObjectContentRequest.builder().bucket(BUCKET_NAME).key(KEY).expression(query).expressionType(ExpressionType.SQL).inputSerialization(inputSerialization).outputSerialization(outputSerialization).build();
    return s3Async.selectObjectContent(select, handler);
}
Also used : SelectObjectContentRequest(software.amazon.awssdk.services.s3.model.SelectObjectContentRequest) InputSerialization(software.amazon.awssdk.services.s3.model.InputSerialization) OutputSerialization(software.amazon.awssdk.services.s3.model.OutputSerialization)

Example 12 with InputSerialization

use of com.amazonaws.services.s3.model.InputSerialization in project pxf by greenplum-db.

the class S3SelectAccessorTest method testFileHeaderInfoIsNone.

@Test
public void testFileHeaderInfoIsNone() {
    RequestContext context = getDefaultRequestContext();
    context.addOption("FILE_HEADER", "NONE");
    InputSerialization inputSerialization = new S3SelectAccessor().getInputSerialization(context);
    assertEquals("NONE", inputSerialization.getCsv().getFileHeaderInfo());
}
Also used : InputSerialization(com.amazonaws.services.s3.model.InputSerialization) RequestContext(org.greenplum.pxf.api.model.RequestContext) Test(org.junit.jupiter.api.Test)

Example 13 with InputSerialization

use of com.amazonaws.services.s3.model.InputSerialization in project pxf by greenplum-db.

the class S3SelectAccessorTest method testParquetInputSerialization.

@Test
public void testParquetInputSerialization() {
    RequestContext context = getRequestContext("s3:parquet");
    context.setFormat("parquet");
    InputSerialization inputSerialization = new S3SelectAccessor().getInputSerialization(context);
    assertNotNull(inputSerialization.getParquet());
    assertNull(inputSerialization.getJson());
    assertNull(inputSerialization.getCsv());
}
Also used : InputSerialization(com.amazonaws.services.s3.model.InputSerialization) RequestContext(org.greenplum.pxf.api.model.RequestContext) Test(org.junit.jupiter.api.Test)

Example 14 with InputSerialization

use of com.amazonaws.services.s3.model.InputSerialization in project pxf by greenplum-db.

the class S3SelectAccessorTest method testPipeDelimiter.

@Test
public void testPipeDelimiter() {
    RequestContext context = getDefaultRequestContext();
    context.getGreenplumCSV().withDelimiter("|");
    InputSerialization inputSerialization = new S3SelectAccessor().getInputSerialization(context);
    assertEquals('|', inputSerialization.getCsv().getFieldDelimiter().charValue());
}
Also used : InputSerialization(com.amazonaws.services.s3.model.InputSerialization) RequestContext(org.greenplum.pxf.api.model.RequestContext) Test(org.junit.jupiter.api.Test)

Example 15 with InputSerialization

use of com.amazonaws.services.s3.model.InputSerialization in project pxf by greenplum-db.

the class S3SelectAccessorTest method testCompressionTypeGZIP.

@Test
public void testCompressionTypeGZIP() {
    RequestContext context = getDefaultRequestContext();
    context.addOption("COMPRESSION_CODEC", "GZIP");
    InputSerialization inputSerialization = new S3SelectAccessor().getInputSerialization(context);
    assertEquals("GZIP", inputSerialization.getCompressionType());
}
Also used : InputSerialization(com.amazonaws.services.s3.model.InputSerialization) RequestContext(org.greenplum.pxf.api.model.RequestContext) Test(org.junit.jupiter.api.Test)

Aggregations

InputSerialization (com.amazonaws.services.s3.model.InputSerialization)20 RequestContext (org.greenplum.pxf.api.model.RequestContext)14 Test (org.junit.jupiter.api.Test)14 OutputSerialization (com.amazonaws.services.s3.model.OutputSerialization)5 SelectObjectContentRequest (com.amazonaws.services.s3.model.SelectObjectContentRequest)5 CSVInput (com.amazonaws.services.s3.model.CSVInput)4 CSVOutput (com.amazonaws.services.s3.model.CSVOutput)4 URI (java.net.URI)4 BZip2Codec (org.apache.hadoop.io.compress.BZip2Codec)3 CompressionCodec (org.apache.hadoop.io.compress.CompressionCodec)3 GzipCodec (org.apache.hadoop.io.compress.GzipCodec)3 PrestoException (com.facebook.presto.spi.PrestoException)2 InputSerialization (software.amazon.awssdk.services.s3.model.InputSerialization)2 OutputSerialization (software.amazon.awssdk.services.s3.model.OutputSerialization)2 SelectObjectContentRequest (software.amazon.awssdk.services.s3.model.SelectObjectContentRequest)2 ParquetInput (com.amazonaws.services.s3.model.ParquetInput)1 PrestoException (io.prestosql.spi.PrestoException)1 SQLException (java.sql.SQLException)1