Examples with S3DatasetProperties org.talend.components.simplefileio.s3.S3DatasetProperties used on opensource projects

Example 16 with S3DatasetProperties

use of org.talend.components.simplefileio.s3.S3DatasetProperties in project components by Talend.

the class S3SparkRuntimeTestIT method testCsv_noEncryption.

/**
 * Basic Csv test.
 */
@Test
@Ignore("columns name different, can't editable")
public void testCsv_noEncryption() throws IOException {
    S3DatasetProperties datasetProps = s3.createS3DatasetProperties();
    datasetProps.format.setValue(SimpleFileIOFormat.CSV);
    datasetProps.recordDelimiter.setValue(SimpleFileIODatasetProperties.RecordDelimiterType.LF);
    datasetProps.fieldDelimiter.setValue(SimpleFileIODatasetProperties.FieldDelimiterType.SEMICOLON);
    test_noEncryption(datasetProps);
}

Example 17 with S3DatasetProperties

use of org.talend.components.simplefileio.s3.S3DatasetProperties in project components by Talend.

the class S3SparkRuntimeTestIT method testAvro_cseKmsEncryption.

/**
 * Basic Avro test with cseKmsEncryption.
 */
@Ignore("cse not yet supported.")
@Test
public void testAvro_cseKmsEncryption() throws IOException {
    S3DatasetProperties datasetProps = s3.createS3DatasetProperties(false, true);
    datasetProps.format.setValue(SimpleFileIOFormat.AVRO);
    test_noEncryption(datasetProps);
}

Example 18 with S3DatasetProperties

use of org.talend.components.simplefileio.s3.S3DatasetProperties in project components by Talend.

the class S3OutputPropertiesTest method setup.

@Before
public void setup() {
    properties = new S3OutputProperties("test");
    S3DatastoreProperties datastoreProperties = new S3DatastoreProperties("test");
    datastoreProperties.init();
    S3DatasetProperties datasetProperties = new S3DatasetProperties("test");
    datasetProperties.init();
    datasetProperties.setDatastoreProperties(datastoreProperties);
    properties.setDatasetProperties(datasetProperties);
    properties.init();
}

Example 19 with S3DatasetProperties

use of org.talend.components.simplefileio.s3.S3DatasetProperties in project components by Talend.

the class S3RoundTripRuntimeTestIT method testAvro_cseKmsEncryption.

/**
 * Basic Avro test with cseKmsEncryption.
 */
@Ignore("cse not yet supported.")
@Test
public void testAvro_cseKmsEncryption() throws IOException {
    S3DatasetProperties datasetProps = s3.createS3DatasetProperties(false, true);
    datasetProps.format.setValue(SimpleFileIOFormat.AVRO);
    test_noEncryption(datasetProps);
}

Example 20 with S3DatasetProperties

use of org.talend.components.simplefileio.s3.S3DatasetProperties in project components by Talend.

the class S3RoundTripRuntimeTestIT method testParquet_noEncryption.

/**
 * Basic Parquet test.
 */
@Test
public void testParquet_noEncryption() throws IOException {
    S3DatasetProperties datasetProps = s3.createS3DatasetProperties();
    datasetProps.format.setValue(SimpleFileIOFormat.PARQUET);
    test_noEncryption(datasetProps);
}