Search in sources :

Example 6 with DatasetRuntime

use of org.talend.components.common.dataset.runtime.DatasetRuntime in project components by Talend.

the class SandboxedSimpleFileIODatasetRuntimeTest method testBasic.

@Test
public void testBasic() throws Exception {
    File input = folder.newFile("stuff.csv");
    try (FileWriter fw = new FileWriter(input)) {
        fw.write("1;one");
    }
    SimpleFileIODatasetProperties props = createDatasetProperties();
    props.path.setValue(input.toURI().toString());
    final List<IndexedRecord> consumed = new ArrayList<>();
    RuntimeInfo ri = def.getRuntimeInfo(props);
    try (SandboxedInstance si = RuntimeUtil.createRuntimeClass(ri, getClass().getClassLoader())) {
        DatasetRuntime runtime = (DatasetRuntime) si.getInstance();
        runtime.initialize(null, props);
        assertThat(runtime, not(nullValue()));
        Schema s = runtime.getSchema();
        assertThat(s, not(nullValue()));
        runtime.getSample(100, new Consumer<IndexedRecord>() {

            @Override
            public void accept(IndexedRecord ir) {
                consumed.add(ir);
            }
        });
    }
    assertThat(consumed, hasSize(1));
    assertThat(consumed.get(0).get(0), is((Object) "1"));
    assertThat(consumed.get(0).get(1), is((Object) "one"));
}
Also used : IndexedRecord(org.apache.avro.generic.IndexedRecord) RuntimeInfo(org.talend.daikon.runtime.RuntimeInfo) DatasetRuntime(org.talend.components.common.dataset.runtime.DatasetRuntime) FileWriter(java.io.FileWriter) Schema(org.apache.avro.Schema) ArrayList(java.util.ArrayList) SandboxedInstance(org.talend.daikon.sandbox.SandboxedInstance) File(java.io.File) SimpleFileIODatasetProperties(org.talend.components.simplefileio.SimpleFileIODatasetProperties) Test(org.junit.Test)

Example 7 with DatasetRuntime

use of org.talend.components.common.dataset.runtime.DatasetRuntime in project components by Talend.

the class SimpleFileIODatasetRuntimeTest method testBasic.

@Test
public void testBasic() throws Exception {
    File input = folder.newFile("stuff.csv");
    try (FileWriter fw = new FileWriter(input)) {
        fw.write("1;one");
    }
    SimpleFileIODatasetProperties props = createDatasetProperties();
    props.path.setValue(input.toURI().toString());
    final List<IndexedRecord> consumed = new ArrayList<>();
    RuntimeInfo ri = def.getRuntimeInfo(props);
    try (SandboxedInstance si = RuntimeUtil.createRuntimeClass(ri, getClass().getClassLoader())) {
        DatasetRuntime runtime = (DatasetRuntime) si.getInstance();
        runtime.initialize(null, props);
        assertThat(runtime, not(nullValue()));
        Schema s = runtime.getSchema();
        assertThat(s, not(nullValue()));
        runtime.getSample(100, new Consumer<IndexedRecord>() {

            @Override
            public void accept(IndexedRecord ir) {
                consumed.add(ir);
            }
        });
    }
    assertThat(consumed, hasSize(1));
    assertThat(consumed.get(0).get(0), is((Object) "1"));
    assertThat(consumed.get(0).get(1), is((Object) "one"));
}
Also used : IndexedRecord(org.apache.avro.generic.IndexedRecord) RuntimeInfo(org.talend.daikon.runtime.RuntimeInfo) DatasetRuntime(org.talend.components.common.dataset.runtime.DatasetRuntime) FileWriter(java.io.FileWriter) Schema(org.apache.avro.Schema) ArrayList(java.util.ArrayList) SandboxedInstance(org.talend.daikon.sandbox.SandboxedInstance) File(java.io.File) SimpleFileIODatasetProperties(org.talend.components.simplefileio.SimpleFileIODatasetProperties) Test(org.junit.Test)

Example 8 with DatasetRuntime

use of org.talend.components.common.dataset.runtime.DatasetRuntime in project components by Talend.

the class JDBCDatasetProperties method updateSchema.

public void updateSchema() {
    JDBCDatasetDefinition definition = new JDBCDatasetDefinition();
    RuntimeInfo runtimeInfo = definition.getRuntimeInfo(this);
    try (SandboxedInstance sandboxedInstance = RuntimeUtil.createRuntimeClass(runtimeInfo, getClass().getClassLoader())) {
        DatasetRuntime<JDBCDatasetProperties> runtime = (DatasetRuntime) sandboxedInstance.getInstance();
        runtime.initialize(null, this);
        Schema schema = runtime.getSchema();
        main.schema.setValue(schema);
    }
}
Also used : SandboxedInstance(org.talend.daikon.sandbox.SandboxedInstance) RuntimeInfo(org.talend.daikon.runtime.RuntimeInfo) DatasetRuntime(org.talend.components.common.dataset.runtime.DatasetRuntime) Schema(org.apache.avro.Schema)

Aggregations

DatasetRuntime (org.talend.components.common.dataset.runtime.DatasetRuntime)8 SandboxedInstance (org.talend.daikon.sandbox.SandboxedInstance)8 RuntimeInfo (org.talend.daikon.runtime.RuntimeInfo)7 ArrayList (java.util.ArrayList)6 Schema (org.apache.avro.Schema)6 IndexedRecord (org.apache.avro.generic.IndexedRecord)6 Test (org.junit.Test)6 File (java.io.File)2 FileWriter (java.io.FileWriter)2 SimpleFileIODatasetProperties (org.talend.components.simplefileio.SimpleFileIODatasetProperties)2 GenericRecord (org.apache.avro.generic.GenericRecord)1 BigQueryDatasetProperties (org.talend.components.bigquery.BigQueryDatasetProperties)1 DatasetProperties (org.talend.components.common.dataset.DatasetProperties)1 DatastoreProperties (org.talend.components.common.datastore.DatastoreProperties)1 JDBCDatasetProperties (org.talend.components.jdbc.dataset.JDBCDatasetProperties)1 KafkaDatasetProperties (org.talend.components.kafka.dataset.KafkaDatasetProperties)1 FixedDatasetDefinition (org.talend.components.localio.fixed.FixedDatasetDefinition)1 FixedDatasetProperties (org.talend.components.localio.fixed.FixedDatasetProperties)1