Search in sources :

Example 16 with Consumer

use of com.urbanairship.connect.java8.Consumer in project components by Talend.

the class SimpleFileIODatasetRuntimeTest method testGetSampleCsv_multipleSources.

@Test
public void testGetSampleCsv_multipleSources() throws Exception {
    RecordSet rs1 = getSimpleTestData(0);
    writeRandomCsvFile(mini.getFs(), "/user/test/input/part-00000", rs1);
    RecordSet rs2 = getSimpleTestData(100);
    writeRandomCsvFile(mini.getFs(), "/user/test/input/part-00001", rs2);
    RecordSet rs3 = getSimpleTestData(100);
    writeRandomCsvFile(mini.getFs(), "/user/test/input/part-00002", rs3);
    String fileSpec = mini.getFs().getUri().resolve("/user/test/input/").toString();
    // Configure the component.
    SimpleFileIODatasetProperties props = createDatasetProperties();
    props.format.setValue(SimpleFileIOFormat.CSV);
    props.path.setValue(fileSpec);
    // Create the runtime.
    SimpleFileIODatasetRuntime runtime = new SimpleFileIODatasetRuntime();
    runtime.initialize(null, props);
    // Attempt to get a sample using the runtime methods.
    {
        final List<IndexedRecord> actual = new ArrayList<>();
        runtime.getSample(15, new Consumer<IndexedRecord>() {

            @Override
            public void accept(IndexedRecord ir) {
                actual.add(ir);
            }
        });
        // Check that the result was limited (15 out of 30 records)
        assertThat(actual, hasSize(15));
    }
    // Run it again to verify that the static state is not retained.
    {
        final List<IndexedRecord> actual = new ArrayList<>();
        runtime.getSample(15, new Consumer<IndexedRecord>() {

            @Override
            public void accept(IndexedRecord ir) {
                actual.add(ir);
            }
        });
        assertThat(actual, hasSize(15));
    }
}
Also used : Consumer(org.talend.daikon.java8.Consumer) IndexedRecord(org.apache.avro.generic.IndexedRecord) ArrayList(java.util.ArrayList) List(java.util.List) RecordSet(org.talend.components.test.RecordSet) SimpleFileIODatasetProperties(org.talend.components.simplefileio.SimpleFileIODatasetProperties) Test(org.junit.Test)

Example 17 with Consumer

use of com.urbanairship.connect.java8.Consumer in project components by Talend.

the class JDBCDatasetTestIT method getSampleAction.

private void getSampleAction(JDBCDatasetProperties dataset) {
    JDBCDatasetRuntime runtime = new JDBCDatasetRuntime();
    runtime.initialize(null, dataset);
    final IndexedRecord[] record = new IndexedRecord[1];
    Consumer<IndexedRecord> storeTheRecords = new Consumer<IndexedRecord>() {

        @Override
        public void accept(IndexedRecord data) {
            record[0] = data;
        }
    };
    runtime.getSample(1, storeTheRecords);
    Assert.assertEquals("1", record[0].get(0));
    Assert.assertEquals("wangwei", record[0].get(1));
}
Also used : JDBCDatasetRuntime(org.talend.components.jdbc.runtime.dataprep.JDBCDatasetRuntime) IndexedRecord(org.apache.avro.generic.IndexedRecord) Consumer(org.talend.daikon.java8.Consumer)

Example 18 with Consumer

use of com.urbanairship.connect.java8.Consumer in project components by Talend.

the class SalesforceDatasetRuntimeTestIT method testGetSampleWithRelationshipQuery.

@Test
public void testGetSampleWithRelationshipQuery() {
    SalesforceDatastoreDefinition def = new SalesforceDatastoreDefinition();
    SalesforceDatastoreProperties datastore = new SalesforceDatastoreProperties("datastore");
    CommonTestUtils.setValueForDatastoreProperties(datastore);
    SalesforceDatasetProperties dataset = (SalesforceDatasetProperties) def.createDatasetProperties(datastore);
    dataset.sourceType.setValue(SalesforceDatasetProperties.SourceType.SOQL_QUERY);
    dataset.query.setValue("SELECT Account.Name from Contact");
    SalesforceDatasetRuntime runtime = new SalesforceDatasetRuntime();
    runtime.initialize(null, dataset);
    final IndexedRecord[] record = new IndexedRecord[1];
    Consumer<IndexedRecord> storeTheRecords = new Consumer<IndexedRecord>() {

        @Override
        public void accept(IndexedRecord data) {
            record[0] = data;
        }
    };
    runtime.getSample(1, storeTheRecords);
    Assert.assertTrue("empty result", record.length > 0);
    Assert.assertNotNull(record[0].getSchema().getField("Account_Name"));
}
Also used : SalesforceDatastoreProperties(org.talend.components.salesforce.datastore.SalesforceDatastoreProperties) IndexedRecord(org.apache.avro.generic.IndexedRecord) Consumer(org.talend.daikon.java8.Consumer) SalesforceDatastoreDefinition(org.talend.components.salesforce.datastore.SalesforceDatastoreDefinition) SalesforceDatasetProperties(org.talend.components.salesforce.dataset.SalesforceDatasetProperties) Test(org.junit.Test)

Aggregations

Test (org.junit.Test)13 Consumer (org.talend.daikon.java8.Consumer)10 Consumer (com.urbanairship.connect.java8.Consumer)8 InvocationOnMock (org.mockito.invocation.InvocationOnMock)8 StartPosition (com.urbanairship.connect.client.model.request.StartPosition)6 CountDownLatch (java.util.concurrent.CountDownLatch)6 Mockito.doAnswer (org.mockito.Mockito.doAnswer)6 Answer (org.mockito.stubbing.Answer)6 JsonObject (com.google.gson.JsonObject)5 IndexedRecord (org.apache.avro.generic.IndexedRecord)5 LinkedBlockingQueue (java.util.concurrent.LinkedBlockingQueue)4 ArrayList (java.util.ArrayList)3 JDBCDatasetRuntime (org.talend.components.jdbc.runtime.dataprep.JDBCDatasetRuntime)3 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)2 AtomicReference (java.util.concurrent.atomic.AtomicReference)2 StreamQueryDescriptor (com.urbanairship.connect.client.model.StreamQueryDescriptor)1 IOException (java.io.IOException)1 BigDecimal (java.math.BigDecimal)1 List (java.util.List)1 NoSuchElementException (java.util.NoSuchElementException)1