Search in sources :

Example 11 with NamedSink

use of com.airhacks.enhydrator.out.NamedSink in project enhydrator by AdamBien.

the class PipelineTest method getJSONToNashornPipeline.

public static Pipeline getJSONToNashornPipeline() {
    DestinationMapper targetMapper = new DestinationMapper();
    targetMapper.addMapping(0, new TargetMapping("*", "*"));
    DatatypeIndexMapper datatypeMapper = new DatatypeIndexMapper();
    datatypeMapper.addMapping(1, Datatype.INTEGER);
    Source source = new ScriptableSource("./src/test/files/languages.json", "./src/test/files/converter.js", "UTF-8");
    NamedSink logSink = new LogSink();
    NamedSink virtualSink = new ScriptableSink("./src/test/scripts/sink.js");
    Pipeline origin = new Pipeline("json", "src/test/scripts", null, source);
    origin.addSink(logSink);
    origin.addSink(virtualSink);
    origin.addPreRowTransformation(targetMapper);
    origin.addPreRowTransformation(datatypeMapper);
    origin.addPreRowTransformation(new NashornRowTransformer("src/test/scripts", "encrypt"));
    origin.addPostRowTransformation(new NashornRowTransformer("src/test/scripts", "compress"));
    origin.addFilter("true");
    origin.addExpression("print($ROW); $ROW");
    return origin;
}
Also used : DestinationMapper(com.airhacks.enhydrator.transform.DestinationMapper) LogSink(com.airhacks.enhydrator.out.LogSink) TargetMapping(com.airhacks.enhydrator.transform.TargetMapping) NashornRowTransformer(com.airhacks.enhydrator.transform.NashornRowTransformer) NamedSink(com.airhacks.enhydrator.out.NamedSink) DatatypeIndexMapper(com.airhacks.enhydrator.transform.DatatypeIndexMapper) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) Source(com.airhacks.enhydrator.in.Source) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource) JDBCSource(com.airhacks.enhydrator.in.JDBCSource) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource) ScriptableSink(com.airhacks.enhydrator.out.ScriptableSink)

Example 12 with NamedSink

use of com.airhacks.enhydrator.out.NamedSink in project enhydrator by AdamBien.

the class PipelineTest method getCSVPipeline.

public static Pipeline getCSVPipeline() {
    DestinationMapper targetMapper = new DestinationMapper();
    targetMapper.addMapping(0, new TargetMapping("*", "*"));
    DatatypeIndexMapper datatypeMapper = new DatatypeIndexMapper();
    datatypeMapper.addMapping(0, Datatype.DOUBLE);
    Source source = new CSVFileSource("./src/test/files/pyramid.csv", ";", "UTF-8", true);
    NamedSink logSink = new LogSink();
    NamedSink jdbcSink = new VirtualSinkSource();
    ColumnTransformation e1 = new ColumnTransformation("name", "convert", true);
    ColumnTransformation e2 = new ColumnTransformation(42, "compress", true);
    Pipeline origin = new Pipeline("csv", "src/test/scripts", "select * from Coffee where name like ? and strength = ?", source);
    origin.addSink(logSink);
    origin.addSink(jdbcSink);
    origin.addQueryParam("arabica");
    origin.addQueryParam(2);
    origin.addEntryTransformation(e1);
    origin.addEntryTransformation(e2);
    origin.addPreRowTransformation(targetMapper);
    origin.addPreRowTransformation(datatypeMapper);
    origin.addEntryTransformation(new ColumnTransformation("1", "function execute(i){return 42;}", false));
    origin.addPreRowTransformation(new NashornRowTransformer("src/test/scripts", "encrypt"));
    origin.addPostRowTransformation(new NashornRowTransformer("src/test/scripts", "compress"));
    origin.addFilter("true");
    origin.addExpression("print($ROW); $ROW");
    return origin;
}
Also used : DestinationMapper(com.airhacks.enhydrator.transform.DestinationMapper) LogSink(com.airhacks.enhydrator.out.LogSink) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) TargetMapping(com.airhacks.enhydrator.transform.TargetMapping) NashornRowTransformer(com.airhacks.enhydrator.transform.NashornRowTransformer) NamedSink(com.airhacks.enhydrator.out.NamedSink) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) DatatypeIndexMapper(com.airhacks.enhydrator.transform.DatatypeIndexMapper) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) Source(com.airhacks.enhydrator.in.Source) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource) JDBCSource(com.airhacks.enhydrator.in.JDBCSource) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource)

Example 13 with NamedSink

use of com.airhacks.enhydrator.out.NamedSink in project enhydrator by AdamBien.

the class PumpIT method getMockedSink.

static NamedSink getMockedSink() {
    NamedSink mock = mock(NamedSink.class);
    when(mock.getName()).thenReturn("*");
    return mock;
}
Also used : NamedSink(com.airhacks.enhydrator.out.NamedSink)

Example 14 with NamedSink

use of com.airhacks.enhydrator.out.NamedSink in project enhydrator by AdamBien.

the class PumpIT method postPreprocessor.

@Test
public void postPreprocessor() {
    CoffeeTestFixture.insertCoffee("arabica", 2, "hawai", Roast.LIGHT, "nice", "whole");
    CoffeeTestFixture.insertCoffee("niceone", 3, "russia", Roast.MEDIUM, "awful", "java beans");
    NamedSink consumer = getMockedSink();
    Pump pump = new Pump.Engine().from(source).endWith(l -> l).to(consumer).sqlQuery("select * from Coffee").build();
    pump.start();
    verify(consumer, times(2)).processRow(any(Row.class));
}
Also used : CoreMatchers.is(org.hamcrest.CoreMatchers.is) PipelineTest(com.airhacks.enhydrator.flexpipe.PipelineTest) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) Mockito.times(org.mockito.Mockito.times) Mockito.when(org.mockito.Mockito.when) Function(java.util.function.Function) ArrayList(java.util.ArrayList) Mockito.verify(org.mockito.Mockito.verify) Assert.assertThat(org.junit.Assert.assertThat) Matchers.any(org.mockito.Matchers.any) List(java.util.List) Mockito.never(org.mockito.Mockito.never) JDBCSource(com.airhacks.enhydrator.in.JDBCSource) After(org.junit.After) NamedSink(com.airhacks.enhydrator.out.NamedSink) Pipeline(com.airhacks.enhydrator.flexpipe.Pipeline) Row(com.airhacks.enhydrator.in.Row) Before(org.junit.Before) Mockito.mock(org.mockito.Mockito.mock) NamedSink(com.airhacks.enhydrator.out.NamedSink) Row(com.airhacks.enhydrator.in.Row) PipelineTest(com.airhacks.enhydrator.flexpipe.PipelineTest) Test(org.junit.Test)

Aggregations

NamedSink (com.airhacks.enhydrator.out.NamedSink)14 PipelineTest (com.airhacks.enhydrator.flexpipe.PipelineTest)9 Row (com.airhacks.enhydrator.in.Row)9 Test (org.junit.Test)9 JDBCSource (com.airhacks.enhydrator.in.JDBCSource)8 Pipeline (com.airhacks.enhydrator.flexpipe.Pipeline)4 LogSink (com.airhacks.enhydrator.out.LogSink)4 DestinationMapper (com.airhacks.enhydrator.transform.DestinationMapper)4 NashornRowTransformer (com.airhacks.enhydrator.transform.NashornRowTransformer)4 TargetMapping (com.airhacks.enhydrator.transform.TargetMapping)4 ArrayList (java.util.ArrayList)4 List (java.util.List)4 Function (java.util.function.Function)4 CoreMatchers.is (org.hamcrest.CoreMatchers.is)4 After (org.junit.After)4 Assert.assertThat (org.junit.Assert.assertThat)4 Assert.assertTrue (org.junit.Assert.assertTrue)4 Before (org.junit.Before)4 Matchers.any (org.mockito.Matchers.any)4 Mockito.mock (org.mockito.Mockito.mock)4