Search in sources :

Example 1 with DatatypeIndexMapper

use of com.airhacks.enhydrator.transform.DatatypeIndexMapper in project enhydrator by AdamBien.

the class PipelineTest method getJSONPipeline.

public static Pipeline getJSONPipeline() {
    DestinationMapper targetMapper = new DestinationMapper();
    targetMapper.addMapping(0, new TargetMapping("*", "*"));
    DatatypeIndexMapper datatypeMapper = new DatatypeIndexMapper();
    datatypeMapper.addMapping(1, Datatype.INTEGER);
    Source source = new ScriptableSource("./src/test/files/languages.json", "./src/test/files/converter.js", "UTF-8");
    NamedSink logSink = new LogSink();
    NamedSink virtualSink = new VirtualSinkSource();
    Pipeline origin = new Pipeline("json", "src/test/scripts", null, source);
    origin.addSink(logSink);
    origin.addSink(virtualSink);
    origin.addPreRowTransformation(targetMapper);
    origin.addPreRowTransformation(datatypeMapper);
    origin.addPreRowTransformation(new NashornRowTransformer("src/test/scripts", "encrypt"));
    origin.addPostRowTransformation(new NashornRowTransformer("src/test/scripts", "compress"));
    origin.addFilter("true");
    origin.addExpression("print($ROW); $ROW");
    return origin;
}
Also used : DestinationMapper(com.airhacks.enhydrator.transform.DestinationMapper) LogSink(com.airhacks.enhydrator.out.LogSink) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) TargetMapping(com.airhacks.enhydrator.transform.TargetMapping) NashornRowTransformer(com.airhacks.enhydrator.transform.NashornRowTransformer) NamedSink(com.airhacks.enhydrator.out.NamedSink) DatatypeIndexMapper(com.airhacks.enhydrator.transform.DatatypeIndexMapper) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) Source(com.airhacks.enhydrator.in.Source) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource) JDBCSource(com.airhacks.enhydrator.in.JDBCSource) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource)

Example 2 with DatatypeIndexMapper

use of com.airhacks.enhydrator.transform.DatatypeIndexMapper in project enhydrator by AdamBien.

the class PipelineTest method getJSONToNashornPipeline.

public static Pipeline getJSONToNashornPipeline() {
    DestinationMapper targetMapper = new DestinationMapper();
    targetMapper.addMapping(0, new TargetMapping("*", "*"));
    DatatypeIndexMapper datatypeMapper = new DatatypeIndexMapper();
    datatypeMapper.addMapping(1, Datatype.INTEGER);
    Source source = new ScriptableSource("./src/test/files/languages.json", "./src/test/files/converter.js", "UTF-8");
    NamedSink logSink = new LogSink();
    NamedSink virtualSink = new ScriptableSink("./src/test/scripts/sink.js");
    Pipeline origin = new Pipeline("json", "src/test/scripts", null, source);
    origin.addSink(logSink);
    origin.addSink(virtualSink);
    origin.addPreRowTransformation(targetMapper);
    origin.addPreRowTransformation(datatypeMapper);
    origin.addPreRowTransformation(new NashornRowTransformer("src/test/scripts", "encrypt"));
    origin.addPostRowTransformation(new NashornRowTransformer("src/test/scripts", "compress"));
    origin.addFilter("true");
    origin.addExpression("print($ROW); $ROW");
    return origin;
}
Also used : DestinationMapper(com.airhacks.enhydrator.transform.DestinationMapper) LogSink(com.airhacks.enhydrator.out.LogSink) TargetMapping(com.airhacks.enhydrator.transform.TargetMapping) NashornRowTransformer(com.airhacks.enhydrator.transform.NashornRowTransformer) NamedSink(com.airhacks.enhydrator.out.NamedSink) DatatypeIndexMapper(com.airhacks.enhydrator.transform.DatatypeIndexMapper) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) Source(com.airhacks.enhydrator.in.Source) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource) JDBCSource(com.airhacks.enhydrator.in.JDBCSource) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource) ScriptableSink(com.airhacks.enhydrator.out.ScriptableSink)

Example 3 with DatatypeIndexMapper

use of com.airhacks.enhydrator.transform.DatatypeIndexMapper in project enhydrator by AdamBien.

the class PipelineTest method getCSVPipeline.

public static Pipeline getCSVPipeline() {
    DestinationMapper targetMapper = new DestinationMapper();
    targetMapper.addMapping(0, new TargetMapping("*", "*"));
    DatatypeIndexMapper datatypeMapper = new DatatypeIndexMapper();
    datatypeMapper.addMapping(0, Datatype.DOUBLE);
    Source source = new CSVFileSource("./src/test/files/pyramid.csv", ";", "UTF-8", true);
    NamedSink logSink = new LogSink();
    NamedSink jdbcSink = new VirtualSinkSource();
    ColumnTransformation e1 = new ColumnTransformation("name", "convert", true);
    ColumnTransformation e2 = new ColumnTransformation(42, "compress", true);
    Pipeline origin = new Pipeline("csv", "src/test/scripts", "select * from Coffee where name like ? and strength = ?", source);
    origin.addSink(logSink);
    origin.addSink(jdbcSink);
    origin.addQueryParam("arabica");
    origin.addQueryParam(2);
    origin.addEntryTransformation(e1);
    origin.addEntryTransformation(e2);
    origin.addPreRowTransformation(targetMapper);
    origin.addPreRowTransformation(datatypeMapper);
    origin.addEntryTransformation(new ColumnTransformation("1", "function execute(i){return 42;}", false));
    origin.addPreRowTransformation(new NashornRowTransformer("src/test/scripts", "encrypt"));
    origin.addPostRowTransformation(new NashornRowTransformer("src/test/scripts", "compress"));
    origin.addFilter("true");
    origin.addExpression("print($ROW); $ROW");
    return origin;
}
Also used : DestinationMapper(com.airhacks.enhydrator.transform.DestinationMapper) LogSink(com.airhacks.enhydrator.out.LogSink) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) TargetMapping(com.airhacks.enhydrator.transform.TargetMapping) NashornRowTransformer(com.airhacks.enhydrator.transform.NashornRowTransformer) NamedSink(com.airhacks.enhydrator.out.NamedSink) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) DatatypeIndexMapper(com.airhacks.enhydrator.transform.DatatypeIndexMapper) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) Source(com.airhacks.enhydrator.in.Source) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource) JDBCSource(com.airhacks.enhydrator.in.JDBCSource) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource)

Aggregations

CSVFileSource (com.airhacks.enhydrator.in.CSVFileSource)3 JDBCSource (com.airhacks.enhydrator.in.JDBCSource)3 ScriptableSource (com.airhacks.enhydrator.in.ScriptableSource)3 Source (com.airhacks.enhydrator.in.Source)3 VirtualSinkSource (com.airhacks.enhydrator.in.VirtualSinkSource)3 LogSink (com.airhacks.enhydrator.out.LogSink)3 NamedSink (com.airhacks.enhydrator.out.NamedSink)3 DatatypeIndexMapper (com.airhacks.enhydrator.transform.DatatypeIndexMapper)3 DestinationMapper (com.airhacks.enhydrator.transform.DestinationMapper)3 NashornRowTransformer (com.airhacks.enhydrator.transform.NashornRowTransformer)3 TargetMapping (com.airhacks.enhydrator.transform.TargetMapping)3 ScriptableSink (com.airhacks.enhydrator.out.ScriptableSink)1