Search in sources :

Example 11 with LogSink

use of com.airhacks.enhydrator.out.LogSink in project enhydrator by AdamBien.

the class CSVImportTest method getSource.

VirtualSinkSource getSource(final String fileName) {
    Source source = new CSVFileSource(fileName, ";", "UTF-8", true);
    VirtualSinkSource vss = new VirtualSinkSource();
    Pump pump = new Pump.Engine().from(source).to(vss).to(new LogSink()).build();
    pump.start();
    return vss;
}
Also used : VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) LogSink(com.airhacks.enhydrator.out.LogSink) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) Source(com.airhacks.enhydrator.in.Source) Pump(com.airhacks.enhydrator.Pump)

Example 12 with LogSink

use of com.airhacks.enhydrator.out.LogSink in project enhydrator by AdamBien.

the class CsvCharsetTest method fromJSONToCSVWithCharset.

@Test
public void fromJSONToCSVWithCharset() throws FileNotFoundException {
    CSVFileSource source = new CSVFileSource(INPUT + "utf8-characters.csv", sourceDelimiter, sourceCharsetName, sourceContainsHeaders);
    String fileName = getFileName();
    final CSVFileSink csvFileSink = new CSVFileSink(sinkName, fileName, sinkDelimiter, sinkUseNamesAsHeaders, sinkAppend, "UTF-8");
    // do the test
    Pump pumpOut = new Pump.Engine().from(source).to(new LogSink()).to(csvFileSink).build();
    pumpOut.start();
}
Also used : LogSink(com.airhacks.enhydrator.out.LogSink) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) Pump(com.airhacks.enhydrator.Pump) CSVFileSink(com.airhacks.enhydrator.out.CSVFileSink) Test(org.junit.Test)

Example 13 with LogSink

use of com.airhacks.enhydrator.out.LogSink in project enhydrator by AdamBien.

the class FromJsonToCSVTest method filterAndCastFromCSVFileToLog.

@Test
public void filterAndCastFromCSVFileToLog() {
    Source source = new CSVFileSource(INPUT + "/languages.csv", ";", "utf-8", true);
    VirtualSinkSource sink = new VirtualSinkSource();
    Pump pump = new Pump.Engine().from(source).filter("$ROW.getColumnValue('language') === 'java'").startWith(new DatatypeNameMapper().addMapping("rank", Datatype.INTEGER)).to(sink).to(new LogSink()).build();
    Memory memory = pump.start();
    assertFalse(memory.areErrorsOccured());
    assertThat(memory.getProcessedRowCount(), is(5l));
    //expecting only "java" language
    assertThat(sink.getNumberOfRows(), is(1));
    String languageValue = (String) sink.getRow(0).getColumnValue("language");
    assertThat(languageValue, is("java"));
    //expecting "java" having rank 1 as Integer
    Object rankValue = sink.getRow(0).getColumnValue("rank");
    assertTrue(rankValue instanceof Integer);
    assertThat((Integer) rankValue, is(1));
}
Also used : VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) LogSink(com.airhacks.enhydrator.out.LogSink) Memory(com.airhacks.enhydrator.transform.Memory) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) DatatypeNameMapper(com.airhacks.enhydrator.transform.DatatypeNameMapper) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) Source(com.airhacks.enhydrator.in.Source) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) Pump(com.airhacks.enhydrator.Pump) Test(org.junit.Test)

Aggregations

LogSink (com.airhacks.enhydrator.out.LogSink)13 VirtualSinkSource (com.airhacks.enhydrator.in.VirtualSinkSource)11 CSVFileSource (com.airhacks.enhydrator.in.CSVFileSource)9 Source (com.airhacks.enhydrator.in.Source)9 Pump (com.airhacks.enhydrator.Pump)8 Test (org.junit.Test)7 ScriptableSource (com.airhacks.enhydrator.in.ScriptableSource)6 JDBCSource (com.airhacks.enhydrator.in.JDBCSource)5 NamedSink (com.airhacks.enhydrator.out.NamedSink)5 Row (com.airhacks.enhydrator.in.Row)4 DestinationMapper (com.airhacks.enhydrator.transform.DestinationMapper)4 NashornRowTransformer (com.airhacks.enhydrator.transform.NashornRowTransformer)4 TargetMapping (com.airhacks.enhydrator.transform.TargetMapping)4 DatatypeIndexMapper (com.airhacks.enhydrator.transform.DatatypeIndexMapper)3 DatatypeNameMapper (com.airhacks.enhydrator.transform.DatatypeNameMapper)3 Memory (com.airhacks.enhydrator.transform.Memory)3 SkipFirstRow (com.airhacks.enhydrator.transform.SkipFirstRow)3 CSVFileSink (com.airhacks.enhydrator.out.CSVFileSink)2 ColumnCopier (com.airhacks.enhydrator.transform.ColumnCopier)2 CoreMatchers.is (org.hamcrest.CoreMatchers.is)2