Search in sources :

Example 1 with LogSink

use of com.airhacks.enhydrator.out.LogSink in project enhydrator by AdamBien.

the class FromJsonToCSVTest method fromJSONToCSV.

@Test
public void fromJSONToCSV() throws FileNotFoundException {
    Source source = new ScriptableSource(INPUT + "languages.json", INPUT + "converter.js", "UTF-8");
    final CSVFileSink csvFileSink = new CSVFileSink("*", getFileName(), ";", true, false);
    Pump pump = new Pump.Engine().from(source).to(new LogSink()).to(csvFileSink).build();
    pump.start();
}
Also used : LogSink(com.airhacks.enhydrator.out.LogSink) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) Source(com.airhacks.enhydrator.in.Source) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) Pump(com.airhacks.enhydrator.Pump) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource) CSVFileSink(com.airhacks.enhydrator.out.CSVFileSink) Test(org.junit.Test)

Example 2 with LogSink

use of com.airhacks.enhydrator.out.LogSink in project enhydrator by AdamBien.

the class CopyTableIT method readTable.

@Test
public void readTable() {
    final String firstName = "arabica";
    final String secondName = "niceone";
    CoffeeTestFixture.insertCoffee(firstName, 2, "hawai", Roast.LIGHT, "nice", "whole");
    CoffeeTestFixture.insertCoffee(secondName, 3, "russia", Roast.MEDIUM, "awful", "java beans");
    VirtualSinkSource vss = new VirtualSinkSource();
    Pump pump = new Pump.Engine().flowListener(l -> Logger.getLogger("plainCopy").info(l)).from(this.source).to(new LogSink("*")).to(vss).sqlQuery("select * from Coffee").build();
    pump.start();
    int numberOfRows = vss.getNumberOfRows();
    assertThat(numberOfRows, is(2));
    System.out.println(vss.toString());
    Row first = vss.getRow(0);
    Object name = first.getColumnValue("NAME");
    assertThat(firstName, is(name));
    Row last = vss.getRow(1);
    name = last.getColumnValue("NAME");
    assertThat(secondName, is(name));
}
Also used : LogSink(com.airhacks.enhydrator.out.LogSink) CoreMatchers.is(org.hamcrest.CoreMatchers.is) Persistence(javax.persistence.Persistence) Test(org.junit.Test) Logger(java.util.logging.Logger) Assert.assertThat(org.junit.Assert.assertThat) JDBCSinkTest(com.airhacks.enhydrator.out.JDBCSinkTest) List(java.util.List) JDBCSource(com.airhacks.enhydrator.in.JDBCSource) Ignore(org.junit.Ignore) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) After(org.junit.After) NamedSink(com.airhacks.enhydrator.out.NamedSink) Row(com.airhacks.enhydrator.in.Row) Before(org.junit.Before) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) LogSink(com.airhacks.enhydrator.out.LogSink) Row(com.airhacks.enhydrator.in.Row) Test(org.junit.Test) JDBCSinkTest(com.airhacks.enhydrator.out.JDBCSinkTest)

Example 3 with LogSink

use of com.airhacks.enhydrator.out.LogSink in project enhydrator by AdamBien.

the class ConvertToIntAndCopyTest method init.

@Before
public void init() {
    this.input = new VirtualSinkSource();
    this.output = new VirtualSinkSource();
    //map column answer to Integer
    final DatatypeNameMapper datatypeMapper = new DatatypeNameMapper();
    datatypeMapper.addMapping("answer", Datatype.INTEGER);
    //Copy column question to answer and origin
    ColumnCopier columnCopier = new ColumnCopier();
    columnCopier.addMapping("question", "answer", "origin");
    this.pump = new Pump.Engine().startWith(columnCopier).startWith(datatypeMapper).with("answer", (c) -> {
        Integer columnValue = (Integer) c;
        return columnValue * 2;
    }).from(input).to(output).to(new LogSink()).build();
}
Also used : LogSink(com.airhacks.enhydrator.out.LogSink) Memory(com.airhacks.enhydrator.transform.Memory) CoreMatchers.is(org.hamcrest.CoreMatchers.is) ColumnCopier(com.airhacks.enhydrator.transform.ColumnCopier) Column(com.airhacks.enhydrator.in.Column) Test(org.junit.Test) Assert.assertThat(org.junit.Assert.assertThat) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) Datatype(com.airhacks.enhydrator.transform.Datatype) DatatypeNameMapper(com.airhacks.enhydrator.transform.DatatypeNameMapper) Pump(com.airhacks.enhydrator.Pump) Row(com.airhacks.enhydrator.in.Row) Before(org.junit.Before) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) LogSink(com.airhacks.enhydrator.out.LogSink) ColumnCopier(com.airhacks.enhydrator.transform.ColumnCopier) DatatypeNameMapper(com.airhacks.enhydrator.transform.DatatypeNameMapper) Before(org.junit.Before)

Example 4 with LogSink

use of com.airhacks.enhydrator.out.LogSink in project enhydrator by AdamBien.

the class JSONImportTest method getSource.

VirtualSinkSource getSource(final String fileName) throws FileNotFoundException {
    final Path pathToContent = Paths.get(fileName);
    Reader script = new FileReader("./src/test/files/converter.js");
    Source source = new ScriptableSource(pathToContent, script, "UTF-8");
    VirtualSinkSource vss = new VirtualSinkSource();
    Pump pump = new Pump.Engine().from(source).to(vss).to(new LogSink()).build();
    pump.start();
    return vss;
}
Also used : Path(java.nio.file.Path) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) LogSink(com.airhacks.enhydrator.out.LogSink) Reader(java.io.Reader) FileReader(java.io.FileReader) FileReader(java.io.FileReader) Source(com.airhacks.enhydrator.in.Source) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) Pump(com.airhacks.enhydrator.Pump) ScriptableSource(com.airhacks.enhydrator.in.ScriptableSource)

Example 5 with LogSink

use of com.airhacks.enhydrator.out.LogSink in project enhydrator by AdamBien.

the class ParentTest method copy.

/**
     * Name;Size;Folder
     */
@Test
public void copy() {
    Source source = new CSVFileSource("./src/test/files/files.csv", ";", "UTF-8", true);
    VirtualSinkSource vss = new VirtualSinkSource();
    Pump pump = new Pump.Engine().from(source).startWith(new SkipFirstRow()).startWith(new NonRecursiveTree("Name", "Folder")).to(vss).to(new LogSink()).build();
    pump.start();
    System.out.println(vss.getRows());
    int numberOfRows = vss.getNumberOfRows();
    assertThat(numberOfRows, is(2));
    Row parentWithChildren = vss.getRow(0);
    assertNotNull(parentWithChildren);
    assertThat(parentWithChildren.getNumberOfColumns(), is(3));
    List<Row> children = parentWithChildren.getChildren();
    assertThat(children.size(), is(2));
    children.forEach(e -> assertThat(e.getNumberOfColumns(), is(2)));
}
Also used : NonRecursiveTree(com.airhacks.enhydrator.functions.NonRecursiveTree) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) LogSink(com.airhacks.enhydrator.out.LogSink) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) SkipFirstRow(com.airhacks.enhydrator.transform.SkipFirstRow) SkipFirstRow(com.airhacks.enhydrator.transform.SkipFirstRow) Row(com.airhacks.enhydrator.in.Row) CSVFileSource(com.airhacks.enhydrator.in.CSVFileSource) Source(com.airhacks.enhydrator.in.Source) VirtualSinkSource(com.airhacks.enhydrator.in.VirtualSinkSource) Pump(com.airhacks.enhydrator.Pump) Test(org.junit.Test)

Aggregations

LogSink (com.airhacks.enhydrator.out.LogSink)13 VirtualSinkSource (com.airhacks.enhydrator.in.VirtualSinkSource)11 CSVFileSource (com.airhacks.enhydrator.in.CSVFileSource)9 Source (com.airhacks.enhydrator.in.Source)9 Pump (com.airhacks.enhydrator.Pump)8 Test (org.junit.Test)7 ScriptableSource (com.airhacks.enhydrator.in.ScriptableSource)6 JDBCSource (com.airhacks.enhydrator.in.JDBCSource)5 NamedSink (com.airhacks.enhydrator.out.NamedSink)5 Row (com.airhacks.enhydrator.in.Row)4 DestinationMapper (com.airhacks.enhydrator.transform.DestinationMapper)4 NashornRowTransformer (com.airhacks.enhydrator.transform.NashornRowTransformer)4 TargetMapping (com.airhacks.enhydrator.transform.TargetMapping)4 DatatypeIndexMapper (com.airhacks.enhydrator.transform.DatatypeIndexMapper)3 DatatypeNameMapper (com.airhacks.enhydrator.transform.DatatypeNameMapper)3 Memory (com.airhacks.enhydrator.transform.Memory)3 SkipFirstRow (com.airhacks.enhydrator.transform.SkipFirstRow)3 CSVFileSink (com.airhacks.enhydrator.out.CSVFileSink)2 ColumnCopier (com.airhacks.enhydrator.transform.ColumnCopier)2 CoreMatchers.is (org.hamcrest.CoreMatchers.is)2