Examples with Pipeline loghub.Pipeline used on opensource projects

Example 16 with Pipeline

use of loghub.Pipeline in project LogHub by fbacchella.

the class TestEtl method test8.

@Test
public void test8() throws ProcessorException, InterruptedException, ConfigException, IOException {
    Properties conf = Tools.loadConf("etl.conf");
    for (Pipeline pipe : conf.pipelines) {
        Assert.assertTrue("configuration failed", pipe.configure(conf));
    }
    Event sent = Tools.getEvent();
    sent.setTimestamp(new Date(1));
    Tools.runProcessing(sent, conf.namedPipeLine.get("timestamp"), conf);
    Assert.assertEquals(new Date(0), sent.getTimestamp());
    Assert.assertEquals(new Date(1), sent.get("reception_time"));
}

Example 17 with Pipeline

use of loghub.Pipeline in project LogHub by fbacchella.

the class TestEtl method test6.

@Test
public void test6() throws ProcessorException, InterruptedException, ConfigException, IOException {
    Properties conf = Tools.loadConf("etl.conf");
    for (Pipeline pipe : conf.pipelines) {
        Assert.assertTrue("configuration failed", pipe.configure(conf));
    }
    Event sent = Tools.getEvent();
    sent.put("count", "1");
    Tools.runProcessing(sent, conf.namedPipeLine.get("second"), conf);
    Assert.assertEquals("conversion not expected", 1, sent.get("count"));
}

Example 18 with Pipeline

use of loghub.Pipeline in project LogHub by fbacchella.

the class TestMapper method test4.

@Test
public void test4() throws ProcessorException, InterruptedException, ConfigException, IOException {
    Properties conf = Tools.loadConf("map.conf");
    for (Pipeline pipe : conf.pipelines) {
        Assert.assertTrue("configuration failed", pipe.configure(conf));
    }
    Event sent = Tools.getEvent();
    sent.put("a", 2L);
    Tools.runProcessing(sent, conf.namedPipeLine.get("mapper2"), conf);
    Assert.assertEquals("conversion not expected", "c", sent.get("a"));
}

Example 19 with Pipeline

use of loghub.Pipeline in project LogHub by fbacchella.

the class TestMapper method test1.

@Test
public void test1() throws ProcessorException, InterruptedException, ConfigException, IOException {
    Properties conf = Tools.loadConf("map.conf");
    for (Pipeline pipe : conf.pipelines) {
        Assert.assertTrue("configuration failed", pipe.configure(conf));
    }
    Event sent = Tools.getEvent();
    sent.put("a", 1);
    Tools.runProcessing(sent, conf.namedPipeLine.get("mapper1"), conf);
    Assert.assertEquals("conversion not expected", "b", sent.get("d"));
}

Example 20 with Pipeline

use of loghub.Pipeline in project LogHub by fbacchella.

the class TestMapper method test2.

@Test
public void test2() throws ProcessorException, InterruptedException, ConfigException, IOException {
    Properties conf = Tools.loadConf("map.conf");
    for (Pipeline pipe : conf.pipelines) {
        Assert.assertTrue("configuration failed", pipe.configure(conf));
    }
    Event sent = Tools.getEvent();
    sent.put("a", 1);
    Tools.runProcessing(sent, conf.namedPipeLine.get("mapper2"), conf);
    Assert.assertEquals("conversion not expected", "b", sent.get("a"));
}