Search in sources :

Example 1 with FilterOperator

use of org.apache.apex.malhar.lib.filter.FilterOperator in project apex-malhar by apache.

the class Application method populateDAG.

@Override
public void populateDAG(DAG dag, Configuration conf) {
    FSRecordReaderModule recordReader = dag.addModule("recordReader", FSRecordReaderModule.class);
    CsvParser csvParser = dag.addOperator("csvParser", CsvParser.class);
    FilterOperator filterOperator = dag.addOperator("filterOperator", new FilterOperator());
    CsvFormatter selectedFormatter = dag.addOperator("selectedFormatter", new CsvFormatter());
    CsvFormatter rejectedFormatter = dag.addOperator("rejectedFormatter", new CsvFormatter());
    StringFileOutputOperator selectedOutput = dag.addOperator("selectedOutput", new StringFileOutputOperator());
    StringFileOutputOperator rejectedOutput = dag.addOperator("rejectedOutput", new StringFileOutputOperator());
    dag.addStream("record", recordReader.records, csvParser.in);
    dag.addStream("pojo", csvParser.out, filterOperator.input);
    dag.addStream("pojoSelected", filterOperator.truePort, selectedFormatter.in);
    dag.addStream("pojoRejected", filterOperator.falsePort, rejectedFormatter.in);
    dag.addStream("csvSelected", selectedFormatter.out, selectedOutput.input);
    dag.addStream("csvRejected", rejectedFormatter.out, rejectedOutput.input);
}
Also used : FilterOperator(org.apache.apex.malhar.lib.filter.FilterOperator) FSRecordReaderModule(org.apache.apex.malhar.lib.fs.FSRecordReaderModule) CsvFormatter(org.apache.apex.malhar.contrib.formatter.CsvFormatter) CsvParser(org.apache.apex.malhar.contrib.parser.CsvParser) StringFileOutputOperator(org.apache.apex.malhar.lib.fs.GenericFileOutputOperator.StringFileOutputOperator)

Aggregations

CsvFormatter (org.apache.apex.malhar.contrib.formatter.CsvFormatter)1 CsvParser (org.apache.apex.malhar.contrib.parser.CsvParser)1 FilterOperator (org.apache.apex.malhar.lib.filter.FilterOperator)1 FSRecordReaderModule (org.apache.apex.malhar.lib.fs.FSRecordReaderModule)1 StringFileOutputOperator (org.apache.apex.malhar.lib.fs.GenericFileOutputOperator.StringFileOutputOperator)1