Search in sources :

Example 26 with MutableLong

use of org.apache.commons.lang.mutable.MutableLong in project knime-core by knime.

the class BigGroupByTable method initMissingValuesMap.

private void initMissingValuesMap() {
    m_missingValuesMap = new HashMap<>();
    ColumnAggregator[] colAggregators = getColAggregators();
    for (ColumnAggregator ca : colAggregators) {
        m_missingValuesMap.put(ca.getOriginalColName(), new MutableLong(0L));
    }
}
Also used : MutableLong(org.apache.commons.lang.mutable.MutableLong) ColumnAggregator(org.knime.base.data.aggregation.ColumnAggregator)

Example 27 with MutableLong

use of org.apache.commons.lang.mutable.MutableLong in project apex-malhar by apache.

the class FSOutputOperatorBenchmark method populateDAG.

@Override
public void populateDAG(DAG dag, Configuration conf) {
    String filePath = "HDFSOutputOperatorBenchmarkingApp/" + System.currentTimeMillis();
    dag.setAttribute(DAG.STREAMING_WINDOW_SIZE_MILLIS, 1000);
    RandomWordGenerator wordGenerator = dag.addOperator("wordGenerator", RandomWordGenerator.class);
    dag.getOperatorMeta("wordGenerator").getMeta(wordGenerator.output).getAttributes().put(PortContext.QUEUE_CAPACITY, 10000);
    dag.getOperatorMeta("wordGenerator").getAttributes().put(OperatorContext.APPLICATION_WINDOW_COUNT, 1);
    FSByteOutputOperator hdfsOutputOperator = dag.addOperator("hdfsOutputOperator", new FSByteOutputOperator());
    hdfsOutputOperator.setFilePath(filePath);
    dag.getOperatorMeta("hdfsOutputOperator").getAttributes().put(OperatorContext.COUNTERS_AGGREGATOR, new BasicCounters.LongAggregator<MutableLong>());
    dag.addStream("Generator2HDFSOutput", wordGenerator.output, hdfsOutputOperator.input);
}
Also used : RandomWordGenerator(org.apache.apex.malhar.lib.testbench.RandomWordGenerator) BasicCounters(org.apache.apex.malhar.lib.counters.BasicCounters) MutableLong(org.apache.commons.lang.mutable.MutableLong)

Aggregations

MutableLong (org.apache.commons.lang.mutable.MutableLong)27 IOException (java.io.IOException)8 Path (org.apache.hadoop.fs.Path)5 ObjectMapperString (com.datatorrent.common.util.ObjectMapperString)3 Map (java.util.Map)3 BasicCounters (org.apache.apex.malhar.lib.counters.BasicCounters)3 DefaultPartition (com.datatorrent.api.DefaultPartition)2 IncompatibleVersionException (com.datatorrent.stram.client.WebServicesVersionConversion.IncompatibleVersionException)2 TupleRecorder (com.datatorrent.stram.debug.TupleRecorder)2 VCFHeader (htsjdk.variant.vcf.VCFHeader)2 VCFHeaderLine (htsjdk.variant.vcf.VCFHeaderLine)2 BufferedReader (java.io.BufferedReader)2 InputStreamReader (java.io.InputStreamReader)2 HashMap (java.util.HashMap)2 ExecutionException (java.util.concurrent.ExecutionException)2 FileStatus (org.apache.hadoop.fs.FileStatus)2 JSONException (org.codehaus.jettison.json.JSONException)2 JSONObject (org.codehaus.jettison.json.JSONObject)2 DataRow (org.knime.core.data.DataRow)2 ColumnBufferedDataTableSorter (org.knime.core.data.sort.ColumnBufferedDataTableSorter)2