use of edu.iu.dsc.tws.comms.stream.SKeyedPartition in project twister2 by DSC-SPIDAL.
the class SKeyedPartitionExample method compute.
@Override
protected void compute(WorkerEnvironment workerEnv) {
LogicalPlanBuilder logicalPlanBuilder = LogicalPlanBuilder.plan(jobParameters.getSources(), jobParameters.getTargets(), workerEnv).withFairDistribution();
// create the communication
partition = new SKeyedPartition(workerEnv.getCommunicator(), logicalPlanBuilder, MessageTypes.INTEGER, MessageTypes.INTEGER_ARRAY, new PartitionReceiver(), new HashingSelector());
this.resultsVerifier = new ResultsVerifier<>(inputDataArray, (ints, args) -> new Tuple<>(-1, ints), new TupleComparator<>(// any int
(d1, d2) -> true, IntArrayComparator.getInstance()));
Set<Integer> tasksOfExecutor = logicalPlanBuilder.getSourcesOnThisWorker();
// now initialize the workers
for (int t : tasksOfExecutor) {
// the map thread where data is produced
Thread mapThread = new Thread(new MapWorker(t));
mapThread.start();
}
}
Aggregations