Search in sources :

Example 21 with Edge

use of edu.iu.dsc.tws.api.compute.graph.Edge in project twister2 by DSC-SPIDAL.

the class AbstractKeyedOpsConfig method buildEdge.

@Override
Edge buildEdge() {
    Edge edge = super.buildEdge();
    edge.setKeyed(true);
    edge.setKeyType(opKeyType);
    edge.setPartitioner(this.tPartitioner);
    return edge;
}
Also used : Edge(edu.iu.dsc.tws.api.compute.graph.Edge)

Example 22 with Edge

use of edu.iu.dsc.tws.api.compute.graph.Edge in project twister2 by DSC-SPIDAL.

the class RowPartitionTLink method getEdge.

@Override
public Edge getEdge() {
    Edge e = new Edge(getId(), OperationNames.TABLE_PARTITION, MessageTypes.ARROW_TABLE);
    if (partitionFunction != null) {
        e.setPartitioner(partitionFunction);
    }
    e.addProperty(CommunicationContext.ROW_SCHEMA, ((RowSchema) getSchema()).toArrowSchema());
    e.addProperty(CommunicationContext.USE_DISK, this.useDisk);
    TLinkUtils.generateCommsSchema(getSchema(), e);
    return e;
}
Also used : Edge(edu.iu.dsc.tws.api.compute.graph.Edge)

Example 23 with Edge

use of edu.iu.dsc.tws.api.compute.graph.Edge in project twister2 by DSC-SPIDAL.

the class AllGatherTLink method getEdge.

@Override
public Edge getEdge() {
    Edge e = new Edge(getId(), OperationNames.ALLGATHER, this.getSchema().getDataType());
    e.addProperty(CommunicationContext.USE_DISK, this.useDisk);
    TLinkUtils.generateCommsSchema(getSchema(), e);
    return e;
}
Also used : Edge(edu.iu.dsc.tws.api.compute.graph.Edge)

Example 24 with Edge

use of edu.iu.dsc.tws.api.compute.graph.Edge in project twister2 by DSC-SPIDAL.

the class GatherTLink method getEdge.

@Override
public Edge getEdge() {
    Edge e = new Edge(getId(), OperationNames.GATHER, getSchema().getDataType());
    e.addProperty(CommunicationContext.USE_DISK, this.useDisk);
    TLinkUtils.generateCommsSchema(getSchema(), e);
    return e;
}
Also used : Edge(edu.iu.dsc.tws.api.compute.graph.Edge)

Example 25 with Edge

use of edu.iu.dsc.tws.api.compute.graph.Edge in project twister2 by DSC-SPIDAL.

the class KeyedPartitionTLink method getEdge.

@Override
public Edge getEdge() {
    Edge e = new Edge(getId(), OperationNames.KEYED_PARTITION, this.getSchema().getDataType());
    e.setKeyed(true);
    e.setKeyType(this.getSchema().getKeyType());
    e.setPartitioner(partitionFunction);
    e.addProperty(CommunicationContext.USE_DISK, this.useDisk);
    TLinkUtils.generateKeyedCommsSchema(getSchema(), e);
    return e;
}
Also used : Edge(edu.iu.dsc.tws.api.compute.graph.Edge)

Aggregations

Edge (edu.iu.dsc.tws.api.compute.graph.Edge)32 HashSet (java.util.HashSet)2 HashBasedTable (com.google.common.collect.HashBasedTable)1 Table (com.google.common.collect.Table)1 CheckpointingClient (edu.iu.dsc.tws.api.checkpointing.CheckpointingClient)1 Communicator (edu.iu.dsc.tws.api.comms.Communicator)1 LogicalPlan (edu.iu.dsc.tws.api.comms.LogicalPlan)1 ExecutionPlan (edu.iu.dsc.tws.api.compute.executor.ExecutionPlan)1 ExecutorContext (edu.iu.dsc.tws.api.compute.executor.ExecutorContext)1 IExecutionPlanBuilder (edu.iu.dsc.tws.api.compute.executor.IExecutionPlanBuilder)1 INodeInstance (edu.iu.dsc.tws.api.compute.executor.INodeInstance)1 IParallelOperation (edu.iu.dsc.tws.api.compute.executor.IParallelOperation)1 ComputeGraph (edu.iu.dsc.tws.api.compute.graph.ComputeGraph)1 OperationMode (edu.iu.dsc.tws.api.compute.graph.OperationMode)1 Vertex (edu.iu.dsc.tws.api.compute.graph.Vertex)1 ICompute (edu.iu.dsc.tws.api.compute.nodes.ICompute)1 INode (edu.iu.dsc.tws.api.compute.nodes.INode)1 ISource (edu.iu.dsc.tws.api.compute.nodes.ISource)1 TaskInstancePlan (edu.iu.dsc.tws.api.compute.schedule.elements.TaskInstancePlan)1 TaskSchedulePlan (edu.iu.dsc.tws.api.compute.schedule.elements.TaskSchedulePlan)1