Search in sources :

Example 1 with OutputHint

use of io.dingodb.exec.base.OutputHint in project dingo by dingodb.

the class DingoJobVisitor method visit.

@Override
public Collection<Output> visit(@Nonnull DingoDistributedValues rel) {
    List<Output> outputs = new LinkedList<>();
    String tableName = getSimpleName(rel.getTable());
    final Map<String, Location> partLocations = Services.META.getPartLocations(tableName);
    final PartitionStrategy ps = new SimpleHashStrategy(partLocations.size());
    final TableDefinition td = Services.META.getTableDefinition(tableName);
    Map<String, List<Object[]>> partMap = ps.partTuples(rel.getValues(), td.getKeyMapping());
    for (Map.Entry<String, List<Object[]>> entry : partMap.entrySet()) {
        Object partId = entry.getKey();
        ValuesOperator operator = new ValuesOperator(entry.getValue());
        operator.setId(idGenerator.get());
        OutputHint hint = new OutputHint();
        hint.setPartId(partId);
        Location location = partLocations.get(partId);
        hint.setLocation(location);
        operator.getSoleOutput().setHint(hint);
        Task task = job.getOrCreate(location);
        task.putOperator(operator);
        outputs.addAll(operator.getOutputs());
    }
    return outputs;
}
Also used : Task(io.dingodb.exec.base.Task) LinkedList(java.util.LinkedList) OutputHint(io.dingodb.exec.base.OutputHint) Output(io.dingodb.exec.base.Output) TableDefinition(io.dingodb.common.table.TableDefinition) List(java.util.List) ArrayList(java.util.ArrayList) LinkedList(java.util.LinkedList) SimpleHashStrategy(io.dingodb.exec.partition.SimpleHashStrategy) ValuesOperator(io.dingodb.exec.operator.ValuesOperator) Map(java.util.Map) Location(io.dingodb.meta.Location) PartitionStrategy(io.dingodb.exec.partition.PartitionStrategy)

Example 2 with OutputHint

use of io.dingodb.exec.base.OutputHint in project dingo by dingodb.

the class DingoJobVisitor method visit.

@Override
public Collection<Output> visit(@Nonnull DingoPartModify rel) {
    Collection<Output> inputs = dingo(rel.getInput()).accept(this);
    String tableName = getSimpleName(rel.getTable());
    List<Output> outputs = new LinkedList<>();
    TableDefinition td = Services.META.getTableDefinition(tableName);
    final TableId tableId = new TableId(Services.META.getTableKey(tableName));
    for (Output input : inputs) {
        Task task = input.getTask();
        Operator operator;
        switch(rel.getOperation()) {
            case INSERT:
                operator = new PartInsertOperator(tableId, input.getHint().getPartId(), td.getTupleSchema(), td.getKeyMapping());
                break;
            case UPDATE:
                operator = new PartUpdateOperator(tableId, input.getHint().getPartId(), td.getTupleSchema(), td.getKeyMapping(), TupleMapping.of(td.getColumnIndices(rel.getUpdateColumnList())), rel.getSourceExpressionList().stream().map(RexConverter::toString).collect(Collectors.toList()));
                break;
            case DELETE:
                operator = new PartDeleteOperator(tableId, input.getHint().getPartId(), td.getTupleSchema(), td.getKeyMapping());
                break;
            default:
                throw new IllegalStateException("Operation \"" + rel.getOperation() + "\" is not supported.");
        }
        operator.setId(idGenerator.get());
        task.putOperator(operator);
        input.setLink(operator.getInput(0));
        OutputHint hint = new OutputHint();
        hint.setToSumUp(true);
        operator.getSoleOutput().setHint(hint);
        outputs.addAll(operator.getOutputs());
    }
    return outputs;
}
Also used : TableId(io.dingodb.common.table.TableId) SumUpOperator(io.dingodb.exec.operator.SumUpOperator) AggregateOperator(io.dingodb.exec.operator.AggregateOperator) SortOperator(io.dingodb.exec.operator.SortOperator) RootOperator(io.dingodb.exec.operator.RootOperator) PartScanOperator(io.dingodb.exec.operator.PartScanOperator) SendOperator(io.dingodb.exec.operator.SendOperator) ReceiveOperator(io.dingodb.exec.operator.ReceiveOperator) ProjectOperator(io.dingodb.exec.operator.ProjectOperator) ValuesOperator(io.dingodb.exec.operator.ValuesOperator) PartUpdateOperator(io.dingodb.exec.operator.PartUpdateOperator) ReduceOperator(io.dingodb.exec.operator.ReduceOperator) Operator(io.dingodb.exec.base.Operator) GetByKeysOperator(io.dingodb.exec.operator.GetByKeysOperator) PartitionOperator(io.dingodb.exec.operator.PartitionOperator) PartDeleteOperator(io.dingodb.exec.operator.PartDeleteOperator) PartInsertOperator(io.dingodb.exec.operator.PartInsertOperator) CoalesceOperator(io.dingodb.exec.operator.CoalesceOperator) Task(io.dingodb.exec.base.Task) PartUpdateOperator(io.dingodb.exec.operator.PartUpdateOperator) PartDeleteOperator(io.dingodb.exec.operator.PartDeleteOperator) LinkedList(java.util.LinkedList) PartInsertOperator(io.dingodb.exec.operator.PartInsertOperator) OutputHint(io.dingodb.exec.base.OutputHint) Output(io.dingodb.exec.base.Output) TableDefinition(io.dingodb.common.table.TableDefinition)

Example 3 with OutputHint

use of io.dingodb.exec.base.OutputHint in project dingo by dingodb.

the class PartitionOperator method createOutputs.

public void createOutputs(String tableName, @Nonnull Map<String, Location> partLocations) {
    outputs = new HashMap<>(partLocations.size());
    for (Map.Entry<String, Location> partLocation : partLocations.entrySet()) {
        OutputHint hint = OutputHint.of(tableName, partLocation.getKey());
        hint.setLocation(partLocation.getValue());
        Output output = OutputIml.of(this);
        output.setHint(hint);
        outputs.put(partLocation.getKey(), output);
    }
}
Also used : OutputHint(io.dingodb.exec.base.OutputHint) Output(io.dingodb.exec.base.Output) HashMap(java.util.HashMap) Map(java.util.Map) Location(io.dingodb.meta.Location)

Aggregations

Output (io.dingodb.exec.base.Output)3 OutputHint (io.dingodb.exec.base.OutputHint)3 TableDefinition (io.dingodb.common.table.TableDefinition)2 Task (io.dingodb.exec.base.Task)2 ValuesOperator (io.dingodb.exec.operator.ValuesOperator)2 Location (io.dingodb.meta.Location)2 LinkedList (java.util.LinkedList)2 Map (java.util.Map)2 TableId (io.dingodb.common.table.TableId)1 Operator (io.dingodb.exec.base.Operator)1 AggregateOperator (io.dingodb.exec.operator.AggregateOperator)1 CoalesceOperator (io.dingodb.exec.operator.CoalesceOperator)1 GetByKeysOperator (io.dingodb.exec.operator.GetByKeysOperator)1 PartDeleteOperator (io.dingodb.exec.operator.PartDeleteOperator)1 PartInsertOperator (io.dingodb.exec.operator.PartInsertOperator)1 PartScanOperator (io.dingodb.exec.operator.PartScanOperator)1 PartUpdateOperator (io.dingodb.exec.operator.PartUpdateOperator)1 PartitionOperator (io.dingodb.exec.operator.PartitionOperator)1 ProjectOperator (io.dingodb.exec.operator.ProjectOperator)1 ReceiveOperator (io.dingodb.exec.operator.ReceiveOperator)1