Search in sources :

Example 21 with OrderColumn

use of org.apache.hyracks.algebricks.core.algebra.properties.OrderColumn in project asterixdb by apache.

the class AbstractStableSortPOperator method computeLocalProperties.

public void computeLocalProperties(ILogicalOperator op) {
    OrderOperator ord = (OrderOperator) op;
    List<OrderColumn> orderColumns = new ArrayList<OrderColumn>();
    for (Pair<IOrder, Mutable<ILogicalExpression>> p : ord.getOrderExpressions()) {
        ILogicalExpression expr = p.second.getValue();
        if (expr.getExpressionTag() == LogicalExpressionTag.VARIABLE) {
            VariableReferenceExpression varRef = (VariableReferenceExpression) expr;
            LogicalVariable var = varRef.getVariableReference();
            orderColumns.add(new OrderColumn(var, p.first.getKind()));
        } else {
            throw new IllegalStateException();
        }
    }
    sortColumns = orderColumns.toArray(new OrderColumn[orderColumns.size()]);
    orderProp = new LocalOrderProperty(orderColumns);
}
Also used : LogicalVariable(org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable) Mutable(org.apache.commons.lang3.mutable.Mutable) ILogicalExpression(org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression) IOrder(org.apache.hyracks.algebricks.core.algebra.operators.logical.OrderOperator.IOrder) VariableReferenceExpression(org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression) OrderColumn(org.apache.hyracks.algebricks.core.algebra.properties.OrderColumn) LocalOrderProperty(org.apache.hyracks.algebricks.core.algebra.properties.LocalOrderProperty) ArrayList(java.util.ArrayList) OrderOperator(org.apache.hyracks.algebricks.core.algebra.operators.logical.OrderOperator)

Example 22 with OrderColumn

use of org.apache.hyracks.algebricks.core.algebra.properties.OrderColumn in project asterixdb by apache.

the class HashPartitionMergeExchangePOperator method createConnectorDescriptor.

@Override
public Pair<IConnectorDescriptor, TargetConstraint> createConnectorDescriptor(IConnectorDescriptorRegistry spec, ILogicalOperator op, IOperatorSchema opSchema, JobGenContext context) throws AlgebricksException {
    int[] keys = new int[partitionFields.size()];
    IBinaryHashFunctionFactory[] hashFunctionFactories = new IBinaryHashFunctionFactory[partitionFields.size()];
    IVariableTypeEnvironment env = context.getTypeEnvironment(op);
    {
        int i = 0;
        IBinaryHashFunctionFactoryProvider hashFunProvider = context.getBinaryHashFunctionFactoryProvider();
        for (LogicalVariable v : partitionFields) {
            keys[i] = opSchema.findVariable(v);
            hashFunctionFactories[i] = hashFunProvider.getBinaryHashFunctionFactory(env.getVarType(v));
            ++i;
        }
    }
    ITuplePartitionComputerFactory tpcf = new FieldHashPartitionComputerFactory(keys, hashFunctionFactories);
    int n = orderColumns.size();
    int[] sortFields = new int[n];
    IBinaryComparatorFactory[] comparatorFactories = new IBinaryComparatorFactory[n];
    INormalizedKeyComputerFactoryProvider nkcfProvider = context.getNormalizedKeyComputerFactoryProvider();
    INormalizedKeyComputerFactory nkcf = null;
    int j = 0;
    for (OrderColumn oc : orderColumns) {
        LogicalVariable var = oc.getColumn();
        sortFields[j] = opSchema.findVariable(var);
        Object type = env.getVarType(var);
        IBinaryComparatorFactoryProvider bcfp = context.getBinaryComparatorFactoryProvider();
        comparatorFactories[j] = bcfp.getBinaryComparatorFactory(type, oc.getOrder() == OrderKind.ASC);
        if (j == 0 && nkcfProvider != null && type != null) {
            nkcf = nkcfProvider.getNormalizedKeyComputerFactory(type, oc.getOrder() == OrderKind.ASC);
        }
        j++;
    }
    IConnectorDescriptor conn = new MToNPartitioningMergingConnectorDescriptor(spec, tpcf, sortFields, comparatorFactories, nkcf);
    return new Pair<IConnectorDescriptor, TargetConstraint>(conn, null);
}
Also used : LogicalVariable(org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable) ITuplePartitionComputerFactory(org.apache.hyracks.api.dataflow.value.ITuplePartitionComputerFactory) IConnectorDescriptor(org.apache.hyracks.api.dataflow.IConnectorDescriptor) IBinaryHashFunctionFactoryProvider(org.apache.hyracks.algebricks.data.IBinaryHashFunctionFactoryProvider) MToNPartitioningMergingConnectorDescriptor(org.apache.hyracks.dataflow.std.connectors.MToNPartitioningMergingConnectorDescriptor) OrderColumn(org.apache.hyracks.algebricks.core.algebra.properties.OrderColumn) IBinaryComparatorFactory(org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory) IBinaryHashFunctionFactory(org.apache.hyracks.api.dataflow.value.IBinaryHashFunctionFactory) FieldHashPartitionComputerFactory(org.apache.hyracks.dataflow.common.data.partition.FieldHashPartitionComputerFactory) IBinaryComparatorFactoryProvider(org.apache.hyracks.algebricks.data.IBinaryComparatorFactoryProvider) TargetConstraint(org.apache.hyracks.algebricks.core.algebra.base.IHyracksJobBuilder.TargetConstraint) INormalizedKeyComputerFactory(org.apache.hyracks.api.dataflow.value.INormalizedKeyComputerFactory) INormalizedKeyComputerFactoryProvider(org.apache.hyracks.algebricks.data.INormalizedKeyComputerFactoryProvider) IVariableTypeEnvironment(org.apache.hyracks.algebricks.core.algebra.expressions.IVariableTypeEnvironment) Pair(org.apache.hyracks.algebricks.common.utils.Pair)

Example 23 with OrderColumn

use of org.apache.hyracks.algebricks.core.algebra.properties.OrderColumn in project asterixdb by apache.

the class HashPartitionMergeExchangePOperator method getRequiredPropertiesForChildren.

@Override
public PhysicalRequirements getRequiredPropertiesForChildren(ILogicalOperator op, IPhysicalPropertiesVector reqdByParent, IOptimizationContext context) {
    List<ILocalStructuralProperty> orderProps = new LinkedList<ILocalStructuralProperty>();
    List<OrderColumn> columns = new ArrayList<OrderColumn>();
    for (OrderColumn oc : orderColumns) {
        LogicalVariable var = oc.getColumn();
        columns.add(new OrderColumn(var, oc.getOrder()));
    }
    orderProps.add(new LocalOrderProperty(columns));
    StructuralPropertiesVector[] r = new StructuralPropertiesVector[] { new StructuralPropertiesVector(null, orderProps) };
    return new PhysicalRequirements(r, IPartitioningRequirementsCoordinator.NO_COORDINATION);
}
Also used : LogicalVariable(org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable) StructuralPropertiesVector(org.apache.hyracks.algebricks.core.algebra.properties.StructuralPropertiesVector) OrderColumn(org.apache.hyracks.algebricks.core.algebra.properties.OrderColumn) LocalOrderProperty(org.apache.hyracks.algebricks.core.algebra.properties.LocalOrderProperty) ArrayList(java.util.ArrayList) ILocalStructuralProperty(org.apache.hyracks.algebricks.core.algebra.properties.ILocalStructuralProperty) LinkedList(java.util.LinkedList) PhysicalRequirements(org.apache.hyracks.algebricks.core.algebra.properties.PhysicalRequirements)

Example 24 with OrderColumn

use of org.apache.hyracks.algebricks.core.algebra.properties.OrderColumn in project asterixdb by apache.

the class IntersectPOperator method computeDeliveredProperties.

@Override
public void computeDeliveredProperties(ILogicalOperator iop, IOptimizationContext context) throws AlgebricksException {
    IntersectOperator op = (IntersectOperator) iop;
    IPartitioningProperty pp = op.getInputs().get(0).getValue().getDeliveredPhysicalProperties().getPartitioningProperty();
    HashMap<LogicalVariable, LogicalVariable> varMaps = new HashMap<>(op.getOutputVars().size());
    for (int i = 0; i < op.getOutputVars().size(); i++) {
        varMaps.put(op.getInputVariables(0).get(i), op.getOutputVars().get(i));
    }
    pp.substituteColumnVars(varMaps);
    List<ILocalStructuralProperty> propsLocal = new ArrayList<>();
    List<OrderColumn> orderColumns = new ArrayList<>();
    for (LogicalVariable var : op.getOutputVars()) {
        orderColumns.add(new OrderColumn(var, OrderOperator.IOrder.OrderKind.ASC));
    }
    propsLocal.add(new LocalOrderProperty(orderColumns));
    deliveredProperties = new StructuralPropertiesVector(pp, propsLocal);
}
Also used : LogicalVariable(org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable) StructuralPropertiesVector(org.apache.hyracks.algebricks.core.algebra.properties.StructuralPropertiesVector) HashMap(java.util.HashMap) OrderColumn(org.apache.hyracks.algebricks.core.algebra.properties.OrderColumn) ArrayList(java.util.ArrayList) IPartitioningProperty(org.apache.hyracks.algebricks.core.algebra.properties.IPartitioningProperty) LocalOrderProperty(org.apache.hyracks.algebricks.core.algebra.properties.LocalOrderProperty) IntersectOperator(org.apache.hyracks.algebricks.core.algebra.operators.logical.IntersectOperator) ILocalStructuralProperty(org.apache.hyracks.algebricks.core.algebra.properties.ILocalStructuralProperty)

Example 25 with OrderColumn

use of org.apache.hyracks.algebricks.core.algebra.properties.OrderColumn in project asterixdb by apache.

the class InMemoryStableSortPOperator method contributeRuntimeOperator.

@Override
public void contributeRuntimeOperator(IHyracksJobBuilder builder, JobGenContext context, ILogicalOperator op, IOperatorSchema opSchema, IOperatorSchema[] inputSchemas, IOperatorSchema outerPlanSchema) throws AlgebricksException {
    RecordDescriptor recDescriptor = JobGenHelper.mkRecordDescriptor(context.getTypeEnvironment(op), opSchema, context);
    int n = sortColumns.length;
    int[] sortFields = new int[n];
    IBinaryComparatorFactory[] comps = new IBinaryComparatorFactory[n];
    int i = 0;
    INormalizedKeyComputerFactoryProvider nkcfProvider = context.getNormalizedKeyComputerFactoryProvider();
    INormalizedKeyComputerFactory nkcf = null;
    IVariableTypeEnvironment env = context.getTypeEnvironment(op);
    for (OrderColumn oc : sortColumns) {
        LogicalVariable var = oc.getColumn();
        sortFields[i] = opSchema.findVariable(var);
        Object type = env.getVarType(var);
        OrderKind order = oc.getOrder();
        if (i == 0 && nkcfProvider != null && type != null) {
            nkcf = nkcfProvider.getNormalizedKeyComputerFactory(type, order == OrderKind.ASC);
        }
        IBinaryComparatorFactoryProvider bcfp = context.getBinaryComparatorFactoryProvider();
        comps[i] = bcfp.getBinaryComparatorFactory(type, oc.getOrder() == OrderKind.ASC);
        i++;
    }
    IPushRuntimeFactory runtime = new InMemorySortRuntimeFactory(sortFields, nkcf, comps, null);
    builder.contributeMicroOperator(op, runtime, recDescriptor);
    ILogicalOperator src = op.getInputs().get(0).getValue();
    builder.contributeGraphEdge(src, 0, op, 0);
}
Also used : LogicalVariable(org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable) InMemorySortRuntimeFactory(org.apache.hyracks.algebricks.runtime.operators.sort.InMemorySortRuntimeFactory) RecordDescriptor(org.apache.hyracks.api.dataflow.value.RecordDescriptor) OrderColumn(org.apache.hyracks.algebricks.core.algebra.properties.OrderColumn) ILogicalOperator(org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator) IBinaryComparatorFactory(org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory) IBinaryComparatorFactoryProvider(org.apache.hyracks.algebricks.data.IBinaryComparatorFactoryProvider) IPushRuntimeFactory(org.apache.hyracks.algebricks.runtime.base.IPushRuntimeFactory) INormalizedKeyComputerFactory(org.apache.hyracks.api.dataflow.value.INormalizedKeyComputerFactory) OrderKind(org.apache.hyracks.algebricks.core.algebra.operators.logical.OrderOperator.IOrder.OrderKind) INormalizedKeyComputerFactoryProvider(org.apache.hyracks.algebricks.data.INormalizedKeyComputerFactoryProvider) IVariableTypeEnvironment(org.apache.hyracks.algebricks.core.algebra.expressions.IVariableTypeEnvironment)

Aggregations

OrderColumn (org.apache.hyracks.algebricks.core.algebra.properties.OrderColumn)26 LogicalVariable (org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable)22 ArrayList (java.util.ArrayList)18 LocalOrderProperty (org.apache.hyracks.algebricks.core.algebra.properties.LocalOrderProperty)17 ILocalStructuralProperty (org.apache.hyracks.algebricks.core.algebra.properties.ILocalStructuralProperty)14 StructuralPropertiesVector (org.apache.hyracks.algebricks.core.algebra.properties.StructuralPropertiesVector)12 ILogicalOperator (org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator)7 PhysicalRequirements (org.apache.hyracks.algebricks.core.algebra.properties.PhysicalRequirements)7 LinkedList (java.util.LinkedList)6 IPartitioningProperty (org.apache.hyracks.algebricks.core.algebra.properties.IPartitioningProperty)6 IVariableTypeEnvironment (org.apache.hyracks.algebricks.core.algebra.expressions.IVariableTypeEnvironment)5 AbstractLogicalOperator (org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator)5 IBinaryComparatorFactoryProvider (org.apache.hyracks.algebricks.data.IBinaryComparatorFactoryProvider)5 INormalizedKeyComputerFactoryProvider (org.apache.hyracks.algebricks.data.INormalizedKeyComputerFactoryProvider)5 IBinaryComparatorFactory (org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory)5 INormalizedKeyComputerFactory (org.apache.hyracks.api.dataflow.value.INormalizedKeyComputerFactory)5 Mutable (org.apache.commons.lang3.mutable.Mutable)4 ListSet (org.apache.hyracks.algebricks.common.utils.ListSet)4 Pair (org.apache.hyracks.algebricks.common.utils.Pair)4 ILogicalExpression (org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression)4