Search in sources :

Example 1 with NotImplementedException

use of org.apache.hyracks.algebricks.common.exceptions.NotImplementedException in project asterixdb by apache.

the class AdmDataGen method dataGen.

public void dataGen() throws Exception {
    for (Map.Entry<TypeSignature, IAType> me : typeMap.entrySet()) {
        TypeSignature tn = me.getKey();
        TypeDataGen tdg = typeAnnotMap.get(tn);
        if (tdg.isDataGen()) {
            IAType t = me.getValue();
            if (t.getTypeTag() != ATypeTag.OBJECT) {
                throw new NotImplementedException();
            }
            ARecordType rt = (ARecordType) t;
            RecordDataGenAnnotation dga = firstDataGenAnnotation(rt);
            if (dga == null) {
                throw new Exception("No data generator annotations for type " + tn);
            }
            File outFile = new File(outputDir + File.separator + tdg.getOutputFileName());
            PrintStream outStream = new PrintStream(new BufferedOutputStream(new FileOutputStream(outFile)));
            RecordGenerator rg = new RecordGenerator(rt, dga, "\n");
            rg.init(outStream, dgCtx);
            for (long i = 0; i < tdg.getNumValues(); i++) {
                rg.generate();
            }
            outStream.close();
        }
    }
}
Also used : PrintStream(java.io.PrintStream) RecordDataGenAnnotation(org.apache.asterix.common.annotations.RecordDataGenAnnotation) NotImplementedException(org.apache.hyracks.algebricks.common.exceptions.NotImplementedException) TypeDataGen(org.apache.asterix.common.annotations.TypeDataGen) NotImplementedException(org.apache.hyracks.algebricks.common.exceptions.NotImplementedException) ACIDException(org.apache.asterix.common.exceptions.ACIDException) MetadataException(org.apache.asterix.metadata.MetadataException) AlgebricksException(org.apache.hyracks.algebricks.common.exceptions.AlgebricksException) ParseException(org.apache.asterix.lang.aql.parser.ParseException) AsterixException(org.apache.asterix.common.exceptions.AsterixException) IOException(java.io.IOException) TypeSignature(org.apache.asterix.om.types.TypeSignature) FileOutputStream(java.io.FileOutputStream) Map(java.util.Map) HashMap(java.util.HashMap) ARecordType(org.apache.asterix.om.types.ARecordType) File(java.io.File) BufferedOutputStream(java.io.BufferedOutputStream) IAType(org.apache.asterix.om.types.IAType)

Example 2 with NotImplementedException

use of org.apache.hyracks.algebricks.common.exceptions.NotImplementedException in project asterixdb by apache.

the class ExternalDataUtils method getValueParserFactories.

public static IValueParserFactory[] getValueParserFactories(ARecordType recordType) {
    int n = recordType.getFieldTypes().length;
    IValueParserFactory[] fieldParserFactories = new IValueParserFactory[n];
    for (int i = 0; i < n; i++) {
        ATypeTag tag = null;
        if (recordType.getFieldTypes()[i].getTypeTag() == ATypeTag.UNION) {
            AUnionType unionType = (AUnionType) recordType.getFieldTypes()[i];
            if (!unionType.isUnknownableType()) {
                throw new NotImplementedException("Non-optional UNION type is not supported.");
            }
            tag = unionType.getActualType().getTypeTag();
        } else {
            tag = recordType.getFieldTypes()[i].getTypeTag();
        }
        if (tag == null) {
            throw new NotImplementedException("Failed to get the type information for field " + i + ".");
        }
        fieldParserFactories[i] = getParserFactory(tag);
    }
    return fieldParserFactories;
}
Also used : IValueParserFactory(org.apache.hyracks.dataflow.common.data.parsers.IValueParserFactory) ATypeTag(org.apache.asterix.om.types.ATypeTag) AUnionType(org.apache.asterix.om.types.AUnionType) NotImplementedException(org.apache.hyracks.algebricks.common.exceptions.NotImplementedException)

Example 3 with NotImplementedException

use of org.apache.hyracks.algebricks.common.exceptions.NotImplementedException in project asterixdb by apache.

the class NestedLoopJoinPOperator method getRequiredPropertiesForChildren.

@Override
public PhysicalRequirements getRequiredPropertiesForChildren(ILogicalOperator op, IPhysicalPropertiesVector reqdByParent, IOptimizationContext context) {
    if (partitioningType != JoinPartitioningType.BROADCAST) {
        throw new NotImplementedException(partitioningType + " nested loop joins are not implemented.");
    }
    StructuralPropertiesVector[] pv = new StructuralPropertiesVector[2];
    // TODO: leverage statistics to make better decisions.
    pv[0] = OperatorPropertiesUtil.checkUnpartitionedAndGetPropertiesVector(op, new StructuralPropertiesVector(new RandomPartitioningProperty(context.getComputationNodeDomain()), null));
    pv[1] = OperatorPropertiesUtil.checkUnpartitionedAndGetPropertiesVector(op, new StructuralPropertiesVector(new BroadcastPartitioningProperty(context.getComputationNodeDomain()), null));
    return new PhysicalRequirements(pv, IPartitioningRequirementsCoordinator.NO_COORDINATION);
}
Also used : StructuralPropertiesVector(org.apache.hyracks.algebricks.core.algebra.properties.StructuralPropertiesVector) BroadcastPartitioningProperty(org.apache.hyracks.algebricks.core.algebra.properties.BroadcastPartitioningProperty) NotImplementedException(org.apache.hyracks.algebricks.common.exceptions.NotImplementedException) RandomPartitioningProperty(org.apache.hyracks.algebricks.core.algebra.properties.RandomPartitioningProperty) PhysicalRequirements(org.apache.hyracks.algebricks.core.algebra.properties.PhysicalRequirements)

Example 4 with NotImplementedException

use of org.apache.hyracks.algebricks.common.exceptions.NotImplementedException in project asterixdb by apache.

the class NestedLoopJoinPOperator method contributeRuntimeOperator.

@Override
public void contributeRuntimeOperator(IHyracksJobBuilder builder, JobGenContext context, ILogicalOperator op, IOperatorSchema propagatedSchema, IOperatorSchema[] inputSchemas, IOperatorSchema outerPlanSchema) throws AlgebricksException {
    AbstractBinaryJoinOperator join = (AbstractBinaryJoinOperator) op;
    RecordDescriptor recDescriptor = JobGenHelper.mkRecordDescriptor(context.getTypeEnvironment(op), propagatedSchema, context);
    IOperatorSchema[] conditionInputSchemas = new IOperatorSchema[1];
    conditionInputSchemas[0] = propagatedSchema;
    IExpressionRuntimeProvider expressionRuntimeProvider = context.getExpressionRuntimeProvider();
    IScalarEvaluatorFactory cond = expressionRuntimeProvider.createEvaluatorFactory(join.getCondition().getValue(), context.getTypeEnvironment(op), conditionInputSchemas, context);
    ITuplePairComparatorFactory comparatorFactory = new TuplePairEvaluatorFactory(cond, context.getBinaryBooleanInspectorFactory());
    IOperatorDescriptorRegistry spec = builder.getJobSpec();
    IOperatorDescriptor opDesc = null;
    switch(kind) {
        case INNER:
            {
                opDesc = new NestedLoopJoinOperatorDescriptor(spec, comparatorFactory, recDescriptor, memSize, false, null);
                break;
            }
        case LEFT_OUTER:
            {
                IMissingWriterFactory[] nonMatchWriterFactories = new IMissingWriterFactory[inputSchemas[1].getSize()];
                for (int j = 0; j < nonMatchWriterFactories.length; j++) {
                    nonMatchWriterFactories[j] = context.getMissingWriterFactory();
                }
                opDesc = new NestedLoopJoinOperatorDescriptor(spec, comparatorFactory, recDescriptor, memSize, true, nonMatchWriterFactories);
                break;
            }
        default:
            {
                throw new NotImplementedException();
            }
    }
    contributeOpDesc(builder, (AbstractLogicalOperator) op, opDesc);
    ILogicalOperator src1 = op.getInputs().get(0).getValue();
    builder.contributeGraphEdge(src1, 0, op, 0);
    ILogicalOperator src2 = op.getInputs().get(1).getValue();
    builder.contributeGraphEdge(src2, 0, op, 1);
}
Also used : RecordDescriptor(org.apache.hyracks.api.dataflow.value.RecordDescriptor) IOperatorSchema(org.apache.hyracks.algebricks.core.algebra.operators.logical.IOperatorSchema) NotImplementedException(org.apache.hyracks.algebricks.common.exceptions.NotImplementedException) ILogicalOperator(org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator) IOperatorDescriptorRegistry(org.apache.hyracks.api.job.IOperatorDescriptorRegistry) AbstractBinaryJoinOperator(org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractBinaryJoinOperator) IScalarEvaluatorFactory(org.apache.hyracks.algebricks.runtime.base.IScalarEvaluatorFactory) IExpressionRuntimeProvider(org.apache.hyracks.algebricks.core.algebra.expressions.IExpressionRuntimeProvider) IOperatorDescriptor(org.apache.hyracks.api.dataflow.IOperatorDescriptor) NestedLoopJoinOperatorDescriptor(org.apache.hyracks.dataflow.std.join.NestedLoopJoinOperatorDescriptor) ITuplePairComparatorFactory(org.apache.hyracks.api.dataflow.value.ITuplePairComparatorFactory)

Example 5 with NotImplementedException

use of org.apache.hyracks.algebricks.common.exceptions.NotImplementedException in project asterixdb by apache.

the class JoinMultiComparator method generateHashJoinRuntime.

private IOperatorDescriptor generateHashJoinRuntime(JobGenContext context, IOperatorSchema[] inputSchemas, int[] keysLeft, int[] keysRight, IBinaryHashFunctionFactory[] hashFunFactories, IBinaryComparatorFactory[] comparatorFactories, IPredicateEvaluatorFactory predEvaluatorFactory, RecordDescriptor recDescriptor, IOperatorDescriptorRegistry spec) throws AlgebricksException {
    IOperatorDescriptor opDesc;
    try {
        switch(kind) {
            case INNER:
                opDesc = new HybridHashJoinOperatorDescriptor(spec, getMemSizeInFrames(), maxInputBuildSizeInFrames, aveRecordsPerFrame, getFudgeFactor(), keysLeft, keysRight, hashFunFactories, comparatorFactories, recDescriptor, predEvaluatorFactory, false, null);
                break;
            case LEFT_OUTER:
                IMissingWriterFactory[] nonMatchWriterFactories = new IMissingWriterFactory[inputSchemas[1].getSize()];
                for (int j = 0; j < nonMatchWriterFactories.length; j++) {
                    nonMatchWriterFactories[j] = context.getMissingWriterFactory();
                }
                opDesc = new HybridHashJoinOperatorDescriptor(spec, getMemSizeInFrames(), maxInputBuildSizeInFrames, aveRecordsPerFrame, getFudgeFactor(), keysLeft, keysRight, hashFunFactories, comparatorFactories, recDescriptor, predEvaluatorFactory, true, nonMatchWriterFactories);
                break;
            default:
                throw new NotImplementedException();
        }
    } catch (HyracksDataException e) {
        throw new AlgebricksException(e);
    }
    return opDesc;
}
Also used : IOperatorDescriptor(org.apache.hyracks.api.dataflow.IOperatorDescriptor) IMissingWriterFactory(org.apache.hyracks.api.dataflow.value.IMissingWriterFactory) NotImplementedException(org.apache.hyracks.algebricks.common.exceptions.NotImplementedException) AlgebricksException(org.apache.hyracks.algebricks.common.exceptions.AlgebricksException) HybridHashJoinOperatorDescriptor(org.apache.hyracks.dataflow.std.join.HybridHashJoinOperatorDescriptor) OptimizedHybridHashJoinOperatorDescriptor(org.apache.hyracks.dataflow.std.join.OptimizedHybridHashJoinOperatorDescriptor) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException)

Aggregations

NotImplementedException (org.apache.hyracks.algebricks.common.exceptions.NotImplementedException)22 ILogicalOperator (org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator)9 AlgebricksException (org.apache.hyracks.algebricks.common.exceptions.AlgebricksException)7 ILogicalExpression (org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression)6 LogicalVariable (org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable)6 IOperatorDescriptor (org.apache.hyracks.api.dataflow.IOperatorDescriptor)6 HyracksDataException (org.apache.hyracks.api.exceptions.HyracksDataException)6 ATypeTag (org.apache.asterix.om.types.ATypeTag)5 IOException (java.io.IOException)4 List (java.util.List)4 Map (java.util.Map)4 IAType (org.apache.asterix.om.types.IAType)4 RecordDescriptor (org.apache.hyracks.api.dataflow.value.RecordDescriptor)4 ArrayList (java.util.ArrayList)3 AUnionType (org.apache.asterix.om.types.AUnionType)3 DataOutput (java.io.DataOutput)2 AsterixException (org.apache.asterix.common.exceptions.AsterixException)2 TypeMismatchException (org.apache.asterix.runtime.exceptions.TypeMismatchException)2 AlgebricksPartitionConstraint (org.apache.hyracks.algebricks.common.constraints.AlgebricksPartitionConstraint)2 Pair (org.apache.hyracks.algebricks.common.utils.Pair)2