Search in sources :

Example 21 with AString

use of org.apache.asterix.om.base.AString in project asterixdb by apache.

the class PlanTranslationUtil method createFieldAccessExpression.

private static ScalarFunctionCallExpression createFieldAccessExpression(ILogicalExpression target, List<String> field) {
    FunctionIdentifier functionIdentifier;
    IAObject value;
    if (field.size() > 1) {
        functionIdentifier = BuiltinFunctions.FIELD_ACCESS_NESTED;
        value = new AOrderedList(field);
    } else {
        functionIdentifier = BuiltinFunctions.FIELD_ACCESS_BY_NAME;
        value = new AString(field.get(0));
    }
    IFunctionInfo finfoAccess = FunctionUtil.getFunctionInfo(functionIdentifier);
    return new ScalarFunctionCallExpression(finfoAccess, new MutableObject<>(target), new MutableObject<>(new ConstantExpression(new AsterixConstantValue(value))));
}
Also used : FunctionIdentifier(org.apache.hyracks.algebricks.core.algebra.functions.FunctionIdentifier) AOrderedList(org.apache.asterix.om.base.AOrderedList) IFunctionInfo(org.apache.hyracks.algebricks.core.algebra.functions.IFunctionInfo) AsterixConstantValue(org.apache.asterix.om.constants.AsterixConstantValue) IAObject(org.apache.asterix.om.base.IAObject) ConstantExpression(org.apache.hyracks.algebricks.core.algebra.expressions.ConstantExpression) AString(org.apache.asterix.om.base.AString) ScalarFunctionCallExpression(org.apache.hyracks.algebricks.core.algebra.expressions.ScalarFunctionCallExpression)

Example 22 with AString

use of org.apache.asterix.om.base.AString in project asterixdb by apache.

the class MetadataNode method createExternalFileSearchTuple.

// This method is used to create a search tuple for external data file since the search tuple has an int value
@SuppressWarnings("unchecked")
public ITupleReference createExternalFileSearchTuple(String dataverseName, String datasetName, int fileNumber) throws HyracksDataException {
    ISerializerDeserializer<AString> stringSerde = SerializerDeserializerProvider.INSTANCE.getSerializerDeserializer(BuiltinType.ASTRING);
    ISerializerDeserializer<AInt32> intSerde = SerializerDeserializerProvider.INSTANCE.getSerializerDeserializer(BuiltinType.AINT32);
    AMutableString aString = new AMutableString("");
    ArrayTupleBuilder tupleBuilder = new ArrayTupleBuilder(3);
    //dataverse field
    aString.setValue(dataverseName);
    stringSerde.serialize(aString, tupleBuilder.getDataOutput());
    tupleBuilder.addFieldEndOffset();
    //dataset field
    aString.setValue(datasetName);
    stringSerde.serialize(aString, tupleBuilder.getDataOutput());
    tupleBuilder.addFieldEndOffset();
    //file number field
    intSerde.serialize(new AInt32(fileNumber), tupleBuilder.getDataOutput());
    tupleBuilder.addFieldEndOffset();
    ArrayTupleReference tuple = new ArrayTupleReference();
    tuple.reset(tupleBuilder.getFieldEndOffsets(), tupleBuilder.getByteArray());
    return tuple;
}
Also used : ArrayTupleReference(org.apache.hyracks.dataflow.common.comm.io.ArrayTupleReference) AMutableString(org.apache.asterix.om.base.AMutableString) ArrayTupleBuilder(org.apache.hyracks.dataflow.common.comm.io.ArrayTupleBuilder) AString(org.apache.asterix.om.base.AString) AInt32(org.apache.asterix.om.base.AInt32)

Example 23 with AString

use of org.apache.asterix.om.base.AString in project asterixdb by apache.

the class MetadataNode method createTuple.

// TODO: Can use Hyrack's TupleUtils for this, once we switch to a newer
// Hyracks version.
public static ITupleReference createTuple(String... fields) {
    @SuppressWarnings("unchecked") ISerializerDeserializer<AString> stringSerde = SerializerDeserializerProvider.INSTANCE.getSerializerDeserializer(BuiltinType.ASTRING);
    AMutableString aString = new AMutableString("");
    ArrayTupleBuilder tupleBuilder = new ArrayTupleBuilder(fields.length);
    for (String s : fields) {
        aString.setValue(s);
        try {
            stringSerde.serialize(aString, tupleBuilder.getDataOutput());
        } catch (HyracksDataException e) {
            // This should never happen
            throw new IllegalStateException("Failed to create search tuple!!!! This should never happen", e);
        }
        tupleBuilder.addFieldEndOffset();
    }
    ArrayTupleReference tuple = new ArrayTupleReference();
    tuple.reset(tupleBuilder.getFieldEndOffsets(), tupleBuilder.getByteArray());
    return tuple;
}
Also used : ArrayTupleReference(org.apache.hyracks.dataflow.common.comm.io.ArrayTupleReference) AMutableString(org.apache.asterix.om.base.AMutableString) ArrayTupleBuilder(org.apache.hyracks.dataflow.common.comm.io.ArrayTupleBuilder) AString(org.apache.asterix.om.base.AString) AMutableString(org.apache.asterix.om.base.AMutableString) AString(org.apache.asterix.om.base.AString) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException)

Example 24 with AString

use of org.apache.asterix.om.base.AString in project asterixdb by apache.

the class LangExpressionToPlanTranslator method visit.

@Override
public Pair<ILogicalOperator, LogicalVariable> visit(FieldAccessor fa, Mutable<ILogicalOperator> tupSource) throws CompilationException {
    Pair<ILogicalExpression, Mutable<ILogicalOperator>> p = langExprToAlgExpression(fa.getExpr(), tupSource);
    LogicalVariable v = context.newVarFromExpression(fa);
    AbstractFunctionCallExpression fldAccess = new ScalarFunctionCallExpression(FunctionUtil.getFunctionInfo(BuiltinFunctions.FIELD_ACCESS_BY_NAME));
    fldAccess.getArguments().add(new MutableObject<>(p.first));
    ILogicalExpression faExpr = new ConstantExpression(new AsterixConstantValue(new AString(fa.getIdent().getValue())));
    fldAccess.getArguments().add(new MutableObject<>(faExpr));
    AssignOperator a = new AssignOperator(v, new MutableObject<>(fldAccess));
    a.getInputs().add(p.second);
    return new Pair<>(a, v);
}
Also used : LogicalVariable(org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable) Mutable(org.apache.commons.lang3.mutable.Mutable) ILogicalExpression(org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression) AsterixConstantValue(org.apache.asterix.om.constants.AsterixConstantValue) AbstractFunctionCallExpression(org.apache.hyracks.algebricks.core.algebra.expressions.AbstractFunctionCallExpression) ConstantExpression(org.apache.hyracks.algebricks.core.algebra.expressions.ConstantExpression) AssignOperator(org.apache.hyracks.algebricks.core.algebra.operators.logical.AssignOperator) AString(org.apache.asterix.om.base.AString) ScalarFunctionCallExpression(org.apache.hyracks.algebricks.core.algebra.expressions.ScalarFunctionCallExpression) GbyVariableExpressionPair(org.apache.asterix.lang.common.expression.GbyVariableExpressionPair) Pair(org.apache.hyracks.algebricks.common.utils.Pair) QuantifiedPair(org.apache.asterix.lang.common.struct.QuantifiedPair)

Example 25 with AString

use of org.apache.asterix.om.base.AString in project asterixdb by apache.

the class StaticTypeCastUtil method staticRecordTypeCast.

/**
     * This method statically cast the type of records from their current type to the required type.
     *
     * @param func
     *            The record constructor expression.
     * @param reqType
     *            The required type.
     * @param inputType
     *            The current type.
     * @param env
     *            The type environment.
     * @throws AlgebricksException
     */
private static boolean staticRecordTypeCast(AbstractFunctionCallExpression func, ARecordType reqType, ARecordType inputType, IVariableTypeEnvironment env) throws AlgebricksException {
    if (!(func.getFunctionIdentifier() == BuiltinFunctions.OPEN_RECORD_CONSTRUCTOR || func.getFunctionIdentifier() == BuiltinFunctions.CLOSED_RECORD_CONSTRUCTOR)) {
        return false;
    }
    IAType[] reqFieldTypes = reqType.getFieldTypes();
    String[] reqFieldNames = reqType.getFieldNames();
    IAType[] inputFieldTypes = inputType.getFieldTypes();
    String[] inputFieldNames = inputType.getFieldNames();
    int[] fieldPermutation = new int[reqFieldTypes.length];
    boolean[] nullFields = new boolean[reqFieldTypes.length];
    boolean[] openFields = new boolean[inputFieldTypes.length];
    Arrays.fill(nullFields, false);
    Arrays.fill(openFields, true);
    Arrays.fill(fieldPermutation, -1);
    // forward match: match from actual to required
    boolean matched = false;
    for (int i = 0; i < inputFieldNames.length; i++) {
        String fieldName = inputFieldNames[i];
        IAType fieldType = inputFieldTypes[i];
        if (2 * i + 1 > func.getArguments().size()) {
            // it is not a record constructor function
            return false;
        }
        // 2*i+1 is the index of field value expression
        ILogicalExpression arg = func.getArguments().get(2 * i + 1).getValue();
        matched = false;
        for (int j = 0; j < reqFieldNames.length; j++) {
            String reqFieldName = reqFieldNames[j];
            IAType reqFieldType = reqFieldTypes[j];
            if (fieldName.equals(reqFieldName)) {
                //type matched
                if (fieldType.equals(reqFieldType)) {
                    fieldPermutation[j] = i;
                    openFields[i] = false;
                    matched = true;
                    if (arg.getExpressionTag() == LogicalExpressionTag.FUNCTION_CALL) {
                        ScalarFunctionCallExpression scalarFunc = (ScalarFunctionCallExpression) arg;
                        rewriteFuncExpr(scalarFunc, reqFieldType, fieldType, env);
                    }
                    break;
                }
                // match the optional field
                if (NonTaggedFormatUtil.isOptional(reqFieldType)) {
                    IAType itemType = ((AUnionType) reqFieldType).getActualType();
                    reqFieldType = itemType;
                    if (fieldType.equals(BuiltinType.AMISSING) || fieldType.equals(itemType)) {
                        fieldPermutation[j] = i;
                        openFields[i] = false;
                        matched = true;
                        // rewrite record expr
                        if (arg.getExpressionTag() == LogicalExpressionTag.FUNCTION_CALL) {
                            ScalarFunctionCallExpression scalarFunc = (ScalarFunctionCallExpression) arg;
                            rewriteFuncExpr(scalarFunc, reqFieldType, fieldType, env);
                        }
                        break;
                    }
                }
                // delay that to runtime by calling the not-null function
                if (NonTaggedFormatUtil.isOptional(fieldType)) {
                    IAType itemType = ((AUnionType) fieldType).getActualType();
                    if (reqFieldType.equals(itemType)) {
                        fieldPermutation[j] = i;
                        openFields[i] = false;
                        matched = true;
                        ScalarFunctionCallExpression notNullFunc = new ScalarFunctionCallExpression(FunctionUtil.getFunctionInfo(BuiltinFunctions.CHECK_UNKNOWN));
                        notNullFunc.getArguments().add(new MutableObject<ILogicalExpression>(arg));
                        //wrap the not null function to the original function
                        func.getArguments().get(2 * i + 1).setValue(notNullFunc);
                        break;
                    }
                }
                // match the record field: need cast
                if (arg.getExpressionTag() == LogicalExpressionTag.FUNCTION_CALL) {
                    ScalarFunctionCallExpression scalarFunc = (ScalarFunctionCallExpression) arg;
                    rewriteFuncExpr(scalarFunc, reqFieldType, fieldType, env);
                    fieldPermutation[j] = i;
                    openFields[i] = false;
                    matched = true;
                    break;
                }
            }
        }
        // the input has extra fields
        if (!matched && !reqType.isOpen()) {
            throw new AlgebricksException("static type mismatch: the input record includes an extra closed field " + fieldName + ":" + fieldType + "! Please check the field name and type.");
        }
    }
    // backward match: match from required to actual
    for (int i = 0; i < reqFieldNames.length; i++) {
        String reqFieldName = reqFieldNames[i];
        IAType reqFieldType = reqFieldTypes[i];
        matched = false;
        for (int j = 0; j < inputFieldNames.length; j++) {
            String fieldName = inputFieldNames[j];
            IAType fieldType = inputFieldTypes[j];
            if (!fieldName.equals(reqFieldName)) {
                continue;
            }
            // the entry index of fieldPermuatons is req field index
            if (!openFields[j]) {
                matched = true;
                break;
            }
            // match the optional field
            if (!NonTaggedFormatUtil.isOptional(reqFieldType)) {
                continue;
            }
            IAType itemType = ((AUnionType) reqFieldType).getActualType();
            if (fieldType.equals(BuiltinType.AMISSING) || fieldType.equals(itemType)) {
                matched = true;
                break;
            }
        }
        if (matched) {
            continue;
        }
        if (NonTaggedFormatUtil.isOptional(reqFieldType)) {
            // add a null field
            nullFields[i] = true;
        } else {
            // no matched field in the input for a required closed field
            if (inputType.isOpen()) {
                //if the input type is open, return false, give that to dynamic type cast to defer the error to the runtime
                return false;
            } else {
                throw new AlgebricksException("static type mismatch: the input record misses a required closed field " + reqFieldName + ":" + reqFieldType + "! Please check the field name and type.");
            }
        }
    }
    List<Mutable<ILogicalExpression>> arguments = func.getArguments();
    List<Mutable<ILogicalExpression>> originalArguments = new ArrayList<Mutable<ILogicalExpression>>();
    originalArguments.addAll(arguments);
    arguments.clear();
    // re-order the closed part and fill in null fields
    for (int i = 0; i < fieldPermutation.length; i++) {
        int pos = fieldPermutation[i];
        if (pos >= 0) {
            arguments.add(originalArguments.get(2 * pos));
            arguments.add(originalArguments.get(2 * pos + 1));
        }
        if (nullFields[i]) {
            // add a null field
            arguments.add(new MutableObject<ILogicalExpression>(new ConstantExpression(new AsterixConstantValue(new AString(reqFieldNames[i])))));
            arguments.add(new MutableObject<ILogicalExpression>(new ConstantExpression(new AsterixConstantValue(ANull.NULL))));
        }
    }
    // add the open part
    for (int i = 0; i < openFields.length; i++) {
        if (openFields[i]) {
            arguments.add(originalArguments.get(2 * i));
            Mutable<ILogicalExpression> expRef = originalArguments.get(2 * i + 1);
            injectCastToRelaxType(expRef, inputFieldTypes[i], env);
            arguments.add(expRef);
        }
    }
    return true;
}
Also used : AUnionType(org.apache.asterix.om.types.AUnionType) ConstantExpression(org.apache.hyracks.algebricks.core.algebra.expressions.ConstantExpression) AlgebricksException(org.apache.hyracks.algebricks.common.exceptions.AlgebricksException) ArrayList(java.util.ArrayList) AString(org.apache.asterix.om.base.AString) Mutable(org.apache.commons.lang3.mutable.Mutable) ILogicalExpression(org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression) AsterixConstantValue(org.apache.asterix.om.constants.AsterixConstantValue) AString(org.apache.asterix.om.base.AString) IAType(org.apache.asterix.om.types.IAType) ScalarFunctionCallExpression(org.apache.hyracks.algebricks.core.algebra.expressions.ScalarFunctionCallExpression)

Aggregations

AString (org.apache.asterix.om.base.AString)58 AsterixConstantValue (org.apache.asterix.om.constants.AsterixConstantValue)21 ConstantExpression (org.apache.hyracks.algebricks.core.algebra.expressions.ConstantExpression)20 ArrayList (java.util.ArrayList)19 ILogicalExpression (org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression)18 AMutableString (org.apache.asterix.om.base.AMutableString)16 ArrayBackedValueStorage (org.apache.hyracks.data.std.util.ArrayBackedValueStorage)16 AOrderedList (org.apache.asterix.om.base.AOrderedList)13 Mutable (org.apache.commons.lang3.mutable.Mutable)12 RecordBuilder (org.apache.asterix.builders.RecordBuilder)11 AInt32 (org.apache.asterix.om.base.AInt32)11 ScalarFunctionCallExpression (org.apache.hyracks.algebricks.core.algebra.expressions.ScalarFunctionCallExpression)10 IARecordBuilder (org.apache.asterix.builders.IARecordBuilder)9 IACursor (org.apache.asterix.om.base.IACursor)9 IAObject (org.apache.asterix.om.base.IAObject)9 IAType (org.apache.asterix.om.types.IAType)9 AlgebricksException (org.apache.hyracks.algebricks.common.exceptions.AlgebricksException)9 LogicalVariable (org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable)9 AbstractFunctionCallExpression (org.apache.hyracks.algebricks.core.algebra.expressions.AbstractFunctionCallExpression)9 ARecord (org.apache.asterix.om.base.ARecord)8