Search in sources :

Example 6 with BinaryEntry

use of org.apache.hyracks.data.std.util.BinaryEntry in project asterixdb by apache.

the class SimilarityJaccardEvaluator method probeHashMap.

protected int probeHashMap(AbstractAsterixListIterator probeIter, int buildListSize, int probeListSize) throws HyracksDataException {
    // Probe phase: Probe items from second list, and compute intersection size.
    int intersectionSize = 0;
    while (probeIter.hasNext()) {
        byte[] buf = probeIter.getData();
        int off = probeIter.getPos();
        int len = probeIter.getItemLen();
        keyEntry.set(buf, off, len);
        BinaryEntry entry = hashMap.get(keyEntry);
        if (entry != null) {
            // Increment second value.
            int firstValInt = IntegerPointable.getInteger(entry.getBuf(), entry.getOffset());
            // Irrelevant for the intersection size.
            if (firstValInt == 0) {
                continue;
            }
            int secondValInt = IntegerPointable.getInteger(entry.getBuf(), entry.getOffset() + 4);
            // Subtract old min value.
            intersectionSize -= (firstValInt < secondValInt) ? firstValInt : secondValInt;
            secondValInt++;
            // Add new min value.
            intersectionSize += (firstValInt < secondValInt) ? firstValInt : secondValInt;
            IntegerPointable.setInteger(entry.getBuf(), entry.getOffset() + 4, secondValInt);
        }
        probeIter.next();
    }
    return intersectionSize;
}
Also used : BinaryEntry(org.apache.hyracks.data.std.util.BinaryEntry)

Example 7 with BinaryEntry

use of org.apache.hyracks.data.std.util.BinaryEntry in project asterixdb by apache.

the class SimilarityJaccardEvaluator method buildHashMap.

protected void buildHashMap(AbstractAsterixListIterator buildIter) throws HyracksDataException {
    // Build phase: Add items into hash map, starting with first list.
    // Value in map is a pair of integers. Set first integer to 1.
    IntegerPointable.setInteger(valEntry.getBuf(), 0, 1);
    while (buildIter.hasNext()) {
        byte[] buf = buildIter.getData();
        int off = buildIter.getPos();
        int len = buildIter.getItemLen();
        keyEntry.set(buf, off, len);
        BinaryEntry entry = hashMap.put(keyEntry, valEntry);
        if (entry != null) {
            // Increment value.
            int firstValInt = IntegerPointable.getInteger(entry.getBuf(), entry.getOffset());
            IntegerPointable.setInteger(entry.getBuf(), entry.getOffset(), firstValInt + 1);
        }
        buildIter.next();
    }
}
Also used : BinaryEntry(org.apache.hyracks.data.std.util.BinaryEntry)

Aggregations

BinaryEntry (org.apache.hyracks.data.std.util.BinaryEntry)7 IVisitablePointable (org.apache.asterix.om.pointables.base.IVisitablePointable)3 ATypeTag (org.apache.asterix.om.types.ATypeTag)2 DataOutput (java.io.DataOutput)1 RecordBuilder (org.apache.asterix.builders.RecordBuilder)1 AsterixException (org.apache.asterix.common.exceptions.AsterixException)1 RuntimeDataException (org.apache.asterix.common.exceptions.RuntimeDataException)1 AListVisitablePointable (org.apache.asterix.om.pointables.AListVisitablePointable)1 ARecordVisitablePointable (org.apache.asterix.om.pointables.ARecordVisitablePointable)1 PointableAllocator (org.apache.asterix.om.pointables.PointableAllocator)1 RuntimeRecordTypeInfo (org.apache.asterix.om.types.runtime.RuntimeRecordTypeInfo)1 BinaryHashMap (org.apache.asterix.runtime.evaluators.functions.BinaryHashMap)1 PointableHelper (org.apache.asterix.runtime.evaluators.functions.PointableHelper)1 InvalidDataFormatException (org.apache.asterix.runtime.exceptions.InvalidDataFormatException)1 TypeMismatchException (org.apache.asterix.runtime.exceptions.TypeMismatchException)1 IScalarEvaluator (org.apache.hyracks.algebricks.runtime.base.IScalarEvaluator)1 IScalarEvaluatorFactory (org.apache.hyracks.algebricks.runtime.base.IScalarEvaluatorFactory)1 IHyracksTaskContext (org.apache.hyracks.api.context.IHyracksTaskContext)1 IBinaryComparator (org.apache.hyracks.api.dataflow.value.IBinaryComparator)1 IBinaryHashFunction (org.apache.hyracks.api.dataflow.value.IBinaryHashFunction)1