Search in sources :

Example 56 with SketchesArgumentException

use of org.apache.datasketches.SketchesArgumentException in project sketches-core by DataSketches.

the class Sketch method heapifyUpdateFromMemory.

/**
 * Instantiates a Heap Update Sketch from Memory. Only SerVer3. SerVer 1 & 2 already handled.
 * @param srcMem <a href="{@docRoot}/resources/dictionary.html#mem">See Memory</a>
 * @param expectedSeed the seed used to validate the given Memory image.
 * <a href="{@docRoot}/resources/dictionary.html#seed">See Update Hash Seed</a>.
 * @return a Sketch
 */
private static final Sketch heapifyUpdateFromMemory(final Memory srcMem, final long expectedSeed) {
    final long cap = srcMem.getCapacity();
    if (cap < 8) {
        throw new SketchesArgumentException("Corrupted: valid sketch must be at least 8 bytes.");
    }
    final byte familyID = srcMem.getByte(FAMILY_BYTE);
    final Family family = idToFamily(familyID);
    if (family == Family.ALPHA) {
        final int flags = PreambleUtil.extractFlags(srcMem);
        final boolean compactFlag = (flags & COMPACT_FLAG_MASK) != 0;
        if (compactFlag) {
            throw new SketchesArgumentException("Corrupted: ALPHA family image: cannot be compact");
        }
        return HeapAlphaSketch.heapifyInstance(srcMem, expectedSeed);
    }
    if (family == Family.QUICKSELECT) {
        return HeapQuickSelectSketch.heapifyInstance(srcMem, expectedSeed);
    }
    throw new SketchesArgumentException("Sketch cannot heapify family: " + family + " as a Sketch");
}
Also used : SketchesArgumentException(org.apache.datasketches.SketchesArgumentException) Family.idToFamily(org.apache.datasketches.Family.idToFamily) Family(org.apache.datasketches.Family)

Example 57 with SketchesArgumentException

use of org.apache.datasketches.SketchesArgumentException in project sketches-core by DataSketches.

the class AdoubleIntersectionTest method checkExactIntersectionWithTheta.

@Test
public void checkExactIntersectionWithTheta() {
    final UpdateSketch thSkNull = null;
    final UpdateSketch thSkEmpty = new UpdateSketchBuilder().build();
    final UpdateSketch thSk10 = new UpdateSketchBuilder().build();
    final UpdateSketch thSk15 = new UpdateSketchBuilder().build();
    for (int i = 0; i < 10; i++) {
        thSk10.update(i);
    }
    // overlap = 5
    for (int i = 0; i < 10; i++) {
        thSk15.update(i + 5);
    }
    DoubleSummary dsum = new DoubleSummaryFactory(mode).newSummary();
    final Intersection<DoubleSummary> intersection = new Intersection<>(new DoubleSummarySetOperations(mode, mode));
    CompactSketch<DoubleSummary> result;
    try {
        intersection.getResult();
        fail();
    }// OK.
     catch (final SketchesStateException e) {
    }
    try {
        intersection.intersect(thSkNull, dsum);
        fail();
    }// OK
     catch (final SketchesArgumentException e) {
    }
    intersection.intersect(thSkEmpty, dsum);
    result = intersection.getResult();
    // Empty after empty first call
    Assert.assertTrue(result.isEmpty());
    intersection.reset();
    intersection.intersect(thSk10, dsum);
    result = intersection.getResult();
    // Returns valid first call
    Assert.assertEquals(result.getEstimate(), 10.0);
    intersection.reset();
    // Valid first call
    intersection.intersect(thSk10, dsum);
    intersection.intersect(thSkEmpty, dsum);
    result = intersection.getResult();
    // Returns Empty after empty second call
    Assert.assertTrue(result.isEmpty());
    intersection.reset();
    intersection.intersect(thSk10, dsum);
    intersection.intersect(thSk15, dsum);
    result = intersection.getResult();
    // Returns intersection
    Assert.assertEquals(result.getEstimate(), 5.0);
    intersection.reset();
    dsum = null;
    try {
        intersection.intersect(thSk10, dsum);
        fail();
    } catch (final SketchesArgumentException e) {
    }
}
Also used : Intersection(org.apache.datasketches.tuple.Intersection) SketchesArgumentException(org.apache.datasketches.SketchesArgumentException) UpdateSketchBuilder(org.apache.datasketches.theta.UpdateSketchBuilder) SketchesStateException(org.apache.datasketches.SketchesStateException) UpdateSketch(org.apache.datasketches.theta.UpdateSketch) Test(org.testng.annotations.Test)

Example 58 with SketchesArgumentException

use of org.apache.datasketches.SketchesArgumentException in project sketches-core by DataSketches.

the class AdoubleUnionTest method checkUnionUpdateWithTheta.

@Test
public void checkUnionUpdateWithTheta() {
    final Union<DoubleSummary> union = new Union<>(new DoubleSummarySetOperations(mode, mode));
    UpdateSketch usk = null;
    DoubleSummary dsum = null;
    try {
        union.union(usk, dsum);
        fail();
    } catch (final SketchesArgumentException e) {
    }
    usk = new UpdateSketchBuilder().build();
    try {
        union.union(usk, dsum);
        fail();
    } catch (final SketchesArgumentException e) {
    }
    dsum = new DoubleSummaryFactory(mode).newSummary();
    for (int i = 0; i < 10; i++) {
        usk.update(i);
    }
    union.union(usk, dsum);
    Assert.assertEquals(union.getResult().getEstimate(), 10.0);
}
Also used : SketchesArgumentException(org.apache.datasketches.SketchesArgumentException) UpdateSketchBuilder(org.apache.datasketches.theta.UpdateSketchBuilder) Union(org.apache.datasketches.tuple.Union) UpdateSketch(org.apache.datasketches.theta.UpdateSketch) Test(org.testng.annotations.Test)

Example 59 with SketchesArgumentException

use of org.apache.datasketches.SketchesArgumentException in project sketches-core by DataSketches.

the class AnotB method aNotB.

/**
 * Returns the A-and-not-B set operation on the two given Tuple sketches.
 *
 * <p>This a stateless operation and has no impact on the internal state of this operator.
 * Thus, this is not an accumulating update and is independent of the {@link #setA(Sketch)},
 * {@link #notB(Sketch)}, {@link #notB(org.apache.datasketches.theta.Sketch)}, and
 * {@link #getResult(boolean)} methods.</p>
 *
 * <p>If either argument is null an exception is thrown.</p>
 *
 * <p>Rationale: In mathematics a "null set" is a set with no members, which we call an empty set.
 * That is distinctly different from the java <i>null</i>, which represents a nonexistent object.
 * In most cases it is a programming error due to some object that was not properly initialized.
 * With a null as the first argument, we cannot know what the user's intent is.
 * With a null as the second argument, we can't ignore it as we must return a result and there is
 * no following possible viable arguments for the second argument.
 * Since it is very likely that a <i>null</i> is a programming error, we throw an exception.</p>
 *
 * @param skA The incoming Tuple sketch for the first argument
 * @param skB The incoming Tuple sketch for the second argument
 * @param <S> Type of Summary
 * @return the result as an unordered {@link CompactSketch}
 */
@SuppressWarnings("unchecked")
public static <S extends Summary> CompactSketch<S> aNotB(final Sketch<S> skA, final Sketch<S> skB) {
    if (skA == null || skB == null) {
        throw new SketchesArgumentException("Neither argument may be null for this stateless operation.");
    }
    final long thetaLongA = skA.getThetaLong();
    final int countA = skA.getRetainedEntries();
    final boolean emptyA = skA.isEmpty();
    final long thetaLongB = skB.getThetaLong();
    final int countB = skB.getRetainedEntries();
    final boolean emptyB = skB.isEmpty();
    final int id = SetOperationCornerCases.createCornerCaseId(thetaLongA, countA, emptyA, thetaLongB, countB, emptyB);
    final CornerCase cCase = CornerCase.caseIdToCornerCase(id);
    final AnotbAction anotbAction = cCase.getAnotbAction();
    CompactSketch<S> result = null;
    switch(anotbAction) {
        case EMPTY_1_0_T:
            {
                result = new CompactSketch<>(null, null, Long.MAX_VALUE, true);
                break;
            }
        case DEGEN_MIN_0_F:
            {
                final long thetaLong = min(thetaLongA, thetaLongB);
                result = new CompactSketch<>(null, null, thetaLong, false);
                break;
            }
        case DEGEN_THA_0_F:
            {
                result = new CompactSketch<>(null, null, thetaLongA, false);
                break;
            }
        case TRIM_A:
            {
                final DataArrays<S> daA = getCopyOfDataArraysTuple(skA);
                final long[] hashArrA = daA.hashArr;
                final S[] summaryArrA = daA.summaryArr;
                final long minThetaLong = min(thetaLongA, thetaLongB);
                final DataArrays<S> da = trimAndCopyDataArrays(hashArrA, summaryArrA, minThetaLong, false);
                result = new CompactSketch<>(da.hashArr, da.summaryArr, minThetaLong, skA.empty_);
                break;
            }
        case SKETCH_A:
            {
                final DataArrays<S> daA = getCopyOfDataArraysTuple(skA);
                result = new CompactSketch<>(daA.hashArr, daA.summaryArr, thetaLongA, skA.empty_);
                break;
            }
        case FULL_ANOTB:
            {
                // both A and B should have valid entries.
                final DataArrays<S> daA = getCopyOfDataArraysTuple(skA);
                final long minThetaLong = min(thetaLongA, thetaLongB);
                final DataArrays<S> daR = getCopyOfResultArraysTuple(minThetaLong, daA.hashArr.length, daA.hashArr, daA.summaryArr, skB);
                final int countR = (daR.hashArr == null) ? 0 : daR.hashArr.length;
                if (countR == 0) {
                    result = new CompactSketch<>(null, null, minThetaLong, minThetaLong == Long.MAX_VALUE);
                } else {
                    result = new CompactSketch<>(daR.hashArr, daR.summaryArr, minThetaLong, false);
                }
            }
    }
    return result;
}
Also used : SketchesArgumentException(org.apache.datasketches.SketchesArgumentException) CornerCase(org.apache.datasketches.SetOperationCornerCases.CornerCase) AnotbAction(org.apache.datasketches.SetOperationCornerCases.AnotbAction)

Example 60 with SketchesArgumentException

use of org.apache.datasketches.SketchesArgumentException in project sketches-core by DataSketches.

the class AnotB method aNotB.

/**
 * Returns the A-and-not-B set operation on a Tuple sketch and a Theta sketch.
 *
 * <p>This a stateless operation and has no impact on the internal state of this operator.
 * Thus, this is not an accumulating update and is independent of the {@link #setA(Sketch)},
 * {@link #notB(Sketch)}, {@link #notB(org.apache.datasketches.theta.Sketch)}, and
 * {@link #getResult(boolean)} methods.</p>
 *
 * <p>If either argument is null an exception is thrown.</p>
 *
 * <p>Rationale: In mathematics a "null set" is a set with no members, which we call an empty set.
 * That is distinctly different from the java <i>null</i>, which represents a nonexistent object.
 * In most cases it is a programming error due to some object that was not properly initialized.
 * With a null as the first argument, we cannot know what the user's intent is.
 * With a null as the second argument, we can't ignore it as we must return a result and there is
 * no following possible viable arguments for the second argument.
 * Since it is very likely that a <i>null</i> is a programming error for either argument
 * we throw a an exception.</p>
 *
 * @param skA The incoming Tuple sketch for the first argument
 * @param skB The incoming Theta sketch for the second argument
 * @param <S> Type of Summary
 * @return the result as an unordered {@link CompactSketch}
 */
@SuppressWarnings("unchecked")
public static <S extends Summary> CompactSketch<S> aNotB(final Sketch<S> skA, final org.apache.datasketches.theta.Sketch skB) {
    if (skA == null || skB == null) {
        throw new SketchesArgumentException("Neither argument may be null for this stateless operation.");
    }
    final long thetaLongA = skA.getThetaLong();
    final int countA = skA.getRetainedEntries();
    final boolean emptyA = skA.isEmpty();
    final long thetaLongB = skB.getThetaLong();
    final int countB = skB.getRetainedEntries();
    final boolean emptyB = skB.isEmpty();
    final int id = SetOperationCornerCases.createCornerCaseId(thetaLongA, countA, emptyA, thetaLongB, countB, emptyB);
    final CornerCase cCase = CornerCase.caseIdToCornerCase(id);
    final AnotbAction anotbAction = cCase.getAnotbAction();
    CompactSketch<S> result = null;
    switch(anotbAction) {
        case EMPTY_1_0_T:
            {
                result = new CompactSketch<>(null, null, Long.MAX_VALUE, true);
                break;
            }
        case DEGEN_MIN_0_F:
            {
                final long thetaLong = min(thetaLongA, thetaLongB);
                result = new CompactSketch<>(null, null, thetaLong, false);
                break;
            }
        case DEGEN_THA_0_F:
            {
                result = new CompactSketch<>(null, null, thetaLongA, false);
                break;
            }
        case TRIM_A:
            {
                final DataArrays<S> daA = getCopyOfDataArraysTuple(skA);
                final long[] hashArrA = daA.hashArr;
                final S[] summaryArrA = daA.summaryArr;
                final long minThetaLong = min(thetaLongA, thetaLongB);
                final DataArrays<S> da = trimAndCopyDataArrays(hashArrA, summaryArrA, minThetaLong, false);
                result = new CompactSketch<>(da.hashArr, da.summaryArr, minThetaLong, skA.empty_);
                break;
            }
        case SKETCH_A:
            {
                final DataArrays<S> daA = getCopyOfDataArraysTuple(skA);
                result = new CompactSketch<>(daA.hashArr, daA.summaryArr, thetaLongA, skA.empty_);
                break;
            }
        case FULL_ANOTB:
            {
                // both A and B should have valid entries.
                final DataArrays<S> daA = getCopyOfDataArraysTuple(skA);
                final long minThetaLong = min(thetaLongA, thetaLongB);
                final DataArrays<S> daR = getCopyOfResultArraysTheta(minThetaLong, daA.hashArr.length, daA.hashArr, daA.summaryArr, skB);
                final int countR = (daR.hashArr == null) ? 0 : daR.hashArr.length;
                if (countR == 0) {
                    result = new CompactSketch<>(null, null, minThetaLong, minThetaLong == Long.MAX_VALUE);
                } else {
                    result = new CompactSketch<>(daR.hashArr, daR.summaryArr, minThetaLong, false);
                }
            }
    }
    return result;
}
Also used : SketchesArgumentException(org.apache.datasketches.SketchesArgumentException) CornerCase(org.apache.datasketches.SetOperationCornerCases.CornerCase) AnotbAction(org.apache.datasketches.SetOperationCornerCases.AnotbAction)

Aggregations

SketchesArgumentException (org.apache.datasketches.SketchesArgumentException)64 WritableMemory (org.apache.datasketches.memory.WritableMemory)38 Test (org.testng.annotations.Test)29 Family (org.apache.datasketches.Family)14 Memory (org.apache.datasketches.memory.Memory)13 Family.idToFamily (org.apache.datasketches.Family.idToFamily)10 SketchesReadOnlyException (org.apache.datasketches.SketchesReadOnlyException)7 WritableHandle (org.apache.datasketches.memory.WritableHandle)6 ResizeFactor (org.apache.datasketches.ResizeFactor)5 ArrayOfLongsSerDe (org.apache.datasketches.ArrayOfLongsSerDe)4 AnotbAction (org.apache.datasketches.SetOperationCornerCases.AnotbAction)3 CornerCase (org.apache.datasketches.SetOperationCornerCases.CornerCase)3 PreambleUtil.extractResizeFactor (org.apache.datasketches.sampling.PreambleUtil.extractResizeFactor)3 ArrayList (java.util.ArrayList)2 SketchesStateException (org.apache.datasketches.SketchesStateException)2 PreambleUtil.insertLgResizeFactor (org.apache.datasketches.theta.PreambleUtil.insertLgResizeFactor)2 UpdateSketch (org.apache.datasketches.theta.UpdateSketch)2 UpdateSketchBuilder (org.apache.datasketches.theta.UpdateSketchBuilder)2 BigDecimal (java.math.BigDecimal)1 ArrayOfBooleansSerDe (org.apache.datasketches.ArrayOfBooleansSerDe)1