Search in sources :

Example 46 with Memory

use of org.apache.datasketches.memory.Memory in project sketches-core by DataSketches.

the class HeapifyWrapSerVer1and2Test method checkHeapifyCompactSketchAssumedDifferentSeed.

@Test
public void checkHeapifyCompactSketchAssumedDifferentSeed() {
    final int k = 64;
    final long seed = 128L;
    final short seedHash = Util.computeSeedHash(seed);
    UpdateSketch sv3usk = UpdateSketch.builder().setNominalEntries(k).setSeed(seed).build();
    for (int i = 0; i < k; i++) {
        sv3usk.update(i);
    }
    CompactSketch sv3csk = sv3usk.compact();
    Memory sv3cskMem = Memory.wrap(sv3csk.toByteArray());
    CompactSketch sv3cskResult;
    // SV3 test
    sv3cskResult = Sketches.heapifyCompactSketch(sv3cskMem);
    assertEquals(sv3cskResult.getEstimate(), sv3usk.getEstimate());
    assertEquals(sv3cskResult.getSeedHash(), seedHash);
    // SV2 test
    Memory sv2cskMem = BackwardConversions.convertSerVer3toSerVer2(sv3csk, seed);
    sv3cskResult = Sketches.heapifyCompactSketch(sv2cskMem);
    assertEquals(sv3cskResult.getEstimate(), sv3usk.getEstimate());
    assertEquals(sv3cskResult.getSeedHash(), seedHash);
    // SV1 test
    Memory sv1cskMem = BackwardConversions.convertSerVer3toSerVer1(sv3csk);
    sv3cskResult = Sketches.heapifyCompactSketch(sv1cskMem);
    assertEquals(sv3cskResult.getEstimate(), sv3usk.getEstimate());
    assertEquals(sv3cskResult.getSeedHash(), defaultSeedHash);
}
Also used : Memory(org.apache.datasketches.memory.Memory) WritableMemory(org.apache.datasketches.memory.WritableMemory) Test(org.testng.annotations.Test)

Example 47 with Memory

use of org.apache.datasketches.memory.Memory in project sketches-core by DataSketches.

the class ReadOnlyMemoryTest method heapifyCompactOrderedSketch.

@Test
public void heapifyCompactOrderedSketch() {
    UpdateSketch updateSketch = UpdateSketch.builder().build();
    updateSketch.update(1);
    Memory mem = Memory.wrap(ByteBuffer.wrap(updateSketch.compact().toByteArray()).asReadOnlyBuffer().order(ByteOrder.nativeOrder()));
    Sketch sketch = Sketch.heapify(mem);
    assertEquals(sketch.getEstimate(), 1.0);
}
Also used : Memory(org.apache.datasketches.memory.Memory) Test(org.testng.annotations.Test)

Example 48 with Memory

use of org.apache.datasketches.memory.Memory in project sketches-core by DataSketches.

the class ReadOnlyMemoryTest method wrapAndTryUpdatingUnion.

@Test
public void wrapAndTryUpdatingUnion() {
    Union u1 = SetOperation.builder().buildUnion();
    u1.update(1);
    Memory mem = Memory.wrap(ByteBuffer.wrap(u1.toByteArray()).asReadOnlyBuffer().order(ByteOrder.nativeOrder()));
    Union u2 = (Union) Sketches.wrapSetOperation(mem);
    Union u3 = Sketches.wrapUnion(mem);
    Assert.assertEquals(u2.getResult().getEstimate(), 1.0);
    Assert.assertEquals(u3.getResult().getEstimate(), 1.0);
    try {
        u2.update(2);
        fail();
    } catch (SketchesReadOnlyException e) {
    // expected
    }
    try {
        u3.update(2);
        fail();
    } catch (SketchesReadOnlyException e) {
    // expected
    }
}
Also used : Memory(org.apache.datasketches.memory.Memory) SketchesReadOnlyException(org.apache.datasketches.SketchesReadOnlyException) Test(org.testng.annotations.Test)

Example 49 with Memory

use of org.apache.datasketches.memory.Memory in project sketches-core by DataSketches.

the class ReadOnlyMemoryTest method wrapAndTryUpdatingUpdateSketch.

@Test
public void wrapAndTryUpdatingUpdateSketch() {
    UpdateSketch updateSketch = UpdateSketch.builder().build();
    updateSketch.update(1);
    Memory mem = Memory.wrap(ByteBuffer.wrap(updateSketch.toByteArray()).asReadOnlyBuffer().order(ByteOrder.nativeOrder()));
    UpdateSketch sketch = (UpdateSketch) Sketch.wrap(mem);
    assertEquals(sketch.getEstimate(), 1.0);
    boolean thrown = false;
    try {
        sketch.update(2);
    } catch (SketchesReadOnlyException e) {
        thrown = true;
    }
    Assert.assertTrue(thrown);
}
Also used : Memory(org.apache.datasketches.memory.Memory) SketchesReadOnlyException(org.apache.datasketches.SketchesReadOnlyException) Test(org.testng.annotations.Test)

Example 50 with Memory

use of org.apache.datasketches.memory.Memory in project sketches-core by DataSketches.

the class ReadOnlyMemoryTest method wrapCompactUnorderedSketch.

@Test
public void wrapCompactUnorderedSketch() {
    UpdateSketch updateSketch = UpdateSketch.builder().build();
    updateSketch.update(1);
    Memory mem = Memory.wrap(ByteBuffer.wrap(updateSketch.compact(false, null).toByteArray()).asReadOnlyBuffer().order(ByteOrder.nativeOrder()));
    Sketch sketch = Sketch.wrap(mem);
    assertEquals(sketch.getEstimate(), 1.0);
}
Also used : Memory(org.apache.datasketches.memory.Memory) Test(org.testng.annotations.Test)

Aggregations

Memory (org.apache.datasketches.memory.Memory)213 Test (org.testng.annotations.Test)180 WritableMemory (org.apache.datasketches.memory.WritableMemory)164 SketchesArgumentException (org.apache.datasketches.SketchesArgumentException)17 ArrayOfLongsSerDe (org.apache.datasketches.ArrayOfLongsSerDe)14 SketchesReadOnlyException (org.apache.datasketches.SketchesReadOnlyException)11 File (java.io.File)10 ArrayOfStringsSerDe (org.apache.datasketches.ArrayOfStringsSerDe)10 Util.getResourceFile (org.apache.datasketches.Util.getResourceFile)10 MapHandle (org.apache.datasketches.memory.MapHandle)10 WritableHandle (org.apache.datasketches.memory.WritableHandle)6 SharedLocal (org.apache.datasketches.theta.ConcurrentHeapQuickSelectSketchTest.SharedLocal)3 ArrayOfNumbersSerDe (org.apache.datasketches.ArrayOfNumbersSerDe)2 CloseableIterator (org.apache.druid.java.util.common.parsers.CloseableIterator)2 IntIterator (it.unimi.dsi.fastutil.ints.IntIterator)1 HashSet (java.util.HashSet)1 Family (org.apache.datasketches.Family)1 SketchesStateException (org.apache.datasketches.SketchesStateException)1 CompressedState.importFromMemory (org.apache.datasketches.cpc.CompressedState.importFromMemory)1 DefaultMemoryRequestServer (org.apache.datasketches.memory.DefaultMemoryRequestServer)1