Search in sources :

Example 51 with OffsetRange

use of org.apache.beam.sdk.io.range.OffsetRange in project beam by apache.

the class OffsetRangeTrackerTest method testCheckpointJustStarted.

@Test
public void testCheckpointJustStarted() throws Exception {
    OffsetRangeTracker tracker = new OffsetRangeTracker(new OffsetRange(100, 200));
    assertTrue(tracker.tryClaim(100L));
    OffsetRange checkpoint = tracker.trySplit(0).getResidual();
    assertEquals(new OffsetRange(100, 101), tracker.currentRestriction());
    assertEquals(new OffsetRange(101, 200), checkpoint);
    tracker.checkDone();
}
Also used : OffsetRange(org.apache.beam.sdk.io.range.OffsetRange) Test(org.junit.Test)

Example 52 with OffsetRange

use of org.apache.beam.sdk.io.range.OffsetRange in project beam by apache.

the class OffsetRangeTrackerTest method testTrySplitAfterCheckpoint.

@Test
public void testTrySplitAfterCheckpoint() throws Exception {
    OffsetRangeTracker tracker = new OffsetRangeTracker(new OffsetRange(100, 200));
    tracker.tryClaim(105L);
    tracker.trySplit(0);
    assertNull(tracker.trySplit(0.1));
}
Also used : OffsetRange(org.apache.beam.sdk.io.range.OffsetRange) Test(org.junit.Test)

Example 53 with OffsetRange

use of org.apache.beam.sdk.io.range.OffsetRange in project beam by apache.

the class OffsetRangeTrackerTest method testTrySplitAtEmptyRange.

@Test
public void testTrySplitAtEmptyRange() throws Exception {
    OffsetRangeTracker tracker = new OffsetRangeTracker(new OffsetRange(100, 100));
    assertNull(tracker.trySplit(0));
    assertNull(tracker.trySplit(0.1));
    assertNull(tracker.trySplit(1));
}
Also used : OffsetRange(org.apache.beam.sdk.io.range.OffsetRange) Test(org.junit.Test)

Example 54 with OffsetRange

use of org.apache.beam.sdk.io.range.OffsetRange in project beam by apache.

the class OffsetRangeTrackerTest method testTryClaim.

@Test
public void testTryClaim() throws Exception {
    OffsetRange range = new OffsetRange(100, 200);
    OffsetRangeTracker tracker = new OffsetRangeTracker(range);
    assertEquals(range, tracker.currentRestriction());
    assertTrue(tracker.tryClaim(100L));
    assertTrue(tracker.tryClaim(150L));
    assertTrue(tracker.tryClaim(199L));
    assertFalse(tracker.tryClaim(200L));
}
Also used : OffsetRange(org.apache.beam.sdk.io.range.OffsetRange) Test(org.junit.Test)

Example 55 with OffsetRange

use of org.apache.beam.sdk.io.range.OffsetRange in project beam by apache.

the class OffsetRangeTrackerTest method testCheckpointRegular.

@Test
public void testCheckpointRegular() throws Exception {
    OffsetRangeTracker tracker = new OffsetRangeTracker(new OffsetRange(100, 200));
    assertTrue(tracker.tryClaim(105L));
    assertTrue(tracker.tryClaim(110L));
    OffsetRange checkpoint = tracker.trySplit(0).getResidual();
    assertEquals(new OffsetRange(100, 111), tracker.currentRestriction());
    assertEquals(new OffsetRange(111, 200), checkpoint);
    tracker.checkDone();
}
Also used : OffsetRange(org.apache.beam.sdk.io.range.OffsetRange) Test(org.junit.Test)

Aggregations

OffsetRange (org.apache.beam.sdk.io.range.OffsetRange)63 Test (org.junit.Test)53 Instant (org.joda.time.Instant)8 ArrayList (java.util.ArrayList)5 OffsetRangeTracker (org.apache.beam.sdk.transforms.splittabledofn.OffsetRangeTracker)5 Progress (org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker.Progress)5 ProcessContinuation (org.apache.beam.sdk.transforms.DoFn.ProcessContinuation)4 PartitionMetadata (org.apache.beam.sdk.io.gcp.spanner.changestreams.model.PartitionMetadata)3 DoFn (org.apache.beam.sdk.transforms.DoFn)3 BigDecimal (java.math.BigDecimal)2 RestrictionTracker (org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker)2 Offset (com.google.cloud.pubsublite.Offset)1 SuppressFBWarnings (edu.umd.cs.findbugs.annotations.SuppressFBWarnings)1 Scope (io.opencensus.common.Scope)1 AttributeValue (io.opencensus.trace.AttributeValue)1 Tracer (io.opencensus.trace.Tracer)1 Tracing (io.opencensus.trace.Tracing)1 Serializable (java.io.Serializable)1 Map (java.util.Map)1 Optional (java.util.Optional)1