Search in sources :

Example 11 with ChildPartitionsRecord

use of org.apache.beam.sdk.io.gcp.spanner.changestreams.model.ChildPartitionsRecord in project beam by apache.

the class ChildPartitionsRecordActionTest method testRestrictionClaimedAndIsMergeCaseAndChildNotExists.

@Test
public void testRestrictionClaimedAndIsMergeCaseAndChildNotExists() {
    final String partitionToken = "partitionToken";
    final String anotherPartitionToken = "anotherPartitionToken";
    final String childPartitionToken = "childPartition1";
    final HashSet<String> parentTokens = Sets.newHashSet(partitionToken, anotherPartitionToken);
    final long heartbeat = 30L;
    final Timestamp startTimestamp = Timestamp.ofTimeMicroseconds(10L);
    final Timestamp endTimestamp = Timestamp.ofTimeMicroseconds(20L);
    final PartitionMetadata partition = mock(PartitionMetadata.class);
    final ChildPartitionsRecord record = new ChildPartitionsRecord(startTimestamp, "recordSequence", Collections.singletonList(new ChildPartition(childPartitionToken, parentTokens)), null);
    when(partition.getEndTimestamp()).thenReturn(endTimestamp);
    when(partition.getHeartbeatMillis()).thenReturn(heartbeat);
    when(partition.getPartitionToken()).thenReturn(partitionToken);
    when(tracker.tryClaim(10L)).thenReturn(true);
    when(transaction.getPartition(childPartitionToken)).thenReturn(null);
    final Optional<ProcessContinuation> maybeContinuation = action.run(partition, record, tracker, watermarkEstimator);
    assertEquals(Optional.empty(), maybeContinuation);
    verify(watermarkEstimator).setWatermark(new Instant(startTimestamp.toSqlTimestamp().getTime()));
    verify(transaction).insert(PartitionMetadata.newBuilder().setPartitionToken(childPartitionToken).setParentTokens(parentTokens).setStartTimestamp(startTimestamp).setEndTimestamp(endTimestamp).setHeartbeatMillis(heartbeat).setState(CREATED).setWatermark(startTimestamp).build());
}
Also used : ChildPartition(org.apache.beam.sdk.io.gcp.spanner.changestreams.model.ChildPartition) Instant(org.joda.time.Instant) PartitionMetadata(org.apache.beam.sdk.io.gcp.spanner.changestreams.model.PartitionMetadata) ChildPartitionsRecord(org.apache.beam.sdk.io.gcp.spanner.changestreams.model.ChildPartitionsRecord) Timestamp(com.google.cloud.Timestamp) ProcessContinuation(org.apache.beam.sdk.transforms.DoFn.ProcessContinuation) Test(org.junit.Test)

Aggregations

ChildPartitionsRecord (org.apache.beam.sdk.io.gcp.spanner.changestreams.model.ChildPartitionsRecord)11 ProcessContinuation (org.apache.beam.sdk.transforms.DoFn.ProcessContinuation)9 Test (org.junit.Test)9 ChildPartition (org.apache.beam.sdk.io.gcp.spanner.changestreams.model.ChildPartition)8 PartitionMetadata (org.apache.beam.sdk.io.gcp.spanner.changestreams.model.PartitionMetadata)8 Timestamp (com.google.cloud.Timestamp)7 Struct (com.google.cloud.spanner.Struct)6 Instant (org.joda.time.Instant)5 ChangeStreamResultSet (org.apache.beam.sdk.io.gcp.spanner.changestreams.dao.ChangeStreamResultSet)3 Scope (io.opencensus.common.Scope)2 ChangeStreamResultSetMetadata (org.apache.beam.sdk.io.gcp.spanner.changestreams.dao.ChangeStreamResultSetMetadata)2 VisibleForTesting (org.apache.beam.vendor.guava.v26_0_jre.com.google.common.annotations.VisibleForTesting)2 SpannerException (com.google.cloud.spanner.SpannerException)1 AttributeValue (io.opencensus.trace.AttributeValue)1 Tracer (io.opencensus.trace.Tracer)1 Tracing (io.opencensus.trace.Tracing)1 Optional (java.util.Optional)1 Nullable (javax.annotation.Nullable)1 ChangeStreamMetrics (org.apache.beam.sdk.io.gcp.spanner.changestreams.ChangeStreamMetrics)1 PARTITION_ID_ATTRIBUTE_LABEL (org.apache.beam.sdk.io.gcp.spanner.changestreams.ChangeStreamMetrics.PARTITION_ID_ATTRIBUTE_LABEL)1