Search in sources :

Example 6 with ChangeEventContext

use of com.google.cloud.teleport.v2.templates.datastream.ChangeEventContext in project DataflowTemplates by GoogleCloudPlatform.

the class OracleChangeEventContextTest method canGenerateShadowTableMutationForBackfillEvent.

@Test
public void canGenerateShadowTableMutationForBackfillEvent() throws Exception {
    // Test Ddl
    Ddl ddl = ChangeEventConvertorTest.getTestDdl();
    // Test Change Event
    JSONObject changeEvent = ChangeEventConvertorTest.getTestChangeEvent("Users2");
    changeEvent.put(DatastreamConstants.ORACLE_TIMESTAMP_KEY, eventTimestamp);
    changeEvent.put(DatastreamConstants.ORACLE_SCN_KEY, JSONObject.NULL);
    changeEvent.put(DatastreamConstants.EVENT_SOURCE_TYPE_KEY, DatastreamConstants.ORACLE_SOURCE_TYPE);
    ChangeEventContext changeEventContext = ChangeEventContextFactory.createChangeEventContext(getJsonNode(changeEvent.toString()), ddl, "shadow_", DatastreamConstants.ORACLE_SOURCE_TYPE);
    Mutation shadowMutation = changeEventContext.getShadowTableMutation();
    Map<String, Value> actual = shadowMutation.asMap();
    // Expected result
    Map<String, Value> expected = ChangeEventConvertorTest.getExpectedMapForTestChangeEvent();
    expected.put(DatastreamConstants.ORACLE_TIMESTAMP_SHADOW_INFO.getLeft(), Value.int64(eventTimestamp));
    expected.put(DatastreamConstants.ORACLE_SCN_SHADOW_INFO.getLeft(), Value.int64(-1));
    // Verify if OracleChangeEventContext was actually created.
    assertThat(changeEventContext, instanceOf(OracleChangeEventContext.class));
    // Verify shadow mutation
    assertThat(actual, is(expected));
    assertEquals(shadowMutation.getTable(), "shadow_Users2");
    assertEquals(shadowMutation.getOperation(), Mutation.Op.INSERT_OR_UPDATE);
}
Also used : JSONObject(org.json.JSONObject) Value(com.google.cloud.spanner.Value) Mutation(com.google.cloud.spanner.Mutation) Ddl(com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl) Test(org.junit.Test)

Example 7 with ChangeEventContext

use of com.google.cloud.teleport.v2.templates.datastream.ChangeEventContext in project DataflowTemplates by GoogleCloudPlatform.

the class OracleChangeEventContextTest method canGenerateShadowTableMutationForBackfillEventWithMissingKeys.

@Test
public void canGenerateShadowTableMutationForBackfillEventWithMissingKeys() throws Exception {
    // Test Ddl
    Ddl ddl = ChangeEventConvertorTest.getTestDdl();
    // Test Change Event
    JSONObject changeEvent = ChangeEventConvertorTest.getTestChangeEvent("Users2");
    changeEvent.put(DatastreamConstants.ORACLE_TIMESTAMP_KEY, eventTimestamp);
    changeEvent.put(DatastreamConstants.EVENT_SOURCE_TYPE_KEY, DatastreamConstants.ORACLE_SOURCE_TYPE);
    ChangeEventContext changeEventContext = ChangeEventContextFactory.createChangeEventContext(getJsonNode(changeEvent.toString()), ddl, "shadow_", DatastreamConstants.ORACLE_SOURCE_TYPE);
    Mutation shadowMutation = changeEventContext.getShadowTableMutation();
    Map<String, Value> actual = shadowMutation.asMap();
    // Expected result
    Map<String, Value> expected = ChangeEventConvertorTest.getExpectedMapForTestChangeEvent();
    expected.put(DatastreamConstants.ORACLE_TIMESTAMP_SHADOW_INFO.getLeft(), Value.int64(eventTimestamp));
    expected.put(DatastreamConstants.ORACLE_SCN_SHADOW_INFO.getLeft(), Value.int64(-1));
    // Verify if OracleChangeEventContext was actually created.
    assertThat(changeEventContext, instanceOf(OracleChangeEventContext.class));
    // Verify shadow mutation
    assertThat(actual, is(expected));
    assertEquals(shadowMutation.getTable(), "shadow_Users2");
    assertEquals(shadowMutation.getOperation(), Mutation.Op.INSERT_OR_UPDATE);
}
Also used : JSONObject(org.json.JSONObject) Value(com.google.cloud.spanner.Value) Mutation(com.google.cloud.spanner.Mutation) Ddl(com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl) Test(org.junit.Test)

Aggregations

Ddl (com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl)7 Mutation (com.google.cloud.spanner.Mutation)6 Value (com.google.cloud.spanner.Value)6 JSONObject (org.json.JSONObject)6 Test (org.junit.Test)6 DeserializationFeature (com.fasterxml.jackson.databind.DeserializationFeature)1 JsonNode (com.fasterxml.jackson.databind.JsonNode)1 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 Timestamp (com.google.cloud.Timestamp)1 SpannerException (com.google.cloud.spanner.SpannerException)1 TransactionCallable (com.google.cloud.spanner.TransactionRunner.TransactionCallable)1 ChangeEventContext (com.google.cloud.teleport.v2.templates.datastream.ChangeEventContext)1 ChangeEventContextFactory (com.google.cloud.teleport.v2.templates.datastream.ChangeEventContextFactory)1 ChangeEventConvertorException (com.google.cloud.teleport.v2.templates.datastream.ChangeEventConvertorException)1 ChangeEventSequence (com.google.cloud.teleport.v2.templates.datastream.ChangeEventSequence)1 ChangeEventSequenceFactory (com.google.cloud.teleport.v2.templates.datastream.ChangeEventSequenceFactory)1 InvalidChangeEventException (com.google.cloud.teleport.v2.templates.datastream.InvalidChangeEventException)1 FailsafeElement (com.google.cloud.teleport.v2.values.FailsafeElement)1 Preconditions (com.google.common.base.Preconditions)1 PrintWriter (java.io.PrintWriter)1