Search in sources :

Example 11 with Ddl

use of com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl in project DataflowTemplates by GoogleCloudPlatform.

the class ChangeEventConvertorTest method cannotConvertChangeEventWithInvalidFloat64ToShadowMutation.

@Test(expected = ChangeEventConvertorException.class)
public void cannotConvertChangeEventWithInvalidFloat64ToShadowMutation() throws Exception {
    Ddl ddl = getTestDdl();
    JSONObject changeEvent = getTestChangeEvent("Users2");
    changeEvent.put("float64_field", "asdfas");
    JsonNode ce = parseChangeEvent(changeEvent.toString());
    Mutation mutation = ChangeEventConvertor.changeEventToShadowTableMutationBuilder(ddl, ce, "shadow_").build();
// Expect an Exception to be thrown with Invalid Float64
}
Also used : JSONObject(org.json.JSONObject) JsonNode(com.fasterxml.jackson.databind.JsonNode) Mutation(com.google.cloud.spanner.Mutation) Ddl(com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl) Test(org.junit.Test)

Example 12 with Ddl

use of com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl in project DataflowTemplates by GoogleCloudPlatform.

the class ChangeEventConvertorTest method cannotConvertChangeEventWithInvalidTimestampToShadowMutation.

@Test(expected = ChangeEventConvertorException.class)
public void cannotConvertChangeEventWithInvalidTimestampToShadowMutation() throws Exception {
    Ddl ddl = getTestDdl();
    JSONObject changeEvent = getTestChangeEvent("Users2");
    changeEvent.put("timestamp_field", "2020-12-asdf");
    JsonNode ce = parseChangeEvent(changeEvent.toString());
    Mutation mutation = ChangeEventConvertor.changeEventToShadowTableMutationBuilder(ddl, ce, "shadow_").build();
// Expect an Exception to be thrown with Invalid timestamp
}
Also used : JSONObject(org.json.JSONObject) JsonNode(com.fasterxml.jackson.databind.JsonNode) Mutation(com.google.cloud.spanner.Mutation) Ddl(com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl) Test(org.junit.Test)

Example 13 with Ddl

use of com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl in project DataflowTemplates by GoogleCloudPlatform.

the class ChangeEventConvertorTest method cannotConvertChangeEventWithInvalidInt64ToMutation.

@Test(expected = ChangeEventConvertorException.class)
public void cannotConvertChangeEventWithInvalidInt64ToMutation() throws Exception {
    Ddl ddl = getTestDdl();
    JSONObject changeEvent = getTestChangeEvent("Users");
    changeEvent.put("int64_field", "asdfas");
    JsonNode ce = parseChangeEvent(changeEvent.toString());
    Mutation mutation = ChangeEventConvertor.changeEventToMutation(ddl, ce);
}
Also used : JSONObject(org.json.JSONObject) JsonNode(com.fasterxml.jackson.databind.JsonNode) Mutation(com.google.cloud.spanner.Mutation) Ddl(com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl) Test(org.junit.Test)

Example 14 with Ddl

use of com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl in project DataflowTemplates by GoogleCloudPlatform.

the class ChangeEventConvertorTest method cannotConvertValidChangeEventWithNullKeyColumnsToMutation.

@Test(expected = ChangeEventConvertorException.class)
public void cannotConvertValidChangeEventWithNullKeyColumnsToMutation() throws Exception {
    Ddl ddl = getTestDdl();
    JSONObject changeEvent = new JSONObject();
    changeEvent.put("first_name", "A");
    changeEvent.put("last_name", JSONObject.NULL);
    changeEvent.put(DatastreamConstants.EVENT_TABLE_NAME_KEY, "Users");
    JsonNode ce = parseChangeEvent(changeEvent.toString());
    Mutation mutation = ChangeEventConvertor.changeEventToMutation(ddl, ce);
}
Also used : JSONObject(org.json.JSONObject) JsonNode(com.fasterxml.jackson.databind.JsonNode) Mutation(com.google.cloud.spanner.Mutation) Ddl(com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl) Test(org.junit.Test)

Example 15 with Ddl

use of com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl in project DataflowTemplates by GoogleCloudPlatform.

the class ChangeEventConvertorTest method canConvertValidChangeEventToMutation.

@Test
public void canConvertValidChangeEventToMutation() throws Exception {
    Ddl ddl = getTestDdl();
    JSONObject changeEvent = getTestChangeEvent("Users");
    JsonNode ce = parseChangeEvent(changeEvent.toString());
    Mutation mutation = ChangeEventConvertor.changeEventToMutation(ddl, ce);
    Map<String, Value> actual = mutation.asMap();
    Map<String, Value> expected = getExpectedMapForTestChangeEvent();
    assertThat(actual, is(expected));
    assertEquals(mutation.getTable(), "Users");
    assertEquals(mutation.getOperation(), Mutation.Op.INSERT_OR_UPDATE);
}
Also used : JSONObject(org.json.JSONObject) Value(com.google.cloud.spanner.Value) JsonNode(com.fasterxml.jackson.databind.JsonNode) Mutation(com.google.cloud.spanner.Mutation) Ddl(com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl) Test(org.junit.Test)

Aggregations

Ddl (com.google.cloud.teleport.v2.templates.spanner.ddl.Ddl)40 Test (org.junit.Test)31 JSONObject (org.json.JSONObject)26 JsonNode (com.fasterxml.jackson.databind.JsonNode)23 Mutation (com.google.cloud.spanner.Mutation)22 Value (com.google.cloud.spanner.Value)12 Table (com.google.cloud.teleport.v2.templates.spanner.ddl.Table)9 Key (com.google.cloud.spanner.Key)6 SpannerConfig (org.apache.beam.sdk.io.gcp.spanner.SpannerConfig)6 Set (java.util.Set)5 Collectors (java.util.stream.Collectors)5 Column (com.google.cloud.teleport.v2.templates.spanner.ddl.Column)4 IndexColumn (com.google.cloud.teleport.v2.templates.spanner.ddl.IndexColumn)4 PipelineResult (org.apache.beam.sdk.PipelineResult)4 Type (com.google.cloud.spanner.Type)3 IntegrationTest (com.google.cloud.teleport.v2.spanner.IntegrationTest)3 List (java.util.List)3 ObjectNode (com.fasterxml.jackson.databind.node.ObjectNode)2 DatabaseClient (com.google.cloud.spanner.DatabaseClient)2 ReadOnlyTransaction (com.google.cloud.spanner.ReadOnlyTransaction)2