Search in sources :

Example 1 with Mutation

use of com.google.cloud.spanner.Mutation in project beam by apache.

the class SpannerIOTest method singleMutationPipeline.

@Test
@Category(NeedsRunner.class)
public void singleMutationPipeline() throws Exception {
    Mutation mutation = Mutation.newInsertOrUpdateBuilder("test").set("one").to(2).build();
    PCollection<Mutation> mutations = pipeline.apply(Create.of(mutation));
    mutations.apply(SpannerIO.write().withProjectId("test-project").withInstanceId("test-instance").withDatabaseId("test-database").withServiceFactory(serviceFactory));
    pipeline.run();
    verify(serviceFactory.mockSpanner()).getDatabaseClient(DatabaseId.of("test-project", "test-instance", "test-database"));
    verify(serviceFactory.mockDatabaseClient(), times(1)).writeAtLeastOnce(argThat(new IterableOfSize(1)));
}
Also used : Mutation(com.google.cloud.spanner.Mutation) Category(org.junit.experimental.categories.Category) Test(org.junit.Test)

Example 2 with Mutation

use of com.google.cloud.spanner.Mutation in project beam by apache.

the class SpannerIOTest method batchingGroups.

@Test
public void batchingGroups() throws Exception {
    Mutation one = Mutation.newInsertOrUpdateBuilder("test").set("one").to(1).build();
    Mutation two = Mutation.newInsertOrUpdateBuilder("test").set("two").to(2).build();
    Mutation three = Mutation.newInsertOrUpdateBuilder("test").set("three").to(3).build();
    // Have a room to accumulate one more item.
    long batchSize = MutationSizeEstimator.sizeOf(one) + 1;
    SpannerIO.Write write = SpannerIO.write().withProjectId("test-project").withInstanceId("test-instance").withDatabaseId("test-database").withBatchSizeBytes(batchSize).withServiceFactory(serviceFactory);
    SpannerIO.SpannerWriteFn writerFn = new SpannerIO.SpannerWriteFn(write);
    DoFnTester<Mutation, Void> fnTester = DoFnTester.of(writerFn);
    fnTester.processBundle(Arrays.asList(one, two, three));
    verify(serviceFactory.mockSpanner()).getDatabaseClient(DatabaseId.of("test-project", "test-instance", "test-database"));
    verify(serviceFactory.mockDatabaseClient(), times(1)).writeAtLeastOnce(argThat(new IterableOfSize(2)));
    verify(serviceFactory.mockDatabaseClient(), times(1)).writeAtLeastOnce(argThat(new IterableOfSize(1)));
}
Also used : Mutation(com.google.cloud.spanner.Mutation) Test(org.junit.Test)

Example 3 with Mutation

use of com.google.cloud.spanner.Mutation in project beam by apache.

the class SpannerIOTest method noBatching.

@Test
public void noBatching() throws Exception {
    Mutation one = Mutation.newInsertOrUpdateBuilder("test").set("one").to(1).build();
    Mutation two = Mutation.newInsertOrUpdateBuilder("test").set("two").to(2).build();
    SpannerIO.Write write = SpannerIO.write().withProjectId("test-project").withInstanceId("test-instance").withDatabaseId("test-database").withBatchSizeBytes(// turn off batching.
    0).withServiceFactory(serviceFactory);
    SpannerIO.SpannerWriteFn writerFn = new SpannerIO.SpannerWriteFn(write);
    DoFnTester<Mutation, Void> fnTester = DoFnTester.of(writerFn);
    fnTester.processBundle(Arrays.asList(one, two));
    verify(serviceFactory.mockSpanner()).getDatabaseClient(DatabaseId.of("test-project", "test-instance", "test-database"));
    verify(serviceFactory.mockDatabaseClient(), times(2)).writeAtLeastOnce(argThat(new IterableOfSize(1)));
}
Also used : Mutation(com.google.cloud.spanner.Mutation) Test(org.junit.Test)

Example 4 with Mutation

use of com.google.cloud.spanner.Mutation in project google-cloud-java by GoogleCloudPlatform.

the class ITLargeReadTest method setUpDatabase.

@BeforeClass
public static void setUpDatabase() {
    db = env.getTestHelper().createTestDatabase("CREATE TABLE TestTable (" + "  Key           INT64 NOT NULL," + "  Data          BYTES(MAX)," + "  Fingerprint   INT64," + "  Size          INT64," + ") PRIMARY KEY (Key)");
    hasher = Hashing.goodFastHash(64);
    client = env.getTestHelper().getDatabaseClient(db);
    List<Mutation> mutations = new ArrayList<>();
    Random rnd = new Random();
    int totalSize = 0;
    int i = 0;
    for (int rowSize : rowSizes()) {
        numRows++;
        byte[] data = new byte[rowSize];
        rnd.nextBytes(data);
        mutations.add(Mutation.newInsertOrUpdateBuilder(TABLE_NAME).set("Key").to(i).set("Data").to(ByteArray.copyFrom(data)).set("Fingerprint").to(hasher.hashBytes(data).asLong()).set("Size").to(rowSize).build());
        totalSize += rowSize;
        i++;
        if (totalSize >= WRITE_BATCH_SIZE) {
            client.write(mutations);
            mutations.clear();
            totalSize = 0;
        }
    }
    client.write(mutations);
}
Also used : Random(java.util.Random) ArrayList(java.util.ArrayList) Mutation(com.google.cloud.spanner.Mutation) BeforeClass(org.junit.BeforeClass)

Example 5 with Mutation

use of com.google.cloud.spanner.Mutation in project spring-cloud-gcp by spring-cloud.

the class SpannerMutationFactoryImplTests method updateTest.

@Test
public void updateTest() {
    Mutation mutation = this.spannerMutationFactory.update(new TestEntity(), null);
    assertEquals("custom_test_table", mutation.getTable());
    assertEquals(Op.UPDATE, mutation.getOperation());
}
Also used : Mutation(com.google.cloud.spanner.Mutation) Test(org.junit.Test)

Aggregations

Mutation (com.google.cloud.spanner.Mutation)70 Test (org.junit.Test)53 ArrayList (java.util.ArrayList)11 CommitResponse (com.google.cloud.spanner.CommitResponse)4 KeySet (com.google.cloud.spanner.KeySet)4 ReadQueryUpdateTransactionOption (com.google.cloud.spanner.Options.ReadQueryUpdateTransactionOption)4 Timestamp (com.google.cloud.Timestamp)3 DatabaseClient (com.google.cloud.spanner.DatabaseClient)3 ResultSet (com.google.cloud.spanner.ResultSet)3 TransactionContext (com.google.cloud.spanner.TransactionContext)3 BatchableMutationFilterFn (org.apache.beam.sdk.io.gcp.spanner.SpannerIO.BatchableMutationFilterFn)3 Sleeper (org.apache.beam.sdk.util.Sleeper)3 Database (com.google.cloud.spanner.Database)2 DatabaseAdminClient (com.google.cloud.spanner.DatabaseAdminClient)2 Key (com.google.cloud.spanner.Key)2 WriteBuilder (com.google.cloud.spanner.Mutation.WriteBuilder)2 SpannerException (com.google.cloud.spanner.SpannerException)2 TransactionRunner (com.google.cloud.spanner.TransactionRunner)2 CreateDatabaseMetadata (com.google.spanner.admin.database.v1.CreateDatabaseMetadata)2 BigDecimal (java.math.BigDecimal)2