Search in sources :

Example 1 with Table

use of com.google.bigtable.admin.v2.Table in project beam by apache.

the class BigtableServiceImpl method tableExists.

@Override
public boolean tableExists(String tableId) throws IOException {
    try (BigtableSession session = new BigtableSession(options)) {
        GetTableRequest getTable = GetTableRequest.newBuilder().setName(options.getInstanceName().toTableNameStr(tableId)).build();
        session.getTableAdminClient().getTable(getTable);
        return true;
    } catch (StatusRuntimeException e) {
        if (e.getStatus().getCode() == Code.NOT_FOUND) {
            return false;
        }
        String message = String.format("Error checking whether table %s (BigtableOptions %s) exists", tableId, options);
        LOG.error(message, e);
        throw new IOException(message, e);
    }
}
Also used : GetTableRequest(com.google.bigtable.admin.v2.GetTableRequest) StatusRuntimeException(io.grpc.StatusRuntimeException) BigtableSession(com.google.cloud.bigtable.grpc.BigtableSession) ByteString(com.google.protobuf.ByteString) IOException(java.io.IOException)

Example 2 with Table

use of com.google.bigtable.admin.v2.Table in project beam by apache.

the class BigtableWriteIT method testE2EBigtableWrite.

@Test
public void testE2EBigtableWrite() throws Exception {
    final String tableName = bigtableOptions.getInstanceName().toTableNameStr(tableId);
    final String instanceName = bigtableOptions.getInstanceName().toString();
    final int numRows = 1000;
    final List<KV<ByteString, ByteString>> testData = generateTableData(numRows);
    createEmptyTable(instanceName, tableId);
    Pipeline p = Pipeline.create(options);
    p.apply(GenerateSequence.from(0).to(numRows)).apply(ParDo.of(new DoFn<Long, KV<ByteString, Iterable<Mutation>>>() {

        @ProcessElement
        public void processElement(ProcessContext c) {
            int index = c.element().intValue();
            Iterable<Mutation> mutations = ImmutableList.of(Mutation.newBuilder().setSetCell(Mutation.SetCell.newBuilder().setValue(testData.get(index).getValue()).setFamilyName(COLUMN_FAMILY_NAME)).build());
            c.output(KV.of(testData.get(index).getKey(), mutations));
        }
    })).apply(BigtableIO.write().withBigtableOptions(bigtableOptions).withTableId(tableId));
    p.run();
    // Test number of column families and column family name equality
    Table table = getTable(tableName);
    assertThat(table.getColumnFamiliesMap().keySet(), Matchers.hasSize(1));
    assertThat(table.getColumnFamiliesMap(), Matchers.hasKey(COLUMN_FAMILY_NAME));
    // Test table data equality
    List<KV<ByteString, ByteString>> tableData = getTableData(tableName);
    assertThat(tableData, Matchers.containsInAnyOrder(testData.toArray()));
}
Also used : Table(com.google.bigtable.admin.v2.Table) ByteString(com.google.protobuf.ByteString) KV(org.apache.beam.sdk.values.KV) Mutation(com.google.bigtable.v2.Mutation) TestPipeline(org.apache.beam.sdk.testing.TestPipeline) Pipeline(org.apache.beam.sdk.Pipeline) Test(org.junit.Test)

Aggregations

ByteString (com.google.protobuf.ByteString)2 GetTableRequest (com.google.bigtable.admin.v2.GetTableRequest)1 Table (com.google.bigtable.admin.v2.Table)1 Mutation (com.google.bigtable.v2.Mutation)1 BigtableSession (com.google.cloud.bigtable.grpc.BigtableSession)1 StatusRuntimeException (io.grpc.StatusRuntimeException)1 IOException (java.io.IOException)1 Pipeline (org.apache.beam.sdk.Pipeline)1 TestPipeline (org.apache.beam.sdk.testing.TestPipeline)1 KV (org.apache.beam.sdk.values.KV)1 Test (org.junit.Test)1