Search in sources :

Example 36 with ReadRowsRequest

use of com.google.cloud.bigquery.storage.v1beta2.ReadRowsRequest in project java-bigtable by googleapis.

the class ReadRowsUserCallableTest method testRequestConverted.

@Test
public void testRequestConverted() {
    ServerStreamingStashCallable<ReadRowsRequest, Row> innerCallable = new ServerStreamingStashCallable<>();
    ReadRowsUserCallable<Row> callable = new ReadRowsUserCallable<>(innerCallable, REQUEST_CONTEXT);
    Query query = Query.create("fake-table");
    callable.call(query);
    Truth.assertThat(innerCallable.getActualRequest()).isEqualTo(query.toProto(REQUEST_CONTEXT));
}
Also used : Query(com.google.cloud.bigtable.data.v2.models.Query) ServerStreamingStashCallable(com.google.cloud.bigtable.gaxx.testing.FakeStreamingApi.ServerStreamingStashCallable) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest) Row(com.google.cloud.bigtable.data.v2.models.Row) Test(org.junit.Test)

Example 37 with ReadRowsRequest

use of com.google.cloud.bigquery.storage.v1beta2.ReadRowsRequest in project java-bigtable by googleapis.

the class RowMergingCallableTest method invalidMarkerInCell.

@Test
public void invalidMarkerInCell() {
    FakeStreamingApi.ServerStreamingStashCallable<ReadRowsRequest, ReadRowsResponse> inner = new ServerStreamingStashCallable<>(Lists.newArrayList(ReadRowsResponse.newBuilder().addChunks(CellChunk.newBuilder().setRowKey(ByteString.copyFromUtf8("key1")).setFamilyName(StringValue.newBuilder().setValue("family")).setQualifier(BytesValue.newBuilder().setValue(ByteString.EMPTY)).setTimestampMicros(1_000).setValue(ByteString.copyFromUtf8("a")).setValueSize(2)).build(), // send a scan marker
    ReadRowsResponse.newBuilder().setLastScannedRowKey(ByteString.copyFromUtf8("key1")).build(), // finish the cell & row
    ReadRowsResponse.newBuilder().addChunks(CellChunk.newBuilder().setValue(ByteString.copyFromUtf8("b")).setValueSize(0).setCommitRow(true)).build()));
    RowMergingCallable<Row> rowMergingCallable = new RowMergingCallable<>(inner, new DefaultRowAdapter());
    Throwable actualError = null;
    try {
        rowMergingCallable.all().call(ReadRowsRequest.getDefaultInstance());
    } catch (Throwable t) {
        actualError = t;
    }
    Truth.assertThat(actualError).isInstanceOf(IllegalStateException.class);
}
Also used : FakeStreamingApi(com.google.cloud.bigtable.gaxx.testing.FakeStreamingApi) ReadRowsResponse(com.google.bigtable.v2.ReadRowsResponse) ServerStreamingStashCallable(com.google.cloud.bigtable.gaxx.testing.FakeStreamingApi.ServerStreamingStashCallable) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest) Row(com.google.cloud.bigtable.data.v2.models.Row) DefaultRowAdapter(com.google.cloud.bigtable.data.v2.models.DefaultRowAdapter) Test(org.junit.Test)

Example 38 with ReadRowsRequest

use of com.google.cloud.bigquery.storage.v1beta2.ReadRowsRequest in project java-bigtable by googleapis.

the class RowMergingCallableTest method scanMarker.

@Test
public void scanMarker() {
    FakeStreamingApi.ServerStreamingStashCallable<ReadRowsRequest, ReadRowsResponse> inner = new ServerStreamingStashCallable<>(Lists.newArrayList(// send a scan marker
    ReadRowsResponse.newBuilder().setLastScannedRowKey(ByteString.copyFromUtf8("key1")).build()));
    RowMergingCallable<Row> rowMergingCallable = new RowMergingCallable<>(inner, new DefaultRowAdapter());
    List<Row> results = rowMergingCallable.all().call(ReadRowsRequest.getDefaultInstance());
    Truth.assertThat(results).containsExactly(Row.create(ByteString.copyFromUtf8("key1"), Lists.<RowCell>newArrayList()));
}
Also used : FakeStreamingApi(com.google.cloud.bigtable.gaxx.testing.FakeStreamingApi) ReadRowsResponse(com.google.bigtable.v2.ReadRowsResponse) ServerStreamingStashCallable(com.google.cloud.bigtable.gaxx.testing.FakeStreamingApi.ServerStreamingStashCallable) RowCell(com.google.cloud.bigtable.data.v2.models.RowCell) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest) Row(com.google.cloud.bigtable.data.v2.models.Row) DefaultRowAdapter(com.google.cloud.bigtable.data.v2.models.DefaultRowAdapter) Test(org.junit.Test)

Example 39 with ReadRowsRequest

use of com.google.cloud.bigquery.storage.v1beta2.ReadRowsRequest in project java-bigtable by googleapis.

the class BigtableDataClientFactoryTest method testCreateWithRefreshingChannel.

@Test
public void testCreateWithRefreshingChannel() throws Exception {
    String[] tableIds = { "fake-table1", "fake-table2" };
    int poolSize = 3;
    BigtableDataSettings.Builder builder = BigtableDataSettings.newBuilderForEmulator(server.getPort()).setProjectId(DEFAULT_PROJECT_ID).setInstanceId(DEFAULT_INSTANCE_ID).setAppProfileId(DEFAULT_APP_PROFILE_ID).setPrimingTableIds(tableIds).setRefreshingChannel(true);
    builder.stubSettings().setCredentialsProvider(credentialsProvider).setStreamWatchdogProvider(watchdogProvider).setExecutorProvider(executorProvider);
    InstantiatingGrpcChannelProvider channelProvider = (InstantiatingGrpcChannelProvider) builder.stubSettings().getTransportChannelProvider();
    InstantiatingGrpcChannelProvider.Builder channelProviderBuilder = channelProvider.toBuilder();
    channelProviderBuilder.setPoolSize(poolSize);
    builder.stubSettings().setTransportChannelProvider(channelProviderBuilder.build());
    BigtableDataClientFactory factory = BigtableDataClientFactory.create(builder.build());
    factory.createDefault();
    factory.createForAppProfile("other-appprofile");
    factory.createForInstance("other-project", "other-instance");
    // Make sure that only 1 instance is created for all clients
    Mockito.verify(credentialsProvider, Mockito.times(1)).getCredentials();
    Mockito.verify(executorProvider, Mockito.times(1)).getExecutor();
    Mockito.verify(watchdogProvider, Mockito.times(1)).getWatchdog();
    // Make sure that the clients are sharing the same ChannelPool
    assertThat(setUpAttributes).hasSize(poolSize);
    // Make sure that prime requests were sent only once per table per connection
    assertThat(service.readRowsRequests).hasSize(poolSize * tableIds.length);
    List<ReadRowsRequest> expectedRequests = new LinkedList<>();
    for (String tableId : tableIds) {
        for (int i = 0; i < poolSize; i++) {
            expectedRequests.add(ReadRowsRequest.newBuilder().setTableName(String.format("projects/%s/instances/%s/tables/%s", DEFAULT_PROJECT_ID, DEFAULT_INSTANCE_ID, tableId)).setAppProfileId(DEFAULT_APP_PROFILE_ID).setRows(RowSet.newBuilder().addRowKeys(ByteString.copyFromUtf8("nonexistent-priming-row"))).setFilter(RowFilter.newBuilder().setBlockAllFilter(true).build()).setRowsLimit(1).build());
        }
    }
    assertThat(service.readRowsRequests).containsExactly(expectedRequests.toArray());
    // Wait for all the connections to close asynchronously
    factory.close();
    long sleepTimeMs = 1000;
    Thread.sleep(sleepTimeMs);
    // Verify that all the channels are closed
    assertThat(terminateAttributes).hasSize(poolSize);
}
Also used : InstantiatingGrpcChannelProvider(com.google.api.gax.grpc.InstantiatingGrpcChannelProvider) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest) ByteString(com.google.protobuf.ByteString) LinkedList(java.util.LinkedList) Test(org.junit.Test)

Example 40 with ReadRowsRequest

use of com.google.cloud.bigquery.storage.v1beta2.ReadRowsRequest in project java-bigtable-hbase by googleapis.

the class TestBigtableVeneerApi method testGetDataClient.

@Test
public void testGetDataClient() throws Exception {
    assertTrue(bigtableApi.getDataClient() instanceof DataClientVeneerApi);
    Query query = Query.create(TABLE_ID).rowKey(ROW_KEY);
    bigtableApi.getDataClient().readRowsAsync(query);
    ReadRowsRequest request = fakeDataService.popLastRequest();
    assertEquals(ROW_KEY, request.getRows().getRowKeys(0).toStringUtf8());
}
Also used : Query(com.google.cloud.bigtable.data.v2.models.Query) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest) Test(org.junit.Test)

Aggregations

Test (org.junit.Test)59 ReadRowsRequest (com.google.cloud.bigquery.storage.v1.ReadRowsRequest)31 ReadRowsRequest (com.google.bigtable.v2.ReadRowsRequest)28 ReadRowsResponse (com.google.cloud.bigquery.storage.v1.ReadRowsResponse)24 ReadSession (com.google.cloud.bigquery.storage.v1.ReadSession)17 ReadRowsRequest (com.google.cloud.bigquery.storage.v1beta2.ReadRowsRequest)16 ReadRowsResponse (com.google.cloud.bigquery.storage.v1beta2.ReadRowsResponse)11 StorageClient (org.apache.beam.sdk.io.gcp.bigquery.BigQueryServices.StorageClient)10 FakeBigQueryServices (org.apache.beam.sdk.io.gcp.testing.FakeBigQueryServices)10 ReadRowsResponse (com.google.bigtable.v2.ReadRowsResponse)9 Row (com.google.cloud.bigtable.data.v2.models.Row)9 CreateReadSessionRequest (com.google.cloud.bigquery.storage.v1.CreateReadSessionRequest)8 TableRow (com.google.api.services.bigquery.model.TableRow)7 Query (com.google.cloud.bigtable.data.v2.models.Query)7 ServerStreamingStashCallable (com.google.cloud.bigtable.gaxx.testing.FakeStreamingApi.ServerStreamingStashCallable)7 ByteString (com.google.protobuf.ByteString)7 GenericRecord (org.apache.avro.generic.GenericRecord)6 TableRowParser (org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.TableRowParser)6 ReadSession (com.google.cloud.bigquery.storage.v1beta2.ReadSession)5 DefaultRowAdapter (com.google.cloud.bigtable.data.v2.models.DefaultRowAdapter)5