Search in sources :

Example 41 with ReadRowsRequest

use of com.google.bigtable.repackaged.com.google.bigtable.v2.ReadRowsRequest in project java-bigtable by googleapis.

the class EnhancedBigtableStub method createBulkReadRowsCallable.

/**
 * Creates a callable chain to handle bulk ReadRows RPCs. This is meant to be used in ReadRows
 * batcher. The chain will:
 *
 * <ul>
 *   <li>Convert a {@link Query} into a {@link com.google.bigtable.v2.ReadRowsRequest}.
 *   <li>Upon receiving the response stream, it will merge the {@link
 *       com.google.bigtable.v2.ReadRowsResponse.CellChunk}s in logical rows. The actual row
 *       implementation can be configured in by the {@code rowAdapter} parameter.
 *   <li>Retry/resume on failure.
 *   <li>Filter out marker rows.
 *   <li>Construct a {@link UnaryCallable} that will buffer the entire stream into memory before
 *       completing. If the stream is empty, then the list will be empty.
 *   <li>Add tracing & metrics.
 * </ul>
 */
private <RowT> UnaryCallable<Query, List<RowT>> createBulkReadRowsCallable(RowAdapter<RowT> rowAdapter) {
    ServerStreamingCallable<ReadRowsRequest, RowT> readRowsCallable = createReadRowsBaseCallable(settings.readRowsSettings(), rowAdapter);
    ServerStreamingCallable<Query, RowT> readRowsUserCallable = new ReadRowsUserCallable<>(readRowsCallable, requestContext);
    SpanName span = getSpanName("ReadRows");
    // The TracedBatcherUnaryCallable has to be wrapped by the TracedUnaryCallable, so that
    // TracedUnaryCallable can inject a tracer for the TracedBatcherUnaryCallable to interact with
    UnaryCallable<Query, List<RowT>> tracedBatcher = new TracedBatcherUnaryCallable<>(readRowsUserCallable.all());
    UnaryCallable<Query, List<RowT>> withHeaderTracer = new HeaderTracerUnaryCallable(tracedBatcher);
    UnaryCallable<Query, List<RowT>> traced = new TracedUnaryCallable<>(withHeaderTracer, clientContext.getTracerFactory(), span);
    return traced.withDefaultCallContext(clientContext.getDefaultCallContext());
}
Also used : ReadRowsUserCallable(com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsUserCallable) SpanName(com.google.api.gax.tracing.SpanName) TracedUnaryCallable(com.google.api.gax.tracing.TracedUnaryCallable) HeaderTracerUnaryCallable(com.google.cloud.bigtable.data.v2.stub.metrics.HeaderTracerUnaryCallable) Query(com.google.cloud.bigtable.data.v2.models.Query) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest) List(java.util.List) ImmutableList(com.google.common.collect.ImmutableList) TracedBatcherUnaryCallable(com.google.cloud.bigtable.data.v2.stub.metrics.TracedBatcherUnaryCallable)

Example 42 with ReadRowsRequest

use of com.google.bigtable.repackaged.com.google.bigtable.v2.ReadRowsRequest in project java-bigtable by googleapis.

the class EnhancedBigtableStub method createReadRowsCallable.

/**
 * Creates a callable chain to handle streaming ReadRows RPCs. The chain will:
 *
 * <ul>
 *   <li>Convert a {@link Query} into a {@link com.google.bigtable.v2.ReadRowsRequest} and
 *       dispatch the RPC.
 *   <li>Upon receiving the response stream, it will merge the {@link
 *       com.google.bigtable.v2.ReadRowsResponse.CellChunk}s in logical rows. The actual row
 *       implementation can be configured in by the {@code rowAdapter} parameter.
 *   <li>Retry/resume on failure.
 *   <li>Filter out marker rows.
 *   <li>Add tracing & metrics.
 * </ul>
 */
public <RowT> ServerStreamingCallable<Query, RowT> createReadRowsCallable(RowAdapter<RowT> rowAdapter) {
    ServerStreamingCallable<ReadRowsRequest, RowT> readRowsCallable = createReadRowsBaseCallable(settings.readRowsSettings(), rowAdapter);
    ServerStreamingCallable<Query, RowT> readRowsUserCallable = new ReadRowsUserCallable<>(readRowsCallable, requestContext);
    SpanName span = getSpanName("ReadRows");
    ServerStreamingCallable<Query, RowT> traced = new TracedServerStreamingCallable<>(readRowsUserCallable, clientContext.getTracerFactory(), span);
    return traced.withDefaultCallContext(clientContext.getDefaultCallContext());
}
Also used : ReadRowsUserCallable(com.google.cloud.bigtable.data.v2.stub.readrows.ReadRowsUserCallable) SpanName(com.google.api.gax.tracing.SpanName) TracedServerStreamingCallable(com.google.api.gax.tracing.TracedServerStreamingCallable) Query(com.google.cloud.bigtable.data.v2.models.Query) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest)

Example 43 with ReadRowsRequest

use of com.google.bigtable.repackaged.com.google.bigtable.v2.ReadRowsRequest in project java-bigtable by googleapis.

the class FilterMarkerRowsCallableTest method testRealRow.

@Test
public void testRealRow() {
    Row row = buildRealRow();
    ServerStreamingStashCallable<ReadRowsRequest, Row> innerCallable = new ServerStreamingStashCallable<>(Lists.newArrayList(row));
    FilterMarkerRowsCallable<Row> filterCallable = new FilterMarkerRowsCallable<>(innerCallable, rowAdapter);
    ServerStream<Row> results = filterCallable.call(ReadRowsRequest.getDefaultInstance());
    assertThat(results).containsExactly(row);
}
Also used : ServerStreamingStashCallable(com.google.cloud.bigtable.gaxx.testing.FakeStreamingApi.ServerStreamingStashCallable) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest) Row(com.google.cloud.bigtable.data.v2.models.Row) Test(org.junit.Test)

Example 44 with ReadRowsRequest

use of com.google.bigtable.repackaged.com.google.bigtable.v2.ReadRowsRequest in project java-bigtable by googleapis.

the class FilterMarkerRowsCallableTest method testMixed.

@Test
public void testMixed() {
    Row row = buildRealRow();
    Row markerRow = buildScanMarker();
    ServerStreamingStashCallable<ReadRowsRequest, Row> innerCallable = new ServerStreamingStashCallable<>(Lists.newArrayList(row, markerRow));
    FilterMarkerRowsCallable<Row> filterCallable = new FilterMarkerRowsCallable<>(innerCallable, rowAdapter);
    ServerStream<Row> results = filterCallable.call(ReadRowsRequest.getDefaultInstance());
    assertThat(results).containsExactly(row);
}
Also used : ServerStreamingStashCallable(com.google.cloud.bigtable.gaxx.testing.FakeStreamingApi.ServerStreamingStashCallable) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest) Row(com.google.cloud.bigtable.data.v2.models.Row) Test(org.junit.Test)

Example 45 with ReadRowsRequest

use of com.google.bigtable.repackaged.com.google.bigtable.v2.ReadRowsRequest in project java-bigtable by googleapis.

the class BigtableChannelPrimerTest method testErrorsAreLogged.

@Test
public void testErrorsAreLogged() {
    fakeService.readRowsCallback = new ApiFunction<ReadRowsRequest, ReadRowsResponse>() {

        @Override
        public ReadRowsResponse apply(ReadRowsRequest req) {
            throw new StatusRuntimeException(Status.FAILED_PRECONDITION);
        }
    };
    primer.primeChannel(channel);
    assertThat(logHandler.logs).hasSize(2);
    for (LogRecord log : logHandler.logs) {
        assertThat(log.getMessage()).contains("FAILED_PRECONDITION");
    }
}
Also used : ReadRowsResponse(com.google.bigtable.v2.ReadRowsResponse) LogRecord(java.util.logging.LogRecord) StatusRuntimeException(io.grpc.StatusRuntimeException) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest) Test(org.junit.Test)

Aggregations

Test (org.junit.Test)59 ReadRowsRequest (com.google.cloud.bigquery.storage.v1.ReadRowsRequest)31 ReadRowsRequest (com.google.bigtable.v2.ReadRowsRequest)28 ReadRowsResponse (com.google.cloud.bigquery.storage.v1.ReadRowsResponse)24 ReadSession (com.google.cloud.bigquery.storage.v1.ReadSession)17 ReadRowsRequest (com.google.cloud.bigquery.storage.v1beta2.ReadRowsRequest)16 ReadRowsResponse (com.google.cloud.bigquery.storage.v1beta2.ReadRowsResponse)10 StorageClient (org.apache.beam.sdk.io.gcp.bigquery.BigQueryServices.StorageClient)10 FakeBigQueryServices (org.apache.beam.sdk.io.gcp.testing.FakeBigQueryServices)10 ReadRowsResponse (com.google.bigtable.v2.ReadRowsResponse)9 Row (com.google.cloud.bigtable.data.v2.models.Row)9 CreateReadSessionRequest (com.google.cloud.bigquery.storage.v1.CreateReadSessionRequest)8 TableRow (com.google.api.services.bigquery.model.TableRow)7 Query (com.google.cloud.bigtable.data.v2.models.Query)7 ServerStreamingStashCallable (com.google.cloud.bigtable.gaxx.testing.FakeStreamingApi.ServerStreamingStashCallable)7 ByteString (com.google.protobuf.ByteString)7 GenericRecord (org.apache.avro.generic.GenericRecord)6 TableRowParser (org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.TableRowParser)6 ReadSession (com.google.cloud.bigquery.storage.v1beta2.ReadSession)5 DefaultRowAdapter (com.google.cloud.bigtable.data.v2.models.DefaultRowAdapter)5