Search in sources :

Example 1 with ReadRowsResponse

use of com.google.cloud.bigquery.storage.v1beta2.ReadRowsResponse in project gapic-generator-java by googleapis.

the class AsyncReadRows method asyncReadRows.

public static void asyncReadRows() throws Exception {
    // It may require modifications to work in your environment.
    try (BaseBigtableDataClient baseBigtableDataClient = BaseBigtableDataClient.create()) {
        ReadRowsRequest request = ReadRowsRequest.newBuilder().setTableName(TableName.of("[PROJECT]", "[INSTANCE]", "[TABLE]").toString()).setAppProfileId("appProfileId704923523").setRows(RowSet.newBuilder().build()).setFilter(RowFilter.newBuilder().build()).setRowsLimit(-944199211).build();
        ServerStream<ReadRowsResponse> stream = baseBigtableDataClient.readRowsCallable().call(request);
        for (ReadRowsResponse response : stream) {
        // Do something when a response is received.
        }
    }
}
Also used : ReadRowsResponse(com.google.bigtable.v2.ReadRowsResponse) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest) BaseBigtableDataClient(com.google.cloud.bigtable.data.v2.BaseBigtableDataClient)

Example 2 with ReadRowsResponse

use of com.google.cloud.bigquery.storage.v1beta2.ReadRowsResponse in project grpc-gcp-java by GoogleCloudPlatform.

the class BigtableIntegrationTest method testReadRows.

@Test
public void testReadRows() throws Exception {
    BigtableBlockingStub stub = getBigtableBlockingStub();
    ReadRowsRequest request = ReadRowsRequest.newBuilder().setTableName(TABLE_NAME).setRows(RowSet.newBuilder().addRowKeys(ByteString.copyFromUtf8("test-row"))).build();
    Iterator<ReadRowsResponse> response = stub.readRows(request);
    assertEquals(1, gcpChannel.channelRefs.size());
}
Also used : ReadRowsResponse(com.google.bigtable.v2.ReadRowsResponse) BigtableBlockingStub(com.google.bigtable.v2.BigtableGrpc.BigtableBlockingStub) ReadRowsRequest(com.google.bigtable.v2.ReadRowsRequest) Test(org.junit.Test)

Example 3 with ReadRowsResponse

use of com.google.cloud.bigquery.storage.v1beta2.ReadRowsResponse in project spark-bigquery-connector by GoogleCloudDataproc.

the class StreamCombiningIterator method next.

/**
 * Consumes the next response and asynchronously request the next response from the observer.
 *
 * @return The next response.
 * @throws NoSuchElementException If the stream has been consumed or cancelled.
 */
@Override
public ReadRowsResponse next() {
    if (!hasNext()) {
        throw new NoSuchElementException();
    }
    try {
        Observer observer = observersQueue.poll();
        Preconditions.checkState(observer != null);
        observer.request();
        @SuppressWarnings("unchecked") ReadRowsResponse tmp = (ReadRowsResponse) last;
        return tmp;
    } finally {
        if (last != EOS) {
            last = null;
        }
    }
}
Also used : ReadRowsResponse(com.google.cloud.bigquery.storage.v1.ReadRowsResponse) ResponseObserver(com.google.api.gax.rpc.ResponseObserver) NoSuchElementException(java.util.NoSuchElementException)

Example 4 with ReadRowsResponse

use of com.google.cloud.bigquery.storage.v1beta2.ReadRowsResponse in project spark-bigquery-connector by GoogleCloudDataproc.

the class BigQueryInputPartitionContext method createPartitionReaderContext.

@Override
public InputPartitionReaderContext<InternalRow> createPartitionReaderContext() {
    ReadRowsRequest.Builder readRowsRequest = ReadRowsRequest.newBuilder().setReadStream(streamName);
    ReadRowsHelper readRowsHelper = new ReadRowsHelper(bigQueryReadClientFactory, readRowsRequest, options);
    Iterator<ReadRowsResponse> readRowsResponses = readRowsHelper.readRows();
    return new BigQueryInputPartitionReaderContext(readRowsResponses, converter, readRowsHelper);
}
Also used : ReadRowsResponse(com.google.cloud.bigquery.storage.v1.ReadRowsResponse) ReadRowsHelper(com.google.cloud.bigquery.connector.common.ReadRowsHelper) ReadRowsRequest(com.google.cloud.bigquery.storage.v1.ReadRowsRequest)

Example 5 with ReadRowsResponse

use of com.google.cloud.bigquery.storage.v1beta2.ReadRowsResponse in project spark-bigquery-connector by GoogleCloudDataproc.

the class BigQueryInputPartitionReaderContextTest method testReadAvro.

@Test
public void testReadAvro() throws Exception {
    TableInfo allTypesTableInfo = allTypesTableInfo();
    ReadRowsResponse.Builder readRowsResponse = ReadRowsResponse.newBuilder();
    TextFormat.merge(ALL_TYPES_TABLE_READ_ROWS_RESPONSE_STR, readRowsResponse);
    Iterator<ReadRowsResponse> readRowsResponses = ImmutableList.of(readRowsResponse.build()).iterator();
    ReadRowsResponseToInternalRowIteratorConverter converter = ReadRowsResponseToInternalRowIteratorConverter.avro(ALL_TYPES_TABLE_BIGQUERY_SCHEMA, ALL_TYPES_TABLE_FIELDS, ALL_TYPES_TABLE_AVRO_RAW_SCHEMA, Optional.empty());
    BigQueryInputPartitionReaderContext reader = new BigQueryInputPartitionReaderContext(readRowsResponses, converter, null);
    assertThat(reader.next()).isTrue();
    InternalRow row = reader.get();
    assertThat(reader.next()).isFalse();
    assertThat(row.numFields()).isEqualTo(15);
    assertThat(row.getString(0)).isEqualTo("hello");
}
Also used : ReadRowsResponseToInternalRowIteratorConverter(com.google.cloud.spark.bigquery.ReadRowsResponseToInternalRowIteratorConverter) ReadRowsResponse(com.google.cloud.bigquery.storage.v1.ReadRowsResponse) InternalRow(org.apache.spark.sql.catalyst.InternalRow) Test(org.junit.Test)

Aggregations

Test (org.junit.Test)44 ReadRowsResponse (com.google.cloud.bigquery.storage.v1.ReadRowsResponse)40 ReadRowsRequest (com.google.cloud.bigquery.storage.v1.ReadRowsRequest)25 ReadRowsResponse (com.google.bigtable.v2.ReadRowsResponse)19 ReadSession (com.google.cloud.bigquery.storage.v1.ReadSession)17 StorageClient (org.apache.beam.sdk.io.gcp.bigquery.BigQueryServices.StorageClient)17 FakeBigQueryServices (org.apache.beam.sdk.io.gcp.testing.FakeBigQueryServices)17 ReadRowsRequest (com.google.bigtable.v2.ReadRowsRequest)15 TableRow (com.google.api.services.bigquery.model.TableRow)14 TableRowParser (org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.TableRowParser)13 ReadRowsResponse (com.google.cloud.bigquery.storage.v1beta2.ReadRowsResponse)11 ReadRowsRequest (com.google.cloud.bigquery.storage.v1beta2.ReadRowsRequest)10 ByteString (com.google.protobuf.ByteString)10 CreateReadSessionRequest (com.google.cloud.bigquery.storage.v1.CreateReadSessionRequest)8 GenericRecord (org.apache.avro.generic.GenericRecord)7 StatusRuntimeException (io.grpc.StatusRuntimeException)6 ReadSession (com.google.cloud.bigquery.storage.v1beta2.ReadSession)5 StreamObserver (io.grpc.stub.StreamObserver)5 Mockito.doAnswer (org.mockito.Mockito.doAnswer)5 InvocationOnMock (org.mockito.invocation.InvocationOnMock)5