Search in sources :

Example 6 with Row

use of com.helger.genericode.v10.Row in project gateway-service by hypertrace.

the class EdsEntityUpdater method update.

public UpdateEntityResponse.Builder update(UpdateEntityRequest updateRequest, UpdateExecutionContext updateExecutionContext) {
    UpdateEntityResponse.Builder responseBuilder = UpdateEntityResponse.newBuilder();
    EntityUpdateRequest eqsUpdateRequest = convertToEqsUpdateRequest(updateRequest);
    if (LOG.isDebugEnabled()) {
        LOG.debug("Sending update request to EDS  ======== \n {}", eqsUpdateRequest);
    }
    Iterator<ResultSetChunk> resultSetChunkIterator = eqsClient.update(eqsUpdateRequest, updateExecutionContext.getRequestHeaders());
    if (!resultSetChunkIterator.hasNext()) {
        return responseBuilder;
    }
    ResultSetChunk chunk = resultSetChunkIterator.next();
    if (LOG.isDebugEnabled()) {
        LOG.debug("Received chunk: " + chunk.toString());
    }
    if (chunk.getRowCount() == 0) {
        return responseBuilder;
    }
    if (chunk.getRowCount() > 1) {
        LOG.warn("Received more than 1 row back. Only the first out of {} rows will be returned in the response", chunk.getRowCount());
    }
    Map<String, AttributeMetadata> resultAttributeMetadata = AttributeMetadataUtil.remapAttributeMetadataByResultKey(updateRequest.getSelectionList(), updateExecutionContext.getAttributeMetadata());
    Row row = chunk.getRow(0);
    Entity.Builder entityBuilder = Entity.newBuilder().setEntityType(updateRequest.getEntityType());
    for (int i = 0; i < chunk.getResultSetMetadata().getColumnMetadataCount(); i++) {
        String resultName = chunk.getResultSetMetadata().getColumnMetadata(i).getColumnName();
        entityBuilder.putAttribute(resultName, EntityServiceAndGatewayServiceConverter.convertQueryValueToGatewayValue(row.getColumn(i), resultAttributeMetadata.get(resultName)));
    }
    responseBuilder.setEntity(entityBuilder);
    return responseBuilder;
}
Also used : Entity(org.hypertrace.gateway.service.v1.entity.Entity) UpdateEntityResponse(org.hypertrace.gateway.service.v1.entity.UpdateEntityResponse) AttributeMetadata(org.hypertrace.core.attribute.service.v1.AttributeMetadata) Row(org.hypertrace.entity.query.service.v1.Row) EntityUpdateRequest(org.hypertrace.entity.query.service.v1.EntityUpdateRequest) ResultSetChunk(org.hypertrace.entity.query.service.v1.ResultSetChunk)

Example 7 with Row

use of com.helger.genericode.v10.Row in project beam by apache.

the class BigtableIOTest method testReadWithReaderStartFailed.

@Test
public void testReadWithReaderStartFailed() throws IOException {
    FailureBigtableService failureService = new FailureBigtableService(FailureOptions.builder().setFailAtStart(true).build());
    BigtableConfig failureConfig = BigtableConfig.builder().setValidate(true).setBigtableService(failureService).build();
    final String table = "TEST-TABLE";
    final int numRows = 100;
    makeTableData(failureService, table, numRows);
    BigtableSource source = new BigtableSource(failureConfig.withTableId(StaticValueProvider.of(table)), BigtableReadOptions.builder().setKeyRanges(ALL_KEY_RANGE).build(), null);
    BoundedReader<Row> reader = source.createReader(TestPipeline.testingPipelineOptions());
    thrown.expect(IOException.class);
    thrown.expectMessage("Fake IOException at start()");
    reader.start();
}
Also used : ByteString(com.google.protobuf.ByteString) Row(com.google.bigtable.v2.Row) BigtableSource(org.apache.beam.sdk.io.gcp.bigtable.BigtableIO.BigtableSource) Test(org.junit.Test)

Example 8 with Row

use of com.helger.genericode.v10.Row in project beam by apache.

the class BigtableIOTest method testGetSplitPointsConsumed.

@Test
public void testGetSplitPointsConsumed() throws Exception {
    final String table = "TEST-TABLE";
    final int numRows = 100;
    int splitPointsConsumed = 0;
    makeTableData(table, numRows);
    BigtableSource source = new BigtableSource(config.withTableId(StaticValueProvider.of(table)), BigtableReadOptions.builder().setKeyRanges(ALL_KEY_RANGE).build(), null);
    BoundedReader<Row> reader = source.createReader(TestPipeline.testingPipelineOptions());
    reader.start();
    // Started, 0 split points consumed
    assertEquals("splitPointsConsumed starting", splitPointsConsumed, reader.getSplitPointsConsumed());
    // Split points consumed increases for each row read
    while (reader.advance()) {
        assertEquals("splitPointsConsumed advancing", ++splitPointsConsumed, reader.getSplitPointsConsumed());
    }
    // Reader marked as done, 100 split points consumed
    assertEquals("splitPointsConsumed done", numRows, reader.getSplitPointsConsumed());
    reader.close();
}
Also used : ByteString(com.google.protobuf.ByteString) Row(com.google.bigtable.v2.Row) BigtableSource(org.apache.beam.sdk.io.gcp.bigtable.BigtableIO.BigtableSource) Test(org.junit.Test)

Example 9 with Row

use of com.helger.genericode.v10.Row in project beam by apache.

the class BigtableIOTest method testReadWithReaderAdvanceFailed.

@Test
public void testReadWithReaderAdvanceFailed() throws IOException {
    FailureBigtableService failureService = new FailureBigtableService(FailureOptions.builder().setFailAtAdvance(true).build());
    BigtableConfig failureConfig = BigtableConfig.builder().setValidate(true).setBigtableService(failureService).build();
    final String table = "TEST-TABLE";
    final int numRows = 100;
    makeTableData(failureService, table, numRows);
    BigtableSource source = new BigtableSource(failureConfig.withTableId(StaticValueProvider.of(table)), BigtableReadOptions.builder().setKeyRanges(ALL_KEY_RANGE).build(), null);
    BoundedReader<Row> reader = source.createReader(TestPipeline.testingPipelineOptions());
    thrown.expect(IOException.class);
    thrown.expectMessage("Fake IOException at advance()");
    reader.start();
    reader.advance();
}
Also used : ByteString(com.google.protobuf.ByteString) Row(com.google.bigtable.v2.Row) BigtableSource(org.apache.beam.sdk.io.gcp.bigtable.BigtableIO.BigtableSource) Test(org.junit.Test)

Example 10 with Row

use of com.helger.genericode.v10.Row in project beam by apache.

the class BigtableIOTest method makeTableData.

/**
 * Helper function to create a table and return the rows that it created with custom service.
 */
private static List<Row> makeTableData(FakeBigtableService fakeService, String tableId, int numRows) {
    fakeService.createTable(tableId);
    Map<ByteString, ByteString> testData = fakeService.getTable(tableId);
    List<Row> testRows = new ArrayList<>(numRows);
    for (int i = 0; i < numRows; ++i) {
        ByteString key = ByteString.copyFromUtf8(String.format("key%09d", i));
        ByteString value = ByteString.copyFromUtf8(String.format("value%09d", i));
        testData.put(key, value);
        testRows.add(makeRow(key, value));
    }
    return testRows;
}
Also used : ByteString(com.google.protobuf.ByteString) ArrayList(java.util.ArrayList) Row(com.google.bigtable.v2.Row)

Aggregations

Test (org.junit.Test)68 Row (org.molgenis.emx2.Row)43 Row (com.google.bigtable.v2.Row)22 ArrayList (java.util.ArrayList)17 ByteString (com.google.protobuf.ByteString)14 Table (org.molgenis.emx2.Table)11 Function (com.google.common.base.Function)10 Row (org.hypertrace.entity.query.service.v1.Row)10 Map (java.util.Map)9 ResultSetChunk (org.hypertrace.entity.query.service.v1.ResultSetChunk)8 Schema (org.molgenis.emx2.Schema)8 Row (com.google.api.ads.admanager.axis.v202108.Row)6 Family (com.google.bigtable.v2.Family)6 List (java.util.List)6 ByteKey (org.apache.beam.sdk.io.range.ByteKey)6 Row (com.google.api.ads.admanager.axis.v202105.Row)5 Row (com.google.api.ads.admanager.axis.v202111.Row)5 Row (com.google.api.ads.admanager.axis.v202202.Row)5 Row (com.google.api.ads.admanager.axis.v202205.Row)5 Row (com.google.api.ads.admanager.jaxws.v202105.Row)5