Search in sources :

Example 21 with ByteKeyRange

use of org.apache.beam.sdk.io.range.ByteKeyRange in project beam by apache.

the class HBaseIOTest method testReadingKeyRangePrefix.

/**
 * Tests reading all rows using key ranges. Tests a prefix [), a suffix (], and a restricted range
 * [] and that some properties hold across them.
 */
@Test
public void testReadingKeyRangePrefix() throws Exception {
    final String table = tmpTable.getName();
    final int numRows = 1001;
    final ByteKey startKey = ByteKey.copyFrom("2".getBytes(StandardCharsets.UTF_8));
    createAndWriteData(table, numRows);
    // Test prefix: [beginning, startKey).
    final ByteKeyRange prefixRange = ByteKeyRange.ALL_KEYS.withEndKey(startKey);
    runReadTestLength(HBaseIO.read().withConfiguration(conf).withTableId(table).withKeyRange(prefixRange), false, 126);
}
Also used : ByteKey(org.apache.beam.sdk.io.range.ByteKey) ByteKeyRange(org.apache.beam.sdk.io.range.ByteKeyRange) Test(org.junit.Test)

Example 22 with ByteKeyRange

use of org.apache.beam.sdk.io.range.ByteKeyRange in project beam by apache.

the class BigtableReadOptions method validate.

void validate() {
    if (getRowFilter() != null && getRowFilter().isAccessible()) {
        checkArgument(getRowFilter().get() != null, "rowFilter can not be null");
    }
    if (getKeyRanges() != null && getKeyRanges().isAccessible()) {
        checkArgument(getKeyRanges().get() != null, "keyRanges can not be null");
        checkArgument(!getKeyRanges().get().isEmpty(), "keyRanges can not be empty");
        for (ByteKeyRange range : getKeyRanges().get()) {
            checkArgument(range != null, "keyRanges cannot hold null range");
        }
    }
}
Also used : ByteKeyRange(org.apache.beam.sdk.io.range.ByteKeyRange)

Aggregations

ByteKeyRange (org.apache.beam.sdk.io.range.ByteKeyRange)22 Test (org.junit.Test)17 ByteKey (org.apache.beam.sdk.io.range.ByteKey)10 ByteString (com.google.protobuf.ByteString)8 BigtableSource (org.apache.beam.sdk.io.gcp.bigtable.BigtableIO.BigtableSource)5 ArrayList (java.util.ArrayList)4 Row (com.google.bigtable.v2.Row)3 HRegionLocation (org.apache.hadoop.hbase.HRegionLocation)2 Connection (org.apache.hadoop.hbase.client.Connection)2 RowFilter (com.google.bigtable.v2.RowFilter)1 DisplayData (org.apache.beam.sdk.transforms.display.DisplayData)1 TableName (org.apache.hadoop.hbase.TableName)1 Result (org.apache.hadoop.hbase.client.Result)1 ResultScanner (org.apache.hadoop.hbase.client.ResultScanner)1 Table (org.apache.hadoop.hbase.client.Table)1