Search in sources :

Example 1 with HBaseConnection

use of com.amazonaws.athena.connectors.hbase.connection.HBaseConnection in project aws-athena-query-federation by awslabs.

the class HbaseSchemaUtilsTest method inferSchema.

@Test
public void inferSchema() throws IOException {
    int numToScan = 4;
    TableName tableName = new TableName("schema", "table");
    List<Result> results = TestUtils.makeResults();
    HBaseConnection mockConnection = mock(HBaseConnection.class);
    ResultScanner mockScanner = mock(ResultScanner.class);
    when(mockScanner.iterator()).thenReturn(results.iterator());
    when(mockConnection.scanTable(anyObject(), any(Scan.class), anyObject())).thenAnswer((InvocationOnMock invocationOnMock) -> {
        ResultProcessor processor = (ResultProcessor) invocationOnMock.getArguments()[2];
        return processor.scan(mockScanner);
    });
    Schema schema = HbaseSchemaUtils.inferSchema(mockConnection, tableName, numToScan);
    Map<String, Types.MinorType> actualFields = new HashMap<>();
    schema.getFields().stream().forEach(next -> actualFields.put(next.getName(), Types.getMinorTypeForArrowType(next.getType())));
    Map<String, Types.MinorType> expectedFields = new HashMap<>();
    TestUtils.makeSchema().build().getFields().stream().forEach(next -> expectedFields.put(next.getName(), Types.getMinorTypeForArrowType(next.getType())));
    for (Map.Entry<String, Types.MinorType> nextExpected : expectedFields.entrySet()) {
        assertNotNull(actualFields.get(nextExpected.getKey()));
        assertEquals(nextExpected.getKey(), nextExpected.getValue(), actualFields.get(nextExpected.getKey()));
    }
    assertEquals(expectedFields.size(), actualFields.size());
    verify(mockConnection, times(1)).scanTable(anyObject(), any(Scan.class), any(ResultProcessor.class));
    verify(mockScanner, times(1)).iterator();
}
Also used : ResultScanner(org.apache.hadoop.hbase.client.ResultScanner) HashMap(java.util.HashMap) Schema(org.apache.arrow.vector.types.pojo.Schema) Result(org.apache.hadoop.hbase.client.Result) TestUtils.makeResult(com.amazonaws.athena.connectors.hbase.TestUtils.makeResult) HBaseConnection(com.amazonaws.athena.connectors.hbase.connection.HBaseConnection) TableName(com.amazonaws.athena.connector.lambda.domain.TableName) InvocationOnMock(org.mockito.invocation.InvocationOnMock) Scan(org.apache.hadoop.hbase.client.Scan) ResultProcessor(com.amazonaws.athena.connectors.hbase.connection.ResultProcessor) HashMap(java.util.HashMap) Map(java.util.Map) Test(org.junit.Test)

Aggregations

TableName (com.amazonaws.athena.connector.lambda.domain.TableName)1 TestUtils.makeResult (com.amazonaws.athena.connectors.hbase.TestUtils.makeResult)1 HBaseConnection (com.amazonaws.athena.connectors.hbase.connection.HBaseConnection)1 ResultProcessor (com.amazonaws.athena.connectors.hbase.connection.ResultProcessor)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1 Schema (org.apache.arrow.vector.types.pojo.Schema)1 Result (org.apache.hadoop.hbase.client.Result)1 ResultScanner (org.apache.hadoop.hbase.client.ResultScanner)1 Scan (org.apache.hadoop.hbase.client.Scan)1 Test (org.junit.Test)1 InvocationOnMock (org.mockito.invocation.InvocationOnMock)1