Search in sources :

Example 1 with KuduTable

use of org.apache.kudu.client.KuduTable in project drill by apache.

the class KuduRecordReader method setup.

@Override
public void setup(OperatorContext context, OutputMutator output) throws ExecutionSetupException {
    this.output = output;
    this.context = context;
    try {
        KuduTable table = client.openTable(scanSpec.getTableName());
        KuduScannerBuilder builder = client.newScannerBuilder(table);
        if (!isStarQuery()) {
            List<String> colNames = Lists.newArrayList();
            for (SchemaPath p : this.getColumns()) {
                colNames.add(p.getAsUnescapedPath());
            }
            builder.setProjectedColumnNames(colNames);
        }
        context.getStats().startWait();
        try {
            scanner = builder.lowerBoundRaw(scanSpec.getStartKey()).exclusiveUpperBoundRaw(scanSpec.getEndKey()).build();
        } finally {
            context.getStats().stopWait();
        }
    } catch (Exception e) {
        throw new ExecutionSetupException(e);
    }
}
Also used : ExecutionSetupException(org.apache.drill.common.exceptions.ExecutionSetupException) KuduScannerBuilder(org.apache.kudu.client.KuduScanner.KuduScannerBuilder) SchemaPath(org.apache.drill.common.expression.SchemaPath) KuduTable(org.apache.kudu.client.KuduTable) UserException(org.apache.drill.common.exceptions.UserException) ExecutionSetupException(org.apache.drill.common.exceptions.ExecutionSetupException) SchemaChangeException(org.apache.drill.exec.exception.SchemaChangeException)

Example 2 with KuduTable

use of org.apache.kudu.client.KuduTable in project drill by apache.

the class TestKuduConnect method createKuduTable.

public static void createKuduTable(String tableName, int tablets, int replicas, int rows) throws Exception {
    try (KuduClient client = new KuduClient.KuduClientBuilder(KUDU_MASTER).build()) {
        ListTablesResponse tables = client.getTablesList(tableName);
        if (!tables.getTablesList().isEmpty()) {
            client.deleteTable(tableName);
        }
        List<ColumnSchema> columns = new ArrayList<>(5);
        columns.add(new ColumnSchema.ColumnSchemaBuilder("key", Type.INT32).key(true).build());
        columns.add(new ColumnSchema.ColumnSchemaBuilder("binary", Type.BINARY).nullable(false).build());
        columns.add(new ColumnSchema.ColumnSchemaBuilder("boolean", Type.BOOL).nullable(true).build());
        columns.add(new ColumnSchema.ColumnSchemaBuilder("float", Type.FLOAT).nullable(false).build());
        columns.add(new ColumnSchema.ColumnSchemaBuilder("string", Type.STRING).nullable(true).build());
        Schema schema = new Schema(columns);
        CreateTableOptions builder = new CreateTableOptions();
        builder.setNumReplicas(replicas);
        builder.setRangePartitionColumns(Arrays.asList("key"));
        for (int i = 1; i < tablets; i++) {
            PartialRow splitRow = schema.newPartialRow();
            splitRow.addInt("key", i * 1000);
            builder.addSplitRow(splitRow);
        }
        client.createTable(tableName, schema, builder);
        KuduTable table = client.openTable(tableName);
        KuduSession session = client.newSession();
        session.setFlushMode(SessionConfiguration.FlushMode.AUTO_FLUSH_SYNC);
        for (int i = 0; i < rows; i++) {
            Insert insert = table.newInsert();
            PartialRow row = insert.getRow();
            row.addInt(0, i);
            row.addBinary(1, ("Row " + i).getBytes());
            row.addBoolean(2, i % 2 == 0);
            row.addFloat(3, i + 0.01f);
            row.addString(4, ("Row " + i));
            session.apply(insert);
        }
        List<String> projectColumns = new ArrayList<>(1);
        projectColumns.add("float");
        KuduScanner scanner = client.newScannerBuilder(table).setProjectedColumnNames(projectColumns).build();
        while (scanner.hasMoreRows()) {
            RowResultIterator results = scanner.nextRows();
            while (results.hasNext()) {
                RowResult result = results.next();
                System.out.println(result.toStringLongFormat());
            }
        }
    }
}
Also used : KuduSession(org.apache.kudu.client.KuduSession) Schema(org.apache.kudu.Schema) ColumnSchema(org.apache.kudu.ColumnSchema) ArrayList(java.util.ArrayList) PartialRow(org.apache.kudu.client.PartialRow) ColumnSchema(org.apache.kudu.ColumnSchema) KuduTable(org.apache.kudu.client.KuduTable) Insert(org.apache.kudu.client.Insert) RowResultIterator(org.apache.kudu.client.RowResultIterator) RowResult(org.apache.kudu.client.RowResult) KuduScanner(org.apache.kudu.client.KuduScanner) KuduClient(org.apache.kudu.client.KuduClient) ListTablesResponse(org.apache.kudu.client.ListTablesResponse) CreateTableOptions(org.apache.kudu.client.CreateTableOptions)

Aggregations

KuduTable (org.apache.kudu.client.KuduTable)2 ArrayList (java.util.ArrayList)1 ExecutionSetupException (org.apache.drill.common.exceptions.ExecutionSetupException)1 UserException (org.apache.drill.common.exceptions.UserException)1 SchemaPath (org.apache.drill.common.expression.SchemaPath)1 SchemaChangeException (org.apache.drill.exec.exception.SchemaChangeException)1 ColumnSchema (org.apache.kudu.ColumnSchema)1 Schema (org.apache.kudu.Schema)1 CreateTableOptions (org.apache.kudu.client.CreateTableOptions)1 Insert (org.apache.kudu.client.Insert)1 KuduClient (org.apache.kudu.client.KuduClient)1 KuduScanner (org.apache.kudu.client.KuduScanner)1 KuduScannerBuilder (org.apache.kudu.client.KuduScanner.KuduScannerBuilder)1 KuduSession (org.apache.kudu.client.KuduSession)1 ListTablesResponse (org.apache.kudu.client.ListTablesResponse)1 PartialRow (org.apache.kudu.client.PartialRow)1 RowResult (org.apache.kudu.client.RowResult)1 RowResultIterator (org.apache.kudu.client.RowResultIterator)1