Search in sources :

Example 6 with KuduPredicate

use of org.apache.kudu.client.KuduPredicate in project gora by apache.

the class KuduStore method exists.

@Override
public boolean exists(K key) throws GoraException {
    try {
        ColumnSchema column = table.getSchema().getColumn(kuduMapping.getPrimaryKey().get(0).getName());
        ArrayList<KuduPredicate> equalPredicate = new ArrayList<>();
        equalPredicate.add(KuduClientUtils.createEqualPredicate(column, key));
        KuduScanner build = createScanner(equalPredicate, new ArrayList<>(), 1);
        RowResult waitFirstResult = KuduClientUtils.waitFirstResult(build);
        build.close();
        return waitFirstResult != null;
    } catch (Exception e) {
        throw new GoraException(e);
    }
}
Also used : RowResult(org.apache.kudu.client.RowResult) GoraException(org.apache.gora.util.GoraException) KuduScanner(org.apache.kudu.client.KuduScanner) ArrayList(java.util.ArrayList) ColumnSchema(org.apache.kudu.ColumnSchema) KuduPredicate(org.apache.kudu.client.KuduPredicate) KuduException(org.apache.kudu.client.KuduException) GoraException(org.apache.gora.util.GoraException) IOException(java.io.IOException)

Example 7 with KuduPredicate

use of org.apache.kudu.client.KuduPredicate in project gora by apache.

the class KuduStore method execute.

@Override
public Result<K, T> execute(Query<K, T> query) throws GoraException {
    String[] avFields = getFieldsToQuery(query.getFields());
    List<String> dbFields = new ArrayList<>();
    for (String af : avFields) {
        dbFields.add(kuduMapping.getFields().get(af).getName());
    }
    try {
        ColumnSchema column = table.getSchema().getColumn(kuduMapping.getPrimaryKey().get(0).getName());
        dbFields.add(kuduMapping.getPrimaryKey().get(0).getName());
        List<KuduPredicate> rangePredicates = KuduClientUtils.createRangePredicate(column, query.getStartKey(), query.getEndKey());
        rangePredicates.add(KuduPredicate.newIsNotNullPredicate(column));
        KuduScanner build = createScanner(rangePredicates, dbFields, query.getLimit());
        return new KuduResult<>(this, query, build);
    } catch (Exception e) {
        throw new GoraException(e);
    }
}
Also used : GoraException(org.apache.gora.util.GoraException) KuduResult(org.apache.gora.kudu.query.KuduResult) KuduScanner(org.apache.kudu.client.KuduScanner) ArrayList(java.util.ArrayList) ColumnSchema(org.apache.kudu.ColumnSchema) KuduPredicate(org.apache.kudu.client.KuduPredicate) KuduException(org.apache.kudu.client.KuduException) GoraException(org.apache.gora.util.GoraException) IOException(java.io.IOException)

Example 8 with KuduPredicate

use of org.apache.kudu.client.KuduPredicate in project gora by apache.

the class KuduStore method get.

@Override
public T get(K key, String[] fields) throws GoraException {
    String[] avFields = getFieldsToQuery(fields);
    List<String> dbFields = new ArrayList<>();
    for (String af : avFields) {
        dbFields.add(kuduMapping.getFields().get(af).getName());
    }
    try {
        ColumnSchema column = table.getSchema().getColumn(kuduMapping.getPrimaryKey().get(0).getName());
        ArrayList<KuduPredicate> equalPredicate = new ArrayList<>();
        equalPredicate.add(KuduClientUtils.createEqualPredicate(column, key));
        KuduScanner build = createScanner(equalPredicate, dbFields, 1);
        RowResult waitGetOneOrZero = KuduClientUtils.waitFirstResult(build);
        T resp = null;
        if (waitGetOneOrZero != null) {
            resp = newInstance(waitGetOneOrZero, fields);
        }
        build.close();
        return resp;
    } catch (Exception ex) {
        throw new GoraException(ex);
    }
}
Also used : RowResult(org.apache.kudu.client.RowResult) GoraException(org.apache.gora.util.GoraException) KuduScanner(org.apache.kudu.client.KuduScanner) ArrayList(java.util.ArrayList) ColumnSchema(org.apache.kudu.ColumnSchema) KuduPredicate(org.apache.kudu.client.KuduPredicate) KuduException(org.apache.kudu.client.KuduException) GoraException(org.apache.gora.util.GoraException) IOException(java.io.IOException)

Example 9 with KuduPredicate

use of org.apache.kudu.client.KuduPredicate in project hive by apache.

the class TestKuduPredicateHandler method testOrPredicates.

@Test
public void testOrPredicates() throws Exception {
    for (ColumnSchema col : SCHEMA.getColumns()) {
        // Skip binary columns because binary predicates are not supported. (HIVE-11370)
        if (col.getName().equals("null") || col.getName().equals("default") || col.getName().equals("binary")) {
            continue;
        }
        PrimitiveTypeInfo typeInfo = toHiveType(col.getType(), col.getTypeAttributes());
        ExprNodeDesc colExpr = new ExprNodeColumnDesc(typeInfo, col.getName(), null, false);
        ExprNodeDesc constExpr = new ExprNodeConstantDesc(typeInfo, ROW.getObject(col.getName()));
        List<ExprNodeDesc> children = Lists.newArrayList();
        children.add(colExpr);
        children.add(constExpr);
        ExprNodeGenericFuncDesc gePredicateExpr = new ExprNodeGenericFuncDesc(typeInfo, new GenericUDFOPEqualOrGreaterThan(), children);
        ExprNodeGenericFuncDesc lePredicateExpr = new ExprNodeGenericFuncDesc(typeInfo, new GenericUDFOPEqualOrLessThan(), children);
        List<ExprNodeDesc> orChildren = Lists.newArrayList();
        orChildren.add(gePredicateExpr);
        orChildren.add(lePredicateExpr);
        ExprNodeGenericFuncDesc predicateExpr = new ExprNodeGenericFuncDesc(typeInfo, new GenericUDFOPOr(), orChildren);
        // Verify KuduPredicateHandler.decompose
        HiveStoragePredicateHandler.DecomposedPredicate decompose = KuduPredicateHandler.decompose(predicateExpr, SCHEMA);
        // OR predicates are currently not supported.
        assertNull(decompose);
        List<KuduPredicate> predicates = expressionToPredicates(predicateExpr);
        assertEquals(0, predicates.size());
        // Also test NOT OR.
        List<ExprNodeDesc> notChildren = Lists.newArrayList();
        notChildren.add(predicateExpr);
        ExprNodeGenericFuncDesc notPredicateExpr = new ExprNodeGenericFuncDesc(typeInfo, new GenericUDFOPNot(), notChildren);
        // Verify KuduPredicateHandler.decompose
        HiveStoragePredicateHandler.DecomposedPredicate decomposeNot = KuduPredicateHandler.decompose(notPredicateExpr, SCHEMA);
        // See note in KuduPredicateHandler.newAnalyzer.
        assertNull(decomposeNot);
        List<KuduPredicate> notPredicates = expressionToPredicates(notPredicateExpr);
        assertEquals(2, notPredicates.size());
    }
}
Also used : HiveStoragePredicateHandler(org.apache.hadoop.hive.ql.metadata.HiveStoragePredicateHandler) ExprNodeConstantDesc(org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc) GenericUDFOPEqualOrLessThan(org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPEqualOrLessThan) ExprNodeGenericFuncDesc(org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) ColumnSchema(org.apache.kudu.ColumnSchema) PrimitiveTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo) GenericUDFOPEqualOrGreaterThan(org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPEqualOrGreaterThan) KuduPredicate(org.apache.kudu.client.KuduPredicate) ExprNodeColumnDesc(org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc) ExprNodeDesc(org.apache.hadoop.hive.ql.plan.ExprNodeDesc) GenericUDFOPNot(org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNot) GenericUDFOPOr(org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPOr) Test(org.junit.Test)

Example 10 with KuduPredicate

use of org.apache.kudu.client.KuduPredicate in project hive by apache.

the class TestKuduPredicateHandler method testInPredicates.

@Test
public void testInPredicates() throws Exception {
    PrimitiveTypeInfo typeInfo = toHiveType(Type.STRING, null);
    ExprNodeDesc colExpr = new ExprNodeColumnDesc(typeInfo, "string", null, false);
    ExprNodeConstantDesc constDesc = new ExprNodeConstantDesc("Alpha");
    ExprNodeConstantDesc constDesc2 = new ExprNodeConstantDesc("Bravo");
    List<ExprNodeDesc> children = Lists.newArrayList();
    children.add(colExpr);
    children.add(constDesc);
    children.add(constDesc2);
    ExprNodeGenericFuncDesc predicateExpr = new ExprNodeGenericFuncDesc(typeInfo, new GenericUDFIn(), children);
    // Verify KuduPredicateHandler.decompose
    HiveStoragePredicateHandler.DecomposedPredicate decompose = KuduPredicateHandler.decompose(predicateExpr, SCHEMA);
    // See note in KuduPredicateHandler.newAnalyzer.
    assertNull(decompose);
    List<KuduPredicate> predicates = expressionToPredicates(predicateExpr);
    assertEquals(1, predicates.size());
    scanWithPredicates(predicates);
    // Also test NOT IN.
    List<ExprNodeDesc> notChildren = Lists.newArrayList();
    notChildren.add(predicateExpr);
    ExprNodeGenericFuncDesc notPredicateExpr = new ExprNodeGenericFuncDesc(typeInfo, new GenericUDFOPNot(), notChildren);
    // Verify KuduPredicateHandler.decompose
    HiveStoragePredicateHandler.DecomposedPredicate decomposeNot = KuduPredicateHandler.decompose(notPredicateExpr, SCHEMA);
    // See note in KuduPredicateHandler.newAnalyzer.
    assertNull(decomposeNot);
    List<KuduPredicate> notPredicates = expressionToPredicates(notPredicateExpr);
    assertEquals(0, notPredicates.size());
}
Also used : HiveStoragePredicateHandler(org.apache.hadoop.hive.ql.metadata.HiveStoragePredicateHandler) ExprNodeConstantDesc(org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc) ExprNodeColumnDesc(org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc) ExprNodeGenericFuncDesc(org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) GenericUDFIn(org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn) ExprNodeDesc(org.apache.hadoop.hive.ql.plan.ExprNodeDesc) GenericUDFOPNot(org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNot) PrimitiveTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo) KuduPredicate(org.apache.kudu.client.KuduPredicate) Test(org.junit.Test)

Aggregations

KuduPredicate (org.apache.kudu.client.KuduPredicate)16 ColumnSchema (org.apache.kudu.ColumnSchema)10 HiveStoragePredicateHandler (org.apache.hadoop.hive.ql.metadata.HiveStoragePredicateHandler)7 ExprNodeColumnDesc (org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc)7 ExprNodeDesc (org.apache.hadoop.hive.ql.plan.ExprNodeDesc)7 ExprNodeGenericFuncDesc (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)7 PrimitiveTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo)7 Test (org.junit.Test)7 ArrayList (java.util.ArrayList)6 ExprNodeConstantDesc (org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc)6 KuduScanner (org.apache.kudu.client.KuduScanner)6 IOException (java.io.IOException)5 GoraException (org.apache.gora.util.GoraException)4 GenericUDFOPNot (org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNot)4 KuduException (org.apache.kudu.client.KuduException)4 GenericUDF (org.apache.hadoop.hive.ql.udf.generic.GenericUDF)3 GenericUDFOPEqualOrGreaterThan (org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPEqualOrGreaterThan)3 RowResult (org.apache.kudu.client.RowResult)3 GenericUDFOPAnd (org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd)2 GenericUDFOPEqualOrLessThan (org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPEqualOrLessThan)2