Search in sources :

Example 11 with UserDefinedFunctionResponse

use of com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse in project aws-athena-query-federation by awslabs.

the class ExampleUserDefinedFunctionHandlerTest method testConcatenateMethod.

@Test
public void testConcatenateMethod() throws Exception {
    Schema inputSchema = SchemaBuilder.newBuilder().addListField("list", Types.MinorType.VARCHAR.getType()).build();
    Schema outputSchema = SchemaBuilder.newBuilder().addField("string", Types.MinorType.VARCHAR.getType()).build();
    Block inputRecords = allocator.createBlock(inputSchema);
    inputRecords.setRowCount(1);
    FieldVector fieldVector = inputRecords.getFieldVector("list");
    List<String> value = Lists.newArrayList("a", "b");
    BlockUtils.setComplexValue(fieldVector, 0, FieldResolver.DEFAULT, value);
    UserDefinedFunctionResponse response = runAndAssertSerialization(inputRecords, outputSchema, "concatenate");
    Block outputRecords = response.getRecords();
    assertEquals(1, outputRecords.getRowCount());
    FieldReader fieldReader = outputRecords.getFieldReader("string");
    ArrowValueProjector arrowValueProjector = ProjectorUtils.createArrowValueProjector(fieldReader);
    assertEquals(exampleUserDefinedFunctionHandler.concatenate(Lists.newArrayList("a", "b")), arrowValueProjector.project(0));
}
Also used : ArrowValueProjector(com.amazonaws.athena.connector.lambda.data.projectors.ArrowValueProjector) Schema(org.apache.arrow.vector.types.pojo.Schema) Block(com.amazonaws.athena.connector.lambda.data.Block) FieldVector(org.apache.arrow.vector.FieldVector) UserDefinedFunctionResponse(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse) FieldReader(org.apache.arrow.vector.complex.reader.FieldReader) Test(org.junit.Test)

Example 12 with UserDefinedFunctionResponse

use of com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse in project aws-athena-query-federation by awslabs.

the class UserDefinedFunctionHandlerTest method testInvocationWithStructType.

@Test
public void testInvocationWithStructType() throws Exception {
    int rowCount = 20;
    UserDefinedFunctionRequest udfRequest = createUDFRequest(rowCount, Map.class, "test_row_type", true, Map.class);
    UserDefinedFunctionResponse udfResponse = handler.processFunction(allocator, udfRequest);
    Block responseBlock = udfResponse.getRecords();
    assertEquals(1, responseBlock.getFieldReaders().size());
    assertEquals(rowCount, responseBlock.getRowCount());
    FieldReader fieldReader = responseBlock.getFieldReaders().get(0);
    for (int pos = 0; pos < rowCount; ++pos) {
        fieldReader.setPosition(pos);
        Map<String, Object> actual = (Map) UnitTestBlockUtils.getValue(fieldReader, pos);
        Map<String, Object> input = ImmutableMap.of("intVal", pos + 100, "doubleVal", pos + 200.2);
        Map<String, Object> expected = handler.test_row_type(input);
        for (Map.Entry<String, Object> entry : expected.entrySet()) {
            String key = entry.getKey();
            assertTrue(actual.containsKey(key));
            assertEquals(expected.get(key), actual.get(key));
        }
    }
}
Also used : UserDefinedFunctionRequest(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest) Block(com.amazonaws.athena.connector.lambda.data.Block) UserDefinedFunctionResponse(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse) FieldReader(org.apache.arrow.vector.complex.reader.FieldReader) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) Test(org.junit.Test)

Example 13 with UserDefinedFunctionResponse

use of com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse in project aws-athena-query-federation by awslabs.

the class UserDefinedFunctionHandlerTest method testInvocationWithListType.

@Test
public void testInvocationWithListType() throws Exception {
    int rowCount = 20;
    UserDefinedFunctionRequest udfRequest = createUDFRequest(rowCount, List.class, "test_list_type", true, List.class);
    UserDefinedFunctionResponse udfResponse = handler.processFunction(allocator, udfRequest);
    Block responseBlock = udfResponse.getRecords();
    assertEquals(1, responseBlock.getFieldReaders().size());
    assertEquals(rowCount, responseBlock.getRowCount());
    FieldReader fieldReader = responseBlock.getFieldReaders().get(0);
    for (int pos = 0; pos < rowCount; ++pos) {
        fieldReader.setPosition(pos);
        List<Integer> result = (List) UnitTestBlockUtils.getValue(fieldReader, pos);
        List<Integer> expected = handler.test_list_type(ImmutableList.of(pos + 100, pos + 200, pos + 300));
        assertArrayEquals(expected.toArray(), result.toArray());
    }
}
Also used : UserDefinedFunctionRequest(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest) Block(com.amazonaws.athena.connector.lambda.data.Block) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) UserDefinedFunctionResponse(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse) FieldReader(org.apache.arrow.vector.complex.reader.FieldReader) Test(org.junit.Test)

Example 14 with UserDefinedFunctionResponse

use of com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse in project aws-athena-query-federation by awslabs.

the class UserDefinedFunctionHandler method processScalarFunction.

private UserDefinedFunctionResponse processScalarFunction(BlockAllocator allocator, UserDefinedFunctionRequest req) throws Exception {
    Method udfMethod = extractScalarFunctionMethod(req);
    Block inputRecords = req.getInputRecords();
    Schema outputSchema = req.getOutputSchema();
    Block outputRecords = processRows(allocator, udfMethod, inputRecords, outputSchema);
    return new UserDefinedFunctionResponse(outputRecords, udfMethod.getName());
}
Also used : Schema(org.apache.arrow.vector.types.pojo.Schema) Block(com.amazonaws.athena.connector.lambda.data.Block) Method(java.lang.reflect.Method) UserDefinedFunctionResponse(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse)

Aggregations

UserDefinedFunctionResponse (com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse)14 Block (com.amazonaws.athena.connector.lambda.data.Block)10 Test (org.junit.Test)10 FieldReader (org.apache.arrow.vector.complex.reader.FieldReader)8 UserDefinedFunctionRequest (com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest)6 Schema (org.apache.arrow.vector.types.pojo.Schema)6 ArrowValueProjector (com.amazonaws.athena.connector.lambda.data.projectors.ArrowValueProjector)4 FederationResponse (com.amazonaws.athena.connector.lambda.request.FederationResponse)2 ByteArrayInputStream (java.io.ByteArrayInputStream)2 BigIntVector (org.apache.arrow.vector.BigIntVector)2 FieldVector (org.apache.arrow.vector.FieldVector)2 TypedSerDeTest (com.amazonaws.athena.connector.lambda.serde.TypedSerDeTest)1 ImmutableList (com.google.common.collect.ImmutableList)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 ByteArrayOutputStream (java.io.ByteArrayOutputStream)1 InputStream (java.io.InputStream)1 Method (java.lang.reflect.Method)1 HashMap (java.util.HashMap)1 List (java.util.List)1 Map (java.util.Map)1