Search in sources :

Example 6 with UserDefinedFunctionRequest

use of com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest in project aws-athena-query-federation by awslabs.

the class CompositeHandler method handleRequest.

/**
 * Handles routing the request to the appropriate Handler, either MetadataHandler or RecordHandler.
 *
 * @param allocator The BlockAllocator to use for Apache Arrow Resources.
 * @param rawReq The request object itself.
 * @param outputStream The OutputStream to which all responses should be written.
 * @param objectMapper The ObjectMapper that can be used for serializing responses.
 * @throws Exception
 * @note that PingRequests are routed to the MetadataHandler even though both MetadataHandler and RecordHandler
 * implemented PingRequest handling.
 */
public final void handleRequest(BlockAllocator allocator, FederationRequest rawReq, OutputStream outputStream, ObjectMapper objectMapper) throws Exception {
    if (rawReq instanceof PingRequest) {
        try (PingResponse response = metadataHandler.doPing((PingRequest) rawReq)) {
            assertNotNull(response);
            objectMapper.writeValue(outputStream, response);
        }
        return;
    }
    if (rawReq instanceof MetadataRequest) {
        metadataHandler.doHandleRequest(allocator, objectMapper, (MetadataRequest) rawReq, outputStream);
    } else if (rawReq instanceof RecordRequest) {
        recordHandler.doHandleRequest(allocator, objectMapper, (RecordRequest) rawReq, outputStream);
    } else if (udfhandler != null && rawReq instanceof UserDefinedFunctionRequest) {
        udfhandler.doHandleRequest(allocator, objectMapper, (UserDefinedFunctionRequest) rawReq, outputStream);
    } else {
        throw new IllegalArgumentException("Unknown request class " + rawReq.getClass());
    }
}
Also used : RecordRequest(com.amazonaws.athena.connector.lambda.records.RecordRequest) PingRequest(com.amazonaws.athena.connector.lambda.request.PingRequest) MetadataRequest(com.amazonaws.athena.connector.lambda.metadata.MetadataRequest) UserDefinedFunctionRequest(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest) PingResponse(com.amazonaws.athena.connector.lambda.request.PingResponse)

Example 7 with UserDefinedFunctionRequest

use of com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest in project aws-athena-query-federation by awslabs.

the class UserDefinedFunctionHandler method handleRequest.

@Override
public final void handleRequest(InputStream inputStream, OutputStream outputStream, Context context) {
    try (BlockAllocator allocator = new BlockAllocatorImpl()) {
        ObjectMapper objectMapper = VersionedObjectMapperFactory.create(allocator);
        try (FederationRequest rawRequest = objectMapper.readValue(inputStream, FederationRequest.class)) {
            if (rawRequest instanceof PingRequest) {
                try (PingResponse response = doPing((PingRequest) rawRequest)) {
                    assertNotNull(response);
                    objectMapper.writeValue(outputStream, response);
                }
                return;
            }
            if (!(rawRequest instanceof UserDefinedFunctionRequest)) {
                throw new RuntimeException("Expected a UserDefinedFunctionRequest but found " + rawRequest.getClass());
            }
            doHandleRequest(allocator, objectMapper, (UserDefinedFunctionRequest) rawRequest, outputStream);
        } catch (Exception ex) {
            throw (ex instanceof RuntimeException) ? (RuntimeException) ex : new RuntimeException(ex);
        }
    }
}
Also used : PingRequest(com.amazonaws.athena.connector.lambda.request.PingRequest) UserDefinedFunctionRequest(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest) BlockAllocatorImpl(com.amazonaws.athena.connector.lambda.data.BlockAllocatorImpl) BlockAllocator(com.amazonaws.athena.connector.lambda.data.BlockAllocator) PingResponse(com.amazonaws.athena.connector.lambda.request.PingResponse) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) InvocationTargetException(java.lang.reflect.InvocationTargetException) FederationRequest(com.amazonaws.athena.connector.lambda.request.FederationRequest)

Example 8 with UserDefinedFunctionRequest

use of com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest in project aws-athena-query-federation by awslabs.

the class UserDefinedFunctionHandlerTest method testInvocationWithStructType.

@Test
public void testInvocationWithStructType() throws Exception {
    int rowCount = 20;
    UserDefinedFunctionRequest udfRequest = createUDFRequest(rowCount, Map.class, "test_row_type", true, Map.class);
    UserDefinedFunctionResponse udfResponse = handler.processFunction(allocator, udfRequest);
    Block responseBlock = udfResponse.getRecords();
    assertEquals(1, responseBlock.getFieldReaders().size());
    assertEquals(rowCount, responseBlock.getRowCount());
    FieldReader fieldReader = responseBlock.getFieldReaders().get(0);
    for (int pos = 0; pos < rowCount; ++pos) {
        fieldReader.setPosition(pos);
        Map<String, Object> actual = (Map) UnitTestBlockUtils.getValue(fieldReader, pos);
        Map<String, Object> input = ImmutableMap.of("intVal", pos + 100, "doubleVal", pos + 200.2);
        Map<String, Object> expected = handler.test_row_type(input);
        for (Map.Entry<String, Object> entry : expected.entrySet()) {
            String key = entry.getKey();
            assertTrue(actual.containsKey(key));
            assertEquals(expected.get(key), actual.get(key));
        }
    }
}
Also used : UserDefinedFunctionRequest(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest) Block(com.amazonaws.athena.connector.lambda.data.Block) UserDefinedFunctionResponse(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse) FieldReader(org.apache.arrow.vector.complex.reader.FieldReader) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) Test(org.junit.Test)

Example 9 with UserDefinedFunctionRequest

use of com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest in project aws-athena-query-federation by awslabs.

the class UserDefinedFunctionHandlerTest method testInvocationWithListType.

@Test
public void testInvocationWithListType() throws Exception {
    int rowCount = 20;
    UserDefinedFunctionRequest udfRequest = createUDFRequest(rowCount, List.class, "test_list_type", true, List.class);
    UserDefinedFunctionResponse udfResponse = handler.processFunction(allocator, udfRequest);
    Block responseBlock = udfResponse.getRecords();
    assertEquals(1, responseBlock.getFieldReaders().size());
    assertEquals(rowCount, responseBlock.getRowCount());
    FieldReader fieldReader = responseBlock.getFieldReaders().get(0);
    for (int pos = 0; pos < rowCount; ++pos) {
        fieldReader.setPosition(pos);
        List<Integer> result = (List) UnitTestBlockUtils.getValue(fieldReader, pos);
        List<Integer> expected = handler.test_list_type(ImmutableList.of(pos + 100, pos + 200, pos + 300));
        assertArrayEquals(expected.toArray(), result.toArray());
    }
}
Also used : UserDefinedFunctionRequest(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest) Block(com.amazonaws.athena.connector.lambda.data.Block) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) UserDefinedFunctionResponse(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse) FieldReader(org.apache.arrow.vector.complex.reader.FieldReader) Test(org.junit.Test)

Example 10 with UserDefinedFunctionRequest

use of com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest in project aws-athena-query-federation by awslabs.

the class UserDefinedFunctionHandlerTest method createUDFRequest.

private UserDefinedFunctionRequest createUDFRequest(int rowCount, Class returnType, String methodName, boolean nonNullData, Class... argumentTypes) {
    Schema inputSchema = buildSchema(argumentTypes);
    Schema outputSchema = buildSchema(returnType);
    Block block = allocator.createBlock(inputSchema);
    block.setRowCount(rowCount);
    if (nonNullData) {
        writeData(block, rowCount);
    }
    return new UserDefinedFunctionRequest(null, block, outputSchema, methodName, SCALAR);
}
Also used : UserDefinedFunctionRequest(com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest) Schema(org.apache.arrow.vector.types.pojo.Schema) Block(com.amazonaws.athena.connector.lambda.data.Block)

Aggregations

UserDefinedFunctionRequest (com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionRequest)11 Block (com.amazonaws.athena.connector.lambda.data.Block)6 UserDefinedFunctionResponse (com.amazonaws.athena.connector.lambda.udf.UserDefinedFunctionResponse)6 Test (org.junit.Test)6 FieldReader (org.apache.arrow.vector.complex.reader.FieldReader)4 FederationRequest (com.amazonaws.athena.connector.lambda.request.FederationRequest)2 PingRequest (com.amazonaws.athena.connector.lambda.request.PingRequest)2 PingResponse (com.amazonaws.athena.connector.lambda.request.PingResponse)2 ByteArrayInputStream (java.io.ByteArrayInputStream)2 Schema (org.apache.arrow.vector.types.pojo.Schema)2 BlockAllocator (com.amazonaws.athena.connector.lambda.data.BlockAllocator)1 BlockAllocatorImpl (com.amazonaws.athena.connector.lambda.data.BlockAllocatorImpl)1 MetadataRequest (com.amazonaws.athena.connector.lambda.metadata.MetadataRequest)1 RecordRequest (com.amazonaws.athena.connector.lambda.records.RecordRequest)1 FederationResponse (com.amazonaws.athena.connector.lambda.request.FederationResponse)1 TypedSerDeTest (com.amazonaws.athena.connector.lambda.serde.TypedSerDeTest)1 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 ImmutableList (com.google.common.collect.ImmutableList)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 ByteArrayOutputStream (java.io.ByteArrayOutputStream)1