Examples with BlockSpiller com.amazonaws.athena.connector.lambda.data.BlockSpiller used on opensource projects

Example 16 with BlockSpiller

use of com.amazonaws.athena.connector.lambda.data.BlockSpiller in project foundry-athena-query-federation-connector by palantir.

the class PartitionValueRoundTripTests method partitionValueRoundTrip.

@ParameterizedTest
@MethodSource
void partitionValueRoundTrip(ArrowType arrowType, PartitionValue partitionValue, FilterValue filterValue) {
    BlockSpiller blockSpiller = new InMemorySingleBlockSpiller(new Schema(ImmutableList.of(new Field(FIELD_NAME, FieldType.nullable(arrowType), ImmutableList.of()))), ConstraintEvaluator.emptyEvaluator());
    blockSpiller.writeRows((block, _rowIdx) -> {
        partitionValue.accept(new PartitionValueWriter(block, FIELD_NAME, 0));
        return 1;
    });
    FieldReader fieldReader = blockSpiller.getBlock().getFieldReader(FIELD_NAME);
    assertThat(arrowType.accept(new FilterValueReader(fieldReader))).isEqualTo(filterValue);
    blockSpiller.close();
}

Example 17 with BlockSpiller

use of com.amazonaws.athena.connector.lambda.data.BlockSpiller in project aws-athena-query-federation by awslabs.

the class ThrottlingInvokerTest method invokeWithThrottleNoSpill.

@Test(expected = FederationThrottleException.class)
public void invokeWithThrottleNoSpill() throws TimeoutException {
    BlockSpiller spiller = mock(BlockSpiller.class);
    ThrottlingInvoker invoker = ThrottlingInvoker.newBuilder().withDecrease(0.5).withIncrease(10).withInitialDelayMs(10).withMaxDelayMs(500).withFilter((Exception ex) -> ex instanceof RuntimeException).withSpiller(spiller).build();
    when(spiller.spilled()).thenReturn(false);
    invoker.invoke(() -> {
        throw new RuntimeException();
    }, 2_000);
}

Example 18 with BlockSpiller

use of com.amazonaws.athena.connector.lambda.data.BlockSpiller in project aws-athena-query-federation by awslabs.

the class HiveMuxRecordHandlerTest method readWithConstraint.

@Test
public void readWithConstraint() throws SQLException {
    BlockSpiller blockSpiller = Mockito.mock(BlockSpiller.class);
    ReadRecordsRequest readRecordsRequest = Mockito.mock(ReadRecordsRequest.class);
    Mockito.when(readRecordsRequest.getCatalogName()).thenReturn("recordHive");
    this.jdbcRecordHandler.readWithConstraint(blockSpiller, readRecordsRequest, queryStatusChecker);
    Mockito.verify(this.hiveRecordHandler, Mockito.times(1)).readWithConstraint(Mockito.eq(blockSpiller), Mockito.eq(readRecordsRequest), Mockito.eq(queryStatusChecker));
}

Example 19 with BlockSpiller

use of com.amazonaws.athena.connector.lambda.data.BlockSpiller in project aws-athena-query-federation by awslabs.

the class HiveMuxRecordHandlerTest method readWithConstraintWithUnsupportedCatalog.

@Test(expected = RuntimeException.class)
public void readWithConstraintWithUnsupportedCatalog() throws SQLException {
    BlockSpiller blockSpiller = Mockito.mock(BlockSpiller.class);
    ReadRecordsRequest readRecordsRequest = Mockito.mock(ReadRecordsRequest.class);
    Mockito.when(readRecordsRequest.getCatalogName()).thenReturn("unsupportedCatalog");
    this.jdbcRecordHandler.readWithConstraint(blockSpiller, readRecordsRequest, queryStatusChecker);
}

Example 20 with BlockSpiller

use of com.amazonaws.athena.connector.lambda.data.BlockSpiller in project aws-athena-query-federation by awslabs.

the class MySqlMuxJdbcRecordHandlerTest method readWithConstraintWithUnsupportedCatalog.

@Test(expected = RuntimeException.class)
public void readWithConstraintWithUnsupportedCatalog() {
    BlockSpiller blockSpiller = Mockito.mock(BlockSpiller.class);
    ReadRecordsRequest readRecordsRequest = Mockito.mock(ReadRecordsRequest.class);
    Mockito.when(readRecordsRequest.getCatalogName()).thenReturn("unsupportedCatalog");
    this.jdbcRecordHandler.readWithConstraint(blockSpiller, readRecordsRequest, queryStatusChecker);
}