Examples with TestingSplitEnumeratorContext org.apache.flink.connector.testutils.source.reader.TestingSplitEnumeratorContext used on opensource projects

Example 1 with TestingSplitEnumeratorContext

use of org.apache.flink.connector.testutils.source.reader.TestingSplitEnumeratorContext in project flink by apache.

the class StaticFileSplitEnumeratorTest method testSplitRequestForRegisteredReader.

@Test
public void testSplitRequestForRegisteredReader() throws Exception {
    final TestingSplitEnumeratorContext<FileSourceSplit> context = new TestingSplitEnumeratorContext<>(4);
    final FileSourceSplit split = createRandomSplit();
    final StaticFileSplitEnumerator enumerator = createEnumerator(context, split);
    context.registerReader(3, "somehost");
    enumerator.addReader(3);
    enumerator.handleSplitRequest(3, "somehost");
    assertThat(enumerator.snapshotState(1L).getSplits(), empty());
    assertThat(context.getSplitAssignments().get(3).getAssignedSplits(), contains(split));
}

Example 2 with TestingSplitEnumeratorContext

use of org.apache.flink.connector.testutils.source.reader.TestingSplitEnumeratorContext in project flink by apache.

the class ContinuousFileSplitEnumeratorTest method testDiscoverSplitWhenNoReaderRegistered.

@Test
public void testDiscoverSplitWhenNoReaderRegistered() throws Exception {
    final TestingFileEnumerator fileEnumerator = new TestingFileEnumerator();
    final TestingSplitEnumeratorContext<FileSourceSplit> context = new TestingSplitEnumeratorContext<>(4);
    final ContinuousFileSplitEnumerator enumerator = createEnumerator(fileEnumerator, context);
    // make one split available and trigger the periodic discovery
    final FileSourceSplit split = createRandomSplit();
    fileEnumerator.addSplits(split);
    context.triggerAllActions();
    assertThat(enumerator.snapshotState(1L).getSplits(), contains(split));
}

Example 3 with TestingSplitEnumeratorContext

use of org.apache.flink.connector.testutils.source.reader.TestingSplitEnumeratorContext in project flink by apache.

the class ContinuousFileSplitEnumeratorTest method testDiscoverWhenReaderRegistered.

@Test
public void testDiscoverWhenReaderRegistered() throws Exception {
    final TestingFileEnumerator fileEnumerator = new TestingFileEnumerator();
    final TestingSplitEnumeratorContext<FileSourceSplit> context = new TestingSplitEnumeratorContext<>(4);
    final ContinuousFileSplitEnumerator enumerator = createEnumerator(fileEnumerator, context);
    // register one reader, and let it request a split
    context.registerReader(2, "localhost");
    enumerator.addReader(2);
    enumerator.handleSplitRequest(2, "localhost");
    // make one split available and trigger the periodic discovery
    final FileSourceSplit split = createRandomSplit();
    fileEnumerator.addSplits(split);
    context.triggerAllActions();
    assertThat(enumerator.snapshotState(1L).getSplits(), empty());
    assertThat(context.getSplitAssignments().get(2).getAssignedSplits(), contains(split));
}

Example 4 with TestingSplitEnumeratorContext

use of org.apache.flink.connector.testutils.source.reader.TestingSplitEnumeratorContext in project flink by apache.

the class ContinuousFileSplitEnumeratorTest method testRequestingReaderUnavailableWhenSplitDiscovered.

@Test
public void testRequestingReaderUnavailableWhenSplitDiscovered() throws Exception {
    final TestingFileEnumerator fileEnumerator = new TestingFileEnumerator();
    final TestingSplitEnumeratorContext<FileSourceSplit> context = new TestingSplitEnumeratorContext<>(4);
    final ContinuousFileSplitEnumerator enumerator = createEnumerator(fileEnumerator, context);
    // register one reader, and let it request a split
    context.registerReader(2, "localhost");
    enumerator.addReader(2);
    enumerator.handleSplitRequest(2, "localhost");
    // remove the reader (like in a failure)
    context.registeredReaders().remove(2);
    // make one split available and trigger the periodic discovery
    final FileSourceSplit split = createRandomSplit();
    fileEnumerator.addSplits(split);
    context.triggerAllActions();
    assertFalse(context.getSplitAssignments().containsKey(2));
    assertThat(enumerator.snapshotState(1L).getSplits(), contains(split));
}

Example 5 with TestingSplitEnumeratorContext

use of org.apache.flink.connector.testutils.source.reader.TestingSplitEnumeratorContext in project flink by apache.

the class StaticFileSplitEnumeratorTest method testSplitRequestForNonRegisteredReader.

@Test
public void testSplitRequestForNonRegisteredReader() throws Exception {
    final TestingSplitEnumeratorContext<FileSourceSplit> context = new TestingSplitEnumeratorContext<>(4);
    final FileSourceSplit split = createRandomSplit();
    final StaticFileSplitEnumerator enumerator = createEnumerator(context, split);
    enumerator.handleSplitRequest(3, "somehost");
    assertFalse(context.getSplitAssignments().containsKey(3));
    assertThat(enumerator.snapshotState(1L).getSplits(), contains(split));
}