Example 6 with ContinuousFileReaderOperator
use of org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator in project flink by apache.
the class ContinuousFileProcessingRescalingTest method getTestHarness.
private OneInputStreamOperatorTestHarness<TimestampedFileInputSplit, String> getTestHarness(BlockingFileInputFormat format, int noOfTasks, int taksIdx) throws Exception {
ContinuousFileReaderOperator<String> reader = new ContinuousFileReaderOperator<>(format);
reader.setOutputType(TypeExtractor.getInputFormatTypes(format), new ExecutionConfig());
OneInputStreamOperatorTestHarness<TimestampedFileInputSplit, String> testHarness = new OneInputStreamOperatorTestHarness<>(reader, 10, noOfTasks, taksIdx);
testHarness.setTimeCharacteristic(TimeCharacteristic.EventTime);
return testHarness;
}
Also used :
TimestampedFileInputSplit(org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit)
ContinuousFileReaderOperator(org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator)
ExecutionConfig(org.apache.flink.api.common.ExecutionConfig)
OneInputStreamOperatorTestHarness(org.apache.flink.streaming.util.OneInputStreamOperatorTestHarness)
Aggregations
ContinuousFileReaderOperator (org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator)6
TimestampedFileInputSplit (org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit)6
Path (org.apache.flink.core.fs.Path)5
OneInputStreamOperatorTestHarness (org.apache.flink.streaming.util.OneInputStreamOperatorTestHarness)5
ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)4
FileInputSplit (org.apache.flink.core.fs.FileInputSplit)4
StreamRecord (org.apache.flink.streaming.runtime.streamrecord.StreamRecord)4
Test (org.junit.Test)4
TextInputFormat (org.apache.flink.api.java.io.TextInputFormat)3
ArrayList (java.util.ArrayList)2
HashMap (java.util.HashMap)2
HashSet (java.util.HashSet)2
List (java.util.List)2
OneShotLatch (org.apache.flink.core.testutils.OneShotLatch)2
Watermark (org.apache.flink.streaming.api.watermark.Watermark)2
IOException (java.io.IOException)1
StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)1
ContinuousFileMonitoringFunction (org.apache.flink.streaming.api.functions.source.ContinuousFileMonitoringFunction)1
OperatorStateHandles (org.apache.flink.streaming.runtime.tasks.OperatorStateHandles)1
