Search in sources :

Example 1 with SourceTask

use of org.apache.kafka.connect.source.SourceTask in project kafka by apache.

the class Worker method buildWorkerTask.

private WorkerTask buildWorkerTask(ConnectorConfig connConfig, ConnectorTaskId id, Task task, TaskStatus.Listener statusListener, TargetState initialState, Converter keyConverter, Converter valueConverter) {
    // Decide which type of worker task we need based on the type of task.
    if (task instanceof SourceTask) {
        TransformationChain<SourceRecord> transformationChain = new TransformationChain<>(connConfig.<SourceRecord>transformations());
        OffsetStorageReader offsetReader = new OffsetStorageReaderImpl(offsetBackingStore, id.connector(), internalKeyConverter, internalValueConverter);
        OffsetStorageWriter offsetWriter = new OffsetStorageWriter(offsetBackingStore, id.connector(), internalKeyConverter, internalValueConverter);
        KafkaProducer<byte[], byte[]> producer = new KafkaProducer<>(producerProps);
        return new WorkerSourceTask(id, (SourceTask) task, statusListener, initialState, keyConverter, valueConverter, transformationChain, producer, offsetReader, offsetWriter, config, time);
    } else if (task instanceof SinkTask) {
        TransformationChain<SinkRecord> transformationChain = new TransformationChain<>(connConfig.<SinkRecord>transformations());
        return new WorkerSinkTask(id, (SinkTask) task, statusListener, initialState, config, keyConverter, valueConverter, transformationChain, time);
    } else {
        log.error("Tasks must be a subclass of either SourceTask or SinkTask", task);
        throw new ConnectException("Tasks must be a subclass of either SourceTask or SinkTask");
    }
}
Also used : OffsetStorageWriter(org.apache.kafka.connect.storage.OffsetStorageWriter) KafkaProducer(org.apache.kafka.clients.producer.KafkaProducer) SinkTask(org.apache.kafka.connect.sink.SinkTask) SinkRecord(org.apache.kafka.connect.sink.SinkRecord) SourceRecord(org.apache.kafka.connect.source.SourceRecord) OffsetStorageReaderImpl(org.apache.kafka.connect.storage.OffsetStorageReaderImpl) SourceTask(org.apache.kafka.connect.source.SourceTask) OffsetStorageReader(org.apache.kafka.connect.storage.OffsetStorageReader) ConnectException(org.apache.kafka.connect.errors.ConnectException)

Aggregations

KafkaProducer (org.apache.kafka.clients.producer.KafkaProducer)1 ConnectException (org.apache.kafka.connect.errors.ConnectException)1 SinkRecord (org.apache.kafka.connect.sink.SinkRecord)1 SinkTask (org.apache.kafka.connect.sink.SinkTask)1 SourceRecord (org.apache.kafka.connect.source.SourceRecord)1 SourceTask (org.apache.kafka.connect.source.SourceTask)1 OffsetStorageReader (org.apache.kafka.connect.storage.OffsetStorageReader)1 OffsetStorageReaderImpl (org.apache.kafka.connect.storage.OffsetStorageReaderImpl)1 OffsetStorageWriter (org.apache.kafka.connect.storage.OffsetStorageWriter)1