Search in sources :

Example 1 with IndexedRecordConverter

use of org.talend.daikon.avro.converter.IndexedRecordConverter in project components by Talend.

the class JmsOutputPTransformRuntime method expand.

@Override
public PDone expand(PCollection<Object> objectPCollection) {
    // TODO remove this method from PCollection<Object> to PCollection<IndexedRecord>, as the incoming type always PCollection<IndexedRecord>
    PCollection<IndexedRecord> indexedCollection = objectPCollection.apply("ExtractIndexedRecord", ParDo.of(new DoFn<Object, IndexedRecord>() {

        IndexedRecordConverter converter;

        @DoFn.ProcessElement
        public void processElement(ProcessContext c) throws Exception {
            if (c.element() == null) {
                return;
            }
            if (converter == null) {
                converter = new AvroRegistry().createIndexedRecordConverter(c.element().getClass());
            }
            c.output((IndexedRecord) converter.convertToAvro(c.element()));
        }
    }));
    indexedCollection.setCoder(LazyAvroCoder.of());
    PCollection<String> jmsCollection = indexedCollection.apply("IndexedRecordToJmsRecord", ParDo.of(new DoFn<IndexedRecord, String>() {

        @DoFn.ProcessElement
        public void processElement(ProcessContext c) throws Exception {
            c.output(c.element().get(0).toString());
        }
    }));
    datastoreRuntime = new JmsDatastoreRuntime();
    datastoreRuntime.initialize(null, properties.datasetRef.getReference().getDatastoreProperties());
    if (messageType.equals(JmsMessageType.QUEUE)) {
        return jmsCollection.apply(JmsIO.write().withConnectionFactory(datastoreRuntime.getConnectionFactory()).withQueue(properties.datasetRef.getReference().queueTopicName.getValue()));
    } else if (messageType.equals(JmsMessageType.TOPIC)) {
        return jmsCollection.apply(JmsIO.write().withConnectionFactory(datastoreRuntime.getConnectionFactory()).withTopic(properties.datasetRef.getReference().queueTopicName.getValue()));
    } else {
        throw new TalendRuntimeException(CommonErrorCodes.UNEXPECTED_ARGUMENT);
    }
}
Also used : TalendRuntimeException(org.talend.daikon.exception.TalendRuntimeException) DoFn(org.apache.beam.sdk.transforms.DoFn) AvroRegistry(org.talend.daikon.avro.AvroRegistry) IndexedRecord(org.apache.avro.generic.IndexedRecord) IndexedRecordConverter(org.talend.daikon.avro.converter.IndexedRecordConverter)

Aggregations

IndexedRecord (org.apache.avro.generic.IndexedRecord)1 DoFn (org.apache.beam.sdk.transforms.DoFn)1 AvroRegistry (org.talend.daikon.avro.AvroRegistry)1 IndexedRecordConverter (org.talend.daikon.avro.converter.IndexedRecordConverter)1 TalendRuntimeException (org.talend.daikon.exception.TalendRuntimeException)1