Search in sources :

Example 1 with AutoChunkProcessor

use of org.talend.sdk.component.runtime.di.AutoChunkProcessor in project component-runtime by Talend.

the class DIBatchSimulationTest method doDi.

private void doDi(final ComponentManager manager, final Collection<Object> sourceData, final Collection<Object> processorData, final Optional<Processor> proc, final Optional<Mapper> mapper) {
    final Map<String, Object> globalMap = new HashMap<>();
    try {
        final Processor processor = proc.orElseThrow(() -> new IllegalStateException("scanning failed"));
        JobStateAware.init(processor, globalMap);
        final Jsonb jsonbProcessor = Jsonb.class.cast(manager.findPlugin(processor.plugin()).get().get(ComponentManager.AllServices.class).getServices().get(Jsonb.class));
        final AutoChunkProcessor processorProcessor = new AutoChunkProcessor(100, processor);
        processorProcessor.start();
        globalMap.put("processorProcessor", processorProcessor);
        final InputsHandler inputsHandlerProcessor = new InputsHandler(jsonbProcessor);
        inputsHandlerProcessor.addConnection("FLOW", row1Struct.class);
        final OutputsHandler outputHandlerProcessor = new OutputsHandler(jsonbProcessor);
        final InputFactory inputsProcessor = inputsHandlerProcessor.asInputFactory();
        final OutputFactory outputsProcessor = outputHandlerProcessor.asOutputFactory();
        final Mapper tempMapperMapper = mapper.orElseThrow(() -> new IllegalStateException("scanning failed"));
        JobStateAware.init(tempMapperMapper, globalMap);
        doRun(manager, sourceData, processorData, globalMap, processorProcessor, inputsHandlerProcessor, outputHandlerProcessor, inputsProcessor, outputsProcessor, tempMapperMapper);
    } finally {
        doClose(globalMap);
    }
}
Also used : InputFactory(org.talend.sdk.component.runtime.output.InputFactory) AutoChunkProcessor(org.talend.sdk.component.runtime.di.AutoChunkProcessor) Processor(org.talend.sdk.component.runtime.output.Processor) HashMap(java.util.HashMap) OutputsHandler(org.talend.sdk.component.runtime.di.OutputsHandler) ToString(lombok.ToString) PartitionMapper(org.talend.sdk.component.api.input.PartitionMapper) ChainedMapper(org.talend.sdk.component.runtime.manager.chain.ChainedMapper) Mapper(org.talend.sdk.component.runtime.input.Mapper) Jsonb(javax.json.bind.Jsonb) AutoChunkProcessor(org.talend.sdk.component.runtime.di.AutoChunkProcessor) JsonObject(javax.json.JsonObject) OutputFactory(org.talend.sdk.component.runtime.output.OutputFactory) InputsHandler(org.talend.sdk.component.runtime.di.InputsHandler)

Example 2 with AutoChunkProcessor

use of org.talend.sdk.component.runtime.di.AutoChunkProcessor in project component-runtime by Talend.

the class DIBatchSimulationTest method doClose.

private void doClose(final Map<String, Object> globalMap) {
    final Mapper mapperMapper = Mapper.class.cast(globalMap.remove("mapperMapper"));
    final Input inputMapper = Input.class.cast(globalMap.remove("inputMapper"));
    try {
        if (inputMapper != null) {
            inputMapper.stop();
        }
    } catch (final RuntimeException re) {
        fail(re.getMessage());
    } finally {
        try {
            if (mapperMapper != null) {
                mapperMapper.stop();
            }
        } catch (final RuntimeException re) {
            fail(re.getMessage());
        }
    }
    final AutoChunkProcessor processorProcessor = AutoChunkProcessor.class.cast(globalMap.remove("processorProcessor"));
    try {
        if (processorProcessor != null) {
            processorProcessor.stop();
        }
    } catch (final RuntimeException re) {
        fail(re.getMessage());
    }
}
Also used : PartitionMapper(org.talend.sdk.component.api.input.PartitionMapper) ChainedMapper(org.talend.sdk.component.runtime.manager.chain.ChainedMapper) Mapper(org.talend.sdk.component.runtime.input.Mapper) Input(org.talend.sdk.component.runtime.input.Input) AutoChunkProcessor(org.talend.sdk.component.runtime.di.AutoChunkProcessor)

Aggregations

PartitionMapper (org.talend.sdk.component.api.input.PartitionMapper)2 AutoChunkProcessor (org.talend.sdk.component.runtime.di.AutoChunkProcessor)2 Mapper (org.talend.sdk.component.runtime.input.Mapper)2 ChainedMapper (org.talend.sdk.component.runtime.manager.chain.ChainedMapper)2 HashMap (java.util.HashMap)1 JsonObject (javax.json.JsonObject)1 Jsonb (javax.json.bind.Jsonb)1 ToString (lombok.ToString)1 InputsHandler (org.talend.sdk.component.runtime.di.InputsHandler)1 OutputsHandler (org.talend.sdk.component.runtime.di.OutputsHandler)1 Input (org.talend.sdk.component.runtime.input.Input)1 InputFactory (org.talend.sdk.component.runtime.output.InputFactory)1 OutputFactory (org.talend.sdk.component.runtime.output.OutputFactory)1 Processor (org.talend.sdk.component.runtime.output.Processor)1