Examples with Pipeline org.opensearch.ingest.Pipeline used on opensource projects

Example 6 with Pipeline

use of org.opensearch.ingest.Pipeline in project OpenSearch by opensearch-project.

the class SimulateExecutionServiceTests method testExecuteVerboseItemExceptionWithIgnoreFailure.

public void testExecuteVerboseItemExceptionWithIgnoreFailure() throws Exception {
    RuntimeException exception = new RuntimeException("processor failed");
    TestProcessor testProcessor = new TestProcessor("processor_0", "mock", null, exception);
    CompoundProcessor processor = new CompoundProcessor(true, Collections.singletonList(testProcessor), Collections.emptyList());
    Pipeline pipeline = new Pipeline("_id", "_description", version, new CompoundProcessor(processor));
    CountDownLatch latch = new CountDownLatch(1);
    AtomicReference<SimulateDocumentResult> holder = new AtomicReference<>();
    executionService.executeDocument(pipeline, ingestDocument, true, (r, e) -> {
        holder.set(r);
        latch.countDown();
    });
    latch.await();
    SimulateDocumentResult actualItemResponse = holder.get();
    assertThat(testProcessor.getInvokedCounter(), equalTo(1));
    assertThat(actualItemResponse, instanceOf(SimulateDocumentVerboseResult.class));
    SimulateDocumentVerboseResult simulateDocumentVerboseResult = (SimulateDocumentVerboseResult) actualItemResponse;
    assertThat(simulateDocumentVerboseResult.getProcessorResults().size(), equalTo(1));
    assertThat(simulateDocumentVerboseResult.getProcessorResults().get(0).getProcessorTag(), equalTo("processor_0"));
    assertThat(simulateDocumentVerboseResult.getProcessorResults().get(0).getFailure(), sameInstance(exception));
    assertVerboseResult(simulateDocumentVerboseResult.getProcessorResults().get(0), pipeline.getId(), ingestDocument);
}

Example 7 with Pipeline

use of org.opensearch.ingest.Pipeline in project geospatial by opensearch-project.

the class PipelineManager method createPipeline.

private void createPipeline(XContentBuilder pipelineRequestXContent, String pipelineID, StepListener<String> createPipelineStep) {
    final BytesReference pipelineRequestBodyBytes = BytesReference.bytes(pipelineRequestXContent);
    final PutPipelineRequest pipelineRequest = new PutPipelineRequest(pipelineID, pipelineRequestBodyBytes, XContentType.JSON);
    client.putPipeline(pipelineRequest, ActionListener.wrap(acknowledgedResponse -> {
        StringBuilder pipelineMessage = new StringBuilder("Created pipeline: ").append(pipelineID);
        LOGGER.info(pipelineMessage.toString());
        createPipelineStep.onResponse(pipelineID);
    }, putPipelineRequestFailedException -> {
        StringBuilder message = new StringBuilder("Failed to create the pipeline: ").append(pipelineID).append(" due to ").append(putPipelineRequestFailedException.getMessage());
        createPipelineStep.onFailure(new IllegalStateException(message.toString()));
    }));
}

Example 8 with Pipeline

use of org.opensearch.ingest.Pipeline in project geospatial by opensearch-project.

the class PipelineManager method delete.

/**
 * Delete pipleine based on given name, notify the status of action using {@link StepListener}
 * @param pipeline pipeline name to be deleted.
 * @param deletePipelineStep  notifies the status of this action
 * @param supplier Exception to be passed to the listener.
 */
public void delete(String pipeline, StepListener<Exception> deletePipelineStep, final Supplier<Exception> supplier) {
    final DeletePipelineRequest pipelineRequest = new DeletePipelineRequest(pipeline);
    client.deletePipeline(pipelineRequest, ActionListener.wrap(acknowledgedResponse -> {
        StringBuilder message = new StringBuilder("Deleted pipeline: ").append(pipeline);
        LOGGER.info(message.toString());
        deletePipelineStep.onResponse(supplier.get());
    }, deletePipelineRequestFailed -> {
        StringBuilder message = new StringBuilder("Failed to delete the pipeline: ").append(pipeline).append(" due to ").append(deletePipelineRequestFailed.getMessage());
        if (supplier.get() != null) {
            message.append("\n").append("Another exception occurred: ").append(supplier.get().getMessage());
        }
        deletePipelineStep.onFailure(new IllegalStateException(message.toString()));
    }));
}

Example 9 with Pipeline

use of org.opensearch.ingest.Pipeline in project OpenSearch by opensearch-project.

the class SimulatePipelineRequest method parseWithPipelineId.

static Parsed parseWithPipelineId(String pipelineId, Map<String, Object> config, boolean verbose, IngestService ingestService) {
    if (pipelineId == null) {
        throw new IllegalArgumentException("param [pipeline] is null");
    }
    Pipeline pipeline = ingestService.getPipeline(pipelineId);
    if (pipeline == null) {
        throw new IllegalArgumentException("pipeline [" + pipelineId + "] does not exist");
    }
    List<IngestDocument> ingestDocumentList = parseDocs(config);
    return new Parsed(pipeline, ingestDocumentList, verbose);
}

Example 10 with Pipeline

use of org.opensearch.ingest.Pipeline in project OpenSearch by opensearch-project.

the class SimulatePipelineRequest method parse.

static Parsed parse(Map<String, Object> config, boolean verbose, IngestService ingestService) throws Exception {
    Map<String, Object> pipelineConfig = ConfigurationUtils.readMap(null, null, config, Fields.PIPELINE);
    Pipeline pipeline = Pipeline.create(SIMULATED_PIPELINE_ID, pipelineConfig, ingestService.getProcessorFactories(), ingestService.getScriptService());
    List<IngestDocument> ingestDocumentList = parseDocs(config);
    return new Parsed(pipeline, ingestDocumentList, verbose);
}