Search in sources :

Example 6 with PipelineConfiguration

use of com.amazon.dataprepper.parser.model.PipelineConfiguration in project data-prepper by opensearch-project.

the class PipelineParser method parseConfiguration.

/**
 * Parses the configuration file into Pipeline
 */
public Map<String, Pipeline> parseConfiguration() {
    try {
        final Map<String, PipelineConfiguration> pipelineConfigurationMap = OBJECT_MAPPER.readValue(new File(pipelineConfigurationFileLocation), new TypeReference<Map<String, PipelineConfiguration>>() {
        });
        final List<String> allPipelineNames = PipelineConfigurationValidator.validateAndGetPipelineNames(pipelineConfigurationMap);
        // LinkedHashMap to preserve insertion order
        final Map<String, Pipeline> pipelineMap = new LinkedHashMap<>();
        pipelineConfigurationMap.forEach((pipelineName, configuration) -> configuration.updateCommonPipelineConfiguration(pipelineName));
        for (String pipelineName : allPipelineNames) {
            if (!pipelineMap.containsKey(pipelineName) && pipelineConfigurationMap.containsKey(pipelineName)) {
                buildPipelineFromConfiguration(pipelineName, pipelineConfigurationMap, pipelineMap);
            }
        }
        return pipelineMap;
    } catch (IOException e) {
        throw new ParseException(format("Failed to parse the configuration file %s", pipelineConfigurationFileLocation), e);
    }
}
Also used : PipelineConfiguration(com.amazon.dataprepper.parser.model.PipelineConfiguration) IOException(java.io.IOException) File(java.io.File) HashMap(java.util.HashMap) LinkedHashMap(java.util.LinkedHashMap) Map(java.util.Map) Pipeline(com.amazon.dataprepper.pipeline.Pipeline) LinkedHashMap(java.util.LinkedHashMap)

Aggregations

PipelineConfiguration (com.amazon.dataprepper.parser.model.PipelineConfiguration)6 PluginSetting (com.amazon.dataprepper.model.configuration.PluginSetting)5 Pipeline (com.amazon.dataprepper.pipeline.Pipeline)2 IOException (java.io.IOException)2 Buffer (com.amazon.dataprepper.model.buffer.Buffer)1 NoPluginFoundException (com.amazon.dataprepper.model.plugin.NoPluginFoundException)1 Sink (com.amazon.dataprepper.model.sink.Sink)1 Source (com.amazon.dataprepper.model.source.Source)1 File (java.io.File)1 HashMap (java.util.HashMap)1 HashSet (java.util.HashSet)1 LinkedHashMap (java.util.LinkedHashMap)1 List (java.util.List)1 Map (java.util.Map)1