Search in sources :

Example 1 with PipelineConnections

use of org.graylog.plugins.pipelineprocessor.rest.PipelineConnections in project graylog2-server by Graylog2.

the class LegacyDefaultStreamMigration method doRun.

@Override
public void doRun() {
    try {
        final PipelineConnections defaultConnections = connectionsService.load(LEGACY_STREAM_ID);
        connectionsService.save(defaultConnections.toBuilder().streamId(Stream.DEFAULT_STREAM_ID).build());
        connectionsService.delete(LEGACY_STREAM_ID);
        clusterConfigService.write(LegacyDefaultStreamMigrated.create(true));
        LOG.info("Pipeline connections to legacy default streams migrated successfully.");
    } catch (NotFoundException e) {
        LOG.info("Legacy default stream has no connections, no migration needed.");
    }
}
Also used : PipelineConnections(org.graylog.plugins.pipelineprocessor.rest.PipelineConnections) NotFoundException(org.graylog2.database.NotFoundException)

Example 2 with PipelineConnections

use of org.graylog.plugins.pipelineprocessor.rest.PipelineConnections in project graylog2-server by Graylog2.

the class MongoDbPipelineStreamConnectionsService method save.

@Override
public PipelineConnections save(PipelineConnections connections) {
    PipelineConnections existingConnections = dbCollection.findOne(DBQuery.is("stream_id", connections.streamId()));
    if (existingConnections == null) {
        existingConnections = PipelineConnections.create(null, connections.streamId(), Collections.emptySet());
    }
    final PipelineConnections toSave = existingConnections.toBuilder().pipelineIds(connections.pipelineIds()).build();
    final WriteResult<PipelineConnections, String> save = dbCollection.save(toSave);
    final PipelineConnections savedConnections = save.getSavedObject();
    clusterBus.post(PipelineConnectionsChangedEvent.create(savedConnections.streamId(), savedConnections.pipelineIds()));
    return savedConnections;
}
Also used : PipelineConnections(org.graylog.plugins.pipelineprocessor.rest.PipelineConnections)

Example 3 with PipelineConnections

use of org.graylog.plugins.pipelineprocessor.rest.PipelineConnections in project graylog2-server by Graylog2.

the class MongoDbPipelineStreamConnectionsService method delete.

@Override
public void delete(String streamId) {
    try {
        final PipelineConnections connections = load(streamId);
        final Set<String> pipelineIds = connections.pipelineIds();
        dbCollection.removeById(connections.id());
        clusterBus.post(PipelineConnectionsChangedEvent.create(streamId, pipelineIds));
    } catch (NotFoundException e) {
        log.debug("No connections found for stream " + streamId);
    }
}
Also used : PipelineConnections(org.graylog.plugins.pipelineprocessor.rest.PipelineConnections) NotFoundException(org.graylog2.database.NotFoundException)

Example 4 with PipelineConnections

use of org.graylog.plugins.pipelineprocessor.rest.PipelineConnections in project graylog2-server by Graylog2.

the class PipelineFacade method resolveNativeEntity.

@Override
public Graph<EntityDescriptor> resolveNativeEntity(EntityDescriptor entityDescriptor) {
    final MutableGraph<EntityDescriptor> mutableGraph = GraphBuilder.directed().build();
    mutableGraph.addNode(entityDescriptor);
    final ModelId modelId = entityDescriptor.id();
    try {
        final PipelineDao pipelineDao = pipelineService.load(modelId.id());
        final String pipelineSource = pipelineDao.source();
        final Collection<String> referencedRules = referencedRules(pipelineSource);
        referencedRules.stream().map(ModelId::of).map(id -> EntityDescriptor.create(id, ModelTypes.PIPELINE_RULE_V1)).forEach(rule -> mutableGraph.putEdge(entityDescriptor, rule));
        final Set<PipelineConnections> pipelineConnections = connectionsService.loadByPipelineId(pipelineDao.id());
        pipelineConnections.stream().map(PipelineConnections::streamId).map(ModelId::of).map(id -> EntityDescriptor.create(id, ModelTypes.STREAM_V1)).forEach(stream -> mutableGraph.putEdge(entityDescriptor, stream));
    } catch (NotFoundException e) {
        LOG.debug("Couldn't find pipeline {}", entityDescriptor, e);
    }
    return ImmutableGraph.copyOf(mutableGraph);
}
Also used : PipelineConnections(org.graylog.plugins.pipelineprocessor.rest.PipelineConnections) EntityDescriptorIds(org.graylog2.contentpacks.EntityDescriptorIds) RuleDao(org.graylog.plugins.pipelineprocessor.db.RuleDao) RuleService(org.graylog.plugins.pipelineprocessor.db.RuleService) ImmutableGraph(com.google.common.graph.ImmutableGraph) NativeEntity(org.graylog2.contentpacks.model.entities.NativeEntity) Tools(org.graylog2.plugin.Tools) LoggerFactory(org.slf4j.LoggerFactory) Entity(org.graylog2.contentpacks.model.entities.Entity) Stage(org.graylog.plugins.pipelineprocessor.ast.Stage) ModelType(org.graylog2.contentpacks.model.ModelType) ValueReference(org.graylog2.contentpacks.model.entities.references.ValueReference) Inject(javax.inject.Inject) PipelineService(org.graylog.plugins.pipelineprocessor.db.PipelineService) EntityExcerpt(org.graylog2.contentpacks.model.entities.EntityExcerpt) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) PipelineStreamConnectionsService(org.graylog.plugins.pipelineprocessor.db.PipelineStreamConnectionsService) Map(java.util.Map) Objects.requireNonNull(java.util.Objects.requireNonNull) JsonNode(com.fasterxml.jackson.databind.JsonNode) PipelineRuleParser(org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser) NotFoundException(org.graylog2.database.NotFoundException) Pipeline(org.graylog.plugins.pipelineprocessor.ast.Pipeline) ImmutableSet(com.google.common.collect.ImmutableSet) Logger(org.slf4j.Logger) ModelId(org.graylog2.contentpacks.model.ModelId) MutableGraph(com.google.common.graph.MutableGraph) EntityDescriptor(org.graylog2.contentpacks.model.entities.EntityDescriptor) Collection(java.util.Collection) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DateTime(org.joda.time.DateTime) MissingNativeEntityException(org.graylog2.contentpacks.exceptions.MissingNativeEntityException) Set(java.util.Set) PipelineDao(org.graylog.plugins.pipelineprocessor.db.PipelineDao) Collectors(java.util.stream.Collectors) GraphBuilder(com.google.common.graph.GraphBuilder) EntityV1(org.graylog2.contentpacks.model.entities.EntityV1) Objects(java.util.Objects) Stream(org.graylog2.plugin.streams.Stream) StreamService(org.graylog2.streams.StreamService) Optional(java.util.Optional) VisibleForTesting(com.google.common.annotations.VisibleForTesting) NativeEntityDescriptor(org.graylog2.contentpacks.model.entities.NativeEntityDescriptor) Collections(java.util.Collections) Graph(com.google.common.graph.Graph) ModelTypes(org.graylog2.contentpacks.model.ModelTypes) EntityDescriptor(org.graylog2.contentpacks.model.entities.EntityDescriptor) NativeEntityDescriptor(org.graylog2.contentpacks.model.entities.NativeEntityDescriptor) PipelineConnections(org.graylog.plugins.pipelineprocessor.rest.PipelineConnections) PipelineDao(org.graylog.plugins.pipelineprocessor.db.PipelineDao) NotFoundException(org.graylog2.database.NotFoundException) ModelId(org.graylog2.contentpacks.model.ModelId)

Example 5 with PipelineConnections

use of org.graylog.plugins.pipelineprocessor.rest.PipelineConnections in project graylog2-server by Graylog2.

the class PipelineFacadeTest method exportEntity.

@Test
public void exportEntity() {
    final PipelineDao pipeline = PipelineDao.builder().id("pipeline-1234").title("title").description("description").source("pipeline \"Test\"\nstage 0 match either\nrule \"debug\"\nend").build();
    final PipelineConnections connections = PipelineConnections.create("id", "stream-1234", Collections.singleton("pipeline-1234"));
    connectionsService.save(connections);
    final EntityDescriptor descriptor = EntityDescriptor.create(pipeline.id(), ModelTypes.PIPELINE_V1);
    final EntityDescriptor streamDescriptor = EntityDescriptor.create("stream-1234", ModelTypes.STREAM_V1);
    final EntityDescriptorIds entityDescriptorIds = EntityDescriptorIds.of(descriptor, streamDescriptor);
    final Entity entity = facade.exportNativeEntity(pipeline, entityDescriptorIds);
    assertThat(entity).isInstanceOf(EntityV1.class);
    assertThat(entity.id()).isEqualTo(ModelId.of(entityDescriptorIds.get(descriptor).orElse(null)));
    assertThat(entity.type()).isEqualTo(ModelTypes.PIPELINE_V1);
    final EntityV1 entityV1 = (EntityV1) entity;
    final PipelineEntity pipelineEntity = objectMapper.convertValue(entityV1.data(), PipelineEntity.class);
    assertThat(pipelineEntity.title()).isEqualTo(ValueReference.of("title"));
    assertThat(pipelineEntity.description()).isEqualTo(ValueReference.of("description"));
    assertThat(pipelineEntity.source().asString(Collections.emptyMap())).startsWith("pipeline \"Test\"");
    assertThat(pipelineEntity.connectedStreams()).containsOnly(ValueReference.of(entityDescriptorIds.get(streamDescriptor).orElse(null)));
}
Also used : EntityV1(org.graylog2.contentpacks.model.entities.EntityV1) PipelineConnections(org.graylog.plugins.pipelineprocessor.rest.PipelineConnections) EntityDescriptor(org.graylog2.contentpacks.model.entities.EntityDescriptor) NativeEntity(org.graylog2.contentpacks.model.entities.NativeEntity) Entity(org.graylog2.contentpacks.model.entities.Entity) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) EntityDescriptorIds(org.graylog2.contentpacks.EntityDescriptorIds) PipelineDao(org.graylog.plugins.pipelineprocessor.db.PipelineDao) Test(org.junit.Test)

Aggregations

PipelineConnections (org.graylog.plugins.pipelineprocessor.rest.PipelineConnections)11 NotFoundException (org.graylog2.database.NotFoundException)6 Pipeline (org.graylog.plugins.pipelineprocessor.ast.Pipeline)3 PipelineDao (org.graylog.plugins.pipelineprocessor.db.PipelineDao)3 PipelineStreamConnectionsService (org.graylog.plugins.pipelineprocessor.db.PipelineStreamConnectionsService)3 PipelineRuleParser (org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser)3 EntityDescriptorIds (org.graylog2.contentpacks.EntityDescriptorIds)3 JsonNode (com.fasterxml.jackson.databind.JsonNode)2 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 ImmutableSet (com.google.common.collect.ImmutableSet)2 Graph (com.google.common.graph.Graph)2 GraphBuilder (com.google.common.graph.GraphBuilder)2 ImmutableGraph (com.google.common.graph.ImmutableGraph)2 MutableGraph (com.google.common.graph.MutableGraph)2 Collection (java.util.Collection)2 Collections (java.util.Collections)2 Map (java.util.Map)2 Objects (java.util.Objects)2 Objects.requireNonNull (java.util.Objects.requireNonNull)2