Search in sources :

Example 1 with PipelineEntity

use of org.graylog2.contentpacks.model.entities.PipelineEntity in project graylog2-server by Graylog2.

the class PipelineFacade method exportNativeEntity.

@VisibleForTesting
Entity exportNativeEntity(PipelineDao pipelineDao, EntityDescriptorIds entityDescriptorIds) {
    final Set<ValueReference> connectedStreams = connectedStreams(pipelineDao.id(), entityDescriptorIds);
    final PipelineEntity pipelineEntity = PipelineEntity.create(ValueReference.of(pipelineDao.title()), ValueReference.of(pipelineDao.description()), ValueReference.of(pipelineDao.source()), connectedStreams);
    final JsonNode data = objectMapper.convertValue(pipelineEntity, JsonNode.class);
    return EntityV1.builder().id(ModelId.of(entityDescriptorIds.getOrThrow(pipelineDao.id(), ModelTypes.PIPELINE_V1))).type(ModelTypes.PIPELINE_V1).data(data).build();
}
Also used : PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) JsonNode(com.fasterxml.jackson.databind.JsonNode) ValueReference(org.graylog2.contentpacks.model.entities.references.ValueReference) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 2 with PipelineEntity

use of org.graylog2.contentpacks.model.entities.PipelineEntity in project graylog2-server by Graylog2.

the class PipelineFacade method resolveForInstallation.

private Graph<Entity> resolveForInstallation(EntityV1 entity, Map<String, ValueReference> parameters, Map<EntityDescriptor, Entity> entities) {
    final MutableGraph<Entity> mutableGraph = GraphBuilder.directed().build();
    mutableGraph.addNode(entity);
    final PipelineEntity pipelineEntity = objectMapper.convertValue(entity.data(), PipelineEntity.class);
    final String source = pipelineEntity.source().asString(parameters);
    final Collection<String> referencedRules = referencedRules(source);
    referencedRules.stream().map(ModelId::of).map(modelId -> EntityDescriptor.create(modelId, ModelTypes.PIPELINE_RULE_V1)).map(entities::get).filter(Objects::nonNull).forEach(ruleEntity -> mutableGraph.putEdge(entity, ruleEntity));
    pipelineEntity.connectedStreams().stream().map(valueReference -> valueReference.asString(parameters)).map(ModelId::of).map(modelId -> EntityDescriptor.create(modelId, ModelTypes.STREAM_V1)).map(entities::get).filter(Objects::nonNull).forEach(streamEntity -> mutableGraph.putEdge(entity, streamEntity));
    return ImmutableGraph.copyOf(mutableGraph);
}
Also used : PipelineConnections(org.graylog.plugins.pipelineprocessor.rest.PipelineConnections) EntityDescriptorIds(org.graylog2.contentpacks.EntityDescriptorIds) RuleDao(org.graylog.plugins.pipelineprocessor.db.RuleDao) RuleService(org.graylog.plugins.pipelineprocessor.db.RuleService) ImmutableGraph(com.google.common.graph.ImmutableGraph) NativeEntity(org.graylog2.contentpacks.model.entities.NativeEntity) Tools(org.graylog2.plugin.Tools) LoggerFactory(org.slf4j.LoggerFactory) Entity(org.graylog2.contentpacks.model.entities.Entity) Stage(org.graylog.plugins.pipelineprocessor.ast.Stage) ModelType(org.graylog2.contentpacks.model.ModelType) ValueReference(org.graylog2.contentpacks.model.entities.references.ValueReference) Inject(javax.inject.Inject) PipelineService(org.graylog.plugins.pipelineprocessor.db.PipelineService) EntityExcerpt(org.graylog2.contentpacks.model.entities.EntityExcerpt) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) PipelineStreamConnectionsService(org.graylog.plugins.pipelineprocessor.db.PipelineStreamConnectionsService) Map(java.util.Map) Objects.requireNonNull(java.util.Objects.requireNonNull) JsonNode(com.fasterxml.jackson.databind.JsonNode) PipelineRuleParser(org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser) NotFoundException(org.graylog2.database.NotFoundException) Pipeline(org.graylog.plugins.pipelineprocessor.ast.Pipeline) ImmutableSet(com.google.common.collect.ImmutableSet) Logger(org.slf4j.Logger) ModelId(org.graylog2.contentpacks.model.ModelId) MutableGraph(com.google.common.graph.MutableGraph) EntityDescriptor(org.graylog2.contentpacks.model.entities.EntityDescriptor) Collection(java.util.Collection) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DateTime(org.joda.time.DateTime) MissingNativeEntityException(org.graylog2.contentpacks.exceptions.MissingNativeEntityException) Set(java.util.Set) PipelineDao(org.graylog.plugins.pipelineprocessor.db.PipelineDao) Collectors(java.util.stream.Collectors) GraphBuilder(com.google.common.graph.GraphBuilder) EntityV1(org.graylog2.contentpacks.model.entities.EntityV1) Objects(java.util.Objects) Stream(org.graylog2.plugin.streams.Stream) StreamService(org.graylog2.streams.StreamService) Optional(java.util.Optional) VisibleForTesting(com.google.common.annotations.VisibleForTesting) NativeEntityDescriptor(org.graylog2.contentpacks.model.entities.NativeEntityDescriptor) Collections(java.util.Collections) Graph(com.google.common.graph.Graph) ModelTypes(org.graylog2.contentpacks.model.ModelTypes) NativeEntity(org.graylog2.contentpacks.model.entities.NativeEntity) Entity(org.graylog2.contentpacks.model.entities.Entity) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) ModelId(org.graylog2.contentpacks.model.ModelId)

Example 3 with PipelineEntity

use of org.graylog2.contentpacks.model.entities.PipelineEntity in project graylog2-server by Graylog2.

the class PipelineFacade method decode.

private NativeEntity<PipelineDao> decode(EntityV1 entity, Map<String, ValueReference> parameters, Map<EntityDescriptor, Object> nativeEntities) {
    final DateTime now = Tools.nowUTC();
    final PipelineEntity pipelineEntity = objectMapper.convertValue(entity.data(), PipelineEntity.class);
    final ValueReference description = pipelineEntity.description();
    final PipelineDao pipelineDao = PipelineDao.builder().title(pipelineEntity.title().asString(parameters)).description(description == null ? null : description.asString(parameters)).source(pipelineEntity.source().asString(parameters)).createdAt(now).modifiedAt(now).build();
    final PipelineDao savedPipelineDao = pipelineService.save(pipelineDao);
    final String pipelineId = requireNonNull(savedPipelineDao.id(), "Saved pipeline ID must not be null");
    final Set<EntityDescriptor> connectedStreamEntities = pipelineEntity.connectedStreams().stream().map(valueReference -> valueReference.asString(parameters)).map(streamId -> EntityDescriptor.create(streamId, ModelTypes.STREAM_V1)).collect(Collectors.toSet());
    final Set<Stream> connectedStreams = connectedStreams(connectedStreamEntities, nativeEntities);
    createPipelineConnections(pipelineId, connectedStreams);
    return NativeEntity.create(entity.id(), pipelineId, TYPE_V1, savedPipelineDao.title(), savedPipelineDao);
}
Also used : PipelineConnections(org.graylog.plugins.pipelineprocessor.rest.PipelineConnections) EntityDescriptorIds(org.graylog2.contentpacks.EntityDescriptorIds) RuleDao(org.graylog.plugins.pipelineprocessor.db.RuleDao) RuleService(org.graylog.plugins.pipelineprocessor.db.RuleService) ImmutableGraph(com.google.common.graph.ImmutableGraph) NativeEntity(org.graylog2.contentpacks.model.entities.NativeEntity) Tools(org.graylog2.plugin.Tools) LoggerFactory(org.slf4j.LoggerFactory) Entity(org.graylog2.contentpacks.model.entities.Entity) Stage(org.graylog.plugins.pipelineprocessor.ast.Stage) ModelType(org.graylog2.contentpacks.model.ModelType) ValueReference(org.graylog2.contentpacks.model.entities.references.ValueReference) Inject(javax.inject.Inject) PipelineService(org.graylog.plugins.pipelineprocessor.db.PipelineService) EntityExcerpt(org.graylog2.contentpacks.model.entities.EntityExcerpt) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) PipelineStreamConnectionsService(org.graylog.plugins.pipelineprocessor.db.PipelineStreamConnectionsService) Map(java.util.Map) Objects.requireNonNull(java.util.Objects.requireNonNull) JsonNode(com.fasterxml.jackson.databind.JsonNode) PipelineRuleParser(org.graylog.plugins.pipelineprocessor.parser.PipelineRuleParser) NotFoundException(org.graylog2.database.NotFoundException) Pipeline(org.graylog.plugins.pipelineprocessor.ast.Pipeline) ImmutableSet(com.google.common.collect.ImmutableSet) Logger(org.slf4j.Logger) ModelId(org.graylog2.contentpacks.model.ModelId) MutableGraph(com.google.common.graph.MutableGraph) EntityDescriptor(org.graylog2.contentpacks.model.entities.EntityDescriptor) Collection(java.util.Collection) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DateTime(org.joda.time.DateTime) MissingNativeEntityException(org.graylog2.contentpacks.exceptions.MissingNativeEntityException) Set(java.util.Set) PipelineDao(org.graylog.plugins.pipelineprocessor.db.PipelineDao) Collectors(java.util.stream.Collectors) GraphBuilder(com.google.common.graph.GraphBuilder) EntityV1(org.graylog2.contentpacks.model.entities.EntityV1) Objects(java.util.Objects) Stream(org.graylog2.plugin.streams.Stream) StreamService(org.graylog2.streams.StreamService) Optional(java.util.Optional) VisibleForTesting(com.google.common.annotations.VisibleForTesting) NativeEntityDescriptor(org.graylog2.contentpacks.model.entities.NativeEntityDescriptor) Collections(java.util.Collections) Graph(com.google.common.graph.Graph) ModelTypes(org.graylog2.contentpacks.model.ModelTypes) EntityDescriptor(org.graylog2.contentpacks.model.entities.EntityDescriptor) NativeEntityDescriptor(org.graylog2.contentpacks.model.entities.NativeEntityDescriptor) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) PipelineDao(org.graylog.plugins.pipelineprocessor.db.PipelineDao) Stream(org.graylog2.plugin.streams.Stream) DateTime(org.joda.time.DateTime) ValueReference(org.graylog2.contentpacks.model.entities.references.ValueReference)

Example 4 with PipelineEntity

use of org.graylog2.contentpacks.model.entities.PipelineEntity in project graylog2-server by Graylog2.

the class PipelineFacadeTest method collectEntity.

@Test
@MongoDBFixtures("PipelineFacadeTest/pipelines.json")
public void collectEntity() {
    final EntityDescriptor descriptor = EntityDescriptor.create("5a85c4854b900afd5d662be3", ModelTypes.PIPELINE_V1);
    final EntityDescriptor streamDescriptor = EntityDescriptor.create("5adf23894b900a0fdb4e517d", ModelTypes.STREAM_V1);
    final EntityDescriptorIds entityDescriptorIds = EntityDescriptorIds.of(descriptor, streamDescriptor);
    final Optional<Entity> collectedEntity = facade.exportEntity(descriptor, entityDescriptorIds);
    assertThat(collectedEntity).isPresent().containsInstanceOf(EntityV1.class);
    final EntityV1 entity = (EntityV1) collectedEntity.orElseThrow(AssertionError::new);
    assertThat(entity.id()).isEqualTo(ModelId.of(entityDescriptorIds.get(descriptor).orElse(null)));
    assertThat(entity.type()).isEqualTo(ModelTypes.PIPELINE_V1);
    final PipelineEntity pipelineEntity = objectMapper.convertValue(entity.data(), PipelineEntity.class);
    assertThat(pipelineEntity.title()).isEqualTo(ValueReference.of("Test"));
    assertThat(pipelineEntity.description()).isEqualTo(ValueReference.of("Description"));
    assertThat(pipelineEntity.source().asString(Collections.emptyMap())).startsWith("pipeline \"Test\"");
    assertThat(pipelineEntity.connectedStreams()).containsOnly(ValueReference.of(entityDescriptorIds.get(streamDescriptor).orElse(null)));
}
Also used : EntityV1(org.graylog2.contentpacks.model.entities.EntityV1) EntityDescriptor(org.graylog2.contentpacks.model.entities.EntityDescriptor) NativeEntity(org.graylog2.contentpacks.model.entities.NativeEntity) Entity(org.graylog2.contentpacks.model.entities.Entity) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) EntityDescriptorIds(org.graylog2.contentpacks.EntityDescriptorIds) MongoDBFixtures(org.graylog.testing.mongodb.MongoDBFixtures) Test(org.junit.Test)

Example 5 with PipelineEntity

use of org.graylog2.contentpacks.model.entities.PipelineEntity in project graylog2-server by Graylog2.

the class PipelineFacadeTest method exportNativeEntity.

@Test
@MongoDBFixtures("PipelineFacadeTest/pipelines.json")
public void exportNativeEntity() {
    final EntityDescriptor descriptor = EntityDescriptor.create("5a85c4854b900afd5d662be3", ModelTypes.PIPELINE_V1);
    final EntityDescriptor streamDescriptor = EntityDescriptor.create("5adf23894b900a0fdb4e517d", ModelTypes.STREAM_V1);
    final EntityDescriptorIds entityDescriptorIds = EntityDescriptorIds.of(descriptor, streamDescriptor);
    final Entity entity = facade.exportEntity(descriptor, entityDescriptorIds).orElseThrow(AssertionError::new);
    assertThat(entity).isInstanceOf(EntityV1.class);
    assertThat(entity.id()).isEqualTo(ModelId.of(entityDescriptorIds.get(descriptor).orElse(null)));
    assertThat(entity.type()).isEqualTo(ModelTypes.PIPELINE_V1);
    final EntityV1 entityV1 = (EntityV1) entity;
    final PipelineEntity pipelineEntity = objectMapper.convertValue(entityV1.data(), PipelineEntity.class);
    assertThat(pipelineEntity.title()).isEqualTo(ValueReference.of("Test"));
    assertThat(pipelineEntity.description()).isEqualTo(ValueReference.of("Description"));
    assertThat(pipelineEntity.source().asString(Collections.emptyMap())).startsWith("pipeline \"Test\"");
    assertThat(pipelineEntity.connectedStreams()).containsOnly(ValueReference.of(entityDescriptorIds.get(streamDescriptor).orElse(null)));
}
Also used : EntityV1(org.graylog2.contentpacks.model.entities.EntityV1) EntityDescriptor(org.graylog2.contentpacks.model.entities.EntityDescriptor) NativeEntity(org.graylog2.contentpacks.model.entities.NativeEntity) Entity(org.graylog2.contentpacks.model.entities.Entity) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) PipelineEntity(org.graylog2.contentpacks.model.entities.PipelineEntity) EntityDescriptorIds(org.graylog2.contentpacks.EntityDescriptorIds) MongoDBFixtures(org.graylog.testing.mongodb.MongoDBFixtures) Test(org.junit.Test)

Aggregations

EntityDescriptor (org.graylog2.contentpacks.model.entities.EntityDescriptor)8 PipelineEntity (org.graylog2.contentpacks.model.entities.PipelineEntity)8 EntityDescriptorIds (org.graylog2.contentpacks.EntityDescriptorIds)7 Entity (org.graylog2.contentpacks.model.entities.Entity)7 EntityV1 (org.graylog2.contentpacks.model.entities.EntityV1)7 NativeEntity (org.graylog2.contentpacks.model.entities.NativeEntity)7 Test (org.junit.Test)6 MongoDBFixtures (org.graylog.testing.mongodb.MongoDBFixtures)5 JsonNode (com.fasterxml.jackson.databind.JsonNode)3 VisibleForTesting (com.google.common.annotations.VisibleForTesting)3 Pipeline (org.graylog.plugins.pipelineprocessor.ast.Pipeline)3 Stage (org.graylog.plugins.pipelineprocessor.ast.Stage)3 PipelineDao (org.graylog.plugins.pipelineprocessor.db.PipelineDao)3 RuleDao (org.graylog.plugins.pipelineprocessor.db.RuleDao)3 PipelineConnections (org.graylog.plugins.pipelineprocessor.rest.PipelineConnections)3 ValueReference (org.graylog2.contentpacks.model.entities.references.ValueReference)3 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)2 ImmutableSet (com.google.common.collect.ImmutableSet)2 Graph (com.google.common.graph.Graph)2 GraphBuilder (com.google.common.graph.GraphBuilder)2