Examples with Output org.graylog2.plugin.streams.Output used on opensource projects

Example 6 with Output

use of org.graylog2.plugin.streams.Output in project graylog2-server by Graylog2.

the class StreamServiceImpl method load.

public Stream load(ObjectId id) throws NotFoundException {
    final DBObject o = get(StreamImpl.class, id);
    if (o == null) {
        throw new NotFoundException("Stream <" + id + "> not found!");
    }
    final List<StreamRule> streamRules = streamRuleService.loadForStreamId(id.toHexString());
    final Set<Output> outputs = loadOutputsForRawStream(o);
    @SuppressWarnings("unchecked") final Map<String, Object> fields = o.toMap();
    return new StreamImpl((ObjectId) o.get("_id"), fields, streamRules, outputs, getIndexSet(o));
}

Example 7 with Output

use of org.graylog2.plugin.streams.Output in project graylog2-server by Graylog2.

the class OutputResource method delete.

@DELETE
@Path("/{outputId}")
@Timed
@ApiOperation(value = "Delete output")
@RequiresPermissions(RestPermissions.OUTPUTS_TERMINATE)
@Produces(MediaType.APPLICATION_JSON)
@ApiResponses(value = { @ApiResponse(code = 404, message = "No such stream/output on this node.") })
@AuditEvent(type = AuditEventTypes.MESSAGE_OUTPUT_DELETE)
public void delete(@ApiParam(name = "outputId", value = "The id of the output that should be deleted", required = true) @PathParam("outputId") String outputId) throws org.graylog2.database.NotFoundException {
    checkPermission(RestPermissions.OUTPUTS_TERMINATE);
    final Output output = outputService.load(outputId);
    outputService.destroy(output);
}

Example 8 with Output

use of org.graylog2.plugin.streams.Output in project graylog2-server by Graylog2.

the class LdapResource method updateGroupMappingSettings.

@PUT
@RequiresPermissions(value = { RestPermissions.LDAPGROUPS_EDIT, RestPermissions.LDAP_EDIT }, logical = OR)
@ApiOperation(value = "Update the LDAP group to Graylog role mapping", notes = "Corresponds directly to the output of GET /system/ldap/settings/groups")
@Path("/settings/groups")
@Consumes(MediaType.APPLICATION_JSON)
@AuditEvent(type = AuditEventTypes.LDAP_GROUP_MAPPING_UPDATE)
public Response updateGroupMappingSettings(@ApiParam(name = "JSON body", required = true, value = "A hash in which the keys are the LDAP group names and values is the Graylog role name.") @NotNull Map<String, String> groupMapping) throws ValidationException {
    final LdapSettings ldapSettings = firstNonNull(ldapSettingsService.load(), ldapSettingsFactory.createEmpty());
    ldapSettings.setGroupMapping(groupMapping);
    ldapSettingsService.save(ldapSettings);
    return Response.noContent().build();
}

Example 9 with Output

use of org.graylog2.plugin.streams.Output in project graylog2-server by Graylog2.

the class OutputBufferProcessor method onEvent.

/**
     * Each message will be written to one or more outputs.
     * <p>
     * The default output is always being used for every message, but optionally the message can be routed to additional
     * outputs, currently based on the stream outputs that are configured in the system.
     * </p>
     * <p>
     * The stream outputs are time limited so one bad output does not impact throughput too much. Essentially this means
     * that the work of writing to the outputs is performed, but the writer threads will not wait forever for stream
     * outputs to finish their work. <b>This might lead to increased memory usage!</b>
     * </p>
     * <p>
     * The default output, however, is allowed to block and is not subject to time limiting. This is important because it
     * can exert back pressure on the processing pipeline this way, making sure we don't run into excessive heap usage.
     * </p>
     *
     * @param event the message to write to outputs
     * @throws Exception
     */
@Override
public void onEvent(MessageEvent event) throws Exception {
    incomingMessages.mark();
    final Message msg = event.getMessage();
    if (msg == null) {
        LOG.debug("Skipping null message.");
        return;
    }
    LOG.debug("Processing message <{}> from OutputBuffer.", msg.getId());
    final Set<MessageOutput> messageOutputs = outputRouter.getStreamOutputsForMessage(msg);
    msg.recordCounter(serverStatus, "matched-outputs", messageOutputs.size());
    final Future<?> defaultOutputCompletion = processMessage(msg, defaultMessageOutput);
    final CountDownLatch streamOutputsDoneSignal = new CountDownLatch(messageOutputs.size());
    for (final MessageOutput output : messageOutputs) {
        processMessage(msg, output, streamOutputsDoneSignal);
    }
    // Wait until all writer threads for stream outputs have finished or timeout is reached.
    if (!streamOutputsDoneSignal.await(configuration.getOutputModuleTimeout(), TimeUnit.MILLISECONDS)) {
        LOG.warn("Timeout reached. Not waiting any longer for stream output writer threads to complete.");
    }
    // this exerts the back pressure to the system
    if (defaultOutputCompletion != null) {
        Uninterruptibles.getUninterruptibly(defaultOutputCompletion);
    } else {
        LOG.error("The default output future was null, this is a bug!");
    }
    if (msg.hasRecordings()) {
        LOG.debug("Message event trace: {}", msg.recordingsAsString());
    }
    outputThroughput.inc();
    LOG.debug("Wrote message <{}> to all outputs. Finished handling.", msg.getId());
    event.clearMessages();
}

Example 10 with Output

use of org.graylog2.plugin.streams.Output in project graylog2-server by Graylog2.

the class OutputSetupService method shutDownRunningOutputs.

private void shutDownRunningOutputs() {
    for (MessageOutput output : outputRegistry.getMessageOutputs()) {
        try {
            // TODO: change to debug
            LOG.info("Stopping output {}", output.getClass().getName());
            output.stop();
        } catch (Exception e) {
            LOG.error("Error stopping output", e);
        }
    }
}