Search in sources :

Example 1 with ShutdownListener

use of com.rabbitmq.client.ShutdownListener in project graylog2-server by Graylog2.

the class AmqpConsumer method connect.

public void connect() throws IOException {
    final ConnectionFactory factory = new ConnectionFactory();
    factory.setHost(hostname);
    factory.setPort(port);
    factory.setVirtualHost(virtualHost);
    factory.setRequestedHeartbeat(heartbeatTimeout);
    if (tls) {
        try {
            LOG.info("Enabling TLS for AMQP input [{}/{}].", sourceInput.getName(), sourceInput.getId());
            factory.useSslProtocol();
        } catch (NoSuchAlgorithmException | KeyManagementException e) {
            throw new IOException("Couldn't enable TLS for AMQP input.", e);
        }
    }
    // Authenticate?
    if (!isNullOrEmpty(username) && !isNullOrEmpty(password)) {
        factory.setUsername(username);
        factory.setPassword(password);
    }
    try {
        connection = factory.newConnection();
    } catch (TimeoutException e) {
        throw new IOException("Timeout while opening new AMQP connection", e);
    }
    channel = connection.createChannel();
    if (null == channel) {
        LOG.error("No channel descriptor available!");
    }
    if (null != channel && prefetchCount > 0) {
        channel.basicQos(prefetchCount);
        LOG.info("AMQP prefetch count overriden to <{}>.", prefetchCount);
    }
    connection.addShutdownListener(new ShutdownListener() {

        @Override
        public void shutdownCompleted(ShutdownSignalException cause) {
            if (cause.isInitiatedByApplication()) {
                LOG.info("Not reconnecting connection, we disconnected explicitly.");
                return;
            }
            while (true) {
                try {
                    LOG.error("AMQP connection lost! Trying reconnect in 1 second.");
                    Uninterruptibles.sleepUninterruptibly(1, TimeUnit.SECONDS);
                    connect();
                    LOG.info("Connected! Re-starting consumer.");
                    run();
                    LOG.info("Consumer running.");
                    break;
                } catch (IOException e) {
                    LOG.error("Could not re-connect to AMQP broker.", e);
                }
            }
        }
    });
}
Also used : ShutdownListener(com.rabbitmq.client.ShutdownListener) ConnectionFactory(com.rabbitmq.client.ConnectionFactory) ShutdownSignalException(com.rabbitmq.client.ShutdownSignalException) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException) IOException(java.io.IOException) KeyManagementException(java.security.KeyManagementException) TimeoutException(java.util.concurrent.TimeoutException)

Example 2 with ShutdownListener

use of com.rabbitmq.client.ShutdownListener in project druid by druid-io.

the class RabbitMQFirehoseFactory method connect.

@Override
public Firehose connect(final ByteBufferInputRowParser firehoseParser) throws IOException {
    ConnectionOptions lyraOptions = new ConnectionOptions(this.connectionFactory);
    Config lyraConfig = new Config().withRecoveryPolicy(new RetryPolicy().withMaxRetries(config.getMaxRetries()).withRetryInterval(Duration.seconds(config.getRetryIntervalSeconds())).withMaxDuration(Duration.seconds(config.getMaxDurationSeconds())));
    String queue = config.getQueue();
    String exchange = config.getExchange();
    String routingKey = config.getRoutingKey();
    boolean durable = config.isDurable();
    boolean exclusive = config.isExclusive();
    boolean autoDelete = config.isAutoDelete();
    final Connection connection = Connections.create(lyraOptions, lyraConfig);
    connection.addShutdownListener(new ShutdownListener() {

        @Override
        public void shutdownCompleted(ShutdownSignalException cause) {
            log.warn(cause, "Connection closed!");
        }
    });
    final Channel channel = connection.createChannel();
    channel.queueDeclare(queue, durable, exclusive, autoDelete, null);
    channel.queueBind(queue, exchange, routingKey);
    channel.addShutdownListener(new ShutdownListener() {

        @Override
        public void shutdownCompleted(ShutdownSignalException cause) {
            log.warn(cause, "Channel closed!");
        }
    });
    // We create a QueueingConsumer that will not auto-acknowledge messages since that
    // happens on commit().
    final QueueingConsumer consumer = new QueueingConsumer(channel);
    channel.basicConsume(queue, false, consumer);
    return new Firehose() {

        /**
       * Storing the latest delivery as a member variable should be safe since this will only be run
       * by a single thread.
       */
        private Delivery delivery;

        /**
       * Store the latest delivery tag to be able to commit (acknowledge) the message delivery up to
       * and including this tag. See commit() for more detail.
       */
        private long lastDeliveryTag;

        @Override
        public boolean hasMore() {
            delivery = null;
            try {
                // Wait for the next delivery. This will block until something is available.
                delivery = consumer.nextDelivery();
                if (delivery != null) {
                    lastDeliveryTag = delivery.getEnvelope().getDeliveryTag();
                    // If delivery is non-null, we report that there is something more to process.
                    return true;
                }
            } catch (InterruptedException e) {
                // A little unclear on how we should handle this.
                // At any rate, we're in an unknown state now so let's log something and return false.
                log.wtf(e, "Got interrupted while waiting for next delivery. Doubt this should ever happen.");
            }
            // nothing more to process.
            return false;
        }

        @Override
        public InputRow nextRow() {
            if (delivery == null) {
                //Just making sure.
                log.wtf("I have nothing in delivery. Method hasMore() should have returned false.");
                return null;
            }
            return firehoseParser.parse(ByteBuffer.wrap(delivery.getBody()));
        }

        @Override
        public Runnable commit() {
            // acknowledge values up to and including that value.
            return new Runnable() {

                // Store (copy) the last delivery tag to "become" thread safe.
                final long deliveryTag = lastDeliveryTag;

                @Override
                public void run() {
                    try {
                        log.info("Acknowledging delivery of messages up to tag: " + deliveryTag);
                        // Acknowledge all messages up to and including the stored delivery tag.
                        channel.basicAck(deliveryTag, true);
                    } catch (IOException e) {
                        log.error(e, "Unable to acknowledge message reception to message queue.");
                    }
                }
            };
        }

        @Override
        public void close() throws IOException {
            log.info("Closing connection to RabbitMQ");
            channel.close();
            connection.close();
        }
    };
}
Also used : Config(net.jodah.lyra.config.Config) Firehose(io.druid.data.input.Firehose) Channel(com.rabbitmq.client.Channel) Connection(com.rabbitmq.client.Connection) IOException(java.io.IOException) ShutdownListener(com.rabbitmq.client.ShutdownListener) ShutdownSignalException(com.rabbitmq.client.ShutdownSignalException) ConnectionOptions(net.jodah.lyra.ConnectionOptions) Delivery(com.rabbitmq.client.QueueingConsumer.Delivery) RetryPolicy(net.jodah.lyra.retry.RetryPolicy)

Aggregations

ShutdownListener (com.rabbitmq.client.ShutdownListener)2 ShutdownSignalException (com.rabbitmq.client.ShutdownSignalException)2 IOException (java.io.IOException)2 Channel (com.rabbitmq.client.Channel)1 Connection (com.rabbitmq.client.Connection)1 ConnectionFactory (com.rabbitmq.client.ConnectionFactory)1 Delivery (com.rabbitmq.client.QueueingConsumer.Delivery)1 Firehose (io.druid.data.input.Firehose)1 KeyManagementException (java.security.KeyManagementException)1 NoSuchAlgorithmException (java.security.NoSuchAlgorithmException)1 TimeoutException (java.util.concurrent.TimeoutException)1 ConnectionOptions (net.jodah.lyra.ConnectionOptions)1 Config (net.jodah.lyra.config.Config)1 RetryPolicy (net.jodah.lyra.retry.RetryPolicy)1