Search in sources :

Example 1 with ChannelBufferOutputStream

use of org.jboss.netty.buffer.ChannelBufferOutputStream in project storm by apache.

the class MessageBatch method buffer.

/**
     * create a buffer containing the encoding of this batch
     */
ChannelBuffer buffer() throws Exception {
    ChannelBufferOutputStream bout = new ChannelBufferOutputStream(ChannelBuffers.directBuffer(encoded_length));
    for (TaskMessage msg : msgs) {
        writeTaskMessage(bout, msg);
    }
    //add a END_OF_BATCH indicator
    ControlMessage.EOB_MESSAGE.write(bout);
    bout.close();
    return bout.buffer();
}
Also used : ChannelBufferOutputStream(org.jboss.netty.buffer.ChannelBufferOutputStream) TaskMessage(org.apache.storm.messaging.TaskMessage)

Example 2 with ChannelBufferOutputStream

use of org.jboss.netty.buffer.ChannelBufferOutputStream in project storm by apache.

the class SaslMessageToken method buffer.

/**
     * encode the current SaslToken Message into a channel buffer
     * SaslTokenMessageRequest is encoded as: identifier .... short(2)
     * payload length .... int payload .... byte[]
     * 
     * @throws IOException
     */
public ChannelBuffer buffer() throws IOException {
    ChannelBufferOutputStream bout = new ChannelBufferOutputStream(ChannelBuffers.directBuffer(encodeLength()));
    int payload_len = 0;
    if (token != null)
        payload_len = token.length;
    bout.writeShort(IDENTIFIER);
    bout.writeInt(payload_len);
    if (payload_len > 0) {
        bout.write(token);
    }
    bout.close();
    return bout.buffer();
}
Also used : ChannelBufferOutputStream(org.jboss.netty.buffer.ChannelBufferOutputStream)

Example 3 with ChannelBufferOutputStream

use of org.jboss.netty.buffer.ChannelBufferOutputStream in project databus by linkedin.

the class NettyTestUtils method streamToChannelBuffer.

public static ChannelBuffer streamToChannelBuffer(DbusEventBuffer buf, Checkpoint cp, int maxSize, DbusEventsStatisticsCollector stats) throws ScnNotFoundException, OffsetNotFoundException, IOException {
    ChannelBuffer tmpBuf = ChannelBuffers.buffer(new DbusEventV1Factory().getByteOrder(), maxSize);
    OutputStream tmpOS = new ChannelBufferOutputStream(tmpBuf);
    WritableByteChannel tmpChannel = java.nio.channels.Channels.newChannel(tmpOS);
    StreamEventsArgs args = new StreamEventsArgs(maxSize).setStatsCollector(stats);
    buf.streamEvents(cp, tmpChannel, args);
    tmpChannel.close();
    return tmpBuf;
}
Also used : ChannelBufferOutputStream(org.jboss.netty.buffer.ChannelBufferOutputStream) ChannelBufferOutputStream(org.jboss.netty.buffer.ChannelBufferOutputStream) OutputStream(java.io.OutputStream) WritableByteChannel(java.nio.channels.WritableByteChannel) DbusEventV1Factory(com.linkedin.databus.core.DbusEventV1Factory) StreamEventsArgs(com.linkedin.databus.core.StreamEventsArgs) ChannelBuffer(org.jboss.netty.buffer.ChannelBuffer)

Example 4 with ChannelBufferOutputStream

use of org.jboss.netty.buffer.ChannelBufferOutputStream in project adbcj by mheath.

the class Encoder method handleDownstream.

public void handleDownstream(ChannelHandlerContext context, ChannelEvent event) throws Exception {
    if (!(event instanceof MessageEvent)) {
        context.sendDownstream(event);
        return;
    }
    MessageEvent e = (MessageEvent) event;
    Object message = e.getMessage();
    boolean singleMessage = message instanceof AbstractFrontendMessage;
    boolean multipleMessages = message instanceof AbstractFrontendMessage[];
    if (!singleMessage && !multipleMessages) {
        context.sendDownstream(event);
        return;
    }
    ChannelBuffer buffer = ChannelBuffers.buffer(1024);
    ChannelBufferOutputStream out = new ChannelBufferOutputStream(buffer);
    if (singleMessage) {
        encoder.encode(out, (AbstractFrontendMessage) e.getMessage());
    } else {
        encoder.encode(out, (AbstractFrontendMessage[]) e.getMessage());
    }
    Channels.write(context, e.getFuture(), buffer);
}
Also used : ChannelBufferOutputStream(org.jboss.netty.buffer.ChannelBufferOutputStream) AbstractFrontendMessage(org.adbcj.postgresql.codec.frontend.AbstractFrontendMessage) MessageEvent(org.jboss.netty.channel.MessageEvent) ChannelBuffer(org.jboss.netty.buffer.ChannelBuffer)

Example 5 with ChannelBufferOutputStream

use of org.jboss.netty.buffer.ChannelBufferOutputStream in project opentsdb by OpenTSDB.

the class QueryExecutor method serialize.

/**
 * Writes the results to a ChannelBuffer to return to the caller. This will
 * iterate over all of the outputs and drop in meta data where appropriate.
 * @throws Exception if something went pear shaped
 */
private Deferred<ChannelBuffer> serialize() throws Exception {
    final long start = System.currentTimeMillis();
    // buffers and an array list to stored the deferreds
    final ChannelBuffer response = ChannelBuffers.dynamicBuffer();
    final OutputStream output_stream = new ChannelBufferOutputStream(response);
    final JsonGenerator json = JSON.getFactory().createGenerator(output_stream);
    json.writeStartObject();
    json.writeFieldName("outputs");
    json.writeStartArray();
    // We want the serializer to execute serially so we need to create a callback
    // chain so that when one DPsResolver is finished, it triggers the next to
    // start serializing.
    final Deferred<Object> cb_chain = new Deferred<Object>();
    // default to the expressions if there, or fall back to the metrics
    final List<Output> outputs;
    if (query.getOutputs() == null || query.getOutputs().isEmpty()) {
        if (query.getExpressions() != null && !query.getExpressions().isEmpty()) {
            outputs = new ArrayList<Output>(query.getExpressions().size());
            for (final Expression exp : query.getExpressions()) {
                outputs.add(Output.Builder().setId(exp.getId()).build());
            }
        } else if (query.getMetrics() != null && !query.getMetrics().isEmpty()) {
            outputs = new ArrayList<Output>(query.getMetrics().size());
            for (final Metric metric : query.getMetrics()) {
                outputs.add(Output.Builder().setId(metric.getId()).build());
            }
        } else {
            throw new IllegalArgumentException("How did we get here?? No metrics or expressions??");
        }
    } else {
        outputs = query.getOutputs();
    }
    for (final Output output : outputs) {
        if (expressions != null) {
            final ExpressionIterator it = expressions.get(output.getId());
            if (it != null) {
                cb_chain.addCallback(new SerializeExpressionIterator(tsdb, json, output, it, ts_query));
                continue;
            }
        }
        if (query.getMetrics() != null && !query.getMetrics().isEmpty()) {
            final TSSubQuery sub = sub_queries.get(output.getId());
            if (sub != null) {
                final TimeSyncedIterator it = new TimeSyncedIterator(output.getId(), sub.getFilterTagKs(), sub_query_results.get(output.getId()));
                cb_chain.addCallback(new SerializeSubIterator(tsdb, json, output, it));
                continue;
            }
        } else {
            LOG.warn("Couldn't find a variable matching: " + output.getId() + " in query " + query);
        }
    }
    /**
     * Final callback to close out the JSON array and return our results
     */
    class FinalCB implements Callback<ChannelBuffer, Object> {

        public ChannelBuffer call(final Object obj) throws Exception {
            json.writeEndArray();
            // ts_query.getQueryStats().setTimeSerialization(
            // DateTime.currentTimeMillis() - start);
            ts_query.getQueryStats().markSerializationSuccessful();
            // dump the original query
            if (true) {
                json.writeFieldName("query");
                json.writeObject(QueryExecutor.this.query);
            }
            // IMPORTANT Make sure the close the JSON array and the generator
            json.writeEndObject();
            json.close();
            return response;
        }
    }
    // trigger the callback chain here
    cb_chain.callback(null);
    return cb_chain.addCallback(new FinalCB());
}
Also used : ChannelBufferOutputStream(org.jboss.netty.buffer.ChannelBufferOutputStream) ExpressionIterator(net.opentsdb.query.expression.ExpressionIterator) ChannelBufferOutputStream(org.jboss.netty.buffer.ChannelBufferOutputStream) OutputStream(java.io.OutputStream) Deferred(com.stumbleupon.async.Deferred) ArrayList(java.util.ArrayList) TimeSyncedIterator(net.opentsdb.query.expression.TimeSyncedIterator) TSSubQuery(net.opentsdb.core.TSSubQuery) ChannelBuffer(org.jboss.netty.buffer.ChannelBuffer) Callback(com.stumbleupon.async.Callback) Expression(net.opentsdb.query.pojo.Expression) Output(net.opentsdb.query.pojo.Output) JsonGenerator(com.fasterxml.jackson.core.JsonGenerator) Metric(net.opentsdb.query.pojo.Metric)

Aggregations

ChannelBufferOutputStream (org.jboss.netty.buffer.ChannelBufferOutputStream)19 ChannelBuffer (org.jboss.netty.buffer.ChannelBuffer)12 IOException (java.io.IOException)5 OutputStream (java.io.OutputStream)3 TaskMessage (backtype.storm.messaging.TaskMessage)2 JsonGenerator (com.fasterxml.jackson.core.JsonGenerator)2 Callback (com.stumbleupon.async.Callback)2 Deferred (com.stumbleupon.async.Deferred)2 PrintWriter (java.io.PrintWriter)2 ClosedChannelException (java.nio.channels.ClosedChannelException)2 ArrayList (java.util.ArrayList)2 TSSubQuery (net.opentsdb.core.TSSubQuery)2 MessageEvent (org.jboss.netty.channel.MessageEvent)2 DbusEventV1Factory (com.linkedin.databus.core.DbusEventV1Factory)1 StreamEventsArgs (com.linkedin.databus.core.StreamEventsArgs)1 HttpException (com.nabalive.framework.web.exception.HttpException)1 WritableByteChannel (java.nio.channels.WritableByteChannel)1 HashMap (java.util.HashMap)1 List (java.util.List)1 Map (java.util.Map)1