Search in sources :

Example 6 with Sink

use of org.apache.flume.Sink in project MSEC by Tencent.

the class TestProfobufSource method testBasic.

@Test
public void testBasic() throws Exception {
    Context context = new Context();
    context.put("bind", "localhost");
    context.put("port", "45673");
    context.put("threads", "1");
    ProtobufSource source = new ProtobufSource();
    source.configure(context);
    Map<String, String> channelContext = new HashMap();
    channelContext.put("capacity", "1000000");
    // for faster tests
    channelContext.put("keep-alive", "0");
    Channel channel = new MemoryChannel();
    Configurables.configure(channel, new Context(channelContext));
    Sink sink = new LoggerSink();
    sink.setChannel(channel);
    sink.start();
    DefaultSinkProcessor proc = new DefaultSinkProcessor();
    proc.setSinks(Collections.singletonList(sink));
    SinkRunner sinkRunner = new SinkRunner(proc);
    sinkRunner.start();
    ChannelSelector rcs = new ReplicatingChannelSelector();
    rcs.setChannels(Collections.singletonList(channel));
    ChannelProcessor chp = new ChannelProcessor(rcs);
    source.setChannelProcessor(chp);
    source.start();
    Thread.sleep(5000);
    source.stop();
    sinkRunner.stop();
    sink.stop();
}
Also used : Context(org.apache.flume.Context) MemoryChannel(org.apache.flume.channel.MemoryChannel) HashMap(java.util.HashMap) MemoryChannel(org.apache.flume.channel.MemoryChannel) Channel(org.apache.flume.Channel) SinkRunner(org.apache.flume.SinkRunner) ChannelProcessor(org.apache.flume.channel.ChannelProcessor) ProtobufSource(org.ngse.source.protobuf.ProtobufSource) DefaultSinkProcessor(org.apache.flume.sink.DefaultSinkProcessor) ReplicatingChannelSelector(org.apache.flume.channel.ReplicatingChannelSelector) LoggerSink(org.apache.flume.sink.LoggerSink) Sink(org.apache.flume.Sink) LoggerSink(org.apache.flume.sink.LoggerSink) ChannelSelector(org.apache.flume.ChannelSelector) ReplicatingChannelSelector(org.apache.flume.channel.ReplicatingChannelSelector) Test(org.junit.Test)

Aggregations

Sink (org.apache.flume.Sink)6 Context (org.apache.flume.Context)5 MemoryChannel (org.apache.flume.channel.MemoryChannel)5 Test (org.junit.Test)5 Transaction (org.apache.flume.Transaction)3 Date (java.util.Date)2 HashMap (java.util.HashMap)2 Channel (org.apache.flume.Channel)2 Event (org.apache.flume.Event)2 DefaultMQPullConsumer (org.apache.rocketmq.client.consumer.DefaultMQPullConsumer)2 PullResult (org.apache.rocketmq.client.consumer.PullResult)2 MessageExt (org.apache.rocketmq.common.message.MessageExt)2 MessageQueue (org.apache.rocketmq.common.message.MessageQueue)2 CountDownLatch (java.util.concurrent.CountDownLatch)1 ChannelSelector (org.apache.flume.ChannelSelector)1 SinkFactory (org.apache.flume.SinkFactory)1 SinkRunner (org.apache.flume.SinkRunner)1 ChannelProcessor (org.apache.flume.channel.ChannelProcessor)1 ReplicatingChannelSelector (org.apache.flume.channel.ReplicatingChannelSelector)1 DefaultSinkFactory (org.apache.flume.sink.DefaultSinkFactory)1