Search in sources :

Example 1 with ReplicatingChannelSelector

use of org.apache.flume.channel.ReplicatingChannelSelector in project MSEC by Tencent.

the class TestProfobufSource method testBasic.

@Test
public void testBasic() throws Exception {
    Context context = new Context();
    context.put("bind", "localhost");
    context.put("port", "45673");
    context.put("threads", "1");
    ProtobufSource source = new ProtobufSource();
    source.configure(context);
    Map<String, String> channelContext = new HashMap();
    channelContext.put("capacity", "1000000");
    // for faster tests
    channelContext.put("keep-alive", "0");
    Channel channel = new MemoryChannel();
    Configurables.configure(channel, new Context(channelContext));
    Sink sink = new LoggerSink();
    sink.setChannel(channel);
    sink.start();
    DefaultSinkProcessor proc = new DefaultSinkProcessor();
    proc.setSinks(Collections.singletonList(sink));
    SinkRunner sinkRunner = new SinkRunner(proc);
    sinkRunner.start();
    ChannelSelector rcs = new ReplicatingChannelSelector();
    rcs.setChannels(Collections.singletonList(channel));
    ChannelProcessor chp = new ChannelProcessor(rcs);
    source.setChannelProcessor(chp);
    source.start();
    Thread.sleep(5000);
    source.stop();
    sinkRunner.stop();
    sink.stop();
}
Also used : Context(org.apache.flume.Context) MemoryChannel(org.apache.flume.channel.MemoryChannel) HashMap(java.util.HashMap) MemoryChannel(org.apache.flume.channel.MemoryChannel) Channel(org.apache.flume.Channel) SinkRunner(org.apache.flume.SinkRunner) ChannelProcessor(org.apache.flume.channel.ChannelProcessor) ProtobufSource(org.ngse.source.protobuf.ProtobufSource) DefaultSinkProcessor(org.apache.flume.sink.DefaultSinkProcessor) ReplicatingChannelSelector(org.apache.flume.channel.ReplicatingChannelSelector) LoggerSink(org.apache.flume.sink.LoggerSink) Sink(org.apache.flume.Sink) LoggerSink(org.apache.flume.sink.LoggerSink) ChannelSelector(org.apache.flume.ChannelSelector) ReplicatingChannelSelector(org.apache.flume.channel.ReplicatingChannelSelector) Test(org.junit.Test)

Example 2 with ReplicatingChannelSelector

use of org.apache.flume.channel.ReplicatingChannelSelector in project logging-log4j2 by apache.

the class FlumeAppenderTest method setUp.

@Before
public void setUp() throws Exception {
    eventSource = new AvroSource();
    channel = new MemoryChannel();
    Configurables.configure(channel, new Context());
    avroLogger = (Logger) LogManager.getLogger("avrologger");
    /*
         * Clear out all other appenders associated with this logger to ensure
         * we're only hitting the Avro appender.
         */
    removeAppenders(avroLogger);
    final Context context = new Context();
    testPort = String.valueOf(AvailablePortFinder.getNextAvailable());
    context.put("port", testPort);
    context.put("bind", "0.0.0.0");
    Configurables.configure(eventSource, context);
    final List<Channel> channels = new ArrayList<>();
    channels.add(channel);
    final ChannelSelector cs = new ReplicatingChannelSelector();
    cs.setChannels(channels);
    eventSource.setChannelProcessor(new ChannelProcessor(cs));
    eventSource.start();
    Assert.assertTrue("Reached start or error", LifecycleController.waitForOneOf(eventSource, LifecycleState.START_OR_ERROR));
    Assert.assertEquals("Server is started", LifecycleState.START, eventSource.getLifecycleState());
}
Also used : ThreadContext(org.apache.logging.log4j.ThreadContext) Context(org.apache.flume.Context) MemoryChannel(org.apache.flume.channel.MemoryChannel) AvroSource(org.apache.flume.source.AvroSource) ReplicatingChannelSelector(org.apache.flume.channel.ReplicatingChannelSelector) MemoryChannel(org.apache.flume.channel.MemoryChannel) Channel(org.apache.flume.Channel) ArrayList(java.util.ArrayList) ReplicatingChannelSelector(org.apache.flume.channel.ReplicatingChannelSelector) ChannelSelector(org.apache.flume.ChannelSelector) ChannelProcessor(org.apache.flume.channel.ChannelProcessor) Before(org.junit.Before)

Aggregations

Channel (org.apache.flume.Channel)2 ChannelSelector (org.apache.flume.ChannelSelector)2 Context (org.apache.flume.Context)2 ChannelProcessor (org.apache.flume.channel.ChannelProcessor)2 MemoryChannel (org.apache.flume.channel.MemoryChannel)2 ReplicatingChannelSelector (org.apache.flume.channel.ReplicatingChannelSelector)2 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 Sink (org.apache.flume.Sink)1 SinkRunner (org.apache.flume.SinkRunner)1 DefaultSinkProcessor (org.apache.flume.sink.DefaultSinkProcessor)1 LoggerSink (org.apache.flume.sink.LoggerSink)1 AvroSource (org.apache.flume.source.AvroSource)1 ThreadContext (org.apache.logging.log4j.ThreadContext)1 Before (org.junit.Before)1 Test (org.junit.Test)1 ProtobufSource (org.ngse.source.protobuf.ProtobufSource)1