Examples with VerifiableProperties kafka.utils.VerifiableProperties used on opensource projects

Example 1 with VerifiableProperties

use of kafka.utils.VerifiableProperties in project microservices by pwillhan.

the class App method start.

private void start(String topic) {
    consumer = Consumer.createJavaConsumerConnector(config);
    /* We tell Kafka how many threads will read each topic. We have one topic and one thread */
    Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
    topicCountMap.put(topic, new Integer(1));
    /* We will use a decoder to get Kafka to convert messages to Strings
        * valid property will be deserializer.encoding with the charset to use.
        * default is UTF8 which works for us */
    StringDecoder decoder = new StringDecoder(new VerifiableProperties());
    /* Kafka will give us a list of streams of messages for each topic.
        In this case, its just one topic with a list of a single stream */
    stream = consumer.createMessageStreams(topicCountMap, decoder, decoder).get(topic).get(0);
}

Example 2 with VerifiableProperties

use of kafka.utils.VerifiableProperties in project jeesuite-libs by vakinge.

the class OldApiTopicConsumer method start.

@Override
public void start() {
    // 重置offset
    if (consumerContext.getOffsetLogHanlder() != null) {
        resetCorrectOffsets();
    }
    Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
    for (String topicName : consumerContext.getMessageHandlers().keySet()) {
        int nThreads = 1;
        topicCountMap.put(topicName, nThreads);
        logger.info("topic[{}] assign fetch Threads {}", topicName, nThreads);
    }
    StringDecoder keyDecoder = new StringDecoder(new VerifiableProperties());
    MessageDecoder valueDecoder = new MessageDecoder(deserializer);
    Map<String, List<KafkaStream<String, Object>>> consumerMap = this.connector.createMessageStreams(topicCountMap, keyDecoder, valueDecoder);
    for (String topicName : consumerContext.getMessageHandlers().keySet()) {
        final List<KafkaStream<String, Object>> streams = consumerMap.get(topicName);
        for (final KafkaStream<String, Object> stream : streams) {
            MessageProcessor processer = new MessageProcessor(topicName, stream);
            this.fetchExecutor.execute(processer);
        }
    }
    // 
    runing.set(true);
}

Example 3 with VerifiableProperties

use of kafka.utils.VerifiableProperties in project incubator-gobblin by apache.

the class MockKafkaStream method createMockStream.

@SuppressWarnings("unchecked")
private static KafkaStream<byte[], byte[]> createMockStream(BlockingQueue<FetchedDataChunk> queue) {
    KafkaStream<byte[], byte[]> stream = (KafkaStream<byte[], byte[]>) Mockito.mock(KafkaStream.class);
    ConsumerIterator<byte[], byte[]> it = new ConsumerIterator<>(queue, -1, new DefaultDecoder(new VerifiableProperties()), new DefaultDecoder(new VerifiableProperties()), "clientId");
    Mockito.when(stream.iterator()).thenReturn(it);
    return stream;
}