Search in sources :

Example 1 with KryoVisibilityBindingSetSerializer

use of org.apache.rya.indexing.pcj.fluo.app.export.kafka.KryoVisibilityBindingSetSerializer in project incubator-rya by apache.

the class KafkaLatencyBenchmark method updateStatsFromKafka.

private void updateStatsFromKafka(final String topic) {
    try (KafkaConsumer<String, VisibilityBindingSet> consumer = new KafkaConsumer<>(options.getKafkaConsumerProperties(), new StringDeserializer(), new KryoVisibilityBindingSetSerializer())) {
        consumer.subscribe(Arrays.asList(topic));
        while (!futureList.isEmpty()) {
            // check kafka at most twice a second.
            final ConsumerRecords<String, VisibilityBindingSet> records = consumer.poll(500);
            handle(records);
        }
    } catch (final Exception e) {
        logger.warn("Exception occurred", e);
    }
}
Also used : KryoVisibilityBindingSetSerializer(org.apache.rya.indexing.pcj.fluo.app.export.kafka.KryoVisibilityBindingSetSerializer) VisibilityBindingSet(org.apache.rya.api.model.VisibilityBindingSet) StringDeserializer(org.apache.kafka.common.serialization.StringDeserializer) KafkaConsumer(org.apache.kafka.clients.consumer.KafkaConsumer) ParameterException(com.beust.jcommander.ParameterException) DatatypeConfigurationException(javax.xml.datatype.DatatypeConfigurationException) RyaClientException(org.apache.rya.api.client.RyaClientException) InstanceDoesNotExistException(org.apache.rya.api.client.InstanceDoesNotExistException) AccumuloSecurityException(org.apache.accumulo.core.client.AccumuloSecurityException) IOException(java.io.IOException) AccumuloException(org.apache.accumulo.core.client.AccumuloException)

Aggregations

ParameterException (com.beust.jcommander.ParameterException)1 IOException (java.io.IOException)1 DatatypeConfigurationException (javax.xml.datatype.DatatypeConfigurationException)1 AccumuloException (org.apache.accumulo.core.client.AccumuloException)1 AccumuloSecurityException (org.apache.accumulo.core.client.AccumuloSecurityException)1 KafkaConsumer (org.apache.kafka.clients.consumer.KafkaConsumer)1 StringDeserializer (org.apache.kafka.common.serialization.StringDeserializer)1 InstanceDoesNotExistException (org.apache.rya.api.client.InstanceDoesNotExistException)1 RyaClientException (org.apache.rya.api.client.RyaClientException)1 VisibilityBindingSet (org.apache.rya.api.model.VisibilityBindingSet)1 KryoVisibilityBindingSetSerializer (org.apache.rya.indexing.pcj.fluo.app.export.kafka.KryoVisibilityBindingSetSerializer)1