Search in sources :

Example 1 with PeriodicNotificationCoordinatorExecutor

use of org.apache.rya.periodic.notification.coordinator.PeriodicNotificationCoordinatorExecutor in project incubator-rya by apache.

the class PeriodicNotificationProviderIT method testProvider.

@Test
public void testProvider() throws MalformedQueryException, InterruptedException, UnsupportedQueryException {
    String sparql = // n
    "prefix function: <http://org.apache.rya/function#> " + // n
    "prefix time: <http://www.w3.org/2006/time#> " + // n
    "select ?id (count(?obs) as ?total) where {" + // n
    "Filter(function:periodic(?time, 1, .25, time:minutes)) " + // n
    "?obs <uri:hasTime> ?time. " + // n
    "?obs <uri:hasId> ?id } group by ?id";
    BlockingQueue<TimestampedNotification> notifications = new LinkedBlockingQueue<>();
    PeriodicNotificationCoordinatorExecutor coord = new PeriodicNotificationCoordinatorExecutor(2, notifications);
    PeriodicNotificationProvider provider = new PeriodicNotificationProvider();
    CreateFluoPcj pcj = new CreateFluoPcj();
    String id = null;
    try (FluoClient fluo = new FluoClientImpl(getFluoConfiguration())) {
        id = pcj.createPcj(FluoQueryUtils.createNewPcjId(), sparql, Sets.newHashSet(), fluo).getQueryId();
        provider.processRegisteredNotifications(coord, fluo.newSnapshot());
    }
    TimestampedNotification notification = notifications.take();
    Assert.assertEquals(5000, notification.getInitialDelay());
    Assert.assertEquals(15000, notification.getPeriod());
    Assert.assertEquals(TimeUnit.MILLISECONDS, notification.getTimeUnit());
    Assert.assertEquals(FluoQueryUtils.convertFluoQueryIdToPcjId(id), notification.getId());
}
Also used : FluoClient(org.apache.fluo.api.client.FluoClient) FluoClientImpl(org.apache.fluo.core.client.FluoClientImpl) PeriodicNotificationCoordinatorExecutor(org.apache.rya.periodic.notification.coordinator.PeriodicNotificationCoordinatorExecutor) TimestampedNotification(org.apache.rya.periodic.notification.notification.TimestampedNotification) CreateFluoPcj(org.apache.rya.indexing.pcj.fluo.api.CreateFluoPcj) LinkedBlockingQueue(java.util.concurrent.LinkedBlockingQueue) PeriodicNotificationProvider(org.apache.rya.periodic.notification.recovery.PeriodicNotificationProvider) Test(org.junit.Test)

Example 2 with PeriodicNotificationCoordinatorExecutor

use of org.apache.rya.periodic.notification.coordinator.PeriodicNotificationCoordinatorExecutor in project incubator-rya by apache.

the class PeriodicCommandNotificationConsumerIT method kafkaNotificationMillisProviderTest.

@Test
public void kafkaNotificationMillisProviderTest() throws InterruptedException {
    BasicConfigurator.configure();
    final BlockingQueue<TimestampedNotification> notifications = new LinkedBlockingQueue<>();
    final Properties props = createKafkaConfig();
    final KafkaProducer<String, CommandNotification> producer = new KafkaProducer<>(props);
    final String topic = rule.getKafkaTopicName();
    rule.createTopic(topic);
    registration = new KafkaNotificationRegistrationClient(topic, producer);
    coord = new PeriodicNotificationCoordinatorExecutor(1, notifications);
    provider = new KafkaNotificationProvider(topic, new StringDeserializer(), new CommandNotificationSerializer(), props, coord, 1);
    provider.start();
    registration.addNotification("1", 1000, 0, TimeUnit.MILLISECONDS);
    Thread.sleep(4000);
    // check that notifications are being added to the blocking queue
    Assert.assertEquals(true, notifications.size() > 0);
    registration.deleteNotification("1");
    Thread.sleep(2000);
    final int size = notifications.size();
    // sleep for 2 seconds to ensure no more messages being produced
    Thread.sleep(2000);
    Assert.assertEquals(size, notifications.size());
    tearDown();
}
Also used : KafkaProducer(org.apache.kafka.clients.producer.KafkaProducer) StringDeserializer(org.apache.kafka.common.serialization.StringDeserializer) TimestampedNotification(org.apache.rya.periodic.notification.notification.TimestampedNotification) LinkedBlockingQueue(java.util.concurrent.LinkedBlockingQueue) Properties(java.util.Properties) CommandNotificationSerializer(org.apache.rya.periodic.notification.serialization.CommandNotificationSerializer) PeriodicNotificationCoordinatorExecutor(org.apache.rya.periodic.notification.coordinator.PeriodicNotificationCoordinatorExecutor) CommandNotification(org.apache.rya.periodic.notification.notification.CommandNotification) KafkaNotificationRegistrationClient(org.apache.rya.periodic.notification.registration.KafkaNotificationRegistrationClient) Test(org.junit.Test)

Example 3 with PeriodicNotificationCoordinatorExecutor

use of org.apache.rya.periodic.notification.coordinator.PeriodicNotificationCoordinatorExecutor in project incubator-rya by apache.

the class PeriodicCommandNotificationConsumerIT method kafkaNotificationProviderTest.

@Test
public void kafkaNotificationProviderTest() throws InterruptedException {
    BasicConfigurator.configure();
    final BlockingQueue<TimestampedNotification> notifications = new LinkedBlockingQueue<>();
    final Properties props = createKafkaConfig();
    final KafkaProducer<String, CommandNotification> producer = new KafkaProducer<>(props);
    final String topic = rule.getKafkaTopicName();
    rule.createTopic(topic);
    registration = new KafkaNotificationRegistrationClient(topic, producer);
    coord = new PeriodicNotificationCoordinatorExecutor(1, notifications);
    provider = new KafkaNotificationProvider(topic, new StringDeserializer(), new CommandNotificationSerializer(), props, coord, 1);
    provider.start();
    registration.addNotification("1", 1, 0, TimeUnit.SECONDS);
    Thread.sleep(4000);
    // check that notifications are being added to the blocking queue
    Assert.assertEquals(true, notifications.size() > 0);
    registration.deleteNotification("1");
    Thread.sleep(2000);
    final int size = notifications.size();
    // sleep for 2 seconds to ensure no more messages being produced
    Thread.sleep(2000);
    Assert.assertEquals(size, notifications.size());
    tearDown();
}
Also used : KafkaProducer(org.apache.kafka.clients.producer.KafkaProducer) StringDeserializer(org.apache.kafka.common.serialization.StringDeserializer) TimestampedNotification(org.apache.rya.periodic.notification.notification.TimestampedNotification) LinkedBlockingQueue(java.util.concurrent.LinkedBlockingQueue) Properties(java.util.Properties) CommandNotificationSerializer(org.apache.rya.periodic.notification.serialization.CommandNotificationSerializer) PeriodicNotificationCoordinatorExecutor(org.apache.rya.periodic.notification.coordinator.PeriodicNotificationCoordinatorExecutor) CommandNotification(org.apache.rya.periodic.notification.notification.CommandNotification) KafkaNotificationRegistrationClient(org.apache.rya.periodic.notification.registration.KafkaNotificationRegistrationClient) Test(org.junit.Test)

Aggregations

LinkedBlockingQueue (java.util.concurrent.LinkedBlockingQueue)3 PeriodicNotificationCoordinatorExecutor (org.apache.rya.periodic.notification.coordinator.PeriodicNotificationCoordinatorExecutor)3 TimestampedNotification (org.apache.rya.periodic.notification.notification.TimestampedNotification)3 Test (org.junit.Test)3 Properties (java.util.Properties)2 KafkaProducer (org.apache.kafka.clients.producer.KafkaProducer)2 StringDeserializer (org.apache.kafka.common.serialization.StringDeserializer)2 CommandNotification (org.apache.rya.periodic.notification.notification.CommandNotification)2 KafkaNotificationRegistrationClient (org.apache.rya.periodic.notification.registration.KafkaNotificationRegistrationClient)2 CommandNotificationSerializer (org.apache.rya.periodic.notification.serialization.CommandNotificationSerializer)2 FluoClient (org.apache.fluo.api.client.FluoClient)1 FluoClientImpl (org.apache.fluo.core.client.FluoClientImpl)1 CreateFluoPcj (org.apache.rya.indexing.pcj.fluo.api.CreateFluoPcj)1 PeriodicNotificationProvider (org.apache.rya.periodic.notification.recovery.PeriodicNotificationProvider)1