Search in sources :

Example 1 with CommandNotification

use of org.apache.rya.periodic.notification.notification.CommandNotification in project incubator-rya by apache.

the class KafkaNotificationRegistrationClient method addNotification.

@Override
public void addNotification(final String id, final long period, final long delay, final TimeUnit unit) {
    final Notification notification = PeriodicNotification.builder().id(id).period(period).initialDelay(delay).timeUnit(unit).build();
    processNotification(new CommandNotification(Command.ADD, notification));
}
Also used : CommandNotification(org.apache.rya.periodic.notification.notification.CommandNotification) Notification(org.apache.rya.periodic.notification.api.Notification) PeriodicNotification(org.apache.rya.periodic.notification.notification.PeriodicNotification) CommandNotification(org.apache.rya.periodic.notification.notification.CommandNotification) BasicNotification(org.apache.rya.periodic.notification.notification.BasicNotification)

Example 2 with CommandNotification

use of org.apache.rya.periodic.notification.notification.CommandNotification in project incubator-rya by apache.

the class CommandNotificationTypeAdapter method serialize.

@Override
public JsonElement serialize(CommandNotification arg0, Type arg1, JsonSerializationContext arg2) {
    JsonObject result = new JsonObject();
    result.add("command", new JsonPrimitive(arg0.getCommand().name()));
    Notification notification = arg0.getNotification();
    if (notification instanceof PeriodicNotification) {
        result.add("type", new JsonPrimitive(PeriodicNotification.class.getSimpleName()));
        PeriodicNotificationTypeAdapter adapter = new PeriodicNotificationTypeAdapter();
        result.add("notification", adapter.serialize((PeriodicNotification) notification, PeriodicNotification.class, arg2));
    } else if (notification instanceof BasicNotification) {
        result.add("type", new JsonPrimitive(BasicNotification.class.getSimpleName()));
        BasicNotificationTypeAdapter adapter = new BasicNotificationTypeAdapter();
        result.add("notification", adapter.serialize((BasicNotification) notification, BasicNotification.class, arg2));
    } else {
        throw new IllegalArgumentException("Invalid notification type.");
    }
    return result;
}
Also used : BasicNotification(org.apache.rya.periodic.notification.notification.BasicNotification) JsonPrimitive(com.google.gson.JsonPrimitive) JsonObject(com.google.gson.JsonObject) PeriodicNotification(org.apache.rya.periodic.notification.notification.PeriodicNotification) Notification(org.apache.rya.periodic.notification.api.Notification) PeriodicNotification(org.apache.rya.periodic.notification.notification.PeriodicNotification) CommandNotification(org.apache.rya.periodic.notification.notification.CommandNotification) BasicNotification(org.apache.rya.periodic.notification.notification.BasicNotification)

Example 3 with CommandNotification

use of org.apache.rya.periodic.notification.notification.CommandNotification in project incubator-rya by apache.

the class PeriodicNotificationProvider method processRegisteredNotifications.

/**
 * Registers all of Periodic Query information already contained within Fluo to the
 * {@link NotificationCoordinatorExecutor}.
 * @param coordinator - coordinator that periodic info will be registered with
 * @param sx - snapshot for reading results from Fluo
 */
public void processRegisteredNotifications(NotificationCoordinatorExecutor coordinator, Snapshot sx) {
    coordinator.start();
    Collection<CommandNotification> notifications = getNotifications(sx);
    for (CommandNotification notification : notifications) {
        coordinator.processNextCommandNotification(notification);
    }
}
Also used : CommandNotification(org.apache.rya.periodic.notification.notification.CommandNotification)

Example 4 with CommandNotification

use of org.apache.rya.periodic.notification.notification.CommandNotification in project incubator-rya by apache.

the class PeriodicNotificationConsumer method run.

@Override
public void run() {
    try {
        LOG.info("Configuring KafkaConsumer on thread: {} to subscribe to topic: {}", threadNumber, topic);
        consumer.subscribe(Arrays.asList(topic));
        while (!closed.get()) {
            final ConsumerRecords<String, CommandNotification> records = consumer.poll(10000);
            // Handle new records
            for (final ConsumerRecord<String, CommandNotification> record : records) {
                final CommandNotification notification = record.value();
                LOG.info("Thread {} is adding notification: {}", threadNumber, notification);
                coord.processNextCommandNotification(notification);
            }
        }
        LOG.info("Finished polling.");
    } catch (final WakeupException e) {
        // Ignore exception if closing
        if (!closed.get()) {
            throw e;
        }
    } finally {
        consumer.close();
    }
}
Also used : CommandNotification(org.apache.rya.periodic.notification.notification.CommandNotification) WakeupException(org.apache.kafka.common.errors.WakeupException)

Example 5 with CommandNotification

use of org.apache.rya.periodic.notification.notification.CommandNotification in project incubator-rya by apache.

the class CreateDeletePeriodicPCJ method runTest.

private void runTest(String query, Collection<Statement> statements, int expectedEntries) throws Exception {
    try (FluoClient fluoClient = FluoFactory.newClient(super.getFluoConfiguration())) {
        String topic = "notification_topic";
        PeriodicQueryResultStorage storage = new AccumuloPeriodicQueryResultStorage(super.getAccumuloConnector(), RYA_INSTANCE_NAME);
        PeriodicNotificationClient notificationClient = new KafkaNotificationRegistrationClient(topic, getNotificationProducer("localhost:9092"));
        CreatePeriodicQuery periodicPCJ = new CreatePeriodicQuery(fluoClient, storage);
        String id = periodicPCJ.createPeriodicQuery(query, notificationClient).getQueryId();
        loadData(statements);
        // Ensure the data was loaded.
        final List<Bytes> rows = getFluoTableEntries(fluoClient);
        assertEquals(expectedEntries, rows.size());
        DeletePeriodicQuery deletePeriodic = new DeletePeriodicQuery(fluoClient, storage);
        deletePeriodic.deletePeriodicQuery(FluoQueryUtils.convertFluoQueryIdToPcjId(id), notificationClient);
        getMiniFluo().waitForObservers();
        // Ensure all data related to the query has been removed.
        final List<Bytes> empty_rows = getFluoTableEntries(fluoClient);
        assertEquals(1, empty_rows.size());
        // Ensure that Periodic Service notified to add and delete PeriodicNotification
        Set<CommandNotification> notifications;
        try (KafkaConsumer<String, CommandNotification> consumer = makeNotificationConsumer(topic)) {
            notifications = getKafkaNotifications(topic, 7000, consumer);
        }
        assertEquals(2, notifications.size());
        String notificationId = "";
        boolean addCalled = false;
        boolean deleteCalled = false;
        for (CommandNotification notification : notifications) {
            if (notificationId.length() == 0) {
                notificationId = notification.getId();
            } else {
                assertEquals(notificationId, notification.getId());
            }
            if (notification.getCommand() == Command.ADD) {
                addCalled = true;
            }
            if (notification.getCommand() == Command.DELETE) {
                deleteCalled = true;
            }
        }
        assertEquals(true, addCalled);
        assertEquals(true, deleteCalled);
    }
}
Also used : FluoClient(org.apache.fluo.api.client.FluoClient) AccumuloPeriodicQueryResultStorage(org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPeriodicQueryResultStorage) Bytes(org.apache.fluo.api.data.Bytes) PeriodicNotificationClient(org.apache.rya.periodic.notification.api.PeriodicNotificationClient) DeletePeriodicQuery(org.apache.rya.indexing.pcj.fluo.api.DeletePeriodicQuery) CommandNotification(org.apache.rya.periodic.notification.notification.CommandNotification) CreatePeriodicQuery(org.apache.rya.indexing.pcj.fluo.api.CreatePeriodicQuery) AccumuloPeriodicQueryResultStorage(org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPeriodicQueryResultStorage) PeriodicQueryResultStorage(org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage) KafkaNotificationRegistrationClient(org.apache.rya.periodic.notification.registration.KafkaNotificationRegistrationClient)

Aggregations

CommandNotification (org.apache.rya.periodic.notification.notification.CommandNotification)12 PeriodicNotification (org.apache.rya.periodic.notification.notification.PeriodicNotification)6 Notification (org.apache.rya.periodic.notification.api.Notification)4 BasicNotification (org.apache.rya.periodic.notification.notification.BasicNotification)4 Properties (java.util.Properties)3 StringDeserializer (org.apache.kafka.common.serialization.StringDeserializer)3 TimestampedNotification (org.apache.rya.periodic.notification.notification.TimestampedNotification)3 KafkaNotificationRegistrationClient (org.apache.rya.periodic.notification.registration.KafkaNotificationRegistrationClient)3 CommandNotificationSerializer (org.apache.rya.periodic.notification.serialization.CommandNotificationSerializer)3 Test (org.junit.Test)3 JsonObject (com.google.gson.JsonObject)2 LinkedBlockingQueue (java.util.concurrent.LinkedBlockingQueue)2 KafkaProducer (org.apache.kafka.clients.producer.KafkaProducer)2 PeriodicNotificationCoordinatorExecutor (org.apache.rya.periodic.notification.coordinator.PeriodicNotificationCoordinatorExecutor)2 Command (org.apache.rya.periodic.notification.notification.CommandNotification.Command)2 JsonParseException (com.google.gson.JsonParseException)1 JsonPrimitive (com.google.gson.JsonPrimitive)1 HashSet (java.util.HashSet)1 FluoClient (org.apache.fluo.api.client.FluoClient)1 Bytes (org.apache.fluo.api.data.Bytes)1