Search in sources :

Example 1 with PeriodicNotificationClient

use of org.apache.rya.periodic.notification.api.PeriodicNotificationClient in project incubator-rya by apache.

the class CreateDeletePeriodicPCJ method runTest.

private void runTest(String query, Collection<Statement> statements, int expectedEntries) throws Exception {
    try (FluoClient fluoClient = FluoFactory.newClient(super.getFluoConfiguration())) {
        String topic = "notification_topic";
        PeriodicQueryResultStorage storage = new AccumuloPeriodicQueryResultStorage(super.getAccumuloConnector(), RYA_INSTANCE_NAME);
        PeriodicNotificationClient notificationClient = new KafkaNotificationRegistrationClient(topic, getNotificationProducer("localhost:9092"));
        CreatePeriodicQuery periodicPCJ = new CreatePeriodicQuery(fluoClient, storage);
        String id = periodicPCJ.createPeriodicQuery(query, notificationClient).getQueryId();
        loadData(statements);
        // Ensure the data was loaded.
        final List<Bytes> rows = getFluoTableEntries(fluoClient);
        assertEquals(expectedEntries, rows.size());
        DeletePeriodicQuery deletePeriodic = new DeletePeriodicQuery(fluoClient, storage);
        deletePeriodic.deletePeriodicQuery(FluoQueryUtils.convertFluoQueryIdToPcjId(id), notificationClient);
        getMiniFluo().waitForObservers();
        // Ensure all data related to the query has been removed.
        final List<Bytes> empty_rows = getFluoTableEntries(fluoClient);
        assertEquals(1, empty_rows.size());
        // Ensure that Periodic Service notified to add and delete PeriodicNotification
        Set<CommandNotification> notifications;
        try (KafkaConsumer<String, CommandNotification> consumer = makeNotificationConsumer(topic)) {
            notifications = getKafkaNotifications(topic, 7000, consumer);
        }
        assertEquals(2, notifications.size());
        String notificationId = "";
        boolean addCalled = false;
        boolean deleteCalled = false;
        for (CommandNotification notification : notifications) {
            if (notificationId.length() == 0) {
                notificationId = notification.getId();
            } else {
                assertEquals(notificationId, notification.getId());
            }
            if (notification.getCommand() == Command.ADD) {
                addCalled = true;
            }
            if (notification.getCommand() == Command.DELETE) {
                deleteCalled = true;
            }
        }
        assertEquals(true, addCalled);
        assertEquals(true, deleteCalled);
    }
}
Also used : FluoClient(org.apache.fluo.api.client.FluoClient) AccumuloPeriodicQueryResultStorage(org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPeriodicQueryResultStorage) Bytes(org.apache.fluo.api.data.Bytes) PeriodicNotificationClient(org.apache.rya.periodic.notification.api.PeriodicNotificationClient) DeletePeriodicQuery(org.apache.rya.indexing.pcj.fluo.api.DeletePeriodicQuery) CommandNotification(org.apache.rya.periodic.notification.notification.CommandNotification) CreatePeriodicQuery(org.apache.rya.indexing.pcj.fluo.api.CreatePeriodicQuery) AccumuloPeriodicQueryResultStorage(org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPeriodicQueryResultStorage) PeriodicQueryResultStorage(org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage) KafkaNotificationRegistrationClient(org.apache.rya.periodic.notification.registration.KafkaNotificationRegistrationClient)

Example 2 with PeriodicNotificationClient

use of org.apache.rya.periodic.notification.api.PeriodicNotificationClient in project incubator-rya by apache.

the class AccumuloCreatePeriodicPCJ method updateFluoAppAndRegisterWithKafka.

private String updateFluoAppAndRegisterWithKafka(final String ryaInstance, final String fluoAppName, String sparql, String periodicTopic, String bootStrapServers) throws RepositoryException, MalformedQueryException, SailException, QueryEvaluationException, PcjException, RyaDAOException, UnsupportedQueryException, PeriodicQueryCreationException {
    requireNonNull(sparql);
    requireNonNull(periodicTopic);
    requireNonNull(bootStrapServers);
    final PeriodicQueryResultStorage periodicStorage = new AccumuloPeriodicQueryResultStorage(getConnector(), ryaInstance);
    // Connect to the Fluo application that is updating this instance's PCJs.
    final AccumuloConnectionDetails cd = super.getAccumuloConnectionDetails();
    try (final FluoClient fluoClient = new FluoClientFactory().connect(cd.getUsername(), new String(cd.getUserPass()), cd.getInstanceName(), cd.getZookeepers(), fluoAppName)) {
        // Initialize the PCJ within the Fluo application.
        final CreatePeriodicQuery periodicPcj = new CreatePeriodicQuery(fluoClient, periodicStorage);
        PeriodicNotificationClient periodicClient = new KafkaNotificationRegistrationClient(periodicTopic, createProducer(bootStrapServers));
        return periodicPcj.withRyaIntegration(sparql, periodicClient, getConnector(), ryaInstance).getQueryId();
    }
}
Also used : FluoClient(org.apache.fluo.api.client.FluoClient) AccumuloPeriodicQueryResultStorage(org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPeriodicQueryResultStorage) PeriodicNotificationClient(org.apache.rya.periodic.notification.api.PeriodicNotificationClient) CreatePeriodicQuery(org.apache.rya.indexing.pcj.fluo.api.CreatePeriodicQuery) AccumuloPeriodicQueryResultStorage(org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPeriodicQueryResultStorage) PeriodicQueryResultStorage(org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage) KafkaNotificationRegistrationClient(org.apache.rya.periodic.notification.registration.KafkaNotificationRegistrationClient)

Aggregations

FluoClient (org.apache.fluo.api.client.FluoClient)2 CreatePeriodicQuery (org.apache.rya.indexing.pcj.fluo.api.CreatePeriodicQuery)2 PeriodicQueryResultStorage (org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage)2 AccumuloPeriodicQueryResultStorage (org.apache.rya.indexing.pcj.storage.accumulo.AccumuloPeriodicQueryResultStorage)2 PeriodicNotificationClient (org.apache.rya.periodic.notification.api.PeriodicNotificationClient)2 KafkaNotificationRegistrationClient (org.apache.rya.periodic.notification.registration.KafkaNotificationRegistrationClient)2 Bytes (org.apache.fluo.api.data.Bytes)1 DeletePeriodicQuery (org.apache.rya.indexing.pcj.fluo.api.DeletePeriodicQuery)1 CommandNotification (org.apache.rya.periodic.notification.notification.CommandNotification)1