Search in sources :

Example 11 with QueryChange

use of org.apache.rya.streams.api.queries.QueryChange in project incubator-rya by apache.

the class QueryEventWorkGeneratorTest method notifyUpdate_isNotActive.

@Test
public void notifyUpdate_isNotActive() throws Exception {
    // The signal that will kill the notifying thread.
    final AtomicBoolean shutdownSignal = new AtomicBoolean(false);
    // The queue generated work is offered to.
    final BlockingQueue<QueryEvent> queue = new ArrayBlockingQueue<>(1);
    // The listener that will perform the QueryEventWorkGenerator work.
    final CountDownLatch latch = new CountDownLatch(1);
    latch.countDown();
    final QueryEventWorkGenerator generator = new QueryEventWorkGenerator("rya", latch, queue, 50, TimeUnit.MILLISECONDS, shutdownSignal);
    // A thread that will attempt to notify the generator with an update query change.
    final UUID queryId = UUID.randomUUID();
    final StreamsQuery query = new StreamsQuery(queryId, "query", false, false);
    final Thread notifyThread = new Thread(() -> {
        final QueryChange change = QueryChange.update(queryId, false);
        final ChangeLogEntry<QueryChange> entry = new ChangeLogEntry<>(0, change);
        generator.notify(entry, Optional.of(query));
    });
    // Start the thread.
    notifyThread.start();
    try {
        // Show work was added to the queue and the notifying thread died.
        final QueryEvent event = queue.poll(500, TimeUnit.MILLISECONDS);
        final QueryEvent expected = QueryEvent.stopped("rya", queryId);
        assertEquals(expected, event);
    } finally {
        shutdownSignal.set(true);
        notifyThread.join();
    }
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) ArrayBlockingQueue(java.util.concurrent.ArrayBlockingQueue) StreamsQuery(org.apache.rya.streams.api.entity.StreamsQuery) QueryEventWorkGenerator(org.apache.rya.streams.querymanager.QueryManager.QueryEventWorkGenerator) QueryChange(org.apache.rya.streams.api.queries.QueryChange) ChangeLogEntry(org.apache.rya.streams.api.queries.ChangeLogEntry) QueryEvent(org.apache.rya.streams.querymanager.QueryManager.QueryEvent) CountDownLatch(java.util.concurrent.CountDownLatch) UUID(java.util.UUID) Test(org.junit.Test)

Example 12 with QueryChange

use of org.apache.rya.streams.api.queries.QueryChange in project incubator-rya by apache.

the class QueryEventWorkGeneratorTest method notifyCreate.

@Test
public void notifyCreate() throws Exception {
    // The signal that will kill the notifying thread.
    final AtomicBoolean shutdownSignal = new AtomicBoolean(false);
    // The queue generated work is offered to.
    final BlockingQueue<QueryEvent> queue = new ArrayBlockingQueue<>(1);
    // The listener that will perform the QueryEventWorkGenerator work.
    final CountDownLatch latch = new CountDownLatch(1);
    latch.countDown();
    final QueryEventWorkGenerator generator = new QueryEventWorkGenerator("rya", latch, queue, 50, TimeUnit.MILLISECONDS, shutdownSignal);
    // A thread that will attempt to notify the generator with a created query.
    final UUID queryId = UUID.randomUUID();
    final StreamsQuery query = new StreamsQuery(queryId, "query", true, false);
    final Thread notifyThread = new Thread(() -> {
        final QueryChange change = QueryChange.create(queryId, query.getSparql(), query.isActive(), query.isInsert());
        final ChangeLogEntry<QueryChange> entry = new ChangeLogEntry<>(0, change);
        generator.notify(entry, Optional.of(query));
    });
    // Start the thread.
    notifyThread.start();
    try {
        // Show work was added to the queue and the notifying thread died.
        final QueryEvent event = queue.poll(500, TimeUnit.MILLISECONDS);
        final QueryEvent expected = QueryEvent.executing("rya", new StreamsQuery(queryId, query.getSparql(), query.isActive(), query.isInsert()));
        assertEquals(expected, event);
    } finally {
        shutdownSignal.set(true);
        notifyThread.join();
    }
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) ArrayBlockingQueue(java.util.concurrent.ArrayBlockingQueue) StreamsQuery(org.apache.rya.streams.api.entity.StreamsQuery) QueryEventWorkGenerator(org.apache.rya.streams.querymanager.QueryManager.QueryEventWorkGenerator) QueryChange(org.apache.rya.streams.api.queries.QueryChange) ChangeLogEntry(org.apache.rya.streams.api.queries.ChangeLogEntry) QueryEvent(org.apache.rya.streams.querymanager.QueryManager.QueryEvent) CountDownLatch(java.util.concurrent.CountDownLatch) UUID(java.util.UUID) Test(org.junit.Test)

Example 13 with QueryChange

use of org.apache.rya.streams.api.queries.QueryChange in project incubator-rya by apache.

the class QueryEventWorkGeneratorTest method waitsForSubscriptionWork.

@Test
public void waitsForSubscriptionWork() throws Exception {
    // The signal that will kill the notifying thread.
    final AtomicBoolean shutdownSignal = new AtomicBoolean(false);
    // The queue generated work is offered to.
    final BlockingQueue<QueryEvent> queue = new ArrayBlockingQueue<>(1);
    // The listener that will perform the QueryEventWorkGenerator work.
    final CountDownLatch latch = new CountDownLatch(1);
    final QueryEventWorkGenerator generator = new QueryEventWorkGenerator("rya", latch, queue, 50, TimeUnit.MILLISECONDS, shutdownSignal);
    // A thread that will attempt to notify the generator with a created query.
    final UUID queryId = UUID.randomUUID();
    final StreamsQuery query = new StreamsQuery(queryId, "query", true, false);
    final Thread notifyThread = new Thread(() -> {
        final QueryChange change = QueryChange.create(queryId, query.getSparql(), query.isActive(), query.isInsert());
        final ChangeLogEntry<QueryChange> entry = new ChangeLogEntry<>(0, change);
        generator.notify(entry, Optional.of(query));
    });
    // Start the thread.
    notifyThread.start();
    try {
        // Wait longer than the blocking period and show the thread is still alive and nothing has been added
        // to the work queue.
        Thread.sleep(150);
        assertTrue(notifyThread.isAlive());
        // Count down the latch.
        latch.countDown();
        // Show work was added to the queue and the notifying thread died.
        final QueryEvent event = queue.poll(500, TimeUnit.MILLISECONDS);
        final QueryEvent expected = QueryEvent.executing("rya", new StreamsQuery(queryId, query.getSparql(), query.isActive(), query.isInsert()));
        assertEquals(expected, event);
    } finally {
        shutdownSignal.set(true);
        notifyThread.join();
    }
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) ArrayBlockingQueue(java.util.concurrent.ArrayBlockingQueue) StreamsQuery(org.apache.rya.streams.api.entity.StreamsQuery) QueryEventWorkGenerator(org.apache.rya.streams.querymanager.QueryManager.QueryEventWorkGenerator) QueryChange(org.apache.rya.streams.api.queries.QueryChange) ChangeLogEntry(org.apache.rya.streams.api.queries.ChangeLogEntry) QueryEvent(org.apache.rya.streams.querymanager.QueryManager.QueryEvent) CountDownLatch(java.util.concurrent.CountDownLatch) UUID(java.util.UUID) Test(org.junit.Test)

Example 14 with QueryChange

use of org.apache.rya.streams.api.queries.QueryChange in project incubator-rya by apache.

the class KafkaQueryChangeLogFactory method make.

/**
 * Creates an instance of {@link KafkaQueryChangeLog} using a new {@link Producer} and {@link Consumer}.
 *
 * @param bootstrapServers - Indicates which instance of Kafka that will be connected to. (not null)
 * @param topic - The topic the QueryChangeLog is persisted to. (not null)
 * @return A new instance of {@link KafkaQueryChangeLog}.
 */
public static KafkaQueryChangeLog make(final String bootstrapServers, final String topic) {
    requireNonNull(bootstrapServers);
    requireNonNull(topic);
    final Properties producerProperties = new Properties();
    producerProperties.setProperty(CommonClientConfigs.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
    producerProperties.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
    producerProperties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, QueryChangeSerializer.class.getName());
    final Properties consumerProperties = new Properties();
    consumerProperties.setProperty(CommonClientConfigs.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
    consumerProperties.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
    consumerProperties.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, QueryChangeDeserializer.class.getName());
    final Producer<?, QueryChange> producer = new KafkaProducer<>(producerProperties);
    final Consumer<?, QueryChange> consumer = new KafkaConsumer<>(consumerProperties);
    return new KafkaQueryChangeLog(producer, consumer, topic);
}
Also used : KafkaProducer(org.apache.kafka.clients.producer.KafkaProducer) StringDeserializer(org.apache.kafka.common.serialization.StringDeserializer) QueryChangeSerializer(org.apache.rya.streams.kafka.serialization.queries.QueryChangeSerializer) QueryChangeDeserializer(org.apache.rya.streams.kafka.serialization.queries.QueryChangeDeserializer) QueryChange(org.apache.rya.streams.api.queries.QueryChange) KafkaConsumer(org.apache.kafka.clients.consumer.KafkaConsumer) Properties(java.util.Properties) StringSerializer(org.apache.kafka.common.serialization.StringSerializer)

Example 15 with QueryChange

use of org.apache.rya.streams.api.queries.QueryChange in project incubator-rya by apache.

the class KafkaQueryChangeLogIT method readSingleWrite.

@Test
public void readSingleWrite() throws Exception {
    // Write a single change to the log.
    final QueryChange change = QueryChange.create(UUID.randomUUID(), "query", true, false);
    changeLog.write(change);
    // Read that entry from the log.
    final QueryChange readChange = changeLog.readFromStart().next().getEntry();
    assertEquals(change, readChange);
}
Also used : QueryChange(org.apache.rya.streams.api.queries.QueryChange) Test(org.junit.Test)

Aggregations

QueryChange (org.apache.rya.streams.api.queries.QueryChange)18 Test (org.junit.Test)11 ChangeLogEntry (org.apache.rya.streams.api.queries.ChangeLogEntry)8 UUID (java.util.UUID)7 ArrayBlockingQueue (java.util.concurrent.ArrayBlockingQueue)5 CountDownLatch (java.util.concurrent.CountDownLatch)5 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)5 InMemoryQueryRepository (org.apache.rya.streams.api.queries.InMemoryQueryRepository)5 QueryChangeLog (org.apache.rya.streams.api.queries.QueryChangeLog)5 KafkaQueryChangeLog (org.apache.rya.streams.kafka.queries.KafkaQueryChangeLog)5 QueryEvent (org.apache.rya.streams.querymanager.QueryManager.QueryEvent)5 QueryEventWorkGenerator (org.apache.rya.streams.querymanager.QueryManager.QueryEventWorkGenerator)5 ArrayList (java.util.ArrayList)4 StreamsQuery (org.apache.rya.streams.api.entity.StreamsQuery)4 Before (org.junit.Before)4 TopicPartition (org.apache.kafka.common.TopicPartition)3 QueryChangeLogException (org.apache.rya.streams.api.queries.QueryChangeLog.QueryChangeLogException)3 StringDeserializer (org.apache.kafka.common.serialization.StringDeserializer)2 StringSerializer (org.apache.kafka.common.serialization.StringSerializer)2 VisibilityBindingSetDeserializer (org.apache.rya.streams.kafka.serialization.VisibilityBindingSetDeserializer)2