Search in sources :

Example 21 with TopologyFactory

use of org.apache.rya.streams.kafka.topology.TopologyFactory in project incubator-rya by apache.

the class TemporalFilterIT method showBeforeWorks.

@Test
public void showBeforeWorks() throws Exception {
    // Enumerate some topics that will be re-used
    final String ryaInstance = UUID.randomUUID().toString();
    final UUID queryId = UUID.randomUUID();
    final String statementsTopic = KafkaTopics.statementsTopic(ryaInstance);
    final String resultsTopic = KafkaTopics.queryResultsTopic(ryaInstance, queryId);
    // Get the RDF model objects that will be used to build the query.
    final String sparql = "PREFIX time: <http://www.w3.org/2006/time/> \n" + "PREFIX tempf: <" + TemporalURIs.NAMESPACE + ">\n" + "SELECT * \n" + "WHERE { \n" + "  <urn:time> time:atTime ?date .\n" + " FILTER(tempf:before(?date, \"" + TIME_10.toString() + "\")) " + "}";
    // Setup a topology.
    final TopologyBuilder builder = new TopologyFactory().build(sparql, statementsTopic, resultsTopic, new RandomUUIDFactory());
    // Create the statements that will be input into the query.
    final ValueFactory vf = new ValueFactoryImpl();
    final List<VisibilityStatement> statements = getStatements();
    // Make the expected results.
    final Set<VisibilityBindingSet> expected = new HashSet<>();
    final MapBindingSet bs = new MapBindingSet();
    bs.addBinding("date", vf.createLiteral(TIME.toString()));
    expected.add(new VisibilityBindingSet(bs, "a"));
    // Run the test.
    RyaStreamsTestUtil.runStreamProcessingTest(kafka, statementsTopic, resultsTopic, builder, statements, expected, VisibilityBindingSetDeserializer.class);
}
Also used : VisibilityBindingSet(org.apache.rya.api.model.VisibilityBindingSet) TopologyBuilder(org.apache.kafka.streams.processor.TopologyBuilder) ValueFactoryImpl(org.openrdf.model.impl.ValueFactoryImpl) TopologyFactory(org.apache.rya.streams.kafka.topology.TopologyFactory) ValueFactory(org.openrdf.model.ValueFactory) VisibilityStatement(org.apache.rya.api.model.VisibilityStatement) RandomUUIDFactory(org.apache.rya.api.function.projection.RandomUUIDFactory) MapBindingSet(org.openrdf.query.impl.MapBindingSet) UUID(java.util.UUID) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 22 with TopologyFactory

use of org.apache.rya.streams.kafka.topology.TopologyFactory in project incubator-rya by apache.

the class TemporalFilterIT method showEqualsWorks.

@Test
public void showEqualsWorks() throws Exception {
    // Enumerate some topics that will be re-used
    final String ryaInstance = UUID.randomUUID().toString();
    final UUID queryId = UUID.randomUUID();
    final String statementsTopic = KafkaTopics.statementsTopic(ryaInstance);
    final String resultsTopic = KafkaTopics.queryResultsTopic(ryaInstance, queryId);
    // Get the RDF model objects that will be used to build the query.
    final String sparql = "PREFIX time: <http://www.w3.org/2006/time/> \n" + "PREFIX tempf: <" + TemporalURIs.NAMESPACE + ">\n" + "SELECT * \n" + "WHERE { \n" + "  <urn:time> time:atTime ?date .\n" + " FILTER(tempf:equals(?date, \"" + TIME.toString() + "\")) " + "}";
    // Setup a topology.
    final TopologyBuilder builder = new TopologyFactory().build(sparql, statementsTopic, resultsTopic, new RandomUUIDFactory());
    // Create the statements that will be input into the query.
    final ValueFactory vf = new ValueFactoryImpl();
    final List<VisibilityStatement> statements = getStatements();
    // Make the expected results.
    final Set<VisibilityBindingSet> expected = new HashSet<>();
    final MapBindingSet bs = new MapBindingSet();
    bs.addBinding("date", vf.createLiteral(TIME.toString()));
    expected.add(new VisibilityBindingSet(bs, "a"));
    // Run the test.
    RyaStreamsTestUtil.runStreamProcessingTest(kafka, statementsTopic, resultsTopic, builder, statements, expected, VisibilityBindingSetDeserializer.class);
}
Also used : VisibilityBindingSet(org.apache.rya.api.model.VisibilityBindingSet) TopologyBuilder(org.apache.kafka.streams.processor.TopologyBuilder) ValueFactoryImpl(org.openrdf.model.impl.ValueFactoryImpl) TopologyFactory(org.apache.rya.streams.kafka.topology.TopologyFactory) ValueFactory(org.openrdf.model.ValueFactory) VisibilityStatement(org.apache.rya.api.model.VisibilityStatement) RandomUUIDFactory(org.apache.rya.api.function.projection.RandomUUIDFactory) MapBindingSet(org.openrdf.query.impl.MapBindingSet) UUID(java.util.UUID) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 23 with TopologyFactory

use of org.apache.rya.streams.kafka.topology.TopologyFactory in project incubator-rya by apache.

the class MultiProjectionProcessorIT method showProcessorWorks.

@Test
public void showProcessorWorks() throws Exception {
    // Enumerate some topics that will be re-used
    final String ryaInstance = UUID.randomUUID().toString();
    final UUID queryId = UUID.randomUUID();
    final String statementsTopic = KafkaTopics.statementsTopic(ryaInstance);
    final String resultsTopic = KafkaTopics.queryResultsTopic(ryaInstance, queryId);
    // Create a topology for the Query that will be tested.
    final String sparql = "CONSTRUCT {" + "_:b a <urn:movementObservation> ; " + "<urn:location> ?location ; " + "<urn:direction> ?direction ; " + "}" + "WHERE {" + "?thing <urn:corner> ?location ." + "?thing <urn:compass> ?direction." + "}";
    final String bNodeId = UUID.randomUUID().toString();
    final TopologyBuilder builder = new TopologyFactory().build(sparql, statementsTopic, resultsTopic, () -> bNodeId);
    // Create the statements that will be input into the query.
    final ValueFactory vf = new ValueFactoryImpl();
    final List<VisibilityStatement> statements = new ArrayList<>();
    statements.add(new VisibilityStatement(vf.createStatement(vf.createURI("urn:car1"), vf.createURI("urn:compass"), vf.createURI("urn:NW")), "a"));
    statements.add(new VisibilityStatement(vf.createStatement(vf.createURI("urn:car1"), vf.createURI("urn:corner"), vf.createURI("urn:corner1")), "a"));
    // Make the expected results.
    final Set<VisibilityStatement> expected = new HashSet<>();
    final BNode blankNode = vf.createBNode(bNodeId);
    expected.add(new VisibilityStatement(vf.createStatement(blankNode, RDF.TYPE, vf.createURI("urn:movementObservation")), "a"));
    expected.add(new VisibilityStatement(vf.createStatement(blankNode, vf.createURI("urn:direction"), vf.createURI("urn:NW")), "a"));
    expected.add(new VisibilityStatement(vf.createStatement(blankNode, vf.createURI("urn:location"), vf.createURI("urn:corner1")), "a"));
    // Run the test.
    RyaStreamsTestUtil.runStreamProcessingTest(kafka, statementsTopic, resultsTopic, builder, statements, expected, VisibilityStatementDeserializer.class);
}
Also used : BNode(org.openrdf.model.BNode) TopologyBuilder(org.apache.kafka.streams.processor.TopologyBuilder) ValueFactoryImpl(org.openrdf.model.impl.ValueFactoryImpl) ArrayList(java.util.ArrayList) TopologyFactory(org.apache.rya.streams.kafka.topology.TopologyFactory) ValueFactory(org.openrdf.model.ValueFactory) UUID(java.util.UUID) VisibilityStatement(org.apache.rya.api.model.VisibilityStatement) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 24 with TopologyFactory

use of org.apache.rya.streams.kafka.topology.TopologyFactory in project incubator-rya by apache.

the class StatementPatternProcessorIT method multiplePatterns_multipleStatements.

@Test
public void multiplePatterns_multipleStatements() throws Exception {
    // Enumerate some topics that will be re-used
    final String ryaInstance = UUID.randomUUID().toString();
    final UUID queryId = UUID.randomUUID();
    final String statementsTopic = KafkaTopics.statementsTopic(ryaInstance);
    final String resultsTopic = KafkaTopics.queryResultsTopic(ryaInstance, queryId);
    // Setup a topology.
    final String query = "SELECT * WHERE { " + "?person <urn:talksTo> ?otherPerson ." + "?person ?action <urn:Bob>" + "}";
    final TopologyFactory factory = new TopologyFactory();
    final TopologyBuilder builder = factory.build(query, statementsTopic, resultsTopic, new RandomUUIDFactory());
    // Create some statements where some generates SP results and others do not.
    final ValueFactory vf = new ValueFactoryImpl();
    final List<VisibilityStatement> statements = new ArrayList<>();
    statements.add(new VisibilityStatement(vf.createStatement(vf.createURI("urn:Alice"), vf.createURI("urn:talksTo"), vf.createURI("urn:Bob")), "a"));
    statements.add(new VisibilityStatement(vf.createStatement(vf.createURI("urn:Alice"), vf.createURI("urn:talksTo"), vf.createURI("urn:Charlie")), "a|b"));
    statements.add(new VisibilityStatement(vf.createStatement(vf.createURI("urn:Charlie"), vf.createURI("urn:walksWith"), vf.createURI("urn:Bob")), "b"));
    // Show the correct binding set results from the job.
    final Set<VisibilityBindingSet> expected = new HashSet<>();
    QueryBindingSet bs = new QueryBindingSet();
    bs = new QueryBindingSet();
    bs.addBinding("person", vf.createURI("urn:Alice"));
    bs.addBinding("action", vf.createURI("urn:talksTo"));
    bs.addBinding("otherPerson", vf.createURI("urn:Charlie"));
    expected.add(new VisibilityBindingSet(bs, "a&(a|b)"));
    bs = new QueryBindingSet();
    bs.addBinding("person", vf.createURI("urn:Alice"));
    bs.addBinding("action", vf.createURI("urn:talksTo"));
    bs.addBinding("otherPerson", vf.createURI("urn:Bob"));
    expected.add(new VisibilityBindingSet(bs, "a"));
    // Run the test.
    RyaStreamsTestUtil.runStreamProcessingTest(kafka, statementsTopic, resultsTopic, builder, statements, expected, VisibilityBindingSetDeserializer.class);
}
Also used : VisibilityBindingSet(org.apache.rya.api.model.VisibilityBindingSet) TopologyBuilder(org.apache.kafka.streams.processor.TopologyBuilder) ValueFactoryImpl(org.openrdf.model.impl.ValueFactoryImpl) ArrayList(java.util.ArrayList) TopologyFactory(org.apache.rya.streams.kafka.topology.TopologyFactory) ValueFactory(org.openrdf.model.ValueFactory) VisibilityStatement(org.apache.rya.api.model.VisibilityStatement) QueryBindingSet(org.openrdf.query.algebra.evaluation.QueryBindingSet) RandomUUIDFactory(org.apache.rya.api.function.projection.RandomUUIDFactory) UUID(java.util.UUID) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 25 with TopologyFactory

use of org.apache.rya.streams.kafka.topology.TopologyFactory in project incubator-rya by apache.

the class StatementPatternProcessorIT method singlePattern_singleStatement.

@Test
public void singlePattern_singleStatement() throws Exception {
    // Enumerate some topics that will be re-used
    final String ryaInstance = UUID.randomUUID().toString();
    final UUID queryId = UUID.randomUUID();
    final String statementsTopic = KafkaTopics.statementsTopic(ryaInstance);
    final String resultsTopic = KafkaTopics.queryResultsTopic(ryaInstance, queryId);
    // Setup a topology.
    final String query = "SELECT * WHERE { ?person <urn:talksTo> ?otherPerson }";
    final TopologyFactory factory = new TopologyFactory();
    final TopologyBuilder builder = factory.build(query, statementsTopic, resultsTopic, new RandomUUIDFactory());
    // Create a statement that generate an SP result.
    final ValueFactory vf = new ValueFactoryImpl();
    final List<VisibilityStatement> statements = new ArrayList<>();
    statements.add(new VisibilityStatement(vf.createStatement(vf.createURI("urn:Alice"), vf.createURI("urn:talksTo"), vf.createURI("urn:Bob")), "a"));
    // Show the correct binding set results from the job.
    final Set<VisibilityBindingSet> expected = new HashSet<>();
    final QueryBindingSet bs = new QueryBindingSet();
    bs.addBinding("person", vf.createURI("urn:Alice"));
    bs.addBinding("otherPerson", vf.createURI("urn:Bob"));
    expected.add(new VisibilityBindingSet(bs, "a"));
    // Run the test.
    RyaStreamsTestUtil.runStreamProcessingTest(kafka, statementsTopic, resultsTopic, builder, statements, expected, VisibilityBindingSetDeserializer.class);
}
Also used : VisibilityBindingSet(org.apache.rya.api.model.VisibilityBindingSet) TopologyBuilder(org.apache.kafka.streams.processor.TopologyBuilder) ValueFactoryImpl(org.openrdf.model.impl.ValueFactoryImpl) ArrayList(java.util.ArrayList) TopologyFactory(org.apache.rya.streams.kafka.topology.TopologyFactory) ValueFactory(org.openrdf.model.ValueFactory) VisibilityStatement(org.apache.rya.api.model.VisibilityStatement) QueryBindingSet(org.openrdf.query.algebra.evaluation.QueryBindingSet) RandomUUIDFactory(org.apache.rya.api.function.projection.RandomUUIDFactory) UUID(java.util.UUID) HashSet(java.util.HashSet) Test(org.junit.Test)

Aggregations

UUID (java.util.UUID)26 TopologyFactory (org.apache.rya.streams.kafka.topology.TopologyFactory)26 HashSet (java.util.HashSet)25 VisibilityStatement (org.apache.rya.api.model.VisibilityStatement)25 Test (org.junit.Test)25 ValueFactory (org.openrdf.model.ValueFactory)25 ValueFactoryImpl (org.openrdf.model.impl.ValueFactoryImpl)25 TopologyBuilder (org.apache.kafka.streams.processor.TopologyBuilder)24 VisibilityBindingSet (org.apache.rya.api.model.VisibilityBindingSet)24 RandomUUIDFactory (org.apache.rya.api.function.projection.RandomUUIDFactory)23 ArrayList (java.util.ArrayList)20 MapBindingSet (org.openrdf.query.impl.MapBindingSet)20 QueryBindingSet (org.openrdf.query.algebra.evaluation.QueryBindingSet)4 StreamsQuery (org.apache.rya.streams.api.entity.StreamsQuery)2 InMemoryQueryRepository (org.apache.rya.streams.api.queries.InMemoryQueryRepository)2 QueryRepository (org.apache.rya.streams.api.queries.QueryRepository)2 JCommander (com.beust.jcommander.JCommander)1 ParameterException (com.beust.jcommander.ParameterException)1 Scheduler (com.google.common.util.concurrent.AbstractScheduledService.Scheduler)1 WKTWriter (com.vividsolutions.jts.io.WKTWriter)1