Search in sources :

Example 1 with TopologyBuilderException

use of org.apache.rya.streams.kafka.topology.TopologyBuilderFactory.TopologyBuilderException in project incubator-rya by apache.

the class SingleThreadKafkaStreamsFactory method make.

@Override
public KafkaStreams make(final String ryaInstance, final StreamsQuery query) throws KafkaStreamsFactoryException {
    requireNonNull(ryaInstance);
    requireNonNull(query);
    // Setup the Kafka Stream program.
    final Properties streamsProps = new Properties();
    // Configure the Kafka servers that will be talked to.
    streamsProps.setProperty(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServersConfig);
    // Use the Query ID as the Application ID to ensure we resume where we left off the last time this command was run.
    streamsProps.put(StreamsConfig.APPLICATION_ID_CONFIG, "RyaStreams-Query-" + query.getQueryId());
    // Always start at the beginning of the input topic.
    streamsProps.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
    // Setup the topology that processes the Query.
    final String statementsTopic = KafkaTopics.statementsTopic(ryaInstance);
    final String resultsTopic = KafkaTopics.queryResultsTopic(ryaInstance, query.getQueryId());
    try {
        final TopologyBuilder topologyBuilder = topologyFactory.build(query.getSparql(), statementsTopic, resultsTopic, new RandomUUIDFactory());
        return new KafkaStreams(topologyBuilder, new StreamsConfig(streamsProps));
    } catch (final MalformedQueryException | TopologyBuilderException e) {
        throw new KafkaStreamsFactoryException("Could not create a KafkaStreams processing topology for query " + query.getQueryId(), e);
    }
}
Also used : KafkaStreams(org.apache.kafka.streams.KafkaStreams) TopologyBuilderException(org.apache.rya.streams.kafka.topology.TopologyBuilderFactory.TopologyBuilderException) TopologyBuilder(org.apache.kafka.streams.processor.TopologyBuilder) RandomUUIDFactory(org.apache.rya.api.function.projection.RandomUUIDFactory) MalformedQueryException(org.openrdf.query.MalformedQueryException) Properties(java.util.Properties) StreamsConfig(org.apache.kafka.streams.StreamsConfig)

Aggregations

Properties (java.util.Properties)1 KafkaStreams (org.apache.kafka.streams.KafkaStreams)1 StreamsConfig (org.apache.kafka.streams.StreamsConfig)1 TopologyBuilder (org.apache.kafka.streams.processor.TopologyBuilder)1 RandomUUIDFactory (org.apache.rya.api.function.projection.RandomUUIDFactory)1 TopologyBuilderException (org.apache.rya.streams.kafka.topology.TopologyBuilderFactory.TopologyBuilderException)1 MalformedQueryException (org.openrdf.query.MalformedQueryException)1