Search in sources :

Example 1 with ResultExportException

use of org.apache.rya.indexing.pcj.fluo.app.export.IncrementalBindingSetExporter.ResultExportException in project incubator-rya by apache.

the class KafkaRyaSubGraphExporter method export.

/**
 * Exports the RyaSubGraph to a Kafka topic equivalent to the result returned by {@link RyaSubGraph#getId()}
 * @param subgraph - RyaSubGraph exported to Kafka
 * @param contructID - rowID of result that is exported. Used for logging purposes.
 */
@Override
public void export(final String constructID, final RyaSubGraph subGraph) throws ResultExportException {
    checkNotNull(constructID);
    checkNotNull(subGraph);
    try {
        // Send the result to the topic whose name matches the PCJ ID.
        final ProducerRecord<String, RyaSubGraph> rec = new ProducerRecord<>(subGraph.getId(), subGraph);
        final Future<RecordMetadata> future = producer.send(rec);
        // Don't let the export return until the result has been written to the topic. Otherwise we may lose results.
        future.get();
        log.debug("Producer successfully sent record with id: {} and statements: {}", constructID, subGraph.getStatements());
    } catch (final Throwable e) {
        throw new ResultExportException("A result could not be exported to Kafka.", e);
    }
}
Also used : RecordMetadata(org.apache.kafka.clients.producer.RecordMetadata) RyaSubGraph(org.apache.rya.api.domain.RyaSubGraph) ProducerRecord(org.apache.kafka.clients.producer.ProducerRecord) ResultExportException(org.apache.rya.indexing.pcj.fluo.app.export.IncrementalBindingSetExporter.ResultExportException)

Example 2 with ResultExportException

use of org.apache.rya.indexing.pcj.fluo.app.export.IncrementalBindingSetExporter.ResultExportException in project incubator-rya by apache.

the class ExporterManager method exportBindingSet.

/**
 * Exports BindingSet using the exporters for a given {@link QueryType}.
 * @param exporters - exporters corresponding to a given queryType
 * @param strategies - export strategies used to export results (possibly a subset of those in the exporters map)
 * @param pcjId - id of the query whose results are being exported
 * @param data - serialized BindingSet result
 * @throws ResultExportException
 */
private void exportBindingSet(final Map<ExportStrategy, IncrementalResultExporter> exporters, final Set<ExportStrategy> strategies, final String pcjId, final Bytes data) throws ResultExportException {
    VisibilityBindingSet bs;
    try {
        bs = BS_SERDE.deserialize(data);
        simplifyVisibilities(bs);
    } catch (final Exception e) {
        throw new ResultExportException("Unable to deserialize the given BindingSet.", e);
    }
    try {
        for (final ExportStrategy strategy : strategies) {
            final IncrementalBindingSetExporter exporter = (IncrementalBindingSetExporter) exporters.get(strategy);
            exporter.export(pcjId, bs);
        }
    } catch (final Exception e) {
        throw new ResultExportException("Unable to export the given BindingSet " + bs + " with the given set of ExportStrategies " + strategies, e);
    }
}
Also used : VisibilityBindingSet(org.apache.rya.api.model.VisibilityBindingSet) ExportStrategy(org.apache.rya.api.client.CreatePCJ.ExportStrategy) ResultExportException(org.apache.rya.indexing.pcj.fluo.app.export.IncrementalBindingSetExporter.ResultExportException) ResultExportException(org.apache.rya.indexing.pcj.fluo.app.export.IncrementalBindingSetExporter.ResultExportException) UnsupportedEncodingException(java.io.UnsupportedEncodingException)

Example 3 with ResultExportException

use of org.apache.rya.indexing.pcj.fluo.app.export.IncrementalBindingSetExporter.ResultExportException in project incubator-rya by apache.

the class ExporterManager method exportSubGraph.

/**
 * Exports RyaSubGraph using the exporters for a given {@link QueryType}.
 * @param exporters - exporters corresponding to a given queryType
 * @param strategies - export strategies used to export results (possibly a subset of those in the exporters map)
 * @param pcjId - id of the query whose results are being exported
 * @param data - serialized RyaSubGraph result
 * @throws ResultExportException
 */
private void exportSubGraph(final Map<ExportStrategy, IncrementalResultExporter> exporters, final Set<ExportStrategy> strategies, final String pcjId, final Bytes data) throws ResultExportException {
    final RyaSubGraph subGraph = SG_SERDE.fromBytes(data.toArray());
    try {
        simplifyVisibilities(subGraph);
    } catch (final UnsupportedEncodingException e) {
        throw new ResultExportException("Undable to deserialize provided RyaSubgraph", e);
    }
    try {
        for (final ExportStrategy strategy : strategies) {
            final IncrementalRyaSubGraphExporter exporter = (IncrementalRyaSubGraphExporter) exporters.get(strategy);
            exporter.export(pcjId, subGraph);
        }
    } catch (final Exception e) {
        throw new ResultExportException("Unable to export the given subgraph " + subGraph + " using all of the ExportStrategies " + strategies);
    }
}
Also used : RyaSubGraph(org.apache.rya.api.domain.RyaSubGraph) ExportStrategy(org.apache.rya.api.client.CreatePCJ.ExportStrategy) UnsupportedEncodingException(java.io.UnsupportedEncodingException) ResultExportException(org.apache.rya.indexing.pcj.fluo.app.export.IncrementalBindingSetExporter.ResultExportException) ResultExportException(org.apache.rya.indexing.pcj.fluo.app.export.IncrementalBindingSetExporter.ResultExportException) UnsupportedEncodingException(java.io.UnsupportedEncodingException)

Aggregations

ResultExportException (org.apache.rya.indexing.pcj.fluo.app.export.IncrementalBindingSetExporter.ResultExportException)3 UnsupportedEncodingException (java.io.UnsupportedEncodingException)2 ExportStrategy (org.apache.rya.api.client.CreatePCJ.ExportStrategy)2 RyaSubGraph (org.apache.rya.api.domain.RyaSubGraph)2 ProducerRecord (org.apache.kafka.clients.producer.ProducerRecord)1 RecordMetadata (org.apache.kafka.clients.producer.RecordMetadata)1 VisibilityBindingSet (org.apache.rya.api.model.VisibilityBindingSet)1