Search in sources :

Example 1 with SearchFailedException

use of org.apache.nifi.provenance.index.SearchFailedException in project nifi by apache.

the class QueryTask method readDocuments.

private Tuple<List<ProvenanceEventRecord>, Integer> readDocuments(final TopDocs topDocs, final IndexReader indexReader) {
    // If no topDocs is supplied, just provide a Tuple that has no records and a hit count of 0.
    if (topDocs == null || topDocs.totalHits == 0) {
        return new Tuple<>(Collections.<ProvenanceEventRecord>emptyList(), 0);
    }
    final long start = System.nanoTime();
    final List<Long> eventIds = Arrays.stream(topDocs.scoreDocs).mapToInt(scoreDoc -> scoreDoc.doc).mapToObj(docId -> {
        try {
            return indexReader.document(docId, LUCENE_FIELDS_TO_LOAD);
        } catch (final Exception e) {
            throw new SearchFailedException("Failed to read Provenance Events from Event File", e);
        }
    }).map(doc -> doc.getField(SearchableFields.Identifier.getSearchableFieldName()).numericValue().longValue()).collect(Collectors.toList());
    final long endConvert = System.nanoTime();
    final long ms = TimeUnit.NANOSECONDS.toMillis(endConvert - start);
    logger.debug("Converting documents took {} ms", ms);
    List<ProvenanceEventRecord> events;
    try {
        events = eventStore.getEvents(eventIds, authorizer, transformer);
    } catch (IOException e) {
        throw new SearchFailedException("Unable to retrieve events from the Provenance Store", e);
    }
    final long fetchEventNanos = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - endConvert);
    logger.debug("Fetching {} events from Event Store took {} ms ({} events actually fetched)", eventIds.size(), fetchEventNanos, events.size());
    final int totalHits = topDocs.totalHits;
    return new Tuple<>(events, totalHits);
}
Also used : Query(org.apache.lucene.search.Query) TopDocs(org.apache.lucene.search.TopDocs) EventIndexSearcher(org.apache.nifi.provenance.index.EventIndexSearcher) Arrays(java.util.Arrays) EventTransformer(org.apache.nifi.provenance.authorization.EventTransformer) Logger(org.slf4j.Logger) SearchFailedException(org.apache.nifi.provenance.index.SearchFailedException) LoggerFactory(org.slf4j.LoggerFactory) Set(java.util.Set) IOException(java.io.IOException) SearchableFields(org.apache.nifi.provenance.SearchableFields) Collectors(java.util.stream.Collectors) File(java.io.File) FileNotFoundException(java.io.FileNotFoundException) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) ProvenanceEventRecord(org.apache.nifi.provenance.ProvenanceEventRecord) Tuple(org.apache.nifi.util.Tuple) IndexManager(org.apache.nifi.provenance.lucene.IndexManager) EventStore(org.apache.nifi.provenance.store.EventStore) ProgressiveResult(org.apache.nifi.provenance.ProgressiveResult) Collections(java.util.Collections) IndexReader(org.apache.lucene.index.IndexReader) EventAuthorizer(org.apache.nifi.provenance.authorization.EventAuthorizer) SearchFailedException(org.apache.nifi.provenance.index.SearchFailedException) ProvenanceEventRecord(org.apache.nifi.provenance.ProvenanceEventRecord) IOException(java.io.IOException) Tuple(org.apache.nifi.util.Tuple) SearchFailedException(org.apache.nifi.provenance.index.SearchFailedException) IOException(java.io.IOException) FileNotFoundException(java.io.FileNotFoundException)

Aggregations

File (java.io.File)1 FileNotFoundException (java.io.FileNotFoundException)1 IOException (java.io.IOException)1 Arrays (java.util.Arrays)1 Collections (java.util.Collections)1 List (java.util.List)1 Set (java.util.Set)1 TimeUnit (java.util.concurrent.TimeUnit)1 Collectors (java.util.stream.Collectors)1 IndexReader (org.apache.lucene.index.IndexReader)1 Query (org.apache.lucene.search.Query)1 TopDocs (org.apache.lucene.search.TopDocs)1 ProgressiveResult (org.apache.nifi.provenance.ProgressiveResult)1 ProvenanceEventRecord (org.apache.nifi.provenance.ProvenanceEventRecord)1 SearchableFields (org.apache.nifi.provenance.SearchableFields)1 EventAuthorizer (org.apache.nifi.provenance.authorization.EventAuthorizer)1 EventTransformer (org.apache.nifi.provenance.authorization.EventTransformer)1 EventIndexSearcher (org.apache.nifi.provenance.index.EventIndexSearcher)1 SearchFailedException (org.apache.nifi.provenance.index.SearchFailedException)1 IndexManager (org.apache.nifi.provenance.lucene.IndexManager)1