Search in sources :

Example 1 with Bitmap

use of org.neo4j.kernel.api.impl.labelscan.bitmaps.Bitmap in project neo4j by neo4j.

the class NodeRangeDocumentLabelScanStorageStrategyTest method shouldUpdateDocumentsForReLabeledNodes.

@Test
public void shouldUpdateDocumentsForReLabeledNodes() throws Exception {
    // given
    Document givenDoc = new Document();
    format.addRangeValuesField(givenDoc, 0);
    format.addLabelFields(givenDoc, "7", 0x70L);
    WritableDatabaseLabelScanIndex index = mock(WritableDatabaseLabelScanIndex.class);
    IndexWriter indexWriter = mock(IndexWriter.class);
    WritableIndexPartition partition = newIndexPartitionMock(indexWriter, givenDoc);
    when(index.getPartitions()).thenReturn(Collections.singletonList(partition));
    LabelScanWriter writer = new PartitionedLuceneLabelScanWriter(index, format);
    // when
    writer.write(labelChanges(0, labels(), labels(7, 8)));
    writer.close();
    // then
    Document thenDoc = new Document();
    format.addRangeValuesField(thenDoc, 0);
    format.addLabelFields(thenDoc, "7", 0x71L);
    format.addLabelAndSearchFields(thenDoc, 8, new Bitmap(0x01L));
    verify(indexWriter).updateDocument(eq(format.rangeTerm(0)), match(thenDoc));
}
Also used : Bitmap(org.neo4j.kernel.api.impl.labelscan.bitmaps.Bitmap) WritableDatabaseLabelScanIndex(org.neo4j.kernel.api.impl.labelscan.WritableDatabaseLabelScanIndex) IndexWriter(org.apache.lucene.index.IndexWriter) PartitionedLuceneLabelScanWriter(org.neo4j.kernel.api.impl.labelscan.writer.PartitionedLuceneLabelScanWriter) WritableIndexPartition(org.neo4j.kernel.api.impl.index.partition.WritableIndexPartition) Document(org.apache.lucene.document.Document) PartitionedLuceneLabelScanWriter(org.neo4j.kernel.api.impl.labelscan.writer.PartitionedLuceneLabelScanWriter) LabelScanWriter(org.neo4j.kernel.api.labelscan.LabelScanWriter) Test(org.junit.Test)

Example 2 with Bitmap

use of org.neo4j.kernel.api.impl.labelscan.bitmaps.Bitmap in project neo4j by neo4j.

the class LuceneAllEntriesLabelScanReader method parse.

private LuceneNodeLabelRange parse(int id, Document document) {
    List<IndexableField> fields = document.getFields();
    int expectedLabelFields = fields.size() - 1;
    long[] scratchLabelIds = new long[expectedLabelFields];
    Bitmap[] scratchBitmaps = new Bitmap[expectedLabelFields];
    int i = 0;
    long rangeId = -1;
    for (IndexableField field : fields) {
        if (format.isRangeField(field)) {
            rangeId = format.rangeOf(field);
        } else if (format.isLabelBitmapField(field)) {
            scratchLabelIds[i] = format.labelId(field);
            scratchBitmaps[i] = format.readBitmap(field);
            i++;
        }
    }
    assert rangeId >= 0;
    final long[] labelIds;
    final Bitmap[] bitmaps;
    if (i < expectedLabelFields) {
        labelIds = Arrays.copyOf(scratchLabelIds, i);
        bitmaps = Arrays.copyOf(scratchBitmaps, i);
    } else {
        labelIds = scratchLabelIds;
        bitmaps = scratchBitmaps;
    }
    return LuceneNodeLabelRange.fromBitmapStructure(id, labelIds, getLongs(bitmaps, rangeId));
}
Also used : IndexableField(org.apache.lucene.index.IndexableField) Bitmap(org.neo4j.kernel.api.impl.labelscan.bitmaps.Bitmap)

Example 3 with Bitmap

use of org.neo4j.kernel.api.impl.labelscan.bitmaps.Bitmap in project neo4j by neo4j.

the class PartitionedLuceneLabelScanWriter method readLabelBitMapsInRange.

private Map<Long, /*range*/
Bitmap> readLabelBitMapsInRange(IndexSearcher searcher, long range) throws IOException {
    Map<Long, Bitmap> /*label*/
    fields = new HashMap<>();
    Term documentTerm = format.rangeTerm(range);
    TermQuery query = new TermQuery(documentTerm);
    FirstHitCollector hitCollector = new FirstHitCollector();
    searcher.search(query, hitCollector);
    if (hitCollector.hasMatched()) {
        Document document = searcher.doc(hitCollector.getMatchedDoc());
        for (IndexableField field : document.getFields()) {
            if (!format.isRangeOrLabelField(field)) {
                Long label = Long.valueOf(field.name());
                fields.put(label, format.readBitmap(field));
            }
        }
    }
    return fields;
}
Also used : IndexableField(org.apache.lucene.index.IndexableField) TermQuery(org.apache.lucene.search.TermQuery) Bitmap(org.neo4j.kernel.api.impl.labelscan.bitmaps.Bitmap) HashMap(java.util.HashMap) FirstHitCollector(org.neo4j.kernel.api.impl.index.collector.FirstHitCollector) Term(org.apache.lucene.index.Term) Document(org.apache.lucene.document.Document)

Example 4 with Bitmap

use of org.neo4j.kernel.api.impl.labelscan.bitmaps.Bitmap in project neo4j by neo4j.

the class PartitionedLuceneLabelScanWriter method flush.

private void flush() throws IOException {
    if (currentRange < 0) {
        return;
    }
    AbstractIndexPartition partition = getCurrentPartition();
    try (PartitionSearcher partitionSearcher = partition.acquireSearcher()) {
        IndexSearcher searcher = partitionSearcher.getIndexSearcher();
        Map<Long, Bitmap> /*label*/
        fields = readLabelBitMapsInRange(searcher, currentRange);
        updateFields(updates, fields);
        Document document = new Document();
        format.addRangeValuesField(document, currentRange);
        for (Map.Entry<Long, Bitmap> /*label*/
        field : fields.entrySet()) {
            // one field per label
            Bitmap value = field.getValue();
            if (value.hasContent()) {
                format.addLabelAndSearchFields(document, field.getKey(), value);
            }
        }
        if (isEmpty(document)) {
            partition.getIndexWriter().deleteDocuments(format.rangeTerm(document));
        } else {
            partition.getIndexWriter().updateDocument(format.rangeTerm(document), document);
        }
        updates.clear();
    }
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) Bitmap(org.neo4j.kernel.api.impl.labelscan.bitmaps.Bitmap) PartitionSearcher(org.neo4j.kernel.api.impl.index.partition.PartitionSearcher) Document(org.apache.lucene.document.Document) HashMap(java.util.HashMap) Map(java.util.Map) AbstractIndexPartition(org.neo4j.kernel.api.impl.index.partition.AbstractIndexPartition)

Example 5 with Bitmap

use of org.neo4j.kernel.api.impl.labelscan.bitmaps.Bitmap in project neo4j by neo4j.

the class PartitionedLuceneLabelScanWriter method setLabels.

private void setLabels(Map<Long, Bitmap> fields, NodeLabelUpdate update) {
    for (long label : update.getLabelsAfter()) {
        Bitmap bitmap = fields.get(label);
        if (bitmap == null) {
            fields.put(label, bitmap = new Bitmap());
        }
        format.bitmapFormat().set(bitmap, update.getNodeId(), true);
    }
}
Also used : Bitmap(org.neo4j.kernel.api.impl.labelscan.bitmaps.Bitmap)

Aggregations

Bitmap (org.neo4j.kernel.api.impl.labelscan.bitmaps.Bitmap)6 Document (org.apache.lucene.document.Document)4 HashMap (java.util.HashMap)2 IndexWriter (org.apache.lucene.index.IndexWriter)2 IndexableField (org.apache.lucene.index.IndexableField)2 Test (org.junit.Test)2 WritableIndexPartition (org.neo4j.kernel.api.impl.index.partition.WritableIndexPartition)2 WritableDatabaseLabelScanIndex (org.neo4j.kernel.api.impl.labelscan.WritableDatabaseLabelScanIndex)2 PartitionedLuceneLabelScanWriter (org.neo4j.kernel.api.impl.labelscan.writer.PartitionedLuceneLabelScanWriter)2 LabelScanWriter (org.neo4j.kernel.api.labelscan.LabelScanWriter)2 Map (java.util.Map)1 Term (org.apache.lucene.index.Term)1 IndexSearcher (org.apache.lucene.search.IndexSearcher)1 TermQuery (org.apache.lucene.search.TermQuery)1 FirstHitCollector (org.neo4j.kernel.api.impl.index.collector.FirstHitCollector)1 AbstractIndexPartition (org.neo4j.kernel.api.impl.index.partition.AbstractIndexPartition)1 PartitionSearcher (org.neo4j.kernel.api.impl.index.partition.PartitionSearcher)1