Search in sources :

Example 1 with CloseableIterable

use of org.apache.jackrabbit.oak.plugins.document.util.CloseableIterable in project jackrabbit-oak by apache.

the class MongoDocumentTraverser method getAllDocuments.

public <T extends Document> CloseableIterable<T> getAllDocuments(Collection<T> collection, Predicate<String> filter) {
    if (!disableReadOnlyCheck) {
        checkState(mongoStore.isReadOnly(), "Traverser can only be used with readOnly store");
    }
    MongoCollection<BasicDBObject> dbCollection = mongoStore.getDBCollection(collection);
    // TODO This may lead to reads being routed to secondary depending on MongoURI
    // So caller must ensure that its safe to read from secondary
    Iterable<BasicDBObject> cursor = dbCollection.withReadPreference(mongoStore.getConfiguredReadPreference(collection)).find();
    CloseableIterable<BasicDBObject> closeableCursor = CloseableIterable.wrap(cursor);
    cursor = closeableCursor;
    @SuppressWarnings("Guava") Iterable<T> result = FluentIterable.from(cursor).filter(o -> filter.test((String) o.get(Document.ID))).transform(o -> {
        T doc = mongoStore.convertFromDBObject(collection, o);
        // TODO Review the cache update approach where tracker has to track *all* docs
        if (collection == Collection.NODES) {
            NodeDocument nodeDoc = (NodeDocument) doc;
            getNodeDocCache().put(nodeDoc);
        }
        return doc;
    });
    return CloseableIterable.wrap(result, closeableCursor);
}
Also used : Collection(org.apache.jackrabbit.oak.plugins.document.Collection) CloseableIterable(org.apache.jackrabbit.oak.plugins.document.util.CloseableIterable) MongoCollection(com.mongodb.client.MongoCollection) FluentIterable(com.google.common.collect.FluentIterable) Predicate(java.util.function.Predicate) Document(org.apache.jackrabbit.oak.plugins.document.Document) BasicDBObject(com.mongodb.BasicDBObject) NodeDocumentCache(org.apache.jackrabbit.oak.plugins.document.cache.NodeDocumentCache) NodeDocument(org.apache.jackrabbit.oak.plugins.document.NodeDocument) Preconditions.checkState(com.google.common.base.Preconditions.checkState) BasicDBObject(com.mongodb.BasicDBObject) NodeDocument(org.apache.jackrabbit.oak.plugins.document.NodeDocument)

Example 2 with CloseableIterable

use of org.apache.jackrabbit.oak.plugins.document.util.CloseableIterable in project jackrabbit-oak by apache.

the class MongoMissingLastRevSeeker method getCandidates.

@Override
@Nonnull
public CloseableIterable<NodeDocument> getCandidates(final long startTime) {
    Bson query = Filters.gte(NodeDocument.MODIFIED_IN_SECS, NodeDocument.getModifiedInSecs(startTime));
    Bson sortFields = new BasicDBObject(NodeDocument.MODIFIED_IN_SECS, 1);
    FindIterable<BasicDBObject> cursor = getNodeCollection().withReadPreference(ReadPreference.primary()).find(query).sort(sortFields);
    return CloseableIterable.wrap(transform(cursor, input -> store.convertFromDBObject(NODES, input)));
}
Also used : ReadPreference(com.mongodb.ReadPreference) Iterables.transform(com.google.common.collect.Iterables.transform) MongoCollection(com.mongodb.client.MongoCollection) BasicDBObject(com.mongodb.BasicDBObject) Filters(com.mongodb.client.model.Filters) Bson(org.bson.conversions.Bson) CloseableIterable(org.apache.jackrabbit.oak.plugins.document.util.CloseableIterable) Clock(org.apache.jackrabbit.oak.stats.Clock) MissingLastRevSeeker(org.apache.jackrabbit.oak.plugins.document.MissingLastRevSeeker) FindIterable(com.mongodb.client.FindIterable) ClusterNodeInfo(org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfo) NodeDocument(org.apache.jackrabbit.oak.plugins.document.NodeDocument) Nonnull(javax.annotation.Nonnull) CLUSTER_NODES(org.apache.jackrabbit.oak.plugins.document.Collection.CLUSTER_NODES) NODES(org.apache.jackrabbit.oak.plugins.document.Collection.NODES) BasicDBObject(com.mongodb.BasicDBObject) Bson(org.bson.conversions.Bson) Nonnull(javax.annotation.Nonnull)

Example 3 with CloseableIterable

use of org.apache.jackrabbit.oak.plugins.document.util.CloseableIterable in project jackrabbit-oak by apache.

the class MongoVersionGCSupport method getPossiblyDeletedDocs.

@Override
public CloseableIterable<NodeDocument> getPossiblyDeletedDocs(final long fromModified, final long toModified) {
    // _deletedOnce == true && _modified >= fromModified && _modified < toModified
    Bson query = Filters.and(Filters.eq(DELETED_ONCE, true), Filters.gte(MODIFIED_IN_SECS, getModifiedInSecs(fromModified)), Filters.lt(MODIFIED_IN_SECS, getModifiedInSecs(toModified)));
    FindIterable<BasicDBObject> cursor = getNodeCollection().withReadPreference(ReadPreference.secondaryPreferred()).find(query).batchSize(batchSize);
    return CloseableIterable.wrap(transform(cursor, input -> store.convertFromDBObject(NODES, input)));
}
Also used : ReadPreference(com.mongodb.ReadPreference) MongoCollection(com.mongodb.client.MongoCollection) Iterables.transform(com.google.common.collect.Iterables.transform) LoggerFactory(org.slf4j.LoggerFactory) SD_MAX_REV_TIME_IN_SECS(org.apache.jackrabbit.oak.plugins.document.NodeDocument.SD_MAX_REV_TIME_IN_SECS) MODIFIED_IN_SECS(org.apache.jackrabbit.oak.plugins.document.NodeDocument.MODIFIED_IN_SECS) ArrayList(java.util.ArrayList) Collections.singletonList(java.util.Collections.singletonList) Filters(com.mongodb.client.model.Filters) Bson(org.bson.conversions.Bson) Utils(org.apache.jackrabbit.oak.plugins.document.util.Utils) Lists(com.google.common.collect.Lists) CloseableIterable(org.apache.jackrabbit.oak.plugins.document.util.CloseableIterable) StandardSystemProperty(com.google.common.base.StandardSystemProperty) SD_TYPE(org.apache.jackrabbit.oak.plugins.document.NodeDocument.SD_TYPE) NodeDocument(org.apache.jackrabbit.oak.plugins.document.NodeDocument) SplitDocType(org.apache.jackrabbit.oak.plugins.document.NodeDocument.SplitDocType) RevisionVector(org.apache.jackrabbit.oak.plugins.document.RevisionVector) Nonnull(javax.annotation.Nonnull) PATH(org.apache.jackrabbit.oak.plugins.document.NodeDocument.PATH) Revision(org.apache.jackrabbit.oak.plugins.document.Revision) NODES(org.apache.jackrabbit.oak.plugins.document.Collection.NODES) DELETED_ONCE(org.apache.jackrabbit.oak.plugins.document.NodeDocument.DELETED_ONCE) Function(com.google.common.base.Function) SplitDocumentCleanUp(org.apache.jackrabbit.oak.plugins.document.SplitDocumentCleanUp) Logger(org.slf4j.Logger) NodeDocument.getModifiedInSecs(org.apache.jackrabbit.oak.plugins.document.NodeDocument.getModifiedInSecs) BasicDBObject(com.mongodb.BasicDBObject) Set(java.util.Set) VersionGCSupport(org.apache.jackrabbit.oak.plugins.document.VersionGCSupport) VersionGCStats(org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector.VersionGCStats) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Block(com.mongodb.Block) Clock(org.apache.jackrabbit.oak.stats.Clock) Predicate(com.google.common.base.Predicate) FindIterable(com.mongodb.client.FindIterable) Document(org.apache.jackrabbit.oak.plugins.document.Document) ID(org.apache.jackrabbit.oak.plugins.document.Document.ID) Pattern(java.util.regex.Pattern) Iterables.filter(com.google.common.collect.Iterables.filter) Joiner(com.google.common.base.Joiner) DEFAULT_NO_BRANCH(org.apache.jackrabbit.oak.plugins.document.NodeDocument.SplitDocType.DEFAULT_NO_BRANCH) BasicDBObject(com.mongodb.BasicDBObject) Bson(org.bson.conversions.Bson)

Aggregations

BasicDBObject (com.mongodb.BasicDBObject)3 MongoCollection (com.mongodb.client.MongoCollection)3 NodeDocument (org.apache.jackrabbit.oak.plugins.document.NodeDocument)3 CloseableIterable (org.apache.jackrabbit.oak.plugins.document.util.CloseableIterable)3 Iterables.transform (com.google.common.collect.Iterables.transform)2 ReadPreference (com.mongodb.ReadPreference)2 FindIterable (com.mongodb.client.FindIterable)2 Filters (com.mongodb.client.model.Filters)2 Nonnull (javax.annotation.Nonnull)2 NODES (org.apache.jackrabbit.oak.plugins.document.Collection.NODES)2 Document (org.apache.jackrabbit.oak.plugins.document.Document)2 Clock (org.apache.jackrabbit.oak.stats.Clock)2 Bson (org.bson.conversions.Bson)2 Function (com.google.common.base.Function)1 Joiner (com.google.common.base.Joiner)1 Preconditions.checkState (com.google.common.base.Preconditions.checkState)1 Predicate (com.google.common.base.Predicate)1 StandardSystemProperty (com.google.common.base.StandardSystemProperty)1 FluentIterable (com.google.common.collect.FluentIterable)1 Iterables.filter (com.google.common.collect.Iterables.filter)1