Search in sources :

Example 1 with Metadata

use of oap.storage.Metadata in project oap by oaplatform.

the class MongoStorage method fsync.

@Override
public void fsync() {
    super.fsync();
    val count = new MutableInt();
    Stream.of(data.values().stream().filter(m -> m.modified >= lastFsync)).grouped(bulkSize).forEach(list -> {
        count.add(list.size());
        final List<? extends WriteModel<Metadata<T>>> bulk = Lists.map(list, metadata -> {
            val id = getIdentifier().get(metadata.object);
            return new ReplaceOneModel<>(eq("_id", new ObjectId(id)), metadata, UPDATE_OPTIONS_UPSERT);
        });
        collection.bulkWrite(bulk);
    });
    log.info("[{}] fsync total: {}, modified: {}", collection.getNamespace(), size(), count.intValue());
    lastFsync = System.currentTimeMillis();
}
Also used : lombok.val(lombok.val) ReplaceOneModel(com.mongodb.client.model.ReplaceOneModel) MutableInt(org.apache.commons.lang3.mutable.MutableInt) MongoCollection(com.mongodb.client.MongoCollection) DateTimeUtils(org.joda.time.DateTimeUtils) lombok.val(lombok.val) CodecRegistries(org.bson.codecs.configuration.CodecRegistries) WriteModel(com.mongodb.client.model.WriteModel) CodecRegistry(org.bson.codecs.configuration.CodecRegistry) Consumer(java.util.function.Consumer) Slf4j(lombok.extern.slf4j.Slf4j) MemoryStorage(oap.storage.MemoryStorage) List(java.util.List) Lists(oap.util.Lists) IdentifierBuilder(oap.storage.IdentifierBuilder) Metadata(oap.storage.Metadata) ObjectId(org.bson.types.ObjectId) Optional(java.util.Optional) Filters.eq(com.mongodb.client.model.Filters.eq) TypeReference(com.fasterxml.jackson.core.type.TypeReference) Stream(oap.util.Stream) UpdateOptions(com.mongodb.client.model.UpdateOptions) ObjectId(org.bson.types.ObjectId) MutableInt(org.apache.commons.lang3.mutable.MutableInt) ReplaceOneModel(com.mongodb.client.model.ReplaceOneModel) Metadata(oap.storage.Metadata)

Aggregations

TypeReference (com.fasterxml.jackson.core.type.TypeReference)1 MongoCollection (com.mongodb.client.MongoCollection)1 Filters.eq (com.mongodb.client.model.Filters.eq)1 ReplaceOneModel (com.mongodb.client.model.ReplaceOneModel)1 UpdateOptions (com.mongodb.client.model.UpdateOptions)1 WriteModel (com.mongodb.client.model.WriteModel)1 List (java.util.List)1 Optional (java.util.Optional)1 Consumer (java.util.function.Consumer)1 Slf4j (lombok.extern.slf4j.Slf4j)1 lombok.val (lombok.val)1 IdentifierBuilder (oap.storage.IdentifierBuilder)1 MemoryStorage (oap.storage.MemoryStorage)1 Metadata (oap.storage.Metadata)1 Lists (oap.util.Lists)1 Stream (oap.util.Stream)1 MutableInt (org.apache.commons.lang3.mutable.MutableInt)1 CodecRegistries (org.bson.codecs.configuration.CodecRegistries)1 CodecRegistry (org.bson.codecs.configuration.CodecRegistry)1 ObjectId (org.bson.types.ObjectId)1