Search in sources :

Example 1 with Cursor

use of com.torodb.core.cursors.Cursor in project torodb by torodb.

the class FindImplementation method apply.

@Override
public Status<FindResult> apply(Request req, Command<? super FindArgument, ? super FindResult> command, FindArgument arg, MongodTransaction context) {
    logFindCommand(arg);
    BsonDocument filter = arg.getFilter();
    Cursor<BsonDocument> cursor;
    switch(filter.size()) {
        case 0:
            {
                cursor = context.getTorodTransaction().findAll(req.getDatabase(), arg.getCollection()).asDocCursor().transform(t -> t.getRoot()).transform(ToBsonDocumentTranslator.getInstance());
                break;
            }
        case 1:
            {
                try {
                    cursor = getByAttributeCursor(context.getTorodTransaction(), req.getDatabase(), arg.getCollection(), filter).transform(ToBsonDocumentTranslator.getInstance());
                } catch (CommandFailed ex) {
                    return Status.from(ex);
                }
                break;
            }
        default:
            {
                return Status.from(ErrorCode.COMMAND_FAILED, "The given query is not supported right now");
            }
    }
    if (Long.valueOf(arg.getBatchSize()) > (long) Integer.MAX_VALUE) {
        return Status.from(ErrorCode.COMMAND_FAILED, "Only batchSize equals or lower than " + Integer.MAX_VALUE + " is supported");
    }
    OptionalLong batchSize = arg.getEffectiveBatchSize();
    List<BsonDocument> batch = cursor.getNextBatch(batchSize.isPresent() ? (int) batchSize.getAsLong() : 101);
    cursor.close();
    return Status.ok(new FindResult(CursorResult.createSingleBatchCursor(req.getDatabase(), arg.getCollection(), batch.iterator())));
}
Also used : Request(com.eightkdata.mongowp.server.api.Request) FindResult(com.torodb.mongodb.commands.signatures.general.FindCommand.FindResult) AttributeReference(com.torodb.core.language.AttributeReference) Cursor(com.torodb.core.cursors.Cursor) KvDocument(com.torodb.kvdocument.values.KvDocument) BsonDocument(com.eightkdata.mongowp.bson.BsonDocument) Command(com.eightkdata.mongowp.server.api.Command) ToBsonDocumentTranslator(com.torodb.kvdocument.conversion.mongowp.ToBsonDocumentTranslator) Singleton(javax.inject.Singleton) CursorResult(com.torodb.mongodb.commands.pojos.CursorResult) MongodTransaction(com.torodb.mongodb.core.MongodTransaction) OptionalLong(java.util.OptionalLong) KvValue(com.torodb.kvdocument.values.KvValue) List(java.util.List) Logger(org.apache.logging.log4j.Logger) Status(com.eightkdata.mongowp.Status) FindArgument(com.torodb.mongodb.commands.signatures.general.FindCommand.FindArgument) TorodTransaction(com.torodb.torod.TorodTransaction) Builder(com.torodb.core.language.AttributeReference.Builder) ErrorCode(com.eightkdata.mongowp.ErrorCode) ReadTorodbCommandImpl(com.torodb.mongodb.commands.impl.ReadTorodbCommandImpl) CommandFailed(com.eightkdata.mongowp.exceptions.CommandFailed) LogManager(org.apache.logging.log4j.LogManager) BsonDocument(com.eightkdata.mongowp.bson.BsonDocument) CommandFailed(com.eightkdata.mongowp.exceptions.CommandFailed) OptionalLong(java.util.OptionalLong) FindResult(com.torodb.mongodb.commands.signatures.general.FindCommand.FindResult)

Example 2 with Cursor

use of com.torodb.core.cursors.Cursor in project torodb by torodb.

the class UpdateImplementation method apply.

@Override
public Status<UpdateResult> apply(Request req, Command<? super UpdateArgument, ? super UpdateResult> command, UpdateArgument arg, WriteMongodTransaction context) {
    UpdateStatus updateStatus = new UpdateStatus();
    try {
        if (!context.getTorodTransaction().existsCollection(req.getDatabase(), arg.getCollection())) {
            context.getTorodTransaction().createIndex(req.getDatabase(), arg.getCollection(), Constants.ID_INDEX, ImmutableList.<IndexFieldInfo>of(new IndexFieldInfo(new AttributeReference(Arrays.asList(new Key[] { new ObjectKey(Constants.ID) })), FieldIndexOrdering.ASC.isAscending())), true);
        }
        for (UpdateStatement updateStatement : arg.getStatements()) {
            BsonDocument query = updateStatement.getQuery();
            UpdateAction updateAction = UpdateActionTranslator.translate(updateStatement.getUpdate());
            Cursor<ToroDocument> candidatesCursor;
            switch(query.size()) {
                case 0:
                    {
                        candidatesCursor = context.getTorodTransaction().findAll(req.getDatabase(), arg.getCollection()).asDocCursor();
                        break;
                    }
                case 1:
                    {
                        try {
                            candidatesCursor = findByAttribute(context.getTorodTransaction(), req.getDatabase(), arg.getCollection(), query);
                        } catch (CommandFailed ex) {
                            return Status.from(ex);
                        }
                        break;
                    }
                default:
                    {
                        return Status.from(ErrorCode.COMMAND_FAILED, "The given query is not supported right now");
                    }
            }
            if (candidatesCursor.hasNext()) {
                try {
                    Stream<List<ToroDocument>> candidatesbatchStream;
                    if (updateStatement.isMulti()) {
                        candidatesbatchStream = StreamSupport.stream(Spliterators.spliteratorUnknownSize(candidatesCursor.batch(100), Spliterator.ORDERED), false);
                    } else {
                        candidatesbatchStream = Stream.of(ImmutableList.of(candidatesCursor.next()));
                    }
                    Stream<KvDocument> updatedCandidates = candidatesbatchStream.map(candidates -> {
                        updateStatus.increaseCandidates(candidates.size());
                        context.getTorodTransaction().delete(req.getDatabase(), arg.getCollection(), candidates);
                        return candidates;
                    }).flatMap(l -> l.stream()).map(candidate -> {
                        try {
                            updateStatus.increaseUpdated();
                            return update(updateAction, candidate);
                        } catch (UserException userException) {
                            throw new UserWrappedException(userException);
                        }
                    });
                    context.getTorodTransaction().insert(req.getDatabase(), arg.getCollection(), updatedCandidates);
                } catch (UserWrappedException userWrappedException) {
                    throw userWrappedException.getCause();
                }
            } else if (updateStatement.isUpsert()) {
                KvDocument toInsertCandidate;
                if (updateAction instanceof SetDocumentUpdateAction) {
                    toInsertCandidate = ((SetDocumentUpdateAction) updateAction).getNewValue();
                } else {
                    toInsertCandidate = update(updateAction, new ToroDocument(-1, (KvDocument) MongoWpConverter.translate(query)));
                }
                if (!toInsertCandidate.containsKey(Constants.ID)) {
                    KvDocument.Builder builder = new KvDocument.Builder();
                    for (DocEntry<?> entry : toInsertCandidate) {
                        builder.putValue(entry.getKey(), entry.getValue());
                    }
                    builder.putValue(Constants.ID, MongoWpConverter.translate(objectIdFactory.consumeObjectId()));
                    toInsertCandidate = builder.build();
                }
                updateStatus.increaseCandidates(1);
                updateStatus.increaseCreated(toInsertCandidate.get(Constants.ID));
                Stream<KvDocument> toInsertCandidates = Stream.of(toInsertCandidate);
                context.getTorodTransaction().insert(req.getDatabase(), arg.getCollection(), toInsertCandidates);
            }
        }
    } catch (UserException ex) {
        //TODO: Improve error reporting
        return Status.from(ErrorCode.COMMAND_FAILED, ex.getLocalizedMessage());
    }
    mongodMetrics.getUpdateModified().mark(updateStatus.updated);
    mongodMetrics.getUpdateMatched().mark(updateStatus.candidates);
    mongodMetrics.getUpdateUpserted().mark(updateStatus.upsertResults.size());
    return Status.ok(new UpdateResult(updateStatus.updated, updateStatus.candidates, ImmutableList.copyOf(updateStatus.upsertResults)));
}
Also used : Request(com.eightkdata.mongowp.server.api.Request) UpdateActionTranslator(com.torodb.mongodb.language.UpdateActionTranslator) Arrays(java.util.Arrays) UpdatedToroDocumentBuilder(com.torodb.mongodb.language.update.UpdatedToroDocumentBuilder) FieldIndexOrdering(com.torodb.core.transaction.metainf.FieldIndexOrdering) Spliterators(java.util.Spliterators) BsonDocument(com.eightkdata.mongowp.bson.BsonDocument) WriteTorodbCommandImpl(com.torodb.mongodb.commands.impl.WriteTorodbCommandImpl) UpdateAction(com.torodb.mongodb.language.update.UpdateAction) UpdateStatement(com.torodb.mongodb.commands.signatures.general.UpdateCommand.UpdateStatement) Singleton(javax.inject.Singleton) ToroDocument(com.torodb.core.document.ToroDocument) UpdateResult(com.torodb.mongodb.commands.signatures.general.UpdateCommand.UpdateResult) ArrayList(java.util.ArrayList) ObjectKey(com.torodb.core.language.AttributeReference.ObjectKey) IndexFieldInfo(com.torodb.torod.IndexFieldInfo) Inject(javax.inject.Inject) KvValue(com.torodb.kvdocument.values.KvValue) ImmutableList(com.google.common.collect.ImmutableList) MongoWpConverter(com.torodb.kvdocument.conversion.mongowp.MongoWpConverter) StreamSupport(java.util.stream.StreamSupport) Builder(com.torodb.core.language.AttributeReference.Builder) ErrorCode(com.eightkdata.mongowp.ErrorCode) CommandFailed(com.eightkdata.mongowp.exceptions.CommandFailed) UpdateException(com.torodb.core.exceptions.user.UpdateException) SharedWriteTorodTransaction(com.torodb.torod.SharedWriteTorodTransaction) AttributeReference(com.torodb.core.language.AttributeReference) Constants(com.torodb.mongodb.language.Constants) Cursor(com.torodb.core.cursors.Cursor) KvDocument(com.torodb.kvdocument.values.KvDocument) UpsertResult(com.torodb.mongodb.commands.signatures.general.UpdateCommand.UpsertResult) Command(com.eightkdata.mongowp.server.api.Command) UserException(com.torodb.core.exceptions.user.UserException) SetDocumentUpdateAction(com.torodb.mongodb.language.update.SetDocumentUpdateAction) ObjectIdFactory(com.torodb.mongodb.language.ObjectIdFactory) DocEntry(com.torodb.kvdocument.values.KvDocument.DocEntry) MongodMetrics(com.torodb.mongodb.core.MongodMetrics) WriteMongodTransaction(com.torodb.mongodb.core.WriteMongodTransaction) List(java.util.List) Stream(java.util.stream.Stream) Status(com.eightkdata.mongowp.Status) UserWrappedException(com.torodb.core.exceptions.UserWrappedException) UpdateArgument(com.torodb.mongodb.commands.signatures.general.UpdateCommand.UpdateArgument) Spliterator(java.util.Spliterator) Key(com.torodb.core.language.AttributeReference.Key) KvDocument(com.torodb.kvdocument.values.KvDocument) UpdateStatement(com.torodb.mongodb.commands.signatures.general.UpdateCommand.UpdateStatement) UpdateAction(com.torodb.mongodb.language.update.UpdateAction) SetDocumentUpdateAction(com.torodb.mongodb.language.update.SetDocumentUpdateAction) AttributeReference(com.torodb.core.language.AttributeReference) UpdatedToroDocumentBuilder(com.torodb.mongodb.language.update.UpdatedToroDocumentBuilder) Builder(com.torodb.core.language.AttributeReference.Builder) ObjectKey(com.torodb.core.language.AttributeReference.ObjectKey) DocEntry(com.torodb.kvdocument.values.KvDocument.DocEntry) BsonDocument(com.eightkdata.mongowp.bson.BsonDocument) UserWrappedException(com.torodb.core.exceptions.UserWrappedException) ToroDocument(com.torodb.core.document.ToroDocument) CommandFailed(com.eightkdata.mongowp.exceptions.CommandFailed) ArrayList(java.util.ArrayList) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) Stream(java.util.stream.Stream) IndexFieldInfo(com.torodb.torod.IndexFieldInfo) UserException(com.torodb.core.exceptions.user.UserException) UpdateResult(com.torodb.mongodb.commands.signatures.general.UpdateCommand.UpdateResult) SetDocumentUpdateAction(com.torodb.mongodb.language.update.SetDocumentUpdateAction)

Example 3 with Cursor

use of com.torodb.core.cursors.Cursor in project torodb by torodb.

the class AbstractReadInterface method getCollectionDidsWithFieldsInBatch.

@SuppressFBWarnings(value = { "OBL_UNSATISFIED_OBLIGATION", "ODR_OPEN_DATABASE_RESOURCE" }, justification = "ResultSet is wrapped in a Cursor<Integer>. It's iterated and closed in caller code")
private Cursor<Integer> getCollectionDidsWithFieldsInBatch(DSLContext dsl, MetaDatabase metaDatabase, MetaCollection metaCol, MetaDocPart metaDocPart, Multimap<MetaField, KvValue<?>> valuesMultimap) throws SQLException {
    @SuppressWarnings("checkstyle:LineLength") Provider<Stream<Map.Entry<MetaField, Collection<KvValue<?>>>>> valuesMultimapSortedStreamProvider = () -> valuesMultimap.asMap().entrySet().stream().sorted((e1, e2) -> e1.getKey().getIdentifier().compareTo(e2.getKey().getIdentifier()));
    String statement = getReadCollectionDidsWithFieldInStatement(metaDatabase.getIdentifier(), metaDocPart.getIdentifier(), valuesMultimapSortedStreamProvider.get().map(e -> new Tuple2<String, Integer>(e.getKey().getIdentifier(), e.getValue().size())));
    Connection connection = dsl.configuration().connectionProvider().acquire();
    try {
        PreparedStatement preparedStatement = connection.prepareStatement(statement);
        int parameterIndex = 1;
        Iterator<Map.Entry<MetaField, Collection<KvValue<?>>>> valuesMultimapSortedIterator = valuesMultimapSortedStreamProvider.get().iterator();
        while (valuesMultimapSortedIterator.hasNext()) {
            Map.Entry<MetaField, Collection<KvValue<?>>> valuesMultimapEntry = valuesMultimapSortedIterator.next();
            for (KvValue<?> value : valuesMultimapEntry.getValue()) {
                sqlHelper.setPreparedStatementValue(preparedStatement, parameterIndex, valuesMultimapEntry.getKey().getType(), value);
                parameterIndex++;
            }
        }
        return new DefaultDidCursor(errorHandler, preparedStatement.executeQuery());
    } finally {
        dsl.configuration().connectionProvider().release(connection);
    }
}
Also used : ArrayListMultimap(com.google.common.collect.ArrayListMultimap) Connection(java.sql.Connection) Provider(javax.inject.Provider) IteratorCursor(com.torodb.core.cursors.IteratorCursor) Multimap(com.google.common.collect.Multimap) Seq(org.jooq.lambda.Seq) Singleton(javax.inject.Singleton) ArrayList(java.util.ArrayList) TableRefFactory(com.torodb.core.TableRefFactory) KvValue(com.torodb.kvdocument.values.KvValue) Tuple2(org.jooq.lambda.tuple.Tuple2) SQLException(java.sql.SQLException) MetaDatabase(com.torodb.core.transaction.metainf.MetaDatabase) ResultSetDocPartResult(com.torodb.backend.d2r.ResultSetDocPartResult) ResultSet(java.sql.ResultSet) Map(java.util.Map) DSLContext(org.jooq.DSLContext) Context(com.torodb.backend.ErrorHandler.Context) Nonnull(javax.annotation.Nonnull) MetaCollection(com.torodb.core.transaction.metainf.MetaCollection) Unchecked(org.jooq.lambda.Unchecked) Iterator(java.util.Iterator) Cursor(com.torodb.core.cursors.Cursor) Collection(java.util.Collection) EmptyCursor(com.torodb.core.cursors.EmptyCursor) DocPartTableFields(com.torodb.backend.tables.MetaDocPartTable.DocPartTableFields) PreparedStatement(java.sql.PreparedStatement) Collectors(java.util.stream.Collectors) TableRef(com.torodb.core.TableRef) List(java.util.List) MetaField(com.torodb.core.transaction.metainf.MetaField) Stream(java.util.stream.Stream) DocPartResult(com.torodb.core.d2r.DocPartResult) Entry(java.util.Map.Entry) MetaDocPart(com.torodb.core.transaction.metainf.MetaDocPart) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings) MetaField(com.torodb.core.transaction.metainf.MetaField) Connection(java.sql.Connection) PreparedStatement(java.sql.PreparedStatement) Entry(java.util.Map.Entry) Tuple2(org.jooq.lambda.tuple.Tuple2) MetaCollection(com.torodb.core.transaction.metainf.MetaCollection) Collection(java.util.Collection) Stream(java.util.stream.Stream) Map(java.util.Map) KvValue(com.torodb.kvdocument.values.KvValue) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings)

Aggregations

Cursor (com.torodb.core.cursors.Cursor)3 KvValue (com.torodb.kvdocument.values.KvValue)3 List (java.util.List)3 Singleton (javax.inject.Singleton)3 ErrorCode (com.eightkdata.mongowp.ErrorCode)2 Status (com.eightkdata.mongowp.Status)2 BsonDocument (com.eightkdata.mongowp.bson.BsonDocument)2 CommandFailed (com.eightkdata.mongowp.exceptions.CommandFailed)2 Command (com.eightkdata.mongowp.server.api.Command)2 Request (com.eightkdata.mongowp.server.api.Request)2 AttributeReference (com.torodb.core.language.AttributeReference)2 Builder (com.torodb.core.language.AttributeReference.Builder)2 KvDocument (com.torodb.kvdocument.values.KvDocument)2 ArrayListMultimap (com.google.common.collect.ArrayListMultimap)1 ImmutableList (com.google.common.collect.ImmutableList)1 Multimap (com.google.common.collect.Multimap)1 Context (com.torodb.backend.ErrorHandler.Context)1 ResultSetDocPartResult (com.torodb.backend.d2r.ResultSetDocPartResult)1 DocPartTableFields (com.torodb.backend.tables.MetaDocPartTable.DocPartTableFields)1 TableRef (com.torodb.core.TableRef)1