Search in sources :

Example 11 with MongoException

use of com.eightkdata.mongowp.exceptions.MongoException in project torodb by torodb.

the class RecoveryService method cloneDatabases.

private void cloneDatabases(@Nonnull MongoClient remoteClient) throws CloningException, MongoException, UserException {
    enableDataImportMode();
    try {
        Stream<String> dbNames;
        try (MongoConnection remoteConnection = remoteClient.openConnection()) {
            RemoteCommandResponse<ListDatabasesReply> remoteResponse = remoteConnection.execute(ListDatabasesCommand.INSTANCE, "admin", true, Empty.getInstance());
            if (!remoteResponse.isOk()) {
                throw remoteResponse.asMongoException();
            }
            dbNames = remoteResponse.getCommandReply().get().getDatabases().stream().map(db -> db.getName());
        }
        dbNames.filter(this::isReplicable).forEach(databaseName -> {
            MyWritePermissionSupplier writePermissionSupplier = new MyWritePermissionSupplier(databaseName);
            CloneOptions options = new CloneOptions(true, true, true, false, databaseName, Collections.<String>emptySet(), writePermissionSupplier, (colName) -> replFilters.getCollectionPredicate().test(databaseName, colName), (collection, indexName, unique, keys) -> replFilters.getIndexPredicate().test(databaseName, collection, indexName, unique, keys));
            try {
                cloner.cloneDatabase(databaseName, remoteClient, server, options);
            } catch (MongoException ex) {
                throw new CloningException(ex);
            }
        });
    } finally {
        disableDataImportMode();
    }
}
Also used : RunnableTorodbService(com.torodb.core.services.RunnableTorodbService) ReadOplogTransaction(com.torodb.mongodb.repl.OplogManager.ReadOplogTransaction) MongoConnection(com.eightkdata.mongowp.client.core.MongoConnection) RollbackException(com.torodb.core.transaction.RollbackException) Assisted(com.google.inject.assistedinject.Assisted) OplogOperation(com.eightkdata.mongowp.server.api.oplog.OplogOperation) MongoDbRepl(com.torodb.mongodb.repl.guice.MongoDbRepl) MongoClient(com.eightkdata.mongowp.client.core.MongoClient) Empty(com.eightkdata.mongowp.server.api.tools.Empty) RollbackReplicationException(com.torodb.mongodb.repl.oplogreplier.RollbackReplicationException) MongoException(com.eightkdata.mongowp.exceptions.MongoException) OplogOperationUnsupported(com.eightkdata.mongowp.exceptions.OplogOperationUnsupported) ThreadFactory(java.util.concurrent.ThreadFactory) OplogFetcher(com.torodb.mongodb.repl.oplogreplier.fetcher.OplogFetcher) CancellationException(java.util.concurrent.CancellationException) CloningException(com.torodb.mongodb.utils.DbCloner.CloningException) ListDatabasesCommand(com.torodb.mongodb.commands.signatures.diagnostic.ListDatabasesCommand) OplogApplier(com.torodb.mongodb.repl.oplogreplier.OplogApplier) List(java.util.List) Logger(org.apache.logging.log4j.Logger) Stream(java.util.stream.Stream) StopReplicationException(com.torodb.mongodb.repl.oplogreplier.StopReplicationException) OpTime(com.eightkdata.mongowp.OpTime) OplogStartMissingException(com.eightkdata.mongowp.exceptions.OplogStartMissingException) Supplier(com.google.common.base.Supplier) MongodServer(com.torodb.mongodb.core.MongodServer) UnreachableMongoServerException(com.eightkdata.mongowp.client.core.UnreachableMongoServerException) Inject(javax.inject.Inject) TorodConnection(com.torodb.torod.TorodConnection) NoSyncSourceFoundException(com.torodb.mongodb.repl.exceptions.NoSyncSourceFoundException) ApplierContext(com.torodb.mongodb.repl.oplogreplier.ApplierContext) MongoClientFactory(com.eightkdata.mongowp.client.core.MongoClientFactory) LimitedOplogFetcher(com.torodb.mongodb.repl.oplogreplier.fetcher.LimitedOplogFetcher) RemoteCommandResponse(com.eightkdata.mongowp.client.core.MongoConnection.RemoteCommandResponse) OplogManagerPersistException(com.torodb.mongodb.repl.OplogManager.OplogManagerPersistException) DbCloner(com.torodb.mongodb.utils.DbCloner) Supervisor(com.torodb.core.supervision.Supervisor) Nonnull(javax.annotation.Nonnull) CloneOptions(com.torodb.mongodb.utils.DbCloner.CloneOptions) SharedWriteTorodTransaction(com.torodb.torod.SharedWriteTorodTransaction) TorodbRunnableService(com.torodb.core.annotations.TorodbRunnableService) SupervisorDecision(com.torodb.core.supervision.SupervisorDecision) MongoCursor(com.eightkdata.mongowp.server.api.pojos.MongoCursor) UnexpectedOplogApplierException(com.torodb.mongodb.repl.oplogreplier.OplogApplier.UnexpectedOplogApplierException) UserException(com.torodb.core.exceptions.user.UserException) WriteOplogTransaction(com.torodb.mongodb.repl.OplogManager.WriteOplogTransaction) HostAndPort(com.google.common.net.HostAndPort) Status(com.eightkdata.mongowp.Status) TorodServer(com.torodb.torod.TorodServer) ListDatabasesReply(com.torodb.mongodb.commands.signatures.diagnostic.ListDatabasesCommand.ListDatabasesReply) LogManager(org.apache.logging.log4j.LogManager) Collections(java.util.Collections) CloningException(com.torodb.mongodb.utils.DbCloner.CloningException) MongoException(com.eightkdata.mongowp.exceptions.MongoException) ListDatabasesReply(com.torodb.mongodb.commands.signatures.diagnostic.ListDatabasesCommand.ListDatabasesReply) MongoConnection(com.eightkdata.mongowp.client.core.MongoConnection) CloneOptions(com.torodb.mongodb.utils.DbCloner.CloneOptions)

Example 12 with MongoException

use of com.eightkdata.mongowp.exceptions.MongoException in project torodb by torodb.

the class ReplSyncFetcher method fetch.

/**
   *
   * @param reader
   * @return true iff rollback is needed
   * @throws com.torodb.torod.mongodb.repl.ReplSyncFetcher.StopFetchException
   * @throws com.torodb.torod.mongodb.repl.ReplSyncFetcher.RestartFetchException
   */
private boolean fetch(OplogReader reader) throws StopFetchException, RestartFetchException {
    try {
        MongoCursor<OplogOperation> cursor = reader.queryGte(lastFetchedOpTime);
        Batch<OplogOperation> batch = cursor.fetchBatch();
        postBatchChecks(reader, cursor, batch);
        try {
            if (isRollbackNeeded(reader, batch, lastFetchedOpTime, lastFetchedHash)) {
                return true;
            }
            while (fetchIterationCanContinue()) {
                if (!batch.hasNext()) {
                    preBatchChecks(batch);
                    batch = cursor.fetchBatch();
                    postBatchChecks(reader, cursor, batch);
                    continue;
                }
                if (batch.hasNext()) {
                    OplogOperation nextOp = batch.next();
                    assert nextOp != null;
                    boolean delivered = false;
                    while (!delivered) {
                        try {
                            LOGGER.debug("Delivered op: {}", nextOp);
                            callback.deliver(nextOp);
                            delivered = true;
                            opsReadCounter++;
                        } catch (InterruptedException ex) {
                            LOGGER.warn(serviceName() + " interrupted while a " + "message was being to deliver. Retrying", ex);
                        }
                    }
                    lastFetchedHash = nextOp.getHash();
                    lastFetchedOpTime = nextOp.getOpTime();
                    metrics.getLastOpTimeFetched().setValue(lastFetchedOpTime.toString());
                }
            }
        } finally {
            cursor.close();
        }
    } catch (MongoException ex) {
        throw new RestartFetchException();
    }
    return false;
}
Also used : MongoException(com.eightkdata.mongowp.exceptions.MongoException) OplogOperation(com.eightkdata.mongowp.server.api.oplog.OplogOperation)

Example 13 with MongoException

use of com.eightkdata.mongowp.exceptions.MongoException in project torodb by torodb.

the class OplogOperationApplier method insertIndex.

private void insertIndex(BsonDocument indexDoc, String database, ExclusiveWriteMongodTransaction trans) throws OplogApplyingException {
    try {
        CreateIndexesCommand command = CreateIndexesCommand.INSTANCE;
        IndexOptions indexOptions = IndexOptions.unmarshall(indexDoc);
        CreateIndexesArgument arg = new CreateIndexesArgument(indexOptions.getCollection(), Arrays.asList(new IndexOptions[] { indexOptions }));
        Status executionResult = executeReplCommand(database, command, arg, trans.getTorodTransaction());
        if (!executionResult.isOk()) {
            throw new OplogApplyingException(new MongoException(executionResult));
        }
    } catch (MongoException ex) {
        throw new OplogApplyingException(ex);
    }
}
Also used : Status(com.eightkdata.mongowp.Status) MongoException(com.eightkdata.mongowp.exceptions.MongoException) CreateIndexesArgument(com.torodb.mongodb.commands.signatures.admin.CreateIndexesCommand.CreateIndexesArgument) IndexOptions(com.torodb.mongodb.commands.pojos.index.IndexOptions) CreateIndexesCommand(com.torodb.mongodb.commands.signatures.admin.CreateIndexesCommand)

Example 14 with MongoException

use of com.eightkdata.mongowp.exceptions.MongoException in project torodb by torodb.

the class OplogOperationApplier method applyCmd.

@SuppressWarnings("unchecked")
private void applyCmd(DbCmdOplogOperation op, ExclusiveWriteMongodTransaction trans, ApplierContext applierContext) throws OplogApplyingException {
    LibraryEntry librayEntry = library.find(op.getRequest());
    if (librayEntry == null) {
        throw new OplogApplyingException(new CommandNotFoundException(op.getRequest().isEmpty() ? "?" : op.getRequest().getFirstEntry().getKey()));
    }
    Command command = librayEntry.getCommand();
    if (command == null) {
        BsonDocument document = op.getRequest();
        if (document.isEmpty()) {
            throw new OplogApplyingException(new CommandNotFoundException("Empty document query"));
        }
        String firstKey = document.getFirstEntry().getKey();
        throw new OplogApplyingException(new CommandNotFoundException(firstKey));
    }
    Object arg;
    try {
        arg = command.unmarshallArg(op.getRequest(), librayEntry.getAlias());
    } catch (MongoException ex) {
        throw new OplogApplyingException(ex);
    }
    Status executionResult = executeReplCommand(op.getDatabase(), command, arg, trans.getTorodTransaction());
    if (!executionResult.isOk()) {
        throw new OplogApplyingException(new MongoException(executionResult));
    }
}
Also used : Status(com.eightkdata.mongowp.Status) LibraryEntry(com.eightkdata.mongowp.server.api.CommandsLibrary.LibraryEntry) MongoException(com.eightkdata.mongowp.exceptions.MongoException) BsonDocument(com.eightkdata.mongowp.bson.BsonDocument) DeleteCommand(com.torodb.mongodb.commands.signatures.general.DeleteCommand) Command(com.eightkdata.mongowp.server.api.Command) UpdateCommand(com.torodb.mongodb.commands.signatures.general.UpdateCommand) CreateIndexesCommand(com.torodb.mongodb.commands.signatures.admin.CreateIndexesCommand) InsertCommand(com.torodb.mongodb.commands.signatures.general.InsertCommand) CommandNotFoundException(com.eightkdata.mongowp.exceptions.CommandNotFoundException)

Example 15 with MongoException

use of com.eightkdata.mongowp.exceptions.MongoException in project torodb by torodb.

the class OplogOperationApplier method applyUpdate.

private void applyUpdate(UpdateOplogOperation op, ExclusiveWriteMongodTransaction trans, ApplierContext applierContext) throws OplogApplyingException {
    boolean upsert = op.isUpsert() || applierContext.treatUpdateAsUpsert();
    Status<UpdateResult> status;
    try {
        status = executeTorodCommand(op.getDatabase(), UpdateCommand.INSTANCE, new UpdateArgument(op.getCollection(), Collections.singletonList(new UpdateStatement(op.getFilter(), op.getModification(), upsert, true)), true, WriteConcern.fsync()), trans);
    } catch (MongoException ex) {
        throw new OplogApplyingException(ex);
    }
    if (!status.isOk()) {
        //TODO: improve error code
        throw new OplogApplyingException(new MongoException(status));
    }
    UpdateResult updateResult = status.getResult();
    assert updateResult != null;
    if (!updateResult.isOk()) {
        throw new OplogApplyingException(new MongoException(updateResult.getErrorMessage(), ErrorCode.UNKNOWN_ERROR));
    }
    if (!upsert && updateResult.getModifiedCounter() != 0) {
        LOGGER.info("Oplog update operation with optime {} and hash {} did not find the doc to " + "modify. Filter is {}", op.getOpTime(), op.getHash(), op.getFilter());
    }
    if (upsert && !updateResult.getUpserts().isEmpty()) {
        LOGGER.warn("Replication couldn't find doc for op " + op);
    }
}
Also used : UpdateStatement(com.torodb.mongodb.commands.signatures.general.UpdateCommand.UpdateStatement) MongoException(com.eightkdata.mongowp.exceptions.MongoException) UpdateArgument(com.torodb.mongodb.commands.signatures.general.UpdateCommand.UpdateArgument) UpdateResult(com.torodb.mongodb.commands.signatures.general.UpdateCommand.UpdateResult)

Aggregations

MongoException (com.eightkdata.mongowp.exceptions.MongoException)18 OplogOperation (com.eightkdata.mongowp.server.api.oplog.OplogOperation)6 Request (com.eightkdata.mongowp.server.api.Request)4 UserException (com.torodb.core.exceptions.user.UserException)4 RetrierAbortException (com.torodb.core.retrier.RetrierAbortException)4 RetrierGiveUpException (com.torodb.core.retrier.RetrierGiveUpException)4 RollbackException (com.torodb.core.transaction.RollbackException)4 OpTime (com.eightkdata.mongowp.OpTime)3 Status (com.eightkdata.mongowp.Status)3 BsonDocument (com.eightkdata.mongowp.bson.BsonDocument)3 MongoConnection (com.eightkdata.mongowp.client.core.MongoConnection)3 UnreachableMongoServerException (com.eightkdata.mongowp.client.core.UnreachableMongoServerException)3 BsonDocumentBuilder (com.eightkdata.mongowp.utils.BsonDocumentBuilder)3 HostAndPort (com.google.common.net.HostAndPort)3 WriteMongodTransaction (com.torodb.mongodb.core.WriteMongodTransaction)3 ErrorCode (com.eightkdata.mongowp.ErrorCode)2 DefaultBsonValues (com.eightkdata.mongowp.bson.utils.DefaultBsonValues)2 MongoClient (com.eightkdata.mongowp.client.core.MongoClient)2 NotMasterException (com.eightkdata.mongowp.exceptions.NotMasterException)2 OplogOperationUnsupported (com.eightkdata.mongowp.exceptions.OplogOperationUnsupported)2