Search in sources :

Example 6 with RollbackException

use of com.torodb.core.transaction.RollbackException in project torodb by torodb.

the class ReplSyncApplier method runProtected.

@Override
protected void runProtected() {
    runThread = Thread.currentThread();
    /*
     * TODO: In general, the replication context can be set as not reaplying. But it is not frequent
     * but possible to stop the replication after some oplog ops have been apply but not marked as
     * executed on the oplog manager. For that reason, all oplog ops betwen the last operation that
     * have been marked as applyed and the current last operation on the remote oplog must be
     * executed as replying operations. As it is not possible to do that yet, we have to always
     * apply operations as replying to be safe.
     */
    ApplierContext applierContext = new ApplierContext.Builder().setReapplying(true).setUpdatesAsUpserts(true).build();
    while (isRunning()) {
        OplogOperation lastOperation = null;
        ExclusiveWriteMongodTransaction trans = connection.openExclusiveWriteTransaction();
        try (ExclusiveWriteMongodTransaction transaction = trans) {
            try {
                for (OplogOperation opToApply : callback.takeOps()) {
                    lastOperation = opToApply;
                    LOGGER.trace("Executing {}", opToApply);
                    try {
                        boolean done = false;
                        while (!done) {
                            try {
                                oplogOpApplier.apply(opToApply, transaction, applierContext);
                                transaction.commit();
                                done = true;
                            } catch (RollbackException ex) {
                                LOGGER.debug("Recived a rollback exception while applying an oplog op", ex);
                            }
                        }
                    } catch (OplogApplyingException ex) {
                        if (!callback.failedToApply(opToApply, ex)) {
                            LOGGER.error(serviceName() + " stopped because one operation " + "cannot be executed", ex);
                            break;
                        }
                    } catch (UserException ex) {
                        if (callback.failedToApply(opToApply, ex)) {
                            LOGGER.error(serviceName() + " stopped because one operation " + "cannot be executed", ex);
                            break;
                        }
                    } catch (Throwable ex) {
                        if (callback.failedToApply(opToApply, ex)) {
                            LOGGER.error(serviceName() + " stopped because " + "an unknown error", ex);
                            break;
                        }
                    }
                    callback.markAsApplied(opToApply);
                }
            } catch (InterruptedException ex) {
                LOGGER.debug("Interrupted applier thread while applying an operator");
            }
        }
        if (lastOperation != null) {
            try (WriteOplogTransaction oplogTransaction = oplogManager.createWriteTransaction()) {
                oplogTransaction.addOperation(lastOperation);
            } catch (OplogManagerPersistException ex) {
                if (callback.failedToApply(lastOperation, ex)) {
                    LOGGER.error(serviceName() + " stopped because " + "the last applied operation couldn't " + "be persisted", ex);
                    break;
                }
            }
        }
    }
}
Also used : OplogApplyingException(com.torodb.mongodb.repl.oplogreplier.OplogOperationApplier.OplogApplyingException) OplogManagerPersistException(com.torodb.mongodb.repl.OplogManager.OplogManagerPersistException) ExclusiveWriteMongodTransaction(com.torodb.mongodb.core.ExclusiveWriteMongodTransaction) UserException(com.torodb.core.exceptions.user.UserException) OplogOperation(com.eightkdata.mongowp.server.api.oplog.OplogOperation) RollbackException(com.torodb.core.transaction.RollbackException) WriteOplogTransaction(com.torodb.mongodb.repl.OplogManager.WriteOplogTransaction)

Example 7 with RollbackException

use of com.torodb.core.transaction.RollbackException in project torodb by torodb.

the class SimpleAnalyzedOplogBatchExecutor method visit.

@Override
public OplogOperation visit(CudAnalyzedOplogBatch batch, ApplierContext arg) throws RetrierGiveUpException {
    metrics.getCudBatchSize().update(batch.getOriginalBatch().size());
    try (Context context = metrics.getCudBatchTimer().time()) {
        try {
            execute(batch, arg);
        } catch (UserException | NamespaceJobExecutionException ex) {
            throw new RetrierGiveUpException("Unexpected exception while replying", ex);
        } catch (RollbackException ex) {
            ApplierContext retryingReplingContext = new ApplierContext.Builder().setReapplying(true).setUpdatesAsUpserts(true).build();
            retrier.retry(() -> {
                try {
                    execute(batch, retryingReplingContext);
                    return Empty.getInstance();
                } catch (UserException | NamespaceJobExecutionException ex2) {
                    throw new RetrierAbortException("Unexpected user exception while applying " + "the batch " + batch, ex2);
                }
            }, Hint.CRITICAL, Hint.TIME_SENSIBLE);
        }
    }
    List<OplogOperation> originalBatch = batch.getOriginalBatch();
    return originalBatch.get(originalBatch.size() - 1);
}
Also used : ApplierContext(com.torodb.mongodb.repl.oplogreplier.ApplierContext) Context(com.codahale.metrics.Timer.Context) RetrierAbortException(com.torodb.core.retrier.RetrierAbortException) UserException(com.torodb.core.exceptions.user.UserException) RetrierGiveUpException(com.torodb.core.retrier.RetrierGiveUpException) RollbackException(com.torodb.core.transaction.RollbackException) OplogOperation(com.eightkdata.mongowp.server.api.oplog.OplogOperation) ApplierContext(com.torodb.mongodb.repl.oplogreplier.ApplierContext)

Example 8 with RollbackException

use of com.torodb.core.transaction.RollbackException in project torodb by torodb.

the class AkkaDbCloner method cloneDatabase.

@Override
public void cloneDatabase(String dstDb, MongoClient remoteClient, MongodServer localServer, CloneOptions opts) throws CloningException, NotMasterException, MongoException {
    Preconditions.checkState(isRunning(), "This db cloner is not running");
    if (!remoteClient.isRemote() && opts.getDbToClone().equals(dstDb)) {
        LOGGER.warn("Trying to clone a database to itself! Ignoring it");
        return;
    }
    String fromDb = opts.getDbToClone();
    CursorResult<Entry> listCollections;
    try (MongoConnection remoteConnection = remoteClient.openConnection()) {
        listCollections = ListCollectionsRequester.getListCollections(remoteConnection, fromDb, null);
    } catch (MongoException ex) {
        throw new CloningException("It was impossible to get information from the remote server", ex);
    }
    if (!opts.getWritePermissionSupplier().get()) {
        throw new NotMasterException("Destiny database cannot be written");
    }
    List<Entry> collsToClone = getCollsToClone(listCollections, fromDb, opts);
    if (!opts.getWritePermissionSupplier().get()) {
        throw new NotMasterException("Destiny database cannot be written " + "after get collections info");
    }
    try {
        for (Entry entry : collsToClone) {
            prepareCollection(localServer, dstDb, entry);
        }
    } catch (RollbackException ex) {
        throw new AssertionError("Unexpected rollback exception", ex);
    }
    Materializer materializer = ActorMaterializer.create(getActorSystem());
    try (MongoConnection remoteConnection = remoteClient.openConnection()) {
        if (opts.isCloneData()) {
            for (Entry entry : collsToClone) {
                LOGGER.info("Cloning collection data {}.{} into {}.{}", fromDb, entry.getCollectionName(), dstDb, entry.getCollectionName());
                try {
                    cloneCollection(localServer, remoteConnection, dstDb, opts, materializer, entry);
                } catch (CompletionException completionException) {
                    Throwable cause = completionException.getCause();
                    if (cause instanceof RollbackException) {
                        throw (RollbackException) cause;
                    }
                    throw completionException;
                }
            }
        }
        if (opts.isCloneIndexes()) {
            for (Entry entry : collsToClone) {
                LOGGER.info("Cloning collection indexes {}.{} into {}.{}", fromDb, entry.getCollectionName(), dstDb, entry.getCollectionName());
                try {
                    cloneIndex(localServer, dstDb, dstDb, remoteConnection, opts, entry.getCollectionName(), entry.getCollectionName());
                } catch (CompletionException completionException) {
                    Throwable cause = completionException.getCause();
                    if (cause instanceof RollbackException) {
                        throw (RollbackException) cause;
                    }
                    throw completionException;
                }
            }
        }
    }
}
Also used : MongoException(com.eightkdata.mongowp.exceptions.MongoException) RollbackException(com.torodb.core.transaction.RollbackException) Entry(com.torodb.mongodb.commands.signatures.admin.ListCollectionsCommand.ListCollectionsResult.Entry) CompletionException(java.util.concurrent.CompletionException) NotMasterException(com.eightkdata.mongowp.exceptions.NotMasterException) MongoConnection(com.eightkdata.mongowp.client.core.MongoConnection) ActorMaterializer(akka.stream.ActorMaterializer) Materializer(akka.stream.Materializer)

Example 9 with RollbackException

use of com.torodb.core.transaction.RollbackException in project torodb by torodb.

the class SameThreadInsertPipeline method insert.

@Override
public void insert(Stream<KvDocument> docs) throws RollbackException, UserException {
    D2RTranslationBatchFunction d2rFun = new D2RTranslationBatchFunction(translatorFactory, metaDb, mutableMetaCollection);
    DefaultToBackendFunction r2BackendFun = new DefaultToBackendFunction(jobFactory, metaDb, mutableMetaCollection);
    try {
        Iterators.partition(docs.iterator(), docBatchSize).forEachRemaining(list -> {
            CollectionData collData = d2rFun.apply(list);
            Iterable<BackendTransactionJob> jobs = r2BackendFun.apply(collData);
            jobs.forEach(Unchecked.consumer(job -> job.execute(backendConnection)));
        });
    } catch (UncheckedException ex) {
        Throwable cause = ex.getCause();
        if (cause != null && cause instanceof UserException) {
            throw (UserException) cause;
        }
        throw ex;
    }
}
Also used : CollectionData(com.torodb.core.d2r.CollectionData) Unchecked(org.jooq.lambda.Unchecked) KvDocument(com.torodb.kvdocument.values.KvDocument) UserException(com.torodb.core.exceptions.user.UserException) BackendTransactionJobFactory(com.torodb.core.dsl.backend.BackendTransactionJobFactory) Iterators(com.google.common.collect.Iterators) BackendTransactionJob(com.torodb.core.dsl.backend.BackendTransactionJob) RollbackException(com.torodb.core.transaction.RollbackException) D2RTranslatorFactory(com.torodb.core.d2r.D2RTranslatorFactory) Assisted(com.google.inject.assistedinject.Assisted) Inject(javax.inject.Inject) MutableMetaCollection(com.torodb.core.transaction.metainf.MutableMetaCollection) UncheckedException(org.jooq.lambda.UncheckedException) MetaDatabase(com.torodb.core.transaction.metainf.MetaDatabase) Stream(java.util.stream.Stream) DefaultToBackendFunction(com.torodb.torod.pipeline.DefaultToBackendFunction) WriteBackendTransaction(com.torodb.core.backend.WriteBackendTransaction) Preconditions(com.google.common.base.Preconditions) CollectionData(com.torodb.core.d2r.CollectionData) D2RTranslationBatchFunction(com.torodb.torod.pipeline.D2RTranslationBatchFunction) InsertPipeline(com.torodb.torod.pipeline.InsertPipeline) UncheckedException(org.jooq.lambda.UncheckedException) D2RTranslationBatchFunction(com.torodb.torod.pipeline.D2RTranslationBatchFunction) BackendTransactionJob(com.torodb.core.dsl.backend.BackendTransactionJob) UserException(com.torodb.core.exceptions.user.UserException) DefaultToBackendFunction(com.torodb.torod.pipeline.DefaultToBackendFunction)

Aggregations

RollbackException (com.torodb.core.transaction.RollbackException)9 OplogOperation (com.eightkdata.mongowp.server.api.oplog.OplogOperation)6 UserException (com.torodb.core.exceptions.user.UserException)4 RetrierGiveUpException (com.torodb.core.retrier.RetrierGiveUpException)4 Context (com.codahale.metrics.Timer.Context)3 MongoException (com.eightkdata.mongowp.exceptions.MongoException)3 ApplierContext (com.torodb.mongodb.repl.oplogreplier.ApplierContext)3 Timer (com.codahale.metrics.Timer)2 MongoConnection (com.eightkdata.mongowp.client.core.MongoConnection)2 RetrierAbortException (com.torodb.core.retrier.RetrierAbortException)2 OplogManagerPersistException (com.torodb.mongodb.repl.OplogManager.OplogManagerPersistException)2 WriteOplogTransaction (com.torodb.mongodb.repl.OplogManager.WriteOplogTransaction)2 Mockito.doAnswer (org.mockito.Mockito.doAnswer)2 InvocationOnMock (org.mockito.invocation.InvocationOnMock)2 Answer (org.mockito.stubbing.Answer)2 ActorMaterializer (akka.stream.ActorMaterializer)1 Materializer (akka.stream.Materializer)1 OpTime (com.eightkdata.mongowp.OpTime)1 MongoClient (com.eightkdata.mongowp.client.core.MongoClient)1 UnreachableMongoServerException (com.eightkdata.mongowp.client.core.UnreachableMongoServerException)1