Search in sources :

Example 26 with DOMStoreWriteTransaction

use of org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction in project controller by opendaylight.

the class DistributedDataStoreRemotingIntegrationTest method testWriteTransactionWithSingleShard.

@Test
public void testWriteTransactionWithSingleShard() throws Exception {
    final String testName = "testWriteTransactionWithSingleShard";
    initDatastoresWithCars(testName);
    final String followerCarShardName = "member-2-shard-cars-" + testName;
    DOMStoreWriteTransaction writeTx = followerDistributedDataStore.newWriteOnlyTransaction();
    assertNotNull("newWriteOnlyTransaction returned null", writeTx);
    writeTx.write(CarsModel.BASE_PATH, CarsModel.emptyContainer());
    writeTx.write(CarsModel.CAR_LIST_PATH, CarsModel.newCarMapNode());
    final MapEntryNode car1 = CarsModel.newCarEntry("optima", BigInteger.valueOf(20000));
    final YangInstanceIdentifier car1Path = CarsModel.newCarPath("optima");
    writeTx.merge(car1Path, car1);
    final MapEntryNode car2 = CarsModel.newCarEntry("sportage", BigInteger.valueOf(25000));
    final YangInstanceIdentifier car2Path = CarsModel.newCarPath("sportage");
    writeTx.merge(car2Path, car2);
    followerTestKit.doCommit(writeTx.ready());
    verifyCars(followerDistributedDataStore.newReadOnlyTransaction(), car1, car2);
    verifyCars(leaderDistributedDataStore.newReadOnlyTransaction(), car1, car2);
    // Test delete
    writeTx = followerDistributedDataStore.newWriteOnlyTransaction();
    writeTx.delete(car1Path);
    followerTestKit.doCommit(writeTx.ready());
    verifyExists(followerDistributedDataStore.newReadOnlyTransaction(), car2Path);
    verifyCars(followerDistributedDataStore.newReadOnlyTransaction(), car2);
    verifyCars(leaderDistributedDataStore.newReadOnlyTransaction(), car2);
    // Re-instate the follower member 2 as a single-node to verify replication and recovery.
    // The following is a bit tricky. Before we reinstate the follower we need to ensure it has persisted and
    // applied and all the log entries from the leader. Since we've verified the car data above we know that
    // all the transactions have been applied on the leader so we first read and capture its lastAppliedIndex.
    final AtomicLong leaderLastAppliedIndex = new AtomicLong();
    IntegrationTestKit.verifyShardState(leaderDistributedDataStore, CARS[0], state -> leaderLastAppliedIndex.set(state.getLastApplied()));
    // Now we need to make sure the follower has persisted the leader's lastAppliedIndex via ApplyJournalEntries.
    // However we don't know exactly how many ApplyJournalEntries messages there will be as it can differ between
    // the tell-based and ask-based front-ends. For ask-based there will be exactly 2 ApplyJournalEntries but
    // tell-based persists additional payloads which could be replicated and applied in a batch resulting in
    // either 2 or 3 ApplyJournalEntries. To handle this we read the follower's persisted ApplyJournalEntries
    // until we find the one that encompasses the leader's lastAppliedIndex.
    Stopwatch sw = Stopwatch.createStarted();
    boolean done = false;
    while (!done) {
        final List<ApplyJournalEntries> entries = InMemoryJournal.get(followerCarShardName, ApplyJournalEntries.class);
        for (ApplyJournalEntries aje : entries) {
            if (aje.getToIndex() >= leaderLastAppliedIndex.get()) {
                done = true;
                break;
            }
        }
        assertTrue("Follower did not persist ApplyJournalEntries containing leader's lastAppliedIndex " + leaderLastAppliedIndex + ". Entries persisted: " + entries, sw.elapsed(TimeUnit.SECONDS) <= 5);
        Uninterruptibles.sleepUninterruptibly(50, TimeUnit.MILLISECONDS);
    }
    TestKit.shutdownActorSystem(leaderSystem, Boolean.TRUE);
    TestKit.shutdownActorSystem(followerSystem, Boolean.TRUE);
    final ActorSystem newSystem = newActorSystem("reinstated-member2", "Member2");
    try (AbstractDataStore member2Datastore = new IntegrationTestKit(newSystem, leaderDatastoreContextBuilder, commitTimeout).setupAbstractDataStore(testParameter, testName, "module-shards-member2", true, CARS)) {
        verifyCars(member2Datastore.newReadOnlyTransaction(), car2);
    }
}
Also used : ActorSystem(akka.actor.ActorSystem) AtomicLong(java.util.concurrent.atomic.AtomicLong) DOMStoreWriteTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction) ApplyJournalEntries(org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries) Stopwatch(com.google.common.base.Stopwatch) AddressFromURIString(akka.actor.AddressFromURIString) MapEntryNode(org.opendaylight.yangtools.yang.data.api.schema.MapEntryNode) YangInstanceIdentifier(org.opendaylight.yangtools.yang.data.api.YangInstanceIdentifier) Test(org.junit.Test)

Example 27 with DOMStoreWriteTransaction

use of org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction in project controller by opendaylight.

the class DistributedDataStoreRemotingIntegrationTest method testLeadershipTransferOnShutdown.

@Test
public void testLeadershipTransferOnShutdown() throws Exception {
    // TODO remove when test passes also for ClientBackedDataStore
    Assume.assumeTrue(testParameter.equals(DistributedDataStore.class));
    leaderDatastoreContextBuilder.shardBatchedModificationCount(1);
    followerDatastoreContextBuilder.shardElectionTimeoutFactor(10).customRaftPolicyImplementation(null);
    final String testName = "testLeadershipTransferOnShutdown";
    initDatastores(testName, MODULE_SHARDS_CARS_PEOPLE_1_2_3, CARS_AND_PEOPLE);
    final IntegrationTestKit follower2TestKit = new IntegrationTestKit(follower2System, DatastoreContext.newBuilderFrom(followerDatastoreContextBuilder.build()).operationTimeoutInMillis(100), commitTimeout);
    try (AbstractDataStore follower2DistributedDataStore = follower2TestKit.setupAbstractDataStore(testParameter, testName, MODULE_SHARDS_CARS_PEOPLE_1_2_3, false)) {
        followerTestKit.waitForMembersUp("member-3");
        follower2TestKit.waitForMembersUp("member-1", "member-2");
        // Create and submit a couple tx's so they're pending.
        DOMStoreWriteTransaction writeTx = followerDistributedDataStore.newWriteOnlyTransaction();
        writeTx.write(CarsModel.BASE_PATH, CarsModel.emptyContainer());
        writeTx.write(CarsModel.CAR_LIST_PATH, CarsModel.newCarMapNode());
        writeTx.write(PeopleModel.BASE_PATH, PeopleModel.emptyContainer());
        final DOMStoreThreePhaseCommitCohort cohort1 = writeTx.ready();
        IntegrationTestKit.verifyShardStats(leaderDistributedDataStore, "cars", stats -> assertEquals("getTxCohortCacheSize", 1, stats.getTxCohortCacheSize()));
        writeTx = followerDistributedDataStore.newWriteOnlyTransaction();
        final MapEntryNode car = CarsModel.newCarEntry("optima", BigInteger.valueOf(20000));
        writeTx.write(CarsModel.newCarPath("optima"), car);
        final DOMStoreThreePhaseCommitCohort cohort2 = writeTx.ready();
        IntegrationTestKit.verifyShardStats(leaderDistributedDataStore, "cars", stats -> assertEquals("getTxCohortCacheSize", 2, stats.getTxCohortCacheSize()));
        // Gracefully stop the leader via a Shutdown message.
        sendDatastoreContextUpdate(leaderDistributedDataStore, leaderDatastoreContextBuilder.shardElectionTimeoutFactor(100));
        final FiniteDuration duration = FiniteDuration.create(5, TimeUnit.SECONDS);
        final Future<ActorRef> future = leaderDistributedDataStore.getActorContext().findLocalShardAsync("cars");
        final ActorRef leaderActor = Await.result(future, duration);
        final Future<Boolean> stopFuture = Patterns.gracefulStop(leaderActor, duration, Shutdown.INSTANCE);
        // Commit the 2 transactions. They should finish and succeed.
        followerTestKit.doCommit(cohort1);
        followerTestKit.doCommit(cohort2);
        // Wait for the leader actor stopped.
        final Boolean stopped = Await.result(stopFuture, duration);
        assertEquals("Stopped", Boolean.TRUE, stopped);
        // Verify leadership was transferred by reading the committed data from the other nodes.
        verifyCars(followerDistributedDataStore.newReadOnlyTransaction(), car);
        verifyCars(follower2DistributedDataStore.newReadOnlyTransaction(), car);
    }
}
Also used : DOMStoreWriteTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction) ActorRef(akka.actor.ActorRef) FiniteDuration(scala.concurrent.duration.FiniteDuration) AddressFromURIString(akka.actor.AddressFromURIString) MapEntryNode(org.opendaylight.yangtools.yang.data.api.schema.MapEntryNode) DOMStoreThreePhaseCommitCohort(org.opendaylight.mdsal.dom.spi.store.DOMStoreThreePhaseCommitCohort) Test(org.junit.Test)

Example 28 with DOMStoreWriteTransaction

use of org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction in project controller by opendaylight.

the class TransactionChainProxyTest method testChainedWriteTransactionsWithPreviousTxNotReady.

@Test(expected = IllegalStateException.class)
public void testChainedWriteTransactionsWithPreviousTxNotReady() {
    ActorRef actorRef = setupActorContextWithInitialCreateTransaction(getSystem(), WRITE_ONLY);
    expectBatchedModifications(actorRef, 1);
    try (TransactionChainProxy txChainProxy = new TransactionChainProxy(mockComponentFactory, historyId)) {
        DOMStoreWriteTransaction writeTx1 = txChainProxy.newWriteOnlyTransaction();
        NormalizedNode<?, ?> writeNode1 = ImmutableNodes.containerNode(TestModel.TEST_QNAME);
        writeTx1.write(TestModel.TEST_PATH, writeNode1);
        txChainProxy.newWriteOnlyTransaction();
    }
}
Also used : ActorRef(akka.actor.ActorRef) DOMStoreWriteTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction) Test(org.junit.Test)

Example 29 with DOMStoreWriteTransaction

use of org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction in project controller by opendaylight.

the class TransactionChainProxyTest method testNewWriteOnlyTransaction.

@SuppressWarnings("resource")
@Test
public void testNewWriteOnlyTransaction() {
    DOMStoreTransaction dst = new TransactionChainProxy(mockComponentFactory, historyId).newWriteOnlyTransaction();
    Assert.assertTrue(dst instanceof DOMStoreWriteTransaction);
}
Also used : DOMStoreTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreTransaction) DOMStoreWriteTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction) Test(org.junit.Test)

Example 30 with DOMStoreWriteTransaction

use of org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction in project controller by opendaylight.

the class IntegrationTestKit method testWriteTransaction.

void testWriteTransaction(final AbstractDataStore dataStore, final YangInstanceIdentifier nodePath, final NormalizedNode<?, ?> nodeToWrite) throws Exception {
    // 1. Create a write-only Tx
    DOMStoreWriteTransaction writeTx = dataStore.newWriteOnlyTransaction();
    assertNotNull("newWriteOnlyTransaction returned null", writeTx);
    // 2. Write some data
    writeTx.write(nodePath, nodeToWrite);
    // 3. Ready the Tx for commit
    DOMStoreThreePhaseCommitCohort cohort = writeTx.ready();
    // 4. Commit the Tx
    doCommit(cohort);
    // 5. Verify the data in the store
    DOMStoreReadTransaction readTx = dataStore.newReadOnlyTransaction();
    Optional<NormalizedNode<?, ?>> optional = readTx.read(nodePath).get(5, TimeUnit.SECONDS);
    assertEquals("isPresent", true, optional.isPresent());
    assertEquals("Data node", nodeToWrite, optional.get());
}
Also used : DOMStoreWriteTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction) DOMStoreReadTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreReadTransaction) NormalizedNode(org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode) DOMStoreThreePhaseCommitCohort(org.opendaylight.mdsal.dom.spi.store.DOMStoreThreePhaseCommitCohort)

Aggregations

DOMStoreWriteTransaction (org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction)30 Test (org.junit.Test)25 DOMStoreThreePhaseCommitCohort (org.opendaylight.mdsal.dom.spi.store.DOMStoreThreePhaseCommitCohort)17 DOMStoreReadTransaction (org.opendaylight.mdsal.dom.spi.store.DOMStoreReadTransaction)9 YangInstanceIdentifier (org.opendaylight.yangtools.yang.data.api.YangInstanceIdentifier)9 MapEntryNode (org.opendaylight.yangtools.yang.data.api.schema.MapEntryNode)9 NormalizedNode (org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode)9 AddressFromURIString (akka.actor.AddressFromURIString)7 DOMStoreTransactionChain (org.opendaylight.mdsal.dom.spi.store.DOMStoreTransactionChain)7 CountDownLatch (java.util.concurrent.CountDownLatch)6 ExecutionException (java.util.concurrent.ExecutionException)6 AtomicReference (java.util.concurrent.atomic.AtomicReference)6 ReadFailedException (org.opendaylight.mdsal.common.api.ReadFailedException)6 NoShardLeaderException (org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException)5 DOMStoreReadWriteTransaction (org.opendaylight.mdsal.dom.spi.store.DOMStoreReadWriteTransaction)5 ActorRef (akka.actor.ActorRef)4 IOException (java.io.IOException)4 RequestTimeoutException (org.opendaylight.controller.cluster.access.client.RequestTimeoutException)4 NotInitializedException (org.opendaylight.controller.cluster.datastore.exceptions.NotInitializedException)4 TransactionChainClosedException (org.opendaylight.mdsal.common.api.TransactionChainClosedException)4