Search in sources :

Example 71 with NormalizedNode

use of org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode in project controller by opendaylight.

the class DistributedDataStoreIntegrationTest method testTransactionWritesWithShardNotInitiallyReady.

@SuppressWarnings("checkstyle:IllegalCatch")
private void testTransactionWritesWithShardNotInitiallyReady(final String testName, final boolean writeOnly) throws Exception {
    new IntegrationTestKit(getSystem(), datastoreContextBuilder) {

        {
            final String shardName = "test-1";
            // Setup the InMemoryJournal to block shard recovery to ensure
            // the shard isn't
            // initialized until we create and submit the write the Tx.
            final String persistentID = String.format("member-1-shard-%s-%s", shardName, testName);
            final CountDownLatch blockRecoveryLatch = new CountDownLatch(1);
            InMemoryJournal.addBlockReadMessagesLatch(persistentID, blockRecoveryLatch);
            try (AbstractDataStore dataStore = setupAbstractDataStore(testParameter, testName, false, shardName)) {
                // Create the write Tx
                final DOMStoreWriteTransaction writeTx = writeOnly ? dataStore.newWriteOnlyTransaction() : dataStore.newReadWriteTransaction();
                assertNotNull("newReadWriteTransaction returned null", writeTx);
                // Do some modification operations and ready the Tx on a
                // separate thread.
                final YangInstanceIdentifier listEntryPath = YangInstanceIdentifier.builder(TestModel.OUTER_LIST_PATH).nodeWithKey(TestModel.OUTER_LIST_QNAME, TestModel.ID_QNAME, 1).build();
                final AtomicReference<DOMStoreThreePhaseCommitCohort> txCohort = new AtomicReference<>();
                final AtomicReference<Exception> caughtEx = new AtomicReference<>();
                final CountDownLatch txReady = new CountDownLatch(1);
                final Thread txThread = new Thread(() -> {
                    try {
                        writeTx.write(TestModel.TEST_PATH, ImmutableNodes.containerNode(TestModel.TEST_QNAME));
                        writeTx.merge(TestModel.OUTER_LIST_PATH, ImmutableNodes.mapNodeBuilder(TestModel.OUTER_LIST_QNAME).build());
                        writeTx.write(listEntryPath, ImmutableNodes.mapEntry(TestModel.OUTER_LIST_QNAME, TestModel.ID_QNAME, 1));
                        writeTx.delete(listEntryPath);
                        txCohort.set(writeTx.ready());
                    } catch (Exception e) {
                        caughtEx.set(e);
                    } finally {
                        txReady.countDown();
                    }
                });
                txThread.start();
                // Wait for the Tx operations to complete.
                final boolean done = Uninterruptibles.awaitUninterruptibly(txReady, 5, TimeUnit.SECONDS);
                if (caughtEx.get() != null) {
                    throw caughtEx.get();
                }
                assertEquals("Tx ready", true, done);
                // At this point the Tx operations should be waiting for the
                // shard to initialize so
                // trigger the latch to let the shard recovery to continue.
                blockRecoveryLatch.countDown();
                // Wait for the Tx commit to complete.
                doCommit(txCohort.get());
                // Verify the data in the store
                final DOMStoreReadTransaction readTx = dataStore.newReadOnlyTransaction();
                Optional<NormalizedNode<?, ?>> optional = readTx.read(TestModel.TEST_PATH).get(5, TimeUnit.SECONDS);
                assertEquals("isPresent", true, optional.isPresent());
                optional = readTx.read(TestModel.OUTER_LIST_PATH).get(5, TimeUnit.SECONDS);
                assertEquals("isPresent", true, optional.isPresent());
                optional = readTx.read(listEntryPath).get(5, TimeUnit.SECONDS);
                assertEquals("isPresent", false, optional.isPresent());
            }
        }
    };
}
Also used : DOMStoreWriteTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction) DOMStoreReadTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreReadTransaction) AtomicReference(java.util.concurrent.atomic.AtomicReference) AddressFromURIString(akka.actor.AddressFromURIString) CountDownLatch(java.util.concurrent.CountDownLatch) YangInstanceIdentifier(org.opendaylight.yangtools.yang.data.api.YangInstanceIdentifier) DOMStoreThreePhaseCommitCohort(org.opendaylight.mdsal.dom.spi.store.DOMStoreThreePhaseCommitCohort) ReadFailedException(org.opendaylight.mdsal.common.api.ReadFailedException) NotInitializedException(org.opendaylight.controller.cluster.datastore.exceptions.NotInitializedException) TransactionChainClosedException(org.opendaylight.mdsal.common.api.TransactionChainClosedException) RequestTimeoutException(org.opendaylight.controller.cluster.access.client.RequestTimeoutException) NoShardLeaderException(org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException) IOException(java.io.IOException) TransactionCommitFailedException(org.opendaylight.mdsal.common.api.TransactionCommitFailedException) ExecutionException(java.util.concurrent.ExecutionException) NormalizedNode(org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode)

Example 72 with NormalizedNode

use of org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode in project controller by opendaylight.

the class DistributedDataStoreIntegrationTest method testTransactionChainWithMultipleShards.

@Test
public void testTransactionChainWithMultipleShards() throws Exception {
    new IntegrationTestKit(getSystem(), datastoreContextBuilder) {

        {
            try (AbstractDataStore dataStore = setupAbstractDataStore(testParameter, "testTransactionChainWithMultipleShards", "cars-1", "people-1")) {
                final DOMStoreTransactionChain txChain = dataStore.createTransactionChain();
                DOMStoreWriteTransaction writeTx = txChain.newWriteOnlyTransaction();
                assertNotNull("newWriteOnlyTransaction returned null", writeTx);
                writeTx.write(CarsModel.BASE_PATH, CarsModel.emptyContainer());
                writeTx.write(PeopleModel.BASE_PATH, PeopleModel.emptyContainer());
                writeTx.write(CarsModel.CAR_LIST_PATH, CarsModel.newCarMapNode());
                writeTx.write(PeopleModel.PERSON_LIST_PATH, PeopleModel.newPersonMapNode());
                final DOMStoreThreePhaseCommitCohort cohort1 = writeTx.ready();
                final DOMStoreReadWriteTransaction readWriteTx = txChain.newReadWriteTransaction();
                final MapEntryNode car = CarsModel.newCarEntry("optima", BigInteger.valueOf(20000));
                final YangInstanceIdentifier carPath = CarsModel.newCarPath("optima");
                readWriteTx.write(carPath, car);
                final MapEntryNode person = PeopleModel.newPersonEntry("jack");
                final YangInstanceIdentifier personPath = PeopleModel.newPersonPath("jack");
                readWriteTx.merge(personPath, person);
                Optional<NormalizedNode<?, ?>> optional = readWriteTx.read(carPath).get(5, TimeUnit.SECONDS);
                assertEquals("isPresent", true, optional.isPresent());
                assertEquals("Data node", car, optional.get());
                optional = readWriteTx.read(personPath).get(5, TimeUnit.SECONDS);
                assertEquals("isPresent", true, optional.isPresent());
                assertEquals("Data node", person, optional.get());
                final DOMStoreThreePhaseCommitCohort cohort2 = readWriteTx.ready();
                writeTx = txChain.newWriteOnlyTransaction();
                writeTx.delete(carPath);
                final DOMStoreThreePhaseCommitCohort cohort3 = writeTx.ready();
                final ListenableFuture<Boolean> canCommit1 = cohort1.canCommit();
                final ListenableFuture<Boolean> canCommit2 = cohort2.canCommit();
                doCommit(canCommit1, cohort1);
                doCommit(canCommit2, cohort2);
                doCommit(cohort3);
                txChain.close();
                final DOMStoreReadTransaction readTx = dataStore.newReadOnlyTransaction();
                optional = readTx.read(carPath).get(5, TimeUnit.SECONDS);
                assertEquals("isPresent", false, optional.isPresent());
                optional = readTx.read(personPath).get(5, TimeUnit.SECONDS);
                assertEquals("isPresent", true, optional.isPresent());
                assertEquals("Data node", person, optional.get());
            }
        }
    };
}
Also used : DOMStoreWriteTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction) DOMStoreReadTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreReadTransaction) DOMStoreTransactionChain(org.opendaylight.mdsal.dom.spi.store.DOMStoreTransactionChain) DOMStoreReadWriteTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreReadWriteTransaction) MapEntryNode(org.opendaylight.yangtools.yang.data.api.schema.MapEntryNode) YangInstanceIdentifier(org.opendaylight.yangtools.yang.data.api.YangInstanceIdentifier) DOMStoreThreePhaseCommitCohort(org.opendaylight.mdsal.dom.spi.store.DOMStoreThreePhaseCommitCohort) NormalizedNode(org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode) Test(org.junit.Test)

Example 73 with NormalizedNode

use of org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode in project controller by opendaylight.

the class DistributedDataStoreIntegrationTest method testTransactionReadFailureWithShardNotInitialized.

@Test(expected = NotInitializedException.class)
@SuppressWarnings("checkstyle:IllegalCatch")
public void testTransactionReadFailureWithShardNotInitialized() throws Exception {
    new IntegrationTestKit(getSystem(), datastoreContextBuilder) {

        {
            final String testName = "testTransactionReadFailureWithShardNotInitialized";
            final String shardName = "test-1";
            // Set the shard initialization timeout low for the test.
            datastoreContextBuilder.shardInitializationTimeout(300, TimeUnit.MILLISECONDS);
            // Setup the InMemoryJournal to block shard recovery
            // indefinitely.
            final String persistentID = String.format("member-1-shard-%s-%s", shardName, testName);
            final CountDownLatch blockRecoveryLatch = new CountDownLatch(1);
            InMemoryJournal.addBlockReadMessagesLatch(persistentID, blockRecoveryLatch);
            InMemoryJournal.addEntry(persistentID, 1, "Dummy data so akka will read from persistence");
            try (AbstractDataStore dataStore = setupAbstractDataStore(testParameter, testName, false, shardName)) {
                // Create the read-write Tx
                final DOMStoreReadWriteTransaction readWriteTx = dataStore.newReadWriteTransaction();
                assertNotNull("newReadWriteTransaction returned null", readWriteTx);
                // Do a read on the Tx on a separate thread.
                final AtomicReference<CheckedFuture<Optional<NormalizedNode<?, ?>>, ReadFailedException>> txReadFuture = new AtomicReference<>();
                final AtomicReference<Exception> caughtEx = new AtomicReference<>();
                final CountDownLatch txReadDone = new CountDownLatch(1);
                final Thread txThread = new Thread(() -> {
                    try {
                        readWriteTx.write(TestModel.TEST_PATH, ImmutableNodes.containerNode(TestModel.TEST_QNAME));
                        txReadFuture.set(readWriteTx.read(TestModel.TEST_PATH));
                        readWriteTx.close();
                    } catch (Exception e) {
                        caughtEx.set(e);
                    } finally {
                        txReadDone.countDown();
                    }
                });
                txThread.start();
                // Wait for the Tx operations to complete.
                boolean done = Uninterruptibles.awaitUninterruptibly(txReadDone, 5, TimeUnit.SECONDS);
                if (caughtEx.get() != null) {
                    throw caughtEx.get();
                }
                assertEquals("Tx read done", true, done);
                // have timed out and throw an appropriate exception cause.
                try {
                    txReadFuture.get().checkedGet(5, TimeUnit.SECONDS);
                    fail("Expected NotInitializedException");
                } catch (final ReadFailedException e) {
                    final Throwable root = Throwables.getRootCause(e);
                    Throwables.throwIfUnchecked(root);
                    throw new RuntimeException(root);
                } finally {
                    blockRecoveryLatch.countDown();
                }
            }
        }
    };
}
Also used : ReadFailedException(org.opendaylight.mdsal.common.api.ReadFailedException) DOMStoreReadWriteTransaction(org.opendaylight.mdsal.dom.spi.store.DOMStoreReadWriteTransaction) AtomicReference(java.util.concurrent.atomic.AtomicReference) AddressFromURIString(akka.actor.AddressFromURIString) CountDownLatch(java.util.concurrent.CountDownLatch) ReadFailedException(org.opendaylight.mdsal.common.api.ReadFailedException) NotInitializedException(org.opendaylight.controller.cluster.datastore.exceptions.NotInitializedException) TransactionChainClosedException(org.opendaylight.mdsal.common.api.TransactionChainClosedException) RequestTimeoutException(org.opendaylight.controller.cluster.access.client.RequestTimeoutException) NoShardLeaderException(org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException) IOException(java.io.IOException) TransactionCommitFailedException(org.opendaylight.mdsal.common.api.TransactionCommitFailedException) ExecutionException(java.util.concurrent.ExecutionException) CheckedFuture(com.google.common.util.concurrent.CheckedFuture) NormalizedNode(org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode) Test(org.junit.Test)

Example 74 with NormalizedNode

use of org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode in project controller by opendaylight.

the class DistributedDataStoreRemotingIntegrationTest method testInstallSnapshot.

@Test
public void testInstallSnapshot() throws Exception {
    final String testName = "testInstallSnapshot";
    final String leaderCarShardName = "member-1-shard-cars-" + testName;
    final String followerCarShardName = "member-2-shard-cars-" + testName;
    // Setup a saved snapshot on the leader. The follower will startup with no data and the leader should
    // install a snapshot to sync the follower.
    DataTree tree = new InMemoryDataTreeFactory().create(DataTreeConfiguration.DEFAULT_CONFIGURATION, SchemaContextHelper.full());
    final ContainerNode carsNode = CarsModel.newCarsNode(CarsModel.newCarsMapNode(CarsModel.newCarEntry("optima", BigInteger.valueOf(20000))));
    AbstractShardTest.writeToStore(tree, CarsModel.BASE_PATH, carsNode);
    final NormalizedNode<?, ?> snapshotRoot = AbstractShardTest.readStore(tree, YangInstanceIdentifier.EMPTY);
    final Snapshot initialSnapshot = Snapshot.create(new ShardSnapshotState(new MetadataShardDataTreeSnapshot(snapshotRoot)), Collections.emptyList(), 5, 1, 5, 1, 1, null, null);
    InMemorySnapshotStore.addSnapshot(leaderCarShardName, initialSnapshot);
    InMemorySnapshotStore.addSnapshotSavedLatch(leaderCarShardName);
    InMemorySnapshotStore.addSnapshotSavedLatch(followerCarShardName);
    initDatastoresWithCars(testName);
    final Optional<NormalizedNode<?, ?>> readOptional = leaderDistributedDataStore.newReadOnlyTransaction().read(CarsModel.BASE_PATH).checkedGet(5, TimeUnit.SECONDS);
    assertEquals("isPresent", true, readOptional.isPresent());
    assertEquals("Node", carsNode, readOptional.get());
    verifySnapshot(InMemorySnapshotStore.waitForSavedSnapshot(leaderCarShardName, Snapshot.class), initialSnapshot, snapshotRoot);
    verifySnapshot(InMemorySnapshotStore.waitForSavedSnapshot(followerCarShardName, Snapshot.class), initialSnapshot, snapshotRoot);
}
Also used : MetadataShardDataTreeSnapshot(org.opendaylight.controller.cluster.datastore.persisted.MetadataShardDataTreeSnapshot) Snapshot(org.opendaylight.controller.cluster.raft.persisted.Snapshot) GetShardDataTree(org.opendaylight.controller.cluster.datastore.messages.GetShardDataTree) DataTree(org.opendaylight.yangtools.yang.data.api.schema.tree.DataTree) ShardSnapshotState(org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState) MetadataShardDataTreeSnapshot(org.opendaylight.controller.cluster.datastore.persisted.MetadataShardDataTreeSnapshot) ContainerNode(org.opendaylight.yangtools.yang.data.api.schema.ContainerNode) AddressFromURIString(akka.actor.AddressFromURIString) NormalizedNode(org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode) InMemoryDataTreeFactory(org.opendaylight.yangtools.yang.data.impl.schema.tree.InMemoryDataTreeFactory) Test(org.junit.Test)

Example 75 with NormalizedNode

use of org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode in project controller by opendaylight.

the class TransactionProxyTest method testReadWithPriorRecordingOperationSuccessful.

@Test
public void testReadWithPriorRecordingOperationSuccessful() throws Exception {
    ActorRef actorRef = setupActorContextWithInitialCreateTransaction(getSystem(), READ_WRITE);
    NormalizedNode<?, ?> expectedNode = ImmutableNodes.containerNode(TestModel.TEST_QNAME);
    expectBatchedModifications(actorRef, 1);
    doReturn(readDataReply(expectedNode)).when(mockActorContext).executeOperationAsync(eq(actorSelection(actorRef)), eqReadData(), any(Timeout.class));
    TransactionProxy transactionProxy = new TransactionProxy(mockComponentFactory, READ_WRITE);
    transactionProxy.write(TestModel.TEST_PATH, expectedNode);
    Optional<NormalizedNode<?, ?>> readOptional = transactionProxy.read(TestModel.TEST_PATH).get(5, TimeUnit.SECONDS);
    assertEquals("NormalizedNode isPresent", true, readOptional.isPresent());
    assertEquals("Response NormalizedNode", expectedNode, readOptional.get());
    InOrder inOrder = Mockito.inOrder(mockActorContext);
    inOrder.verify(mockActorContext).executeOperationAsync(eq(actorSelection(actorRef)), isA(BatchedModifications.class), any(Timeout.class));
    inOrder.verify(mockActorContext).executeOperationAsync(eq(actorSelection(actorRef)), eqReadData(), any(Timeout.class));
}
Also used : InOrder(org.mockito.InOrder) ActorRef(akka.actor.ActorRef) Timeout(akka.util.Timeout) NormalizedNode(org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode) BatchedModifications(org.opendaylight.controller.cluster.datastore.messages.BatchedModifications) NormalizedNodeAggregatorTest(org.opendaylight.controller.cluster.datastore.utils.NormalizedNodeAggregatorTest) Test(org.junit.Test)

Aggregations

NormalizedNode (org.opendaylight.yangtools.yang.data.api.schema.NormalizedNode)94 Test (org.junit.Test)55 YangInstanceIdentifier (org.opendaylight.yangtools.yang.data.api.YangInstanceIdentifier)39 ContainerNode (org.opendaylight.yangtools.yang.data.api.schema.ContainerNode)18 Optional (com.google.common.base.Optional)15 MapEntryNode (org.opendaylight.yangtools.yang.data.api.schema.MapEntryNode)11 DOMStoreReadTransaction (org.opendaylight.mdsal.dom.spi.store.DOMStoreReadTransaction)10 ReadFailedException (org.opendaylight.controller.md.sal.common.api.data.ReadFailedException)9 DOMStoreWriteTransaction (org.opendaylight.mdsal.dom.spi.store.DOMStoreWriteTransaction)9 NodeIdentifier (org.opendaylight.yangtools.yang.data.api.YangInstanceIdentifier.NodeIdentifier)9 ActorRef (akka.actor.ActorRef)8 InOrder (org.mockito.InOrder)8 DOMStoreReadWriteTransaction (org.opendaylight.mdsal.dom.spi.store.DOMStoreReadWriteTransaction)8 DOMStoreThreePhaseCommitCohort (org.opendaylight.mdsal.dom.spi.store.DOMStoreThreePhaseCommitCohort)8 ExecutionException (java.util.concurrent.ExecutionException)7 ReadFailedException (org.opendaylight.mdsal.common.api.ReadFailedException)7 DataTreeCandidate (org.opendaylight.yangtools.yang.data.api.schema.tree.DataTreeCandidate)7 Map (java.util.Map)6 CountDownLatch (java.util.concurrent.CountDownLatch)6 AtomicReference (java.util.concurrent.atomic.AtomicReference)6