Search in sources :

Example 11 with Version

use of org.apache.hadoop.yarn.server.records.Version in project hadoop by apache.

the class TestFSRMStateStore method testCheckMajorVersionChange.

@Test(timeout = 60000)
public void testCheckMajorVersionChange() throws Exception {
    HdfsConfiguration conf = new HdfsConfiguration();
    MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).numDataNodes(1).build();
    try {
        fsTester = new TestFSRMStateStoreTester(cluster, false) {

            Version VERSION_INFO = Version.newInstance(Integer.MAX_VALUE, 0);

            @Override
            public Version getCurrentVersion() throws Exception {
                return VERSION_INFO;
            }

            @Override
            public RMStateStore getRMStateStore() throws Exception {
                YarnConfiguration conf = new YarnConfiguration();
                conf.set(YarnConfiguration.FS_RM_STATE_STORE_URI, workingDirPathURI.toString());
                this.store = new TestFileSystemRMStore(conf) {

                    Version storedVersion = null;

                    @Override
                    public Version getCurrentVersion() {
                        return VERSION_INFO;
                    }

                    @Override
                    protected synchronized Version loadVersion() throws Exception {
                        return storedVersion;
                    }

                    @Override
                    protected synchronized void storeVersion() throws Exception {
                        storedVersion = VERSION_INFO;
                    }
                };
                return store;
            }
        };
        // default version
        RMStateStore store = fsTester.getRMStateStore();
        Version defaultVersion = fsTester.getCurrentVersion();
        store.checkVersion();
        Assert.assertEquals(defaultVersion, store.loadVersion());
    } finally {
        cluster.shutdown();
    }
}
Also used : MiniDFSCluster(org.apache.hadoop.hdfs.MiniDFSCluster) Version(org.apache.hadoop.yarn.server.records.Version) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) HdfsConfiguration(org.apache.hadoop.hdfs.HdfsConfiguration) IOException(java.io.IOException) AccessControlException(org.apache.hadoop.security.AccessControlException) Test(org.junit.Test)

Example 12 with Version

use of org.apache.hadoop.yarn.server.records.Version in project hadoop by apache.

the class LeveldbTimelineStateStore method loadVersion.

@VisibleForTesting
Version loadVersion() throws IOException {
    try {
        byte[] data = db.get(TIMELINE_STATE_STORE_VERSION_KEY);
        // if version is not stored previously, treat it as CURRENT_VERSION_INFO.
        if (data == null || data.length == 0) {
            return getCurrentVersion();
        }
        Version version = new VersionPBImpl(YarnServerCommonProtos.VersionProto.parseFrom(data));
        return version;
    } catch (DBException e) {
        throw new IOException(e);
    }
}
Also used : DBException(org.iq80.leveldb.DBException) VersionPBImpl(org.apache.hadoop.yarn.server.records.impl.pb.VersionPBImpl) Version(org.apache.hadoop.yarn.server.records.Version) IOException(java.io.IOException) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 13 with Version

use of org.apache.hadoop.yarn.server.records.Version in project hadoop by apache.

the class TestLeveldbTimelineStateStore method testCheckVersion.

@Test
public void testCheckVersion() throws IOException {
    LeveldbTimelineStateStore store = initAndStartTimelineServiceStateStoreService();
    // default version
    Version defaultVersion = store.getCurrentVersion();
    Assert.assertEquals(defaultVersion, store.loadVersion());
    // compatible version
    Version compatibleVersion = Version.newInstance(defaultVersion.getMajorVersion(), defaultVersion.getMinorVersion() + 2);
    store.storeVersion(compatibleVersion);
    Assert.assertEquals(compatibleVersion, store.loadVersion());
    store.stop();
    // overwrite the compatible version
    store = initAndStartTimelineServiceStateStoreService();
    Assert.assertEquals(defaultVersion, store.loadVersion());
    // incompatible version
    Version incompatibleVersion = Version.newInstance(defaultVersion.getMajorVersion() + 1, defaultVersion.getMinorVersion());
    store.storeVersion(incompatibleVersion);
    store.stop();
    try {
        initAndStartTimelineServiceStateStoreService();
        Assert.fail("Incompatible version, should expect fail here.");
    } catch (ServiceStateException e) {
        Assert.assertTrue("Exception message mismatch", e.getMessage().contains("Incompatible version for timeline state store"));
    }
}
Also used : Version(org.apache.hadoop.yarn.server.records.Version) ServiceStateException(org.apache.hadoop.service.ServiceStateException) Test(org.junit.Test)

Example 14 with Version

use of org.apache.hadoop.yarn.server.records.Version in project hadoop by apache.

the class TestLeveldbTimelineStore method testCheckVersion.

@Test
public void testCheckVersion() throws IOException {
    LeveldbTimelineStore dbStore = (LeveldbTimelineStore) store;
    // default version
    Version defaultVersion = dbStore.getCurrentVersion();
    Assert.assertEquals(defaultVersion, dbStore.loadVersion());
    // compatible version
    Version compatibleVersion = Version.newInstance(defaultVersion.getMajorVersion(), defaultVersion.getMinorVersion() + 2);
    dbStore.storeVersion(compatibleVersion);
    Assert.assertEquals(compatibleVersion, dbStore.loadVersion());
    restartTimelineStore();
    dbStore = (LeveldbTimelineStore) store;
    // overwrite the compatible version
    Assert.assertEquals(defaultVersion, dbStore.loadVersion());
    // incompatible version
    Version incompatibleVersion = Version.newInstance(defaultVersion.getMajorVersion() + 1, defaultVersion.getMinorVersion());
    dbStore.storeVersion(incompatibleVersion);
    try {
        restartTimelineStore();
        Assert.fail("Incompatible version, should expect fail here.");
    } catch (ServiceStateException e) {
        Assert.assertTrue("Exception message mismatch", e.getMessage().contains("Incompatible version for timeline store"));
    }
}
Also used : Version(org.apache.hadoop.yarn.server.records.Version) ServiceStateException(org.apache.hadoop.service.ServiceStateException) Test(org.junit.Test)

Example 15 with Version

use of org.apache.hadoop.yarn.server.records.Version in project hadoop by apache.

the class TestRollingLevelDBTimelineStore method testCheckVersion.

@Test
public void testCheckVersion() throws IOException {
    RollingLevelDBTimelineStore dbStore = (RollingLevelDBTimelineStore) store;
    // default version
    Version defaultVersion = dbStore.getCurrentVersion();
    Assert.assertEquals(defaultVersion, dbStore.loadVersion());
    // compatible version
    Version compatibleVersion = Version.newInstance(defaultVersion.getMajorVersion(), defaultVersion.getMinorVersion() + 2);
    dbStore.storeVersion(compatibleVersion);
    Assert.assertEquals(compatibleVersion, dbStore.loadVersion());
    restartTimelineStore();
    dbStore = (RollingLevelDBTimelineStore) store;
    // overwrite the compatible version
    Assert.assertEquals(defaultVersion, dbStore.loadVersion());
    // incompatible version
    Version incompatibleVersion = Version.newInstance(defaultVersion.getMajorVersion() + 1, defaultVersion.getMinorVersion());
    dbStore.storeVersion(incompatibleVersion);
    try {
        restartTimelineStore();
        Assert.fail("Incompatible version, should expect fail here.");
    } catch (ServiceStateException e) {
        Assert.assertTrue("Exception message mismatch", e.getMessage().contains("Incompatible version for timeline store"));
    }
}
Also used : Version(org.apache.hadoop.yarn.server.records.Version) ServiceStateException(org.apache.hadoop.service.ServiceStateException) Test(org.junit.Test)

Aggregations

Version (org.apache.hadoop.yarn.server.records.Version)23 IOException (java.io.IOException)10 Test (org.junit.Test)8 VersionPBImpl (org.apache.hadoop.yarn.server.records.impl.pb.VersionPBImpl)7 ServiceStateException (org.apache.hadoop.service.ServiceStateException)6 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)3 VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 Path (org.apache.hadoop.fs.Path)2 File (java.io.File)1 Configuration (org.apache.hadoop.conf.Configuration)1 FileStatus (org.apache.hadoop.fs.FileStatus)1 HdfsConfiguration (org.apache.hadoop.hdfs.HdfsConfiguration)1 MiniDFSCluster (org.apache.hadoop.hdfs.MiniDFSCluster)1 DataOutputBuffer (org.apache.hadoop.io.DataOutputBuffer)1 Text (org.apache.hadoop.io.Text)1 JobTokenIdentifier (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier)1 AccessControlException (org.apache.hadoop.security.AccessControlException)1 Token (org.apache.hadoop.security.token.Token)1 ApplicationId (org.apache.hadoop.yarn.api.records.ApplicationId)1 ApplicationInitializationContext (org.apache.hadoop.yarn.server.api.ApplicationInitializationContext)1