Search in sources :

Example 26 with Version

use of org.apache.hadoop.yarn.server.records.Version in project tez by apache.

the class ShuffleHandler method checkVersion.

/**
 * 1) Versioning scheme: major.minor. For e.g. 1.0, 1.1, 1.2...1.25, 2.0 etc.
 * 2) Any incompatible change of DB schema is a major upgrade, and any
 *    compatible change of DB schema is a minor upgrade.
 * 3) Within a minor upgrade, say 1.1 to 1.2:
 *    overwrite the version info and proceed as normal.
 * 4) Within a major upgrade, say 1.2 to 2.0:
 *    throw exception and indicate user to use a separate upgrade tool to
 *    upgrade shuffle info or remove incompatible old state.
 */
private void checkVersion() throws IOException {
    Version loadedVersion = loadVersion();
    LOG.info("Loaded state DB schema version info " + loadedVersion);
    if (loadedVersion.equals(getCurrentVersion())) {
        return;
    }
    if (loadedVersion.isCompatibleTo(getCurrentVersion())) {
        LOG.info("Storing state DB schedma version info " + getCurrentVersion());
        storeVersion();
    } else {
        throw new IOException("Incompatible version for state DB schema: expecting DB schema version " + getCurrentVersion() + ", but loading version " + loadedVersion);
    }
}
Also used : Version(org.apache.hadoop.yarn.server.records.Version) IOException(java.io.IOException)

Aggregations

Version (org.apache.hadoop.yarn.server.records.Version)26 IOException (java.io.IOException)11 Test (org.junit.Test)9 VersionPBImpl (org.apache.hadoop.yarn.server.records.impl.pb.VersionPBImpl)8 ServiceStateException (org.apache.hadoop.service.ServiceStateException)7 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)4 VisibleForTesting (com.google.common.annotations.VisibleForTesting)3 Path (org.apache.hadoop.fs.Path)3 File (java.io.File)2 Configuration (org.apache.hadoop.conf.Configuration)2 DataOutputBuffer (org.apache.hadoop.io.DataOutputBuffer)2 Text (org.apache.hadoop.io.Text)2 Token (org.apache.hadoop.security.token.Token)2 ApplicationId (org.apache.hadoop.yarn.api.records.ApplicationId)2 ApplicationInitializationContext (org.apache.hadoop.yarn.server.api.ApplicationInitializationContext)2 FileStatus (org.apache.hadoop.fs.FileStatus)1 HdfsConfiguration (org.apache.hadoop.hdfs.HdfsConfiguration)1 MiniDFSCluster (org.apache.hadoop.hdfs.MiniDFSCluster)1 JobTokenIdentifier (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier)1 AccessControlException (org.apache.hadoop.security.AccessControlException)1