Search in sources :

Example 1 with HSAdminServer

use of org.apache.hadoop.mapreduce.v2.hs.server.HSAdminServer in project hadoop by apache.

the class TestHSAdminServer method init.

@Before
public void init() throws HadoopIllegalArgumentException, IOException {
    conf = new JobConf();
    conf.set(JHAdminConfig.JHS_ADMIN_ADDRESS, "0.0.0.0:0");
    conf.setClass("hadoop.security.group.mapping", MockUnixGroupsMapping.class, GroupMappingServiceProvider.class);
    conf.setLong("hadoop.security.groups.cache.secs", groupRefreshTimeoutSec);
    conf.setBoolean(CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHORIZATION, securityEnabled);
    Groups.getUserToGroupsMappingService(conf);
    jobHistoryService = mock(JobHistory.class);
    alds = mock(AggregatedLogDeletionService.class);
    hsAdminServer = new HSAdminServer(alds, jobHistoryService) {

        @Override
        protected Configuration createConf() {
            return conf;
        }
    };
    hsAdminServer.init(conf);
    hsAdminServer.start();
    conf.setSocketAddr(JHAdminConfig.JHS_ADMIN_ADDRESS, hsAdminServer.clientRpcServer.getListenerAddress());
    hsAdminClient = new HSAdmin(conf);
}
Also used : HSAdmin(org.apache.hadoop.mapreduce.v2.hs.client.HSAdmin) Configuration(org.apache.hadoop.conf.Configuration) JobHistory(org.apache.hadoop.mapreduce.v2.hs.JobHistory) AggregatedLogDeletionService(org.apache.hadoop.yarn.logaggregation.AggregatedLogDeletionService) JobConf(org.apache.hadoop.mapred.JobConf) Before(org.junit.Before)

Example 2 with HSAdminServer

use of org.apache.hadoop.mapreduce.v2.hs.server.HSAdminServer in project hadoop by apache.

the class JobHistoryServer method serviceInit.

@Override
protected void serviceInit(Configuration conf) throws Exception {
    Configuration config = new YarnConfiguration(conf);
    config.setBoolean(Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY, true);
    // This is required for WebApps to use https if enabled.
    MRWebAppUtil.initialize(getConfig());
    try {
        doSecureLogin(conf);
    } catch (IOException ie) {
        throw new YarnRuntimeException("History Server Failed to login", ie);
    }
    jobHistoryService = new JobHistory();
    historyContext = (HistoryContext) jobHistoryService;
    stateStore = createStateStore(conf);
    this.jhsDTSecretManager = createJHSSecretManager(conf, stateStore);
    clientService = createHistoryClientService();
    aggLogDelService = new AggregatedLogDeletionService();
    hsAdminServer = new HSAdminServer(aggLogDelService, jobHistoryService);
    addService(stateStore);
    addService(new HistoryServerSecretManagerService());
    addService(jobHistoryService);
    addService(clientService);
    addService(aggLogDelService);
    addService(hsAdminServer);
    DefaultMetricsSystem.initialize("JobHistoryServer");
    JvmMetrics jm = JvmMetrics.initSingleton("JobHistoryServer", null);
    pauseMonitor = new JvmPauseMonitor();
    addService(pauseMonitor);
    jm.setPauseMonitor(pauseMonitor);
    super.serviceInit(config);
}
Also used : YarnRuntimeException(org.apache.hadoop.yarn.exceptions.YarnRuntimeException) HSAdminServer(org.apache.hadoop.mapreduce.v2.hs.server.HSAdminServer) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) Configuration(org.apache.hadoop.conf.Configuration) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) JvmMetrics(org.apache.hadoop.metrics2.source.JvmMetrics) AggregatedLogDeletionService(org.apache.hadoop.yarn.logaggregation.AggregatedLogDeletionService) IOException(java.io.IOException) JvmPauseMonitor(org.apache.hadoop.util.JvmPauseMonitor)

Aggregations

Configuration (org.apache.hadoop.conf.Configuration)2 AggregatedLogDeletionService (org.apache.hadoop.yarn.logaggregation.AggregatedLogDeletionService)2 IOException (java.io.IOException)1 JobConf (org.apache.hadoop.mapred.JobConf)1 JobHistory (org.apache.hadoop.mapreduce.v2.hs.JobHistory)1 HSAdmin (org.apache.hadoop.mapreduce.v2.hs.client.HSAdmin)1 HSAdminServer (org.apache.hadoop.mapreduce.v2.hs.server.HSAdminServer)1 JvmMetrics (org.apache.hadoop.metrics2.source.JvmMetrics)1 JvmPauseMonitor (org.apache.hadoop.util.JvmPauseMonitor)1 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)1 YarnRuntimeException (org.apache.hadoop.yarn.exceptions.YarnRuntimeException)1 Before (org.junit.Before)1