Search in sources :

Example 1 with Module

use of com.google.inject.Module in project hadoop by apache.

the class TestContainerLogsPage method testContainerLogPageAccess.

@Test(timeout = 10000)
public void testContainerLogPageAccess() throws IOException {
    // SecureIOUtils require Native IO to be enabled. This test will run
    // only if it is enabled.
    assumeTrue(NativeIO.isAvailable());
    String user = "randomUser" + System.currentTimeMillis();
    File absLogDir = null, appDir = null, containerDir = null, syslog = null;
    try {
        // target log directory
        absLogDir = new File("target", TestContainerLogsPage.class.getSimpleName() + "LogDir").getAbsoluteFile();
        absLogDir.mkdir();
        Configuration conf = new Configuration();
        conf.set(YarnConfiguration.NM_LOG_DIRS, absLogDir.toURI().toString());
        conf.set(CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHENTICATION, "kerberos");
        UserGroupInformation.setConfiguration(conf);
        NodeHealthCheckerService healthChecker = createNodeHealthCheckerService(conf);
        healthChecker.init(conf);
        LocalDirsHandlerService dirsHandler = healthChecker.getDiskHandler();
        // Add an application and the corresponding containers
        RecordFactory recordFactory = RecordFactoryProvider.getRecordFactory(conf);
        long clusterTimeStamp = 1234;
        ApplicationId appId = BuilderUtils.newApplicationId(recordFactory, clusterTimeStamp, 1);
        Application app = mock(Application.class);
        when(app.getAppId()).thenReturn(appId);
        // Making sure that application returns a random user. This is required
        // for SecureIOUtils' file owner check.
        when(app.getUser()).thenReturn(user);
        ApplicationAttemptId appAttemptId = BuilderUtils.newApplicationAttemptId(appId, 1);
        ContainerId container1 = BuilderUtils.newContainerId(recordFactory, appId, appAttemptId, 0);
        // Testing secure read access for log files
        // Creating application and container directory and syslog file.
        appDir = new File(absLogDir, appId.toString());
        appDir.mkdir();
        containerDir = new File(appDir, container1.toString());
        containerDir.mkdir();
        syslog = new File(containerDir, "syslog");
        syslog.createNewFile();
        BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream(syslog));
        out.write("Log file Content".getBytes());
        out.close();
        Context context = mock(Context.class);
        ConcurrentMap<ApplicationId, Application> appMap = new ConcurrentHashMap<ApplicationId, Application>();
        appMap.put(appId, app);
        when(context.getApplications()).thenReturn(appMap);
        ConcurrentHashMap<ContainerId, Container> containers = new ConcurrentHashMap<ContainerId, Container>();
        when(context.getContainers()).thenReturn(containers);
        when(context.getLocalDirsHandler()).thenReturn(dirsHandler);
        MockContainer container = new MockContainer(appAttemptId, new AsyncDispatcher(), conf, user, appId, 1);
        container.setState(ContainerState.RUNNING);
        context.getContainers().put(container1, container);
        ContainersLogsBlock cLogsBlock = new ContainersLogsBlock(context);
        Map<String, String> params = new HashMap<String, String>();
        params.put(YarnWebParams.CONTAINER_ID, container1.toString());
        params.put(YarnWebParams.CONTAINER_LOG_TYPE, "syslog");
        Injector injector = WebAppTests.testPage(ContainerLogsPage.class, ContainersLogsBlock.class, cLogsBlock, params, (Module[]) null);
        PrintWriter spyPw = WebAppTests.getPrintWriter(injector);
        verify(spyPw).write("Exception reading log file. Application submitted by '" + user + "' doesn't own requested log file : syslog");
    } finally {
        if (syslog != null) {
            syslog.delete();
        }
        if (containerDir != null) {
            containerDir.delete();
        }
        if (appDir != null) {
            appDir.delete();
        }
        if (absLogDir != null) {
            absLogDir.delete();
        }
    }
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) NodeHealthCheckerService(org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) HashMap(java.util.HashMap) Container(org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container) ContainerId(org.apache.hadoop.yarn.api.records.ContainerId) Injector(com.google.inject.Injector) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) BufferedOutputStream(java.io.BufferedOutputStream) PrintWriter(java.io.PrintWriter) NMContext(org.apache.hadoop.yarn.server.nodemanager.NodeManager.NMContext) Context(org.apache.hadoop.yarn.server.nodemanager.Context) ContainersLogsBlock(org.apache.hadoop.yarn.server.nodemanager.webapp.ContainerLogsPage.ContainersLogsBlock) ApplicationAttemptId(org.apache.hadoop.yarn.api.records.ApplicationAttemptId) LocalDirsHandlerService(org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService) RecordFactory(org.apache.hadoop.yarn.factories.RecordFactory) AsyncDispatcher(org.apache.hadoop.yarn.event.AsyncDispatcher) FileOutputStream(java.io.FileOutputStream) ApplicationId(org.apache.hadoop.yarn.api.records.ApplicationId) Module(com.google.inject.Module) File(java.io.File) Application(org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application) Test(org.junit.Test)

Example 2 with Module

use of com.google.inject.Module in project hadoop by apache.

the class TestRMWebApp method testControllerIndex.

@Test
public void testControllerIndex() {
    Injector injector = WebAppTests.createMockInjector(TestRMWebApp.class, this, new Module() {

        @Override
        public void configure(Binder binder) {
            binder.bind(ApplicationACLsManager.class).toInstance(new ApplicationACLsManager(new Configuration()));
        }
    });
    RmController c = injector.getInstance(RmController.class);
    c.index();
    assertEquals("Applications", c.get(TITLE, "unknown"));
}
Also used : Binder(com.google.inject.Binder) ApplicationACLsManager(org.apache.hadoop.yarn.server.security.ApplicationACLsManager) CapacitySchedulerConfiguration(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration) Configuration(org.apache.hadoop.conf.Configuration) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) Injector(com.google.inject.Injector) Module(com.google.inject.Module) Test(org.junit.Test)

Example 3 with Module

use of com.google.inject.Module in project hadoop by apache.

the class TestRMWebAppFairScheduler method testFairSchedulerWebAppPage.

@Test
public void testFairSchedulerWebAppPage() {
    List<RMAppState> appStates = Arrays.asList(RMAppState.NEW, RMAppState.NEW_SAVING, RMAppState.SUBMITTED);
    final RMContext rmContext = mockRMContext(appStates);
    Injector injector = WebAppTests.createMockInjector(RMContext.class, rmContext, new Module() {

        @Override
        public void configure(Binder binder) {
            try {
                ResourceManager mockRmWithFairScheduler = mockRm(rmContext);
                binder.bind(ResourceManager.class).toInstance(mockRmWithFairScheduler);
                binder.bind(ApplicationBaseProtocol.class).toInstance(mockRmWithFairScheduler.getClientRMService());
            } catch (IOException e) {
                throw new IllegalStateException(e);
            }
        }
    });
    FairSchedulerPage fsViewInstance = injector.getInstance(FairSchedulerPage.class);
    fsViewInstance.render();
    WebAppTests.flushOutput(injector);
}
Also used : RMContext(org.apache.hadoop.yarn.server.resourcemanager.RMContext) Binder(com.google.inject.Binder) RMAppState(org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppState) Injector(com.google.inject.Injector) ResourceManager(org.apache.hadoop.yarn.server.resourcemanager.ResourceManager) IOException(java.io.IOException) Module(com.google.inject.Module) Test(org.junit.Test)

Example 4 with Module

use of com.google.inject.Module in project hadoop by apache.

the class TestRMWebAppFairScheduler method testFairSchedulerWebAppPageInInconsistentState.

/**
   *  Testing inconsistent state between AbstractYarnScheduler#applications and
   *  RMContext#applications
   */
@Test
public void testFairSchedulerWebAppPageInInconsistentState() {
    List<RMAppState> appStates = Arrays.asList(RMAppState.NEW, RMAppState.NEW_SAVING, RMAppState.SUBMITTED, RMAppState.RUNNING, RMAppState.FINAL_SAVING, RMAppState.ACCEPTED, RMAppState.FINISHED);
    final RMContext rmContext = mockRMContext(appStates);
    Injector injector = WebAppTests.createMockInjector(RMContext.class, rmContext, new Module() {

        @Override
        public void configure(Binder binder) {
            try {
                ResourceManager mockRmWithFairScheduler = mockRmWithApps(rmContext);
                binder.bind(ResourceManager.class).toInstance(mockRmWithFairScheduler);
                binder.bind(ApplicationBaseProtocol.class).toInstance(mockRmWithFairScheduler.getClientRMService());
            } catch (IOException e) {
                throw new IllegalStateException(e);
            }
        }
    });
    FairSchedulerPage fsViewInstance = injector.getInstance(FairSchedulerPage.class);
    try {
        fsViewInstance.render();
    } catch (Exception e) {
        Assert.fail("Failed to render FairSchedulerPage: " + StringUtils.stringifyException(e));
    }
    WebAppTests.flushOutput(injector);
}
Also used : RMContext(org.apache.hadoop.yarn.server.resourcemanager.RMContext) Binder(com.google.inject.Binder) RMAppState(org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppState) Injector(com.google.inject.Injector) ResourceManager(org.apache.hadoop.yarn.server.resourcemanager.ResourceManager) IOException(java.io.IOException) Module(com.google.inject.Module) IOException(java.io.IOException) Test(org.junit.Test)

Example 5 with Module

use of com.google.inject.Module in project hadoop by apache.

the class TestAppPage method testAppBlockRenderWithNullCurrentAppAttempt.

@Test
public void testAppBlockRenderWithNullCurrentAppAttempt() throws Exception {
    final ApplicationId APP_ID = ApplicationId.newInstance(1234L, 0);
    Injector injector;
    // init app
    RMApp app = mock(RMApp.class);
    when(app.getTrackingUrl()).thenReturn("http://host:123");
    when(app.getState()).thenReturn(RMAppState.FAILED);
    when(app.getApplicationId()).thenReturn(APP_ID);
    when(app.getApplicationType()).thenReturn("Type");
    when(app.getUser()).thenReturn("user");
    when(app.getName()).thenReturn("Name");
    when(app.getQueue()).thenReturn("queue");
    when(app.getDiagnostics()).thenReturn(new StringBuilder());
    when(app.getFinalApplicationStatus()).thenReturn(FinalApplicationStatus.FAILED);
    when(app.getFinalApplicationStatus()).thenReturn(FinalApplicationStatus.FAILED);
    when(app.getStartTime()).thenReturn(0L);
    when(app.getFinishTime()).thenReturn(0L);
    when(app.createApplicationState()).thenReturn(YarnApplicationState.FAILED);
    RMAppMetrics appMetrics = new RMAppMetrics(Resource.newInstance(0, 0), 0, 0, 0, 0, 0, 0);
    when(app.getRMAppMetrics()).thenReturn(appMetrics);
    // initialize RM Context, and create RMApp, without creating RMAppAttempt
    final RMContext rmContext = TestRMWebApp.mockRMContext(15, 1, 2, 8);
    rmContext.getRMApps().put(APP_ID, app);
    injector = WebAppTests.createMockInjector(RMContext.class, rmContext, new Module() {

        @Override
        public void configure(Binder binder) {
            try {
                ResourceManager rm = TestRMWebApp.mockRm(rmContext);
                binder.bind(ResourceManager.class).toInstance(rm);
                binder.bind(ApplicationBaseProtocol.class).toInstance(rm.getClientRMService());
            } catch (IOException e) {
                throw new IllegalStateException(e);
            }
        }
    });
    AppBlock instance = injector.getInstance(AppBlock.class);
    instance.set(YarnWebParams.APPLICATION_ID, APP_ID.toString());
    instance.render();
}
Also used : RMApp(org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMApp) RMContext(org.apache.hadoop.yarn.server.resourcemanager.RMContext) ApplicationBaseProtocol(org.apache.hadoop.yarn.api.ApplicationBaseProtocol) RMAppMetrics(org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppMetrics) ResourceManager(org.apache.hadoop.yarn.server.resourcemanager.ResourceManager) IOException(java.io.IOException) Binder(com.google.inject.Binder) AppBlock(org.apache.hadoop.yarn.server.webapp.AppBlock) Injector(com.google.inject.Injector) ApplicationId(org.apache.hadoop.yarn.api.records.ApplicationId) Module(com.google.inject.Module) Test(org.junit.Test)

Aggregations

Module (com.google.inject.Module)386 Injector (com.google.inject.Injector)253 AbstractModule (com.google.inject.AbstractModule)252 Binder (com.google.inject.Binder)63 Test (org.junit.Test)51 PrivateModule (com.google.inject.PrivateModule)46 Provider (com.google.inject.Provider)44 ArrayList (java.util.ArrayList)30 CreationException (com.google.inject.CreationException)26 Guice.createInjector (com.google.inject.Guice.createInjector)25 Map (java.util.Map)24 Properties (java.util.Properties)22 ImmutableList (com.google.common.collect.ImmutableList)20 HashMap (java.util.HashMap)20 HashSet (java.util.HashSet)20 Key (com.google.inject.Key)19 Set (java.util.Set)19 ImmutableSet (com.google.common.collect.ImmutableSet)18 Binding (com.google.inject.Binding)17 List (java.util.List)17