Examples with EmbeddedServletContainer org.apache.oozie.test.EmbeddedServletContainer used on opensource projects

Search in sources :

Example 1 with EmbeddedServletContainer

use of org.apache.oozie.test.EmbeddedServletContainer in project oozie by apache.

the class LocalOozie method start.

/**
 * Start LocalOozie.
 *
 * @throws Exception if LocalOozie could not be started.
 */
public static synchronized void start() throws Exception {
    if (localOozieActive) {
        throw new IllegalStateException("LocalOozie is already initialized");
    }
    String log4jFile = System.getProperty(XLogService.LOG4J_FILE, null);
    String oozieLocalLog = System.getProperty("oozielocal.log", null);
    if (log4jFile == null) {
        System.setProperty(XLogService.LOG4J_FILE, "localoozie-log4j.properties");
    }
    if (oozieLocalLog == null) {
        System.setProperty("oozielocal.log", "./oozielocal.log");
    }
    localOozieActive = true;
    new Services().init();
    if (log4jFile != null) {
        System.setProperty(XLogService.LOG4J_FILE, log4jFile);
    } else {
        System.getProperties().remove(XLogService.LOG4J_FILE);
    }
    if (oozieLocalLog != null) {
        System.setProperty("oozielocal.log", oozieLocalLog);
    } else {
        System.getProperties().remove("oozielocal.log");
    }
    container = new EmbeddedServletContainer("oozie");
    container.addServletEndpoint("/callback", CallbackServlet.class);
    container.start();
    String callbackUrl = container.getServletURL("/callback");
    Services.get().getConf().set(CallbackService.CONF_BASE_URL, callbackUrl);
    XLog.getLog(LocalOozie.class).info("LocalOozie started callback set to [{0}]", callbackUrl);
}
Also used : Services(org.apache.oozie.service.Services) EmbeddedServletContainer(org.apache.oozie.test.EmbeddedServletContainer)

Example 2 with EmbeddedServletContainer

use of org.apache.oozie.test.EmbeddedServletContainer in project oozie by apache.

the class TestZKXLogStreamingService method testStreamingWithMultipleOozieServers.

public void testStreamingWithMultipleOozieServers() throws Exception {
    XLogFilter.reset();
    XLogFilter.defineParameter("USER");
    XLogFilter.defineParameter("GROUP");
    XLogFilter.defineParameter("TOKEN");
    XLogFilter.defineParameter("APP");
    XLogFilter.defineParameter("JOB");
    XLogFilter.defineParameter("ACTION");
    XLogFilter xf = new XLogFilter();
    xf.setParameter("JOB", "0000003-130610102426873-oozie-rkan-W");
    xf.setLogLevel("WARN|INFO");
    File log4jFile = new File(getTestCaseConfDir(), "test-log4j.properties");
    ClassLoader cl = Thread.currentThread().getContextClassLoader();
    InputStream is = cl.getResourceAsStream("test-no-dash-log4j.properties");
    Properties log4jProps = new Properties();
    log4jProps.load(is);
    // prevent conflicts with other tests by changing the log file location
    log4jProps.setProperty("log4j.appender.oozie.File", getTestCaseDir() + "/oozie.log");
    log4jProps.store(new FileOutputStream(log4jFile), "");
    setSystemProperty(XLogService.LOG4J_FILE, log4jFile.getName());
    assertFalse(doStreamDisabledCheck());
    File logFile = new File(Services.get().get(XLogService.class).getOozieLogPath(), Services.get().get(XLogService.class).getOozieLogName());
    logFile.getParentFile().mkdirs();
    FileWriter logWriter = new FileWriter(logFile);
    // local logs
    logWriter.append("2013-06-10 10:25:44,008 WARN HiveActionExecutor:542 SERVER[foo] USER[rkanter] GROUP[-] TOKEN[] " + "APP[hive-wf] JOB[0000003-130610102426873-oozie-rkan-W] ACTION[0000003-130610102426873-oozie-rkan-W@hive-node] " + "credentials is null for the action _L3_").append("\n").append("2013-06-10 10:26:10,008 INFO HiveActionExecutor:539 SERVER[foo] USER[rkanter] GROUP[-] TOKEN[] " + "APP[hive-wf] JOB[0000003-130610102426873-oozie-rkan-W] ACTION[0000003-130610102426873-oozie-rkan-W@hive-node] " + "action completed, external ID [job_201306101021_0005] _L4_").append("\n").append("2013-06-10 10:26:10,341 WARN ActionStartXCommand:542 USER[rkanter] GROUP[-] TOKEN[] " + "APP[hive-wf] JOB[0000003-130610102426873-oozie-rkan-W] ACTION[0000003-130610102426873-oozie-rkan-W@end] " + "[***0000003-130610102426873-oozie-rkan-W@end***]Action updated in DB! _L6_").append("\n");
    logWriter.close();
    // logs to be returned by another "Oozie server"
    DummyLogStreamingServlet.logs = "2013-06-10 10:25:43,575 WARN ActionStartXCommand:542 SERVER[foo] USER[rkanter] GROUP[-] TOKEN[] APP[hive-wf] " + "JOB[0000003-130610102426873-oozie-rkan-W] ACTION[0000003-130610102426873-oozie-rkan-W@:start:] " + "[***0000003-130610102426873-oozie-rkan-W@:start:***]Action status=DONE _L1_" + "\n" + "2013-06-10 10:25:43,575 WARN ActionStartXCommand:542 SERVER[foo] USER[rkanter] GROUP[-] TOKEN[] APP[hive-wf] " + "JOB[0000003-130610102426873-oozie-rkan-W] ACTION[0000003-130610102426873-oozie-rkan-W@:start:] " + "[***0000003-130610102426873-oozie-rkan-W@:start:***]Action updated in DB! _L2_" + "\n" + "2013-06-10 10:26:10,148 INFO HiveActionExecutor:539 SERVER[foo] USER[rkanter] GROUP[-] TOKEN[] APP[hive-wf] " + "JOB[0000003-130610102426873-oozie-rkan-W] ACTION[0000003-130610102426873-oozie-rkan-W@hive-node] action produced" + " output _L5_" + "\n" + // a multiline message with a stack trace
    "2013-06-10 10:26:30,202  WARN ActionStartXCommand:542 - SERVER[foo] USER[rkanter] GROUP[-] TOKEN[] APP[hive-wf] " + "JOB[0000003-130610102426873-oozie-rkan-W] ACTION[0000003-130610102426873-oozie-rkan-W@hive-node] Error starting " + "action [hive-node]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: java.io.IOException: Unknown " + "protocol to name node: org.apache.hadoop.mapred.JobSubmissionProtocol _L7_\n" + "     at org.apache.hadoop.hdfs.server.namenode.NameNode.getProtocolVersion(NameNode.java:156) _L8_\n" + "     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)_L9_\n" + "     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) _L10_\n" + "     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426) _L11_\n" + "] _L12_\n" + "org.apache.oozie.action.ActionExecutorException: JA009: java.io.IOException: Unknown protocol to name node: " + "org.apache.hadoop.mapred.JobSubmissionProtocol _L13_\n" + "     at org.apache.hadoop.hdfs.server.namenode.NameNode.getProtocolVersion(NameNode.java:156) _L14_\n" + "     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) _L15_\n" + "     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) _L16_\n";
    String out = doStreamLog(xf);
    String[] outArr = out.split("\n");
    assertEquals(3, outArr.length);
    assertFalse(out.contains("_L1_"));
    assertFalse(out.contains("_L2_"));
    assertTrue(outArr[0].contains("_L3_"));
    assertTrue(outArr[1].contains("_L4_"));
    assertFalse(out.contains("_L5_"));
    assertTrue(outArr[2].contains("_L6_"));
    // We'll use a DummyZKOozie to create an entry in ZK and then set its url to an (unrelated) servlet that will simply return
    // some log messages
    DummyZKOozie dummyOozie = null;
    EmbeddedServletContainer container = new EmbeddedServletContainer("oozie");
    container.addServletEndpoint("/other-oozie-server/*", DummyLogStreamingServlet.class);
    try {
        container.start();
        dummyOozie = new DummyZKOozie("9876", container.getServletURL("/other-oozie-server/*"));
        DummyLogStreamingServlet.lastQueryString = null;
        out = doStreamLog(xf);
        outArr = out.split("\n");
        assertEquals(16, outArr.length);
        assertTrue(outArr[0].contains("_L1_"));
        assertTrue(outArr[1].contains("_L2_"));
        assertTrue(outArr[2].contains("_L3_"));
        assertTrue(outArr[3].contains("_L4_"));
        assertTrue(outArr[4].contains("_L5_"));
        assertTrue(outArr[5].contains("_L6_"));
        assertTrue(outArr[6].contains("_L7_"));
        assertTrue(outArr[7].contains("_L8_"));
        assertTrue(outArr[8].contains("_L9_"));
        assertTrue(outArr[9].contains("_L10_"));
        assertTrue(outArr[10].contains("_L11_"));
        assertTrue(outArr[11].contains("_L12_"));
        assertTrue(outArr[12].contains("_L13_"));
        assertTrue(outArr[13].contains("_L14_"));
        assertTrue(outArr[14].contains("_L15_"));
        assertTrue(outArr[15].contains("_L16_"));
        assertEquals("show=log&allservers=false", DummyLogStreamingServlet.lastQueryString);
        // If we stop the container but leave the DummyZKOozie running, it will simulate if that server is down but still has
        // info in ZK; we should be able to get the logs from other servers (in this case, this server) and a message about
        // which servers it couldn't reach
        container.stop();
        out = doStreamLog(xf);
        outArr = out.split("\n");
        assertEquals(6, outArr.length);
        assertTrue(outArr[0].startsWith("Unable"));
        assertEquals("9876", outArr[1].trim());
        assertEquals("", outArr[2]);
        assertFalse(out.contains("_L1_"));
        assertFalse(out.contains("_L2_"));
        assertTrue(outArr[3].contains("_L3_"));
        assertTrue(outArr[4].contains("_L4_"));
        assertFalse(out.contains("_L5_"));
        assertTrue(outArr[5].contains("_L6_"));
    } finally {
        if (dummyOozie != null) {
            dummyOozie.teardown();
        }
        container.stop();
    }
}
Also used : XLogFilter(org.apache.oozie.util.XLogFilter) InputStream(java.io.InputStream) FileOutputStream(java.io.FileOutputStream) FileWriter(java.io.FileWriter) EmbeddedServletContainer(org.apache.oozie.test.EmbeddedServletContainer) Properties(java.util.Properties) File(java.io.File)

Example 3 with EmbeddedServletContainer

use of org.apache.oozie.test.EmbeddedServletContainer in project oozie by apache.

the class DagServletTestCase method runTest.

protected void runTest(String[] servletPath, Class[] servletClass, boolean securityEnabled, Callable<Void> assertions) throws Exception {
    Services services = new Services();
    this.servletPath = servletPath[0];
    try {
        String proxyUser = getTestUser();
        services.getConf().set(ProxyUserService.CONF_PREFIX + proxyUser + ProxyUserService.HOSTS, "*");
        services.getConf().set(ProxyUserService.CONF_PREFIX + proxyUser + ProxyUserService.GROUPS, "*");
        services.init();
        services.getConf().setBoolean(AuthorizationService.CONF_SECURITY_ENABLED, securityEnabled);
        Services.get().setService(ForTestAuthorizationService.class);
        Services.get().setService(ForTestWorkflowStoreService.class);
        Services.get().setService(MockDagEngineService.class);
        Services.get().setService(MockCoordinatorEngineService.class);
        container = new EmbeddedServletContainer("oozie");
        for (int i = 0; i < servletPath.length; i++) {
            container.addServletEndpoint(servletPath[i], servletClass[i]);
        }
        container.addFilter("*", HostnameFilter.class);
        container.addFilter("*", AuthFilter.class);
        setSystemProperty("user.name", getTestUser());
        container.start();
        assertions.call();
    } finally {
        this.servletPath = null;
        if (container != null) {
            container.stop();
        }
        services.destroy();
        container = null;
    }
}
Also used : Services(org.apache.oozie.service.Services) EmbeddedServletContainer(org.apache.oozie.test.EmbeddedServletContainer)

Example 4 with EmbeddedServletContainer

use of org.apache.oozie.test.EmbeddedServletContainer in project oozie by apache.

the class TestAuthFilterAuthOozieClient method runTest.

protected void runTest(Callable<Void> assertions, Configuration additionalConf) throws Exception {
    Services services = new Services();
    try {
        services.init();
        if (additionalConf != null) {
            for (Map.Entry<String, String> prop : additionalConf) {
                Services.get().getConf().set(prop.getKey(), prop.getValue());
            }
        }
        Services.get().setService(ForTestAuthorizationService.class);
        Services.get().setService(ForTestWorkflowStoreService.class);
        Services.get().setService(MockDagEngineService.class);
        Services.get().setService(MockCoordinatorEngineService.class);
        container = new EmbeddedServletContainer("oozie");
        container.addServletEndpoint("/versions", HeaderTestingVersionServlet.class);
        String version = "/v" + XOozieClient.WS_PROTOCOL_VERSION;
        container.addServletEndpoint(version + "/admin/*", V1AdminServlet.class);
        container.addFilter("*", HostnameFilter.class);
        container.addFilter("/*", AuthFilter.class);
        container.start();
        assertions.call();
    } finally {
        if (container != null) {
            container.stop();
        }
        services.destroy();
        container = null;
    }
}
Also used : Services(org.apache.oozie.service.Services) EmbeddedServletContainer(org.apache.oozie.test.EmbeddedServletContainer) Map(java.util.Map)

Example 5 with EmbeddedServletContainer

use of org.apache.oozie.test.EmbeddedServletContainer in project oozie by apache.

the class TestBulkMonitorWebServiceAPI method runTest.

private void runTest(String[] servletPath, Class[] servletClass, boolean securityEnabled, Callable<Void> assertions) throws Exception {
    Services services = new Services();
    this.servletPath = servletPath[0];
    try {
        String proxyUser = getTestUser();
        services.getConf().set(ProxyUserService.CONF_PREFIX + proxyUser + ProxyUserService.HOSTS, "*");
        services.getConf().set(ProxyUserService.CONF_PREFIX + proxyUser + ProxyUserService.GROUPS, "*");
        services.init();
        services.getConf().setBoolean(AuthorizationService.CONF_SECURITY_ENABLED, securityEnabled);
        Services.get().setService(ForTestAuthorizationService.class);
        Services.get().setService(BundleEngineService.class);
        container = new EmbeddedServletContainer("oozie");
        for (int i = 0; i < servletPath.length; i++) {
            container.addServletEndpoint(servletPath[i], servletClass[i]);
        }
        container.addFilter("*", HostnameFilter.class);
        container.addFilter("*", AuthFilter.class);
        setSystemProperty("user.name", getTestUser());
        container.start();
        assertions.call();
    } finally {
        this.servletPath = null;
        if (container != null) {
            container.stop();
        }
        services.destroy();
        container = null;
    }
}
Also used : Services(org.apache.oozie.service.Services) EmbeddedServletContainer(org.apache.oozie.test.EmbeddedServletContainer)

Aggregations

EmbeddedServletContainer (org.apache.oozie.test.EmbeddedServletContainer)13 Services (org.apache.oozie.service.Services)8 File (java.io.File)3 FileOutputStream (java.io.FileOutputStream)3 FileWriter (java.io.FileWriter)3 InputStream (java.io.InputStream)3 Properties (java.util.Properties)3 XLogFilter (org.apache.oozie.util.XLogFilter)3 Date (java.util.Date)2 URI (java.net.URI)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1 Configuration (org.apache.hadoop.conf.Configuration)1 Path (org.apache.hadoop.fs.Path)1 ActionService (org.apache.oozie.service.ActionService)1
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial