Search in sources :

Example 56 with XConfiguration

use of org.apache.oozie.util.XConfiguration in project oozie by apache.

the class TestCoordUpdateXCommand method testCoordFromBundleJobChangeDefinition.

public void testCoordFromBundleJobChangeDefinition() throws Exception {
    final XConfiguration jobConf = new XConfiguration();
    String coordJobId = setUpBundleAndGetCoordID(jobConf);
    CoordinatorJobBean job = getCoordJobs(coordJobId);
    Element processedJobXml = XmlUtils.parseXml(job.getJobXml());
    Namespace namespace = processedJobXml.getNamespace();
    String text = ((Element) processedJobXml.getChild("input-events", namespace).getChild("data-in", namespace).getChildren("instance", namespace).get(0)).getText();
    assertEquals(text, "${coord:latest(0)}");
    final Path coordPath1 = new Path(getFsTestCaseDir(), "coord1");
    writeCoordXml(coordPath1, "coord-multiple-input-instance4.xml");
    Configuration newConf = new Configuration();
    newConf.set(OozieClient.USER_NAME, getTestUser());
    CoordUpdateXCommand update = new CoordUpdateXCommand(false, newConf, coordJobId);
    update.call();
    job = getCoordJobs(coordJobId);
    processedJobXml = XmlUtils.parseXml(job.getJobXml());
    namespace = processedJobXml.getNamespace();
    text = ((Element) processedJobXml.getChild("input-events", namespace).getChild("data-in", namespace).getChildren("instance", namespace).get(0)).getText();
    assertEquals(text, "${coord:future(0, 1)}");
}
Also used : Path(org.apache.hadoop.fs.Path) XConfiguration(org.apache.oozie.util.XConfiguration) CoordinatorJobBean(org.apache.oozie.CoordinatorJobBean) XConfiguration(org.apache.oozie.util.XConfiguration) Configuration(org.apache.hadoop.conf.Configuration) Element(org.jdom.Element) Namespace(org.jdom.Namespace)

Example 57 with XConfiguration

use of org.apache.oozie.util.XConfiguration in project oozie by apache.

the class TestCoordUpdateXCommand method testCoordFromBundleJobChangeConf.

public void testCoordFromBundleJobChangeConf() throws Exception {
    final XConfiguration jobConf = new XConfiguration();
    String coordJobId = setUpBundleAndGetCoordID(jobConf);
    jobConf.set("newvalue", "yes");
    CoordUpdateXCommand update = new CoordUpdateXCommand(false, jobConf, coordJobId);
    update.call();
    CoordinatorJobBean job = getCoordJobs(coordJobId);
    Configuration xConf = new XConfiguration(new StringReader(job.getConf()));
    assertEquals(xConf.get("newvalue"), "yes");
    /*
         * testProperty is part of bundle.xml <property> <name>testProperty</name> <value>abc</value> </property>
         */
    jobConf.set("testProperty", "xyz");
    new CoordUpdateXCommand(false, jobConf, coordJobId).call();
    job = getCoordJobs(coordJobId);
    xConf = new XConfiguration(new StringReader(job.getConf()));
    assertEquals(xConf.get("testProperty"), "xyz");
}
Also used : XConfiguration(org.apache.oozie.util.XConfiguration) CoordinatorJobBean(org.apache.oozie.CoordinatorJobBean) XConfiguration(org.apache.oozie.util.XConfiguration) Configuration(org.apache.hadoop.conf.Configuration) StringReader(java.io.StringReader)

Example 58 with XConfiguration

use of org.apache.oozie.util.XConfiguration in project oozie by apache.

the class TestCoordActionStartXCommand method getCoordConf.

@Override
protected Configuration getCoordConf(Path coordAppPath) throws IOException {
    Path wfAppPath = new Path(getFsTestCaseDir(), "app");
    FileSystem fs = getFileSystem();
    fs.mkdirs(new Path(wfAppPath, "lib"));
    File jarFile = IOUtils.createJar(new File(getTestCaseDir()), "test.jar", MapperReducerForTest.class);
    InputStream is = new FileInputStream(jarFile);
    OutputStream os = fs.create(new Path(wfAppPath, "lib/test.jar"));
    IOUtils.copyStream(is, os);
    Path input = new Path(wfAppPath, "input");
    fs.mkdirs(input);
    Writer writer = new OutputStreamWriter(fs.create(new Path(input, "test.txt")));
    writer.write("hello");
    writer.close();
    final String APP1 = "<workflow-app xmlns='uri:oozie:workflow:0.1' name='app'>" + "<start to='end'/>" + "<end name='end'/>" + "</workflow-app>";
    String subWorkflowAppPath = new Path(wfAppPath, "subwf").toString();
    fs.mkdirs(new Path(wfAppPath, "subwf"));
    Writer writer2 = new OutputStreamWriter(fs.create(new Path(subWorkflowAppPath, "workflow.xml")));
    writer2.write(APP1);
    writer2.close();
    Reader reader = IOUtils.getResourceAsReader("wf-url-template.xml", -1);
    Writer writer1 = new OutputStreamWriter(fs.create(new Path(wfAppPath, "workflow.xml")));
    IOUtils.copyCharStream(reader, writer1);
    Properties jobConf = new Properties();
    jobConf.setProperty(OozieClient.COORDINATOR_APP_PATH, coordAppPath.toString());
    jobConf.setProperty(OozieClient.USER_NAME, getTestUser());
    jobConf.setProperty(OozieClient.GROUP_NAME, getTestGroup());
    jobConf.setProperty("myJobTracker", getJobTrackerUri());
    jobConf.setProperty("myNameNode", getNameNodeUri());
    jobConf.setProperty("wfAppPath", new Path(wfAppPath, "workflow.xml").toString());
    jobConf.setProperty("mrclass", MapperReducerForTest.class.getName());
    jobConf.setProperty("delPath", new Path(wfAppPath, "output").toString());
    jobConf.setProperty("subWfApp", new Path(wfAppPath, "subwf/workflow.xml").toString());
    return new XConfiguration(jobConf);
}
Also used : Path(org.apache.hadoop.fs.Path) FileInputStream(java.io.FileInputStream) InputStream(java.io.InputStream) OutputStream(java.io.OutputStream) Reader(java.io.Reader) Properties(java.util.Properties) FileInputStream(java.io.FileInputStream) MapperReducerForTest(org.apache.oozie.action.hadoop.MapperReducerForTest) XConfiguration(org.apache.oozie.util.XConfiguration) FileSystem(org.apache.hadoop.fs.FileSystem) OutputStreamWriter(java.io.OutputStreamWriter) File(java.io.File) OutputStreamWriter(java.io.OutputStreamWriter) PrintWriter(java.io.PrintWriter) FileWriter(java.io.FileWriter) Writer(java.io.Writer)

Example 59 with XConfiguration

use of org.apache.oozie.util.XConfiguration in project oozie by apache.

the class TestCoordActionsKillXCommand method createDBRecords.

private String[] createDBRecords() throws Exception {
    JPAService jpaService = services.get(JPAService.class);
    Date startTime = DateUtils.parseDateOozieTZ("2013-08-01T23:59Z");
    Date endTime = DateUtils.parseDateOozieTZ("2013-08-02T23:59Z");
    CoordinatorJobBean job = addRecordToCoordJobTable(CoordinatorJob.Status.RUNNING, startTime, endTime, false, true, 0);
    CoordinatorActionBean action1 = addRecordToCoordActionTable(job.getId(), 1, CoordinatorAction.Status.RUNNING, "coord-action-get.xml", 0);
    CoordinatorActionBean action2 = addRecordToCoordActionTable(job.getId(), 2, CoordinatorAction.Status.RUNNING, "coord-action-get.xml", 0);
    action2.setNominalTime(DateUtils.parseDateOozieTZ("2009-12-15T02:00Z"));
    action2.setExternalId(null);
    CoordActionQueryExecutor.getInstance().executeUpdate(CoordActionQuery.UPDATE_COORD_ACTION, action2);
    WorkflowJobBean wf = new WorkflowJobBean();
    WorkflowApp app = new LiteWorkflowApp("testApp", "<workflow-app/>", new StartNodeDef(LiteWorkflowStoreService.LiteControlNodeHandler.class, "end")).addNode(new EndNodeDef("end", LiteWorkflowStoreService.LiteControlNodeHandler.class));
    wf.setId(action1.getExternalId());
    wf.setStatus(WorkflowJob.Status.RUNNING);
    WorkflowLib workflowLib = Services.get().get(WorkflowStoreService.class).getWorkflowLibWithNoDB();
    WorkflowInstance wfInstance = workflowLib.createInstance(app, new XConfiguration());
    ((LiteWorkflowInstance) wfInstance).setStatus(WorkflowInstance.Status.RUNNING);
    wf.setWorkflowInstance(wfInstance);
    jpaService.execute(new WorkflowJobInsertJPAExecutor(wf));
    return new String[] { job.getId(), action1.getId(), action2.getId(), wf.getId() };
}
Also used : WorkflowApp(org.apache.oozie.workflow.WorkflowApp) LiteWorkflowApp(org.apache.oozie.workflow.lite.LiteWorkflowApp) EndNodeDef(org.apache.oozie.workflow.lite.EndNodeDef) CoordinatorJobBean(org.apache.oozie.CoordinatorJobBean) WorkflowLib(org.apache.oozie.workflow.WorkflowLib) CoordinatorActionBean(org.apache.oozie.CoordinatorActionBean) WorkflowStoreService(org.apache.oozie.service.WorkflowStoreService) LiteWorkflowStoreService(org.apache.oozie.service.LiteWorkflowStoreService) LiteWorkflowApp(org.apache.oozie.workflow.lite.LiteWorkflowApp) LiteWorkflowInstance(org.apache.oozie.workflow.lite.LiteWorkflowInstance) WorkflowJobInsertJPAExecutor(org.apache.oozie.executor.jpa.WorkflowJobInsertJPAExecutor) LiteWorkflowInstance(org.apache.oozie.workflow.lite.LiteWorkflowInstance) WorkflowInstance(org.apache.oozie.workflow.WorkflowInstance) WorkflowJobBean(org.apache.oozie.WorkflowJobBean) Date(java.util.Date) StartNodeDef(org.apache.oozie.workflow.lite.StartNodeDef) XConfiguration(org.apache.oozie.util.XConfiguration) LiteWorkflowStoreService(org.apache.oozie.service.LiteWorkflowStoreService) JPAService(org.apache.oozie.service.JPAService)

Example 60 with XConfiguration

use of org.apache.oozie.util.XConfiguration in project oozie by apache.

the class TestCoordCommandUtils method createCoordinatorActionBean.

private CoordinatorActionBean createCoordinatorActionBean(CoordinatorJob job) throws IOException {
    CoordinatorActionBean actionBean = new CoordinatorActionBean();
    String actionId = Services.get().get(UUIDService.class).generateChildId(job.getId(), "1");
    actionBean.setJobId(job.getId());
    actionBean.setId(actionId);
    Configuration jobConf = new XConfiguration(new StringReader(job.getConf()));
    actionBean.setRunConf(XmlUtils.prettyPrint(jobConf).toString());
    return actionBean;
}
Also used : XConfiguration(org.apache.oozie.util.XConfiguration) XConfiguration(org.apache.oozie.util.XConfiguration) Configuration(org.apache.hadoop.conf.Configuration) CoordinatorActionBean(org.apache.oozie.CoordinatorActionBean) StringReader(java.io.StringReader) UUIDService(org.apache.oozie.service.UUIDService)

Aggregations

XConfiguration (org.apache.oozie.util.XConfiguration)373 Configuration (org.apache.hadoop.conf.Configuration)241 Path (org.apache.hadoop.fs.Path)106 StringReader (java.io.StringReader)97 File (java.io.File)92 IOException (java.io.IOException)78 WorkflowJobBean (org.apache.oozie.WorkflowJobBean)75 WorkflowActionBean (org.apache.oozie.WorkflowActionBean)70 CommandException (org.apache.oozie.command.CommandException)68 Element (org.jdom.Element)66 Writer (java.io.Writer)58 Date (java.util.Date)50 FileSystem (org.apache.hadoop.fs.FileSystem)48 FileWriter (java.io.FileWriter)45 Reader (java.io.Reader)43 CoordinatorActionBean (org.apache.oozie.CoordinatorActionBean)37 CoordinatorJobBean (org.apache.oozie.CoordinatorJobBean)36 JPAExecutorException (org.apache.oozie.executor.jpa.JPAExecutorException)28 OutputStream (java.io.OutputStream)27 FileOutputStream (java.io.FileOutputStream)25