Examples with CoordActionGetForInputCheckJPAExecutor org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor used on opensource projects

Search in sources :

Example 1 with CoordActionGetForInputCheckJPAExecutor

use of org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor in project oozie by apache.

the class CoordPushDependencyCheckXCommand method loadState.

@Override
protected void loadState() throws CommandException {
    jpaService = Services.get().get(JPAService.class);
    try {
        coordAction = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(actionId));
        if (coordAction != null) {
            coordJob = jpaService.execute(new CoordJobGetJPAExecutor(coordAction.getJobId()));
            LogUtils.setLogInfo(coordAction);
        } else {
            throw new CommandException(ErrorCode.E0605, actionId);
        }
    } catch (JPAExecutorException je) {
        throw new CommandException(je);
    }
}
Also used : JPAExecutorException(org.apache.oozie.executor.jpa.JPAExecutorException) CoordJobGetJPAExecutor(org.apache.oozie.executor.jpa.CoordJobGetJPAExecutor) CoordActionGetForInputCheckJPAExecutor(org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor) CommandException(org.apache.oozie.command.CommandException) JPAService(org.apache.oozie.service.JPAService)

Example 2 with CoordActionGetForInputCheckJPAExecutor

use of org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor in project oozie by apache.

the class CoordActionInputCheckXCommand method loadState.

@Override
protected void loadState() throws CommandException {
    if (jpaService == null) {
        jpaService = Services.get().get(JPAService.class);
    }
    try {
        coordAction = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(actionId));
        coordJob = CoordJobQueryExecutor.getInstance().get(CoordJobQuery.GET_COORD_JOB_INPUT_CHECK, coordAction.getJobId());
    } catch (JPAExecutorException je) {
        throw new CommandException(je);
    }
    LogUtils.setLogInfo(coordAction);
}
Also used : JPAExecutorException(org.apache.oozie.executor.jpa.JPAExecutorException) CoordActionGetForInputCheckJPAExecutor(org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor) CommandException(org.apache.oozie.command.CommandException) JPAService(org.apache.oozie.service.JPAService)

Example 3 with CoordActionGetForInputCheckJPAExecutor

use of org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor in project oozie by apache.

the class TestCoordActionInputCheckXCommand method testActionInputCheckLatestActionCreationTimeWithPushDependency.

public void testActionInputCheckLatestActionCreationTimeWithPushDependency() throws Exception {
    setupServicesForHCatalog(services);
    Services.get().getConf().setBoolean(CoordELFunctions.LATEST_EL_USE_CURRENT_TIME, false);
    services.init();
    String jobId = "0000000-" + new Date().getTime() + "-TestCoordActionInputCheckXCommand-C";
    Date startTime = DateUtils.parseDateOozieTZ("2009-02-15T23:59" + TZ);
    Date endTime = DateUtils.parseDateOozieTZ("2009-02-16T23:59" + TZ);
    CoordinatorJobBean job = addRecordToCoordJobTable(jobId, startTime, endTime, "latest");
    new CoordMaterializeTransitionXCommand(job.getId(), 3600).call();
    // Set push missing dependency
    JPAService jpaService = Services.get().get(JPAService.class);
    CoordinatorActionBean action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    final String pushMissingDependency = getTestCaseFileUri("2009/02/05");
    action.setPushMissingDependencies(pushMissingDependency);
    CoordActionQueryExecutor.getInstance().executeUpdate(CoordActionQueryExecutor.CoordActionQuery.UPDATE_COORD_ACTION_FOR_PUSH_INPUTCHECK, action);
    // Update action creation time
    String actionXML = action.getActionXml();
    String actionCreationTime = "2009-02-15T01:00" + TZ;
    actionXML = actionXML.replaceAll("action-actual-time=\".*\">", "action-actual-time=\"" + actionCreationTime + "\">");
    action.setActionXml(actionXML);
    action.setCreatedTime(DateUtils.parseDateOozieTZ(actionCreationTime));
    CoordActionQueryExecutor.getInstance().executeUpdate(CoordActionQueryExecutor.CoordActionQuery.UPDATE_COORD_ACTION_FOR_INPUTCHECK, action);
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertTrue(action.getActionXml().contains("action-actual-time=\"2009-02-15T01:00"));
    new CoordActionInputCheckXCommand(job.getId() + "@1", job.getId()).call();
    new CoordPushDependencyCheckXCommand(job.getId() + "@1").call();
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertEquals(CoordCommandUtils.RESOLVED_UNRESOLVED_SEPARATOR + "${coord:latestRange(-3,0)}", action.getMissingDependencies());
    assertEquals(pushMissingDependency, action.getPushMissingDependencies());
    // providing some of the dataset dirs required as per coordinator specification with holes
    // before and after action creation time
    createTestCaseSubDir("2009/03/05/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/02/19/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/02/12/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/01/22/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/01/08/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/01/01/_SUCCESS".split("/"));
    // Run input check after making latest available
    new CoordActionInputCheckXCommand(job.getId() + "@1", job.getId()).call();
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertEquals(CoordCommandUtils.RESOLVED_UNRESOLVED_SEPARATOR + "${coord:latestRange(-3,0)}", action.getMissingDependencies());
    assertEquals(pushMissingDependency, action.getPushMissingDependencies());
    // Run input check after making push dependencies available
    createTestCaseSubDir("2009/02/05/_SUCCESS".split("/"));
    new CoordPushDependencyCheckXCommand(job.getId() + "@1").call();
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertEquals("", action.getPushMissingDependencies());
    checkCoordAction(job.getId() + "@1", CoordCommandUtils.RESOLVED_UNRESOLVED_SEPARATOR + "${coord:latestRange(-3,0)}", CoordinatorAction.Status.WAITING);
    new CoordActionInputCheckXCommand(job.getId() + "@1", job.getId()).call();
    // Sleep for sometime as it gets requeued with 10ms delay on failure to acquire write lock
    Thread.sleep(1000);
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertEquals("", action.getMissingDependencies());
    actionXML = action.getActionXml();
    // Datasets only before action creation/actual time should be picked up.
    String resolvedList = getTestCaseFileUri("2009/02/12") + CoordELFunctions.INSTANCE_SEPARATOR + getTestCaseFileUri("2009/02/05") + CoordELFunctions.INSTANCE_SEPARATOR + getTestCaseFileUri("2009/01/22") + CoordELFunctions.INSTANCE_SEPARATOR + getTestCaseFileUri("2009/01/08");
    System.out.println("Expected: " + resolvedList);
    System.out.println("Actual: " + actionXML.substring(actionXML.indexOf("<uris>") + 6, actionXML.indexOf("</uris>")));
    assertEquals(resolvedList, actionXML.substring(actionXML.indexOf("<uris>") + 6, actionXML.indexOf("</uris>")));
}
Also used : CoordinatorJobBean(org.apache.oozie.CoordinatorJobBean) CoordinatorActionBean(org.apache.oozie.CoordinatorActionBean) CoordActionGetForInputCheckJPAExecutor(org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor) JPAService(org.apache.oozie.service.JPAService) Date(java.util.Date)

Example 4 with CoordActionGetForInputCheckJPAExecutor

use of org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor in project oozie by apache.

the class TestCoordActionInputCheckXCommand method testActionInputCheckLatestCurrentTimeWithPushDependency.

public void testActionInputCheckLatestCurrentTimeWithPushDependency() throws Exception {
    setupServicesForHCatalog(services);
    Services.get().getConf().setBoolean(CoordELFunctions.LATEST_EL_USE_CURRENT_TIME, true);
    services.init();
    String jobId = "0000000-" + new Date().getTime() + "-TestCoordActionInputCheckXCommand-C";
    Date startTime = DateUtils.parseDateOozieTZ("2009-02-15T23:59" + TZ);
    Date endTime = DateUtils.parseDateOozieTZ("2009-02-16T23:59" + TZ);
    CoordinatorJobBean job = addRecordToCoordJobTable(jobId, startTime, endTime, "latest");
    new CoordMaterializeTransitionXCommand(job.getId(), 3600).call();
    // Set push missing dependency
    JPAService jpaService = Services.get().get(JPAService.class);
    CoordinatorActionBean action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    final String pushMissingDependency = getTestCaseFileUri("2009/02/05");
    action.setPushMissingDependencies(pushMissingDependency);
    CoordActionQueryExecutor.getInstance().executeUpdate(CoordActionQueryExecutor.CoordActionQuery.UPDATE_COORD_ACTION_FOR_PUSH_INPUTCHECK, action);
    // Update action creation time
    String actionXML = action.getActionXml();
    String actionCreationTime = "2009-02-15T01:00" + TZ;
    actionXML = actionXML.replaceAll("action-actual-time=\".*\">", "action-actual-time=\"" + actionCreationTime + "\">");
    action.setActionXml(actionXML);
    action.setCreatedTime(DateUtils.parseDateOozieTZ(actionCreationTime));
    CoordActionQueryExecutor.getInstance().executeUpdate(CoordActionQueryExecutor.CoordActionQuery.UPDATE_COORD_ACTION_FOR_INPUTCHECK, action);
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertTrue(action.getActionXml().contains("action-actual-time=\"2009-02-15T01:00"));
    new CoordActionInputCheckXCommand(job.getId() + "@1", job.getId()).call();
    new CoordPushDependencyCheckXCommand(job.getId() + "@1").call();
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertEquals(CoordCommandUtils.RESOLVED_UNRESOLVED_SEPARATOR + "${coord:latestRange(-3,0)}", action.getMissingDependencies());
    assertEquals(pushMissingDependency, action.getPushMissingDependencies());
    // providing some of the dataset dirs required as per coordinator specification with holes
    // before and after action creation time
    createTestCaseSubDir("2009/03/05/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/02/19/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/02/12/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/01/22/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/01/08/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/01/05/_SUCCESS".split("/"));
    // Run input check after making latest available
    new CoordActionInputCheckXCommand(job.getId() + "@1", job.getId()).call();
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertEquals(CoordCommandUtils.RESOLVED_UNRESOLVED_SEPARATOR + "${coord:latestRange(-3,0)}", action.getMissingDependencies());
    assertEquals(pushMissingDependency, action.getPushMissingDependencies());
    // Run input check after making push dependencies available
    createTestCaseSubDir("2009/02/05/_SUCCESS".split("/"));
    new CoordPushDependencyCheckXCommand(job.getId() + "@1").call();
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertEquals("", action.getPushMissingDependencies());
    checkCoordAction(job.getId() + "@1", CoordCommandUtils.RESOLVED_UNRESOLVED_SEPARATOR + "${coord:latestRange(-3,0)}", CoordinatorAction.Status.WAITING);
    new CoordActionInputCheckXCommand(job.getId() + "@1", job.getId()).call();
    // Sleep for sometime as it gets requeued with 10ms delay on failure to acquire write lock
    Thread.sleep(1000);
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertEquals("", action.getMissingDependencies());
    actionXML = action.getActionXml();
    // Datasets should be picked up based on current time and not action creation/actual time.
    String resolvedList = getTestCaseFileUri("2009/03/05") + CoordELFunctions.INSTANCE_SEPARATOR + getTestCaseFileUri("2009/02/19") + CoordELFunctions.INSTANCE_SEPARATOR + getTestCaseFileUri("2009/02/12") + CoordELFunctions.INSTANCE_SEPARATOR + getTestCaseFileUri("2009/02/05");
    System.out.println("Expected: " + resolvedList);
    System.out.println("Actual: " + actionXML.substring(actionXML.indexOf("<uris>") + 6, actionXML.indexOf("</uris>")));
    assertEquals(resolvedList, actionXML.substring(actionXML.indexOf("<uris>") + 6, actionXML.indexOf("</uris>")));
}
Also used : CoordinatorJobBean(org.apache.oozie.CoordinatorJobBean) CoordinatorActionBean(org.apache.oozie.CoordinatorActionBean) CoordActionGetForInputCheckJPAExecutor(org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor) JPAService(org.apache.oozie.service.JPAService) Date(java.util.Date)

Example 5 with CoordActionGetForInputCheckJPAExecutor

use of org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor in project oozie by apache.

the class TestCoordActionInputCheckXCommand method testActionInputCheckLatestActionCreationTime.

public void testActionInputCheckLatestActionCreationTime() throws Exception {
    Services.get().getConf().setBoolean(CoordELFunctions.LATEST_EL_USE_CURRENT_TIME, false);
    String jobId = "0000000-" + new Date().getTime() + "-TestCoordActionInputCheckXCommand-C";
    Date startTime = DateUtils.parseDateOozieTZ("2009-02-15T23:59" + TZ);
    Date endTime = DateUtils.parseDateOozieTZ("2009-02-16T23:59" + TZ);
    CoordinatorJobBean job = addRecordToCoordJobTable(jobId, startTime, endTime, "latest");
    new CoordMaterializeTransitionXCommand(job.getId(), 3600).call();
    JPAService jpaService = Services.get().get(JPAService.class);
    CoordinatorActionBean action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertEquals(CoordCommandUtils.RESOLVED_UNRESOLVED_SEPARATOR + "${coord:latestRange(-3,0)}", action.getMissingDependencies());
    // Update action creation time
    String actionXML = action.getActionXml();
    String actionCreationTime = "2009-02-15T01:00" + TZ;
    actionXML = actionXML.replaceAll("action-actual-time=\".*\">", "action-actual-time=\"" + actionCreationTime + "\">");
    action.setActionXml(actionXML);
    action.setCreatedTime(DateUtils.parseDateOozieTZ(actionCreationTime));
    CoordActionQueryExecutor.getInstance().executeUpdate(CoordActionQueryExecutor.CoordActionQuery.UPDATE_COORD_ACTION_FOR_INPUTCHECK, action);
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    assertTrue(action.getActionXml().contains("action-actual-time=\"2009-02-15T01:00"));
    Thread.sleep(1000);
    // providing some of the dataset dirs required as per coordinator specification with holes
    // before and after action creation time
    createTestCaseSubDir("2009/03/05/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/02/19/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/02/12/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/02/05/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/01/22/_SUCCESS".split("/"));
    createTestCaseSubDir("2009/01/08/_SUCCESS".split("/"));
    new CoordActionInputCheckXCommand(job.getId() + "@1", job.getId()).call();
    // Sleep for sometime as it gets requeued with 10ms delay on failure to acquire write lock
    Thread.sleep(1000);
    action = jpaService.execute(new CoordActionGetForInputCheckJPAExecutor(job.getId() + "@1"));
    actionXML = action.getActionXml();
    assertEquals("", action.getMissingDependencies());
    // Datasets only before action creation/actual time should be picked up.
    String resolvedList = getTestCaseFileUri("2009/02/12") + CoordELFunctions.INSTANCE_SEPARATOR + getTestCaseFileUri("2009/02/05") + CoordELFunctions.INSTANCE_SEPARATOR + getTestCaseFileUri("2009/01/22") + CoordELFunctions.INSTANCE_SEPARATOR + getTestCaseFileUri("2009/01/08");
    System.out.println("Expected: " + resolvedList);
    System.out.println("Actual: " + actionXML.substring(actionXML.indexOf("<uris>") + 6, actionXML.indexOf("</uris>")));
    assertEquals(resolvedList, actionXML.substring(actionXML.indexOf("<uris>") + 6, actionXML.indexOf("</uris>")));
}
Also used : CoordinatorJobBean(org.apache.oozie.CoordinatorJobBean) CoordinatorActionBean(org.apache.oozie.CoordinatorActionBean) CoordActionGetForInputCheckJPAExecutor(org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor) JPAService(org.apache.oozie.service.JPAService) Date(java.util.Date)

Aggregations

CoordActionGetForInputCheckJPAExecutor (org.apache.oozie.executor.jpa.CoordActionGetForInputCheckJPAExecutor)6 JPAService (org.apache.oozie.service.JPAService)6 Date (java.util.Date)4 CoordinatorActionBean (org.apache.oozie.CoordinatorActionBean)4 CoordinatorJobBean (org.apache.oozie.CoordinatorJobBean)4 CommandException (org.apache.oozie.command.CommandException)2 JPAExecutorException (org.apache.oozie.executor.jpa.JPAExecutorException)2 CoordActionGetJPAExecutor (org.apache.oozie.executor.jpa.CoordActionGetJPAExecutor)1 CoordJobGetJPAExecutor (org.apache.oozie.executor.jpa.CoordJobGetJPAExecutor)1
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial