Examples with ScheduleProgramInfo co.cask.cdap.api.workflow.ScheduleProgramInfo used on opensource projects

Search in sources :

Example 11 with ScheduleProgramInfo

use of co.cask.cdap.api.workflow.ScheduleProgramInfo in project cdap by caskdata.

the class ApplicationVerificationStage method verifyWorkflowAction.

private void verifyWorkflowAction(ApplicationSpecification appSpec, WorkflowNode node) {
    WorkflowActionNode actionNode = (WorkflowActionNode) node;
    ScheduleProgramInfo program = actionNode.getProgram();
    switch(program.getProgramType()) {
        case MAPREDUCE:
            Preconditions.checkArgument(appSpec.getMapReduce().containsKey(program.getProgramName()), String.format("MapReduce program '%s' is not configured with the Application.", program.getProgramName()));
            break;
        case SPARK:
            Preconditions.checkArgument(appSpec.getSpark().containsKey(program.getProgramName()), String.format("Spark program '%s' is not configured with the Application.", program.getProgramName()));
            break;
        case CUSTOM_ACTION:
            // no-op
            break;
        default:
            throw new RuntimeException(String.format("Unknown Program '%s' in the Workflow.", program.getProgramName()));
    }
}
Also used : WorkflowActionNode(co.cask.cdap.api.workflow.WorkflowActionNode) ScheduleProgramInfo(co.cask.cdap.api.workflow.ScheduleProgramInfo)

Example 12 with ScheduleProgramInfo

use of co.cask.cdap.api.workflow.ScheduleProgramInfo in project cdap by caskdata.

the class WorkflowVerificationTest method verifyGoodWorkflowSpecifications.

private void verifyGoodWorkflowSpecifications(ApplicationSpecification appSpec) {
    WorkflowSpecification spec = appSpec.getWorkflows().get("GoodWorkflow");
    Assert.assertTrue(spec.getNodes().size() == 4);
    List<WorkflowNode> nodes = spec.getNodes();
    WorkflowNode node = nodes.get(0);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    WorkflowActionNode actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.CUSTOM_ACTION, "DummyAction")));
    Assert.assertNotNull(actionNode.getCustomActionSpecification());
    node = nodes.get(1);
    Assert.assertTrue(node.getType() == WorkflowNodeType.FORK);
    WorkflowForkNode fork1 = (WorkflowForkNode) node;
    Assert.assertTrue(fork1.getBranches().size() == 2);
    List<WorkflowNode> fork1Branch1 = fork1.getBranches().get(0);
    Assert.assertTrue(fork1Branch1.size() == 2);
    List<WorkflowNode> fork1Branch2 = fork1.getBranches().get(1);
    Assert.assertTrue(fork1Branch2.size() == 1);
    actionNode = (WorkflowActionNode) fork1Branch1.get(0);
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "DummyMR")));
    Assert.assertNull(actionNode.getActionSpecification());
    WorkflowForkNode fork2 = (WorkflowForkNode) fork1Branch1.get(1);
    List<WorkflowNode> fork2Branch1 = fork2.getBranches().get(0);
    Assert.assertTrue(fork2Branch1.size() == 2);
    List<WorkflowNode> fork2Branch2 = fork2.getBranches().get(1);
    Assert.assertTrue(fork2Branch2.size() == 1);
    actionNode = (WorkflowActionNode) fork2Branch1.get(0);
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.CUSTOM_ACTION, "DummyAction")));
    Assert.assertNotNull(actionNode.getCustomActionSpecification());
    WorkflowForkNode fork3 = (WorkflowForkNode) fork2Branch1.get(1);
    List<WorkflowNode> fork3Branch1 = fork3.getBranches().get(0);
    Assert.assertTrue(fork3Branch1.size() == 2);
    List<WorkflowNode> fork3Branch2 = fork3.getBranches().get(1);
    Assert.assertTrue(fork3Branch2.size() == 1);
    WorkflowForkNode fork4 = (WorkflowForkNode) fork3Branch1.get(0);
    List<WorkflowNode> fork4Branch1 = fork4.getBranches().get(0);
    Assert.assertTrue(fork4Branch1.size() == 2);
    List<WorkflowNode> fork4Branch2 = fork4.getBranches().get(1);
    Assert.assertTrue(fork4Branch2.size() == 1);
    actionNode = (WorkflowActionNode) fork4Branch1.get(0);
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "DummyMR")));
    Assert.assertNull(actionNode.getActionSpecification());
    actionNode = (WorkflowActionNode) fork4Branch1.get(1);
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.CUSTOM_ACTION, "DummyAction")));
    Assert.assertNotNull(actionNode.getCustomActionSpecification());
    actionNode = (WorkflowActionNode) fork4Branch2.get(0);
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "DummyMR")));
    Assert.assertNull(actionNode.getCustomActionSpecification());
    actionNode = (WorkflowActionNode) fork4Branch1.get(0);
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "DummyMR")));
    Assert.assertNull(actionNode.getCustomActionSpecification());
    actionNode = (WorkflowActionNode) fork4Branch2.get(0);
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "DummyMR")));
    Assert.assertNull(actionNode.getCustomActionSpecification());
    actionNode = (WorkflowActionNode) fork2Branch2.get(0);
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.CUSTOM_ACTION, "DummyAction")));
    Assert.assertNotNull(actionNode.getCustomActionSpecification());
    actionNode = (WorkflowActionNode) fork1Branch2.get(0);
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.CUSTOM_ACTION, "DummyAction")));
    Assert.assertNotNull(actionNode.getCustomActionSpecification());
    node = nodes.get(2);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "DummyMR")));
    Assert.assertNull(actionNode.getCustomActionSpecification());
    node = nodes.get(3);
    WorkflowForkNode fork5 = (WorkflowForkNode) node;
    List<WorkflowNode> fork5Branch1 = fork5.getBranches().get(0);
    Assert.assertTrue(fork5Branch1.size() == 1);
    List<WorkflowNode> fork5Branch2 = fork5.getBranches().get(1);
    Assert.assertTrue(fork5Branch2.size() == 1);
    actionNode = (WorkflowActionNode) fork5Branch1.get(0);
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.CUSTOM_ACTION, "DummyAction")));
    Assert.assertNotNull(actionNode.getCustomActionSpecification());
    actionNode = (WorkflowActionNode) fork5Branch2.get(0);
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "DummyMR")));
    Assert.assertNull(actionNode.getCustomActionSpecification());
}
Also used : WorkflowActionNode(co.cask.cdap.api.workflow.WorkflowActionNode) WorkflowSpecification(co.cask.cdap.api.workflow.WorkflowSpecification) ScheduleProgramInfo(co.cask.cdap.api.workflow.ScheduleProgramInfo) WorkflowForkNode(co.cask.cdap.api.workflow.WorkflowForkNode) WorkflowNode(co.cask.cdap.api.workflow.WorkflowNode)

Example 13 with ScheduleProgramInfo

use of co.cask.cdap.api.workflow.ScheduleProgramInfo in project cdap by caskdata.

the class WorkflowVerificationTest method verifyAnotherGoodWorkflowSpecification.

private void verifyAnotherGoodWorkflowSpecification(ApplicationSpecification appSpec) {
    WorkflowSpecification spec = appSpec.getWorkflows().get("AnotherGoodWorkflow");
    List<WorkflowNode> nodes = spec.getNodes();
    Assert.assertTrue(nodes.size() == 4);
    WorkflowNode node = nodes.get(0);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    WorkflowActionNode actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "MR1")));
    node = nodes.get(1);
    Assert.assertTrue(node.getType() == WorkflowNodeType.FORK);
    WorkflowForkNode fork = (WorkflowForkNode) node;
    Assert.assertTrue(fork.getBranches().size() == 2);
    List<WorkflowNode> forkBranch1 = fork.getBranches().get(0);
    Assert.assertTrue(forkBranch1.size() == 3);
    node = forkBranch1.get(0);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "MR2")));
    node = forkBranch1.get(1);
    Assert.assertTrue(node.getType() == WorkflowNodeType.CONDITION);
    WorkflowConditionNode condition = (WorkflowConditionNode) node;
    Assert.assertTrue(condition.getPredicateClassName().contains("MyVerificationPredicate"));
    List<WorkflowNode> ifNodes = condition.getIfBranch();
    Assert.assertTrue(ifNodes.size() == 2);
    List<WorkflowNode> elseNodes = condition.getElseBranch();
    Assert.assertTrue(elseNodes.size() == 2);
    node = ifNodes.get(0);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "MR3")));
    node = ifNodes.get(1);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "MR4")));
    node = elseNodes.get(0);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "MR5")));
    node = elseNodes.get(1);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "MR6")));
    node = forkBranch1.get(2);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "MR7")));
    List<WorkflowNode> forkBranch2 = fork.getBranches().get(1);
    Assert.assertTrue(forkBranch2.size() == 1);
    node = forkBranch2.get(0);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.MAPREDUCE, "MR8")));
    node = nodes.get(2);
    Assert.assertTrue(node.getType() == WorkflowNodeType.CONDITION);
    ifNodes = ((WorkflowConditionNode) node).getIfBranch();
    elseNodes = ((WorkflowConditionNode) node).getElseBranch();
    Assert.assertTrue(ifNodes.size() == 2);
    node = ifNodes.get(0);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.SPARK, "SP1")));
    node = ifNodes.get(1);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.SPARK, "SP2")));
    Assert.assertTrue(elseNodes.size() == 3);
    node = elseNodes.get(0);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.SPARK, "SP3")));
    node = elseNodes.get(1);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.SPARK, "SP4")));
    node = elseNodes.get(2);
    Assert.assertTrue(node.getType() == WorkflowNodeType.FORK);
    WorkflowForkNode anotherFork = (WorkflowForkNode) node;
    Assert.assertTrue(anotherFork.getBranches().size() == 2);
    List<WorkflowNode> anotherForkBranch1 = anotherFork.getBranches().get(0);
    Assert.assertTrue(anotherForkBranch1.size() == 1);
    List<WorkflowNode> anotherForkBranch2 = anotherFork.getBranches().get(1);
    Assert.assertTrue(anotherForkBranch2.size() == 1);
    node = anotherForkBranch1.get(0);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.SPARK, "SP5")));
    node = anotherForkBranch2.get(0);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.SPARK, "SP6")));
    node = nodes.get(3);
    Assert.assertTrue(node.getType() == WorkflowNodeType.ACTION);
    actionNode = (WorkflowActionNode) node;
    Assert.assertTrue(actionNode.getProgram().equals(new ScheduleProgramInfo(SchedulableProgramType.SPARK, "SP7")));
}
Also used : WorkflowActionNode(co.cask.cdap.api.workflow.WorkflowActionNode) WorkflowSpecification(co.cask.cdap.api.workflow.WorkflowSpecification) ScheduleProgramInfo(co.cask.cdap.api.workflow.ScheduleProgramInfo) WorkflowForkNode(co.cask.cdap.api.workflow.WorkflowForkNode) WorkflowNode(co.cask.cdap.api.workflow.WorkflowNode) WorkflowConditionNode(co.cask.cdap.api.workflow.WorkflowConditionNode)

Example 14 with ScheduleProgramInfo

use of co.cask.cdap.api.workflow.ScheduleProgramInfo in project cdap by caskdata.

the class ScheduleSpecificationCodec method deserialize.

@Override
public ScheduleSpecification deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
    JsonObject jsonObj = json.getAsJsonObject();
    JsonElement scheduleTypeJson = jsonObj.get("scheduleType");
    ScheduleType scheduleType;
    if (scheduleTypeJson == null) {
        // For backwards compatibility with spec persisted with older versions than 2.8, we need these lines
        scheduleType = null;
    } else {
        scheduleType = context.deserialize(jsonObj.get("scheduleType"), ScheduleType.class);
    }
    Schedule schedule = null;
    if (scheduleType == null) {
        JsonObject scheduleObj = jsonObj.get("schedule").getAsJsonObject();
        String name = context.deserialize(scheduleObj.get("name"), String.class);
        String description = context.deserialize(scheduleObj.get("description"), String.class);
        String cronEntry = context.deserialize(scheduleObj.get("cronEntry"), String.class);
        schedule = Schedules.builder(name).setDescription(description).createTimeSchedule(cronEntry);
    } else {
        switch(scheduleType) {
            case TIME:
                schedule = context.deserialize(jsonObj.get("schedule"), TimeSchedule.class);
                break;
            case STREAM:
                schedule = context.deserialize(jsonObj.get("schedule"), StreamSizeSchedule.class);
                break;
        }
    }
    ScheduleProgramInfo program = context.deserialize(jsonObj.get("program"), ScheduleProgramInfo.class);
    Map<String, String> properties = deserializeMap(jsonObj.get("properties"), context, String.class);
    return new ScheduleSpecification(schedule, program, properties);
}
Also used : ScheduleType(co.cask.cdap.proto.ScheduleType) JsonElement(com.google.gson.JsonElement) Schedule(co.cask.cdap.api.schedule.Schedule) StreamSizeSchedule(co.cask.cdap.internal.schedule.StreamSizeSchedule) TimeSchedule(co.cask.cdap.internal.schedule.TimeSchedule) JsonObject(com.google.gson.JsonObject) TimeSchedule(co.cask.cdap.internal.schedule.TimeSchedule) StreamSizeSchedule(co.cask.cdap.internal.schedule.StreamSizeSchedule) ScheduleProgramInfo(co.cask.cdap.api.workflow.ScheduleProgramInfo) ScheduleSpecification(co.cask.cdap.api.schedule.ScheduleSpecification)

Aggregations

ScheduleProgramInfo (co.cask.cdap.api.workflow.ScheduleProgramInfo)14 WorkflowActionNode (co.cask.cdap.api.workflow.WorkflowActionNode)7 ScheduleSpecification (co.cask.cdap.api.schedule.ScheduleSpecification)6 TimeSchedule (co.cask.cdap.internal.schedule.TimeSchedule)5 WorkflowNode (co.cask.cdap.api.workflow.WorkflowNode)4 WorkflowSpecification (co.cask.cdap.api.workflow.WorkflowSpecification)4 StreamSizeSchedule (co.cask.cdap.internal.schedule.StreamSizeSchedule)4 Test (org.junit.Test)4 Schedule (co.cask.cdap.api.schedule.Schedule)3 Resources (co.cask.cdap.api.Resources)2 SchedulableProgramType (co.cask.cdap.api.schedule.SchedulableProgramType)2 WorkflowForkNode (co.cask.cdap.api.workflow.WorkflowForkNode)2 FileSet (co.cask.cdap.api.dataset.lib.FileSet)1 Table (co.cask.cdap.api.dataset.table.Table)1 WorkflowConditionNode (co.cask.cdap.api.workflow.WorkflowConditionNode)1 ProgramRunner (co.cask.cdap.app.runtime.ProgramRunner)1 TimeTrigger (co.cask.cdap.internal.app.runtime.schedule.trigger.TimeTrigger)1 DatasetCreationSpec (co.cask.cdap.internal.dataset.DatasetCreationSpec)1 ScheduleCreationSpec (co.cask.cdap.internal.schedule.ScheduleCreationSpec)1 ProgramType (co.cask.cdap.proto.ProgramType)1
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial