Search in sources :

Example 1 with NodeStatus

use of co.cask.cdap.api.workflow.NodeStatus in project cdap by caskdata.

the class DataPipelineTest method testPostAction.

@Test
public void testPostAction() throws Exception {
    ETLBatchConfig etlConfig = ETLBatchConfig.builder("* * * * *").addStage(new ETLStage("source", MockSource.getPlugin("actionInput"))).addStage(new ETLStage("sink", MockSink.getPlugin("actionOutput"))).addPostAction(new ETLStage("tokenWriter", NodeStatesAction.getPlugin("tokenTable"))).addConnection("source", "sink").build();
    AppRequest<ETLBatchConfig> appRequest = new AppRequest<>(APP_ARTIFACT, etlConfig);
    ApplicationId appId = NamespaceId.DEFAULT.app("ActionApp");
    ApplicationManager appManager = deployApplication(appId, appRequest);
    Schema schema = Schema.recordOf("testRecord", Schema.Field.of("name", Schema.of(Schema.Type.STRING)));
    StructuredRecord recordSamuel = StructuredRecord.builder(schema).set("name", "samuel").build();
    StructuredRecord recordBob = StructuredRecord.builder(schema).set("name", "bob").build();
    StructuredRecord recordJane = StructuredRecord.builder(schema).set("name", "jane").build();
    DataSetManager<Table> inputManager = getDataset(NamespaceId.DEFAULT.dataset("actionInput"));
    MockSource.writeInput(inputManager, ImmutableList.of(recordSamuel, recordBob, recordJane));
    WorkflowManager workflowManager = appManager.getWorkflowManager(SmartWorkflow.NAME);
    workflowManager.start();
    workflowManager.waitForRun(ProgramRunStatus.COMPLETED, 5, TimeUnit.MINUTES);
    DataSetManager<Table> tokenTableManager = getDataset(NamespaceId.DEFAULT.dataset("tokenTable"));
    Table tokenTable = tokenTableManager.get();
    NodeStatus status = NodeStatus.valueOf(Bytes.toString(tokenTable.get(Bytes.toBytes("phase-1"), Bytes.toBytes("status"))));
    Assert.assertEquals(NodeStatus.COMPLETED, status);
}
Also used : ETLBatchConfig(co.cask.cdap.etl.proto.v2.ETLBatchConfig) ApplicationManager(co.cask.cdap.test.ApplicationManager) KeyValueTable(co.cask.cdap.api.dataset.lib.KeyValueTable) Table(co.cask.cdap.api.dataset.table.Table) ETLStage(co.cask.cdap.etl.proto.v2.ETLStage) Schema(co.cask.cdap.api.data.schema.Schema) WorkflowManager(co.cask.cdap.test.WorkflowManager) ApplicationId(co.cask.cdap.proto.id.ApplicationId) StructuredRecord(co.cask.cdap.api.data.format.StructuredRecord) NodeStatus(co.cask.cdap.api.workflow.NodeStatus) AppRequest(co.cask.cdap.proto.artifact.AppRequest) Test(org.junit.Test)

Example 2 with NodeStatus

use of co.cask.cdap.api.workflow.NodeStatus in project cdap by caskdata.

the class WorkflowDriver method executeCustomAction.

private void executeCustomAction(final WorkflowActionNode node, InstantiatorFactory instantiator, final ClassLoader classLoader, WorkflowToken token) throws Exception {
    CustomActionExecutor customActionExecutor;
    // Node has CustomActionSpecification, so it must represent the CustomAction added in 3.5.0
    // Create instance of the CustomActionExecutor using CustomActionContext
    WorkflowProgramInfo info = new WorkflowProgramInfo(workflowSpec.getName(), node.getNodeId(), workflowRunId.getRun(), node.getNodeId(), (BasicWorkflowToken) token);
    ProgramOptions actionOptions = new SimpleProgramOptions(programOptions.getProgramId(), programOptions.getArguments(), new BasicArguments(RuntimeArguments.extractScope(ACTION_SCOPE, node.getNodeId(), programOptions.getUserArguments().asMap())));
    BasicCustomActionContext context = new BasicCustomActionContext(program, actionOptions, cConf, node.getCustomActionSpecification(), info, metricsCollectionService, datasetFramework, txClient, discoveryServiceClient, pluginInstantiator, secureStore, secureStoreManager, messagingService);
    customActionExecutor = new CustomActionExecutor(context, instantiator, classLoader);
    status.put(node.getNodeId(), node);
    runtimeStore.addWorkflowNodeState(workflowRunId, new WorkflowNodeStateDetail(node.getNodeId(), NodeStatus.RUNNING));
    Throwable failureCause = null;
    try {
        customActionExecutor.execute();
    } catch (Throwable t) {
        failureCause = t;
        throw t;
    } finally {
        status.remove(node.getNodeId());
        runtimeStore.updateWorkflowToken(workflowRunId, token);
        NodeStatus status = failureCause == null ? NodeStatus.COMPLETED : NodeStatus.FAILED;
        nodeStates.put(node.getNodeId(), new WorkflowNodeState(node.getNodeId(), status, null, failureCause));
        BasicThrowable defaultThrowable = failureCause == null ? null : new BasicThrowable(failureCause);
        runtimeStore.addWorkflowNodeState(workflowRunId, new WorkflowNodeStateDetail(node.getNodeId(), status, null, defaultThrowable));
    }
}
Also used : WorkflowNodeState(co.cask.cdap.api.workflow.WorkflowNodeState) BasicCustomActionContext(co.cask.cdap.internal.app.runtime.customaction.BasicCustomActionContext) BasicThrowable(co.cask.cdap.proto.BasicThrowable) SimpleProgramOptions(co.cask.cdap.internal.app.runtime.SimpleProgramOptions) BasicArguments(co.cask.cdap.internal.app.runtime.BasicArguments) BasicThrowable(co.cask.cdap.proto.BasicThrowable) SimpleProgramOptions(co.cask.cdap.internal.app.runtime.SimpleProgramOptions) ProgramOptions(co.cask.cdap.app.runtime.ProgramOptions) NodeStatus(co.cask.cdap.api.workflow.NodeStatus) WorkflowNodeStateDetail(co.cask.cdap.proto.WorkflowNodeStateDetail)

Aggregations

NodeStatus (co.cask.cdap.api.workflow.NodeStatus)2 StructuredRecord (co.cask.cdap.api.data.format.StructuredRecord)1 Schema (co.cask.cdap.api.data.schema.Schema)1 KeyValueTable (co.cask.cdap.api.dataset.lib.KeyValueTable)1 Table (co.cask.cdap.api.dataset.table.Table)1 WorkflowNodeState (co.cask.cdap.api.workflow.WorkflowNodeState)1 ProgramOptions (co.cask.cdap.app.runtime.ProgramOptions)1 ETLBatchConfig (co.cask.cdap.etl.proto.v2.ETLBatchConfig)1 ETLStage (co.cask.cdap.etl.proto.v2.ETLStage)1 BasicArguments (co.cask.cdap.internal.app.runtime.BasicArguments)1 SimpleProgramOptions (co.cask.cdap.internal.app.runtime.SimpleProgramOptions)1 BasicCustomActionContext (co.cask.cdap.internal.app.runtime.customaction.BasicCustomActionContext)1 BasicThrowable (co.cask.cdap.proto.BasicThrowable)1 WorkflowNodeStateDetail (co.cask.cdap.proto.WorkflowNodeStateDetail)1 AppRequest (co.cask.cdap.proto.artifact.AppRequest)1 ApplicationId (co.cask.cdap.proto.id.ApplicationId)1 ApplicationManager (co.cask.cdap.test.ApplicationManager)1 WorkflowManager (co.cask.cdap.test.WorkflowManager)1 Test (org.junit.Test)1