Search in sources :

Example 1 with BasicWorkflowToken

use of io.cdap.cdap.internal.app.runtime.workflow.BasicWorkflowToken in project cdap by caskdata.

the class MetadataSubscriberServiceTest method testWorkflow.

@Test
public void testWorkflow() throws InterruptedException, ExecutionException, TimeoutException {
    ProgramRunId workflowRunId = workflow1.run(RunIds.generate());
    // Try to read, should have nothing
    Store store = getInjector().getInstance(DefaultStore.class);
    WorkflowToken workflowToken = store.getWorkflowToken(workflow1, workflowRunId.getRun());
    Assert.assertNull(workflowToken.get("key"));
    BasicWorkflowToken token = new BasicWorkflowToken(1024);
    token.setCurrentNode("node1");
    token.put("key", "value");
    // Publish some workflow states
    WorkflowStateWriter workflowStateWriter = getInjector().getInstance(MessagingWorkflowStateWriter.class);
    workflowStateWriter.setWorkflowToken(workflowRunId, token);
    workflowStateWriter.addWorkflowNodeState(workflowRunId, new WorkflowNodeStateDetail("action1", NodeStatus.RUNNING));
    // Verify the WorkflowToken
    Tasks.waitFor("value", () -> Optional.ofNullable(store.getWorkflowToken(workflow1, workflowRunId.getRun()).get("key")).map(Value::toString).orElse(null), 10, TimeUnit.SECONDS, 100, TimeUnit.MILLISECONDS);
    // Verify the workflow node state
    Tasks.waitFor(NodeStatus.RUNNING, () -> store.getWorkflowNodeStates(workflowRunId).stream().findFirst().map(WorkflowNodeStateDetail::getNodeStatus).orElse(null), 10, TimeUnit.SECONDS, 100, TimeUnit.MILLISECONDS);
    // Update the node state
    workflowStateWriter.addWorkflowNodeState(workflowRunId, new WorkflowNodeStateDetail("action1", NodeStatus.COMPLETED));
    // Verify the updated node state
    Tasks.waitFor(NodeStatus.COMPLETED, () -> store.getWorkflowNodeStates(workflowRunId).stream().findFirst().map(WorkflowNodeStateDetail::getNodeStatus).orElse(null), 10, TimeUnit.SECONDS, 100, TimeUnit.MILLISECONDS);
}
Also used : Value(io.cdap.cdap.api.workflow.Value) DefaultStore(io.cdap.cdap.internal.app.store.DefaultStore) Store(io.cdap.cdap.app.store.Store) WorkflowToken(io.cdap.cdap.api.workflow.WorkflowToken) BasicWorkflowToken(io.cdap.cdap.internal.app.runtime.workflow.BasicWorkflowToken) MessagingWorkflowStateWriter(io.cdap.cdap.internal.app.runtime.workflow.MessagingWorkflowStateWriter) WorkflowStateWriter(io.cdap.cdap.internal.app.runtime.workflow.WorkflowStateWriter) BasicWorkflowToken(io.cdap.cdap.internal.app.runtime.workflow.BasicWorkflowToken) ProgramRunId(io.cdap.cdap.proto.id.ProgramRunId) WorkflowNodeStateDetail(io.cdap.cdap.proto.WorkflowNodeStateDetail) Test(org.junit.Test)

Example 2 with BasicWorkflowToken

use of io.cdap.cdap.internal.app.runtime.workflow.BasicWorkflowToken in project cdap by caskdata.

the class SparkExecutionServiceTest method testWorkflowToken.

@Test
public void testWorkflowToken() throws Exception {
    ProgramRunId programRunId = new ProgramRunId("ns", "app", ProgramType.SPARK, "test", RunIds.generate().getId());
    // Start a service with empty workflow token
    BasicWorkflowToken token = new BasicWorkflowToken(10);
    token.setCurrentNode("spark");
    SparkExecutionService service = new SparkExecutionService(locationFactory, InetAddress.getLoopbackAddress().getCanonicalHostName(), programRunId, token);
    service.startAndWait();
    try {
        SparkExecutionClient client = new SparkExecutionClient(service.getBaseURI(), programRunId);
        // Update token via heartbeat
        BasicWorkflowToken clientToken = new BasicWorkflowToken(10);
        clientToken.setCurrentNode("spark");
        for (int i = 0; i < 5; i++) {
            clientToken.put("key", "value" + i);
            client.heartbeat(clientToken);
            // The server side token should get updated
            Assert.assertEquals(Value.of("value" + i), token.get("key", "spark"));
        }
        clientToken.put("completed", "true");
        client.completed(clientToken);
    } finally {
        service.stopAndWait();
    }
    // The token on the service side should get updated after the completed call.
    Map<String, Value> values = token.getAllFromNode("spark");
    Map<String, Value> expected = ImmutableMap.of("key", Value.of("value4"), "completed", Value.of("true"));
    Assert.assertEquals(expected, values);
}
Also used : Value(io.cdap.cdap.api.workflow.Value) BasicWorkflowToken(io.cdap.cdap.internal.app.runtime.workflow.BasicWorkflowToken) ProgramRunId(io.cdap.cdap.proto.id.ProgramRunId) Test(org.junit.Test)

Example 3 with BasicWorkflowToken

use of io.cdap.cdap.internal.app.runtime.workflow.BasicWorkflowToken in project cdap by caskdata.

the class TriggeringScheduleInfoAdapterTest method testSerDeserScheduleInfo.

@Test
public void testSerDeserScheduleInfo() {
    BasicWorkflowToken token = new BasicWorkflowToken(1);
    token.setCurrentNode("node");
    token.put("tokenKey", "tokenVal");
    List<TriggerInfo> triggerInfos = ImmutableList.of(new DefaultProgramStatusTriggerInfo("ns", Specifications.from(new WorkflowAppWithFork()).getName(), ProgramType.WORKFLOW, WorkflowAppWithFork.WorkflowWithFork.class.getSimpleName(), RunIds.generate(), ProgramStatus.COMPLETED, token, Collections.emptyMap()), new DefaultPartitionTriggerInfo("ns", "ds", 10, 11), new DefaultTimeTriggerInfo("1 * * * *", 0L));
    TriggeringScheduleInfo scheduleInfo = new DefaultTriggeringScheduleInfo("schedule", "description", triggerInfos, ImmutableMap.of("key", "value"));
    String scheduleInfoJson = GSON.toJson(scheduleInfo);
    TriggeringScheduleInfo deserializedScheduleInfo = GSON.fromJson(scheduleInfoJson, TriggeringScheduleInfo.class);
    Assert.assertEquals(scheduleInfoJson, GSON.toJson(deserializedScheduleInfo));
    DefaultProgramStatusTriggerInfo expectedProgramStatusTriggerInfo = (DefaultProgramStatusTriggerInfo) triggerInfos.get(0);
    DefaultProgramStatusTriggerInfo deserializedProgramStatusTriggerInfo = (DefaultProgramStatusTriggerInfo) deserializedScheduleInfo.getTriggerInfos().get(0);
    Assert.assertEquals(expectedProgramStatusTriggerInfo.getApplicationName(), deserializedProgramStatusTriggerInfo.getApplicationName());
    Assert.assertEquals(expectedProgramStatusTriggerInfo.getWorkflowToken().getAll(), deserializedProgramStatusTriggerInfo.getWorkflowToken().getAll());
}
Also used : DefaultPartitionTriggerInfo(io.cdap.cdap.internal.app.runtime.schedule.trigger.DefaultPartitionTriggerInfo) WorkflowAppWithFork(io.cdap.cdap.WorkflowAppWithFork) DefaultTimeTriggerInfo(io.cdap.cdap.internal.app.runtime.schedule.trigger.DefaultTimeTriggerInfo) TriggerInfo(io.cdap.cdap.api.schedule.TriggerInfo) DefaultProgramStatusTriggerInfo(io.cdap.cdap.internal.app.runtime.schedule.trigger.DefaultProgramStatusTriggerInfo) DefaultPartitionTriggerInfo(io.cdap.cdap.internal.app.runtime.schedule.trigger.DefaultPartitionTriggerInfo) DefaultTimeTriggerInfo(io.cdap.cdap.internal.app.runtime.schedule.trigger.DefaultTimeTriggerInfo) TriggeringScheduleInfo(io.cdap.cdap.api.schedule.TriggeringScheduleInfo) BasicWorkflowToken(io.cdap.cdap.internal.app.runtime.workflow.BasicWorkflowToken) DefaultProgramStatusTriggerInfo(io.cdap.cdap.internal.app.runtime.schedule.trigger.DefaultProgramStatusTriggerInfo) Test(org.junit.Test)

Aggregations

BasicWorkflowToken (io.cdap.cdap.internal.app.runtime.workflow.BasicWorkflowToken)3 Test (org.junit.Test)3 Value (io.cdap.cdap.api.workflow.Value)2 ProgramRunId (io.cdap.cdap.proto.id.ProgramRunId)2 WorkflowAppWithFork (io.cdap.cdap.WorkflowAppWithFork)1 TriggerInfo (io.cdap.cdap.api.schedule.TriggerInfo)1 TriggeringScheduleInfo (io.cdap.cdap.api.schedule.TriggeringScheduleInfo)1 WorkflowToken (io.cdap.cdap.api.workflow.WorkflowToken)1 Store (io.cdap.cdap.app.store.Store)1 DefaultPartitionTriggerInfo (io.cdap.cdap.internal.app.runtime.schedule.trigger.DefaultPartitionTriggerInfo)1 DefaultProgramStatusTriggerInfo (io.cdap.cdap.internal.app.runtime.schedule.trigger.DefaultProgramStatusTriggerInfo)1 DefaultTimeTriggerInfo (io.cdap.cdap.internal.app.runtime.schedule.trigger.DefaultTimeTriggerInfo)1 MessagingWorkflowStateWriter (io.cdap.cdap.internal.app.runtime.workflow.MessagingWorkflowStateWriter)1 WorkflowStateWriter (io.cdap.cdap.internal.app.runtime.workflow.WorkflowStateWriter)1 DefaultStore (io.cdap.cdap.internal.app.store.DefaultStore)1 WorkflowNodeStateDetail (io.cdap.cdap.proto.WorkflowNodeStateDetail)1