Search in sources :

Example 1 with WorkflowToken

use of co.cask.cdap.api.workflow.WorkflowToken in project cdap by caskdata.

the class WorkflowHttpHandler method getWorkflowToken.

@GET
@Path("/apps/{app-id}/workflows/{workflow-id}/runs/{run-id}/token")
public void getWorkflowToken(HttpRequest request, HttpResponder responder, @PathParam("namespace-id") String namespaceId, @PathParam("app-id") String appId, @PathParam("workflow-id") String workflowId, @PathParam("run-id") String runId, @QueryParam("scope") @DefaultValue("user") String scope, @QueryParam("key") @DefaultValue("") String key) throws NotFoundException {
    WorkflowToken workflowToken = getWorkflowToken(namespaceId, appId, workflowId, runId);
    WorkflowToken.Scope tokenScope = WorkflowToken.Scope.valueOf(scope.toUpperCase());
    WorkflowTokenDetail workflowTokenDetail = WorkflowTokenDetail.of(workflowToken.getAll(tokenScope));
    Type workflowTokenDetailType = new TypeToken<WorkflowTokenDetail>() {
    }.getType();
    if (key.isEmpty()) {
        responder.sendJson(HttpResponseStatus.OK, workflowTokenDetail, workflowTokenDetailType, GSON);
        return;
    }
    List<NodeValue> nodeValueEntries = workflowToken.getAll(key, tokenScope);
    if (nodeValueEntries.isEmpty()) {
        throw new NotFoundException(key);
    }
    responder.sendJson(HttpResponseStatus.OK, WorkflowTokenDetail.of(ImmutableMap.of(key, nodeValueEntries)), workflowTokenDetailType, GSON);
}
Also used : NodeValue(co.cask.cdap.api.workflow.NodeValue) ProgramType(co.cask.cdap.proto.ProgramType) Type(java.lang.reflect.Type) SchedulableProgramType(co.cask.cdap.api.schedule.SchedulableProgramType) WorkflowToken(co.cask.cdap.api.workflow.WorkflowToken) ProgramNotFoundException(co.cask.cdap.common.ProgramNotFoundException) ApplicationNotFoundException(co.cask.cdap.common.ApplicationNotFoundException) InstanceNotFoundException(co.cask.cdap.api.dataset.InstanceNotFoundException) NotFoundException(co.cask.cdap.common.NotFoundException) WorkflowTokenDetail(co.cask.cdap.proto.WorkflowTokenDetail) Path(javax.ws.rs.Path) GET(javax.ws.rs.GET)

Example 2 with WorkflowToken

use of co.cask.cdap.api.workflow.WorkflowToken in project cdap by caskdata.

the class RemoteRuntimeStoreHandler method updateWorkflowToken.

@POST
@Path("/updateWorkflowToken")
public void updateWorkflowToken(HttpRequest request, HttpResponder responder) throws Exception {
    Iterator<MethodArgument> arguments = parseArguments(request);
    ProgramRunId workflowRunId = deserializeNext(arguments);
    WorkflowToken token = deserializeNext(arguments);
    store.updateWorkflowToken(workflowRunId, token);
    responder.sendStatus(HttpResponseStatus.OK);
}
Also used : MethodArgument(co.cask.cdap.common.internal.remote.MethodArgument) WorkflowToken(co.cask.cdap.api.workflow.WorkflowToken) ProgramRunId(co.cask.cdap.proto.id.ProgramRunId) Path(javax.ws.rs.Path) POST(javax.ws.rs.POST)

Example 3 with WorkflowToken

use of co.cask.cdap.api.workflow.WorkflowToken in project cdap by caskdata.

the class PipelineAction method run.

@Override
public void run() throws Exception {
    CustomActionContext context = getContext();
    Map<String, String> properties = context.getSpecification().getProperties();
    BatchPhaseSpec phaseSpec = GSON.fromJson(properties.get(Constants.PIPELINEID), BatchPhaseSpec.class);
    PipelinePhase phase = phaseSpec.getPhase();
    StageInfo stageInfo = phase.iterator().next();
    PluginContext pluginContext = new PipelinePluginContext(context, metrics, phaseSpec.isStageLoggingEnabled(), phaseSpec.isProcessTimingEnabled());
    Action action = pluginContext.newPluginInstance(stageInfo.getName(), new DefaultMacroEvaluator(context.getWorkflowToken(), context.getRuntimeArguments(), context.getLogicalStartTime(), context, context.getNamespace()));
    BasicArguments arguments = new BasicArguments(context);
    ActionContext actionContext = new BasicActionContext(context, metrics, stageInfo.getName(), arguments);
    if (!context.getDataTracer(stageInfo.getName()).isEnabled()) {
        action.run(actionContext);
    }
    WorkflowToken token = context.getWorkflowToken();
    if (token == null) {
        throw new IllegalStateException("WorkflowToken cannot be null when action is executed through Workflow.");
    }
    for (Map.Entry<String, String> entry : arguments.getAddedArguments().entrySet()) {
        token.put(entry.getKey(), entry.getValue());
    }
}
Also used : Action(co.cask.cdap.etl.api.action.Action) CustomAction(co.cask.cdap.api.customaction.CustomAction) AbstractCustomAction(co.cask.cdap.api.customaction.AbstractCustomAction) PipelinePluginContext(co.cask.cdap.etl.common.plugin.PipelinePluginContext) PluginContext(co.cask.cdap.api.plugin.PluginContext) StageInfo(co.cask.cdap.etl.planner.StageInfo) WorkflowToken(co.cask.cdap.api.workflow.WorkflowToken) CustomActionContext(co.cask.cdap.api.customaction.CustomActionContext) ActionContext(co.cask.cdap.etl.api.action.ActionContext) PipelinePhase(co.cask.cdap.etl.common.PipelinePhase) DefaultMacroEvaluator(co.cask.cdap.etl.common.DefaultMacroEvaluator) CustomActionContext(co.cask.cdap.api.customaction.CustomActionContext) BatchPhaseSpec(co.cask.cdap.etl.batch.BatchPhaseSpec) BasicArguments(co.cask.cdap.etl.common.BasicArguments) HashMap(java.util.HashMap) Map(java.util.Map) PipelinePluginContext(co.cask.cdap.etl.common.plugin.PipelinePluginContext)

Example 4 with WorkflowToken

use of co.cask.cdap.api.workflow.WorkflowToken in project cdap by caskdata.

the class WikipediaDataDownloader method destroy.

@Override
public void destroy() {
    WorkflowToken workflowToken = getContext().getWorkflowToken();
    if (workflowToken != null) {
        boolean isSuccessful = getContext().getState().getStatus() == ProgramStatus.COMPLETED;
        workflowToken.put("result", Value.of(isSuccessful));
    }
}
Also used : WorkflowToken(co.cask.cdap.api.workflow.WorkflowToken)

Example 5 with WorkflowToken

use of co.cask.cdap.api.workflow.WorkflowToken in project cdap by caskdata.

the class StreamToDataset method initialize.

@Override
public void initialize() throws Exception {
    MapReduceContext context = getContext();
    Job job = context.getHadoopJob();
    job.setNumReduceTasks(0);
    WorkflowToken workflowToken = context.getWorkflowToken();
    Class<? extends Mapper> mapper = PageTitleToDatasetMapper.class;
    String inputStream = WikipediaPipelineApp.PAGE_TITLES_STREAM;
    String outputDataset = WikipediaPipelineApp.PAGE_TITLES_DATASET;
    if (workflowToken != null) {
        Value likesToDatasetResult = workflowToken.get("result", WikipediaPipelineApp.LIKES_TO_DATASET_MR_NAME);
        if (likesToDatasetResult != null && likesToDatasetResult.getAsBoolean()) {
            // The "likes" stream to the dataset has already run and has been successful in this run so far.
            // Now run raw wikipedia stream to dataset.
            mapper = RawWikiDataToDatasetMapper.class;
            inputStream = WikipediaPipelineApp.RAW_WIKIPEDIA_STREAM;
            outputDataset = WikipediaPipelineApp.RAW_WIKIPEDIA_DATASET;
        }
    }
    LOG.info("Using '{}' as the input stream and '{}' as the output dataset.", inputStream, outputDataset);
    job.setMapperClass(mapper);
    String dataNamespace = context.getRuntimeArguments().get(WikipediaPipelineApp.NAMESPACE_ARG);
    dataNamespace = dataNamespace == null ? getContext().getNamespace() : dataNamespace;
    context.addInput(Input.ofStream(inputStream).fromNamespace(dataNamespace));
    context.addOutput(Output.ofDataset(outputDataset).fromNamespace(dataNamespace));
}
Also used : MapReduceContext(co.cask.cdap.api.mapreduce.MapReduceContext) Value(co.cask.cdap.api.workflow.Value) WorkflowToken(co.cask.cdap.api.workflow.WorkflowToken) Job(org.apache.hadoop.mapreduce.Job)

Aggregations

WorkflowToken (co.cask.cdap.api.workflow.WorkflowToken)10 MapReduceContext (co.cask.cdap.api.mapreduce.MapReduceContext)2 Map (java.util.Map)2 Path (javax.ws.rs.Path)2 Job (org.apache.hadoop.mapreduce.Job)2 AbstractCustomAction (co.cask.cdap.api.customaction.AbstractCustomAction)1 CustomAction (co.cask.cdap.api.customaction.CustomAction)1 CustomActionContext (co.cask.cdap.api.customaction.CustomActionContext)1 DatasetManagementException (co.cask.cdap.api.dataset.DatasetManagementException)1 InstanceNotFoundException (co.cask.cdap.api.dataset.InstanceNotFoundException)1 PluginContext (co.cask.cdap.api.plugin.PluginContext)1 SchedulableProgramType (co.cask.cdap.api.schedule.SchedulableProgramType)1 NodeValue (co.cask.cdap.api.workflow.NodeValue)1 Value (co.cask.cdap.api.workflow.Value)1 WorkflowNode (co.cask.cdap.api.workflow.WorkflowNode)1 ApplicationNotFoundException (co.cask.cdap.common.ApplicationNotFoundException)1 NotFoundException (co.cask.cdap.common.NotFoundException)1 ProgramNotFoundException (co.cask.cdap.common.ProgramNotFoundException)1 MethodArgument (co.cask.cdap.common.internal.remote.MethodArgument)1 Action (co.cask.cdap.etl.api.action.Action)1