Search in sources :

Example 31 with Workflow

use of org.apache.helix.task.Workflow in project helix by apache.

the class AbstractTestClass method createWorkflows.

protected Map<String, Workflow> createWorkflows(String cluster, int numWorkflows) {
    Map<String, Workflow> workflows = new HashMap<>();
    for (int i = 0; i < numWorkflows; i++) {
        Workflow.Builder workflow = new Workflow.Builder(WORKFLOW_PREFIX + i);
        int j = 0;
        for (JobConfig.Builder job : createJobs(cluster, WORKFLOW_PREFIX + i, 3)) {
            workflow.addJob(JOB_PREFIX + j++, job);
        }
        workflows.put(WORKFLOW_PREFIX + i, workflow.build());
        WorkflowContext workflowContext = TaskTestUtil.buildWorkflowContext(WORKFLOW_PREFIX + i, TaskState.IN_PROGRESS, System.currentTimeMillis(), TaskState.COMPLETED, TaskState.COMPLETED, TaskState.IN_PROGRESS);
        _baseAccessor.set(String.format("/%s/%s%s/%s/%s", cluster, PropertyType.PROPERTYSTORE.name(), TaskConstants.REBALANCER_CONTEXT_ROOT, WORKFLOW_PREFIX + i, TaskConstants.CONTEXT_NODE), workflowContext.getRecord(), AccessOption.PERSISTENT);
        _configAccessor.setResourceConfig(cluster, WORKFLOW_PREFIX + i, workflow.getWorkflowConfig());
    }
    return workflows;
}
Also used : HashMap(java.util.HashMap) WorkflowContext(org.apache.helix.task.WorkflowContext) Workflow(org.apache.helix.task.Workflow) JobConfig(org.apache.helix.task.JobConfig)

Example 32 with Workflow

use of org.apache.helix.task.Workflow in project helix by apache.

the class WorkflowsResource method post.

@Override
public Representation post(Representation entity) {
    try {
        String clusterName = (String) getRequest().getAttributes().get("clusterName");
        Form form = new Form(entity);
        // Get the workflow and submit it
        if (form.size() < 1) {
            throw new HelixException("yaml workflow is required!");
        }
        Parameter payload = form.get(0);
        String yamlPayload = payload.getName();
        if (yamlPayload == null) {
            throw new HelixException("yaml workflow is required!");
        }
        String zkAddr = (String) getContext().getAttributes().get(RestAdminApplication.ZKSERVERADDRESS);
        HelixManager manager = HelixManagerFactory.getZKHelixManager(clusterName, null, InstanceType.ADMINISTRATOR, zkAddr);
        manager.connect();
        try {
            Workflow workflow = Workflow.parse(yamlPayload);
            TaskDriver driver = new TaskDriver(manager);
            driver.start(workflow);
        } finally {
            manager.disconnect();
        }
        getResponse().setEntity(getHostedEntitiesRepresentation(clusterName));
        getResponse().setStatus(Status.SUCCESS_OK);
    } catch (Exception e) {
        getResponse().setEntity(ClusterRepresentationUtil.getErrorAsJsonStringFromException(e), MediaType.APPLICATION_JSON);
        getResponse().setStatus(Status.SUCCESS_OK);
        LOG.error("Error in posting " + entity, e);
    }
    return null;
}
Also used : HelixException(org.apache.helix.HelixException) HelixManager(org.apache.helix.HelixManager) Form(org.restlet.data.Form) TaskDriver(org.apache.helix.task.TaskDriver) Parameter(org.restlet.data.Parameter) Workflow(org.apache.helix.task.Workflow) HelixException(org.apache.helix.HelixException) JsonMappingException(org.codehaus.jackson.map.JsonMappingException) IOException(java.io.IOException) JsonGenerationException(org.codehaus.jackson.JsonGenerationException)

Aggregations

Workflow (org.apache.helix.task.Workflow)32 JobConfig (org.apache.helix.task.JobConfig)26 Test (org.testng.annotations.Test)25 JobContext (org.apache.helix.task.JobContext)13 TaskDriver (org.apache.helix.task.TaskDriver)6 WorkflowConfig (org.apache.helix.task.WorkflowConfig)6 WorkflowContext (org.apache.helix.task.WorkflowContext)6 ArrayList (java.util.ArrayList)5 TaskConfig (org.apache.helix.task.TaskConfig)5 HashMap (java.util.HashMap)4 HelixException (org.apache.helix.HelixException)4 TaskPartitionState (org.apache.helix.task.TaskPartitionState)4 IOException (java.io.IOException)3 Map (java.util.Map)3 ZkClient (org.apache.helix.manager.zk.ZkClient)3 Form (org.restlet.data.Form)3 JobQueue (org.apache.helix.task.JobQueue)2 BestPossibleExternalViewVerifier (org.apache.helix.tools.ClusterVerifiers.BestPossibleExternalViewVerifier)2 HelixClusterVerifier (org.apache.helix.tools.ClusterVerifiers.HelixClusterVerifier)2 JsonGenerationException (org.codehaus.jackson.JsonGenerationException)2