Search in sources :

Example 11 with ActivityClusterGraph

use of org.apache.hyracks.api.job.ActivityClusterGraph in project asterixdb by apache.

the class JobManager method createJobLogObject.

private ObjectNode createJobLogObject(final JobRun run) {
    ObjectMapper om = new ObjectMapper();
    ObjectNode jobLogObject = om.createObjectNode();
    ActivityClusterGraph acg = run.getActivityClusterGraph();
    jobLogObject.set("activity-cluster-graph", acg.toJSON());
    jobLogObject.set("job-run", run.toJSON());
    return jobLogObject;
}
Also used : ObjectNode(com.fasterxml.jackson.databind.node.ObjectNode) ActivityClusterGraph(org.apache.hyracks.api.job.ActivityClusterGraph) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper)

Example 12 with ActivityClusterGraph

use of org.apache.hyracks.api.job.ActivityClusterGraph in project asterixdb by apache.

the class DistributeJobWork method doRun.

@Override
protected void doRun() throws Exception {
    try {
        final CCServiceContext ccServiceCtx = ccs.getContext();
        ccs.getPreDistributedJobStore().checkForExistingDistributedJobDescriptor(jobId);
        IActivityClusterGraphGeneratorFactory acggf = (IActivityClusterGraphGeneratorFactory) DeploymentUtils.deserialize(acggfBytes, null, ccServiceCtx);
        IActivityClusterGraphGenerator acgg = acggf.createActivityClusterGraphGenerator(jobId, ccServiceCtx, EnumSet.noneOf(JobFlag.class));
        ActivityClusterGraph acg = acgg.initialize();
        ccs.getPreDistributedJobStore().addDistributedJobDescriptor(jobId, acg, acggf.getJobSpecification(), acgg.getConstraints());
        ccServiceCtx.notifyJobCreation(jobId, acggf.getJobSpecification());
        byte[] acgBytes = JavaSerializationUtils.serialize(acg);
        INodeManager nodeManager = ccs.getNodeManager();
        for (NodeControllerState node : nodeManager.getAllNodeControllerStates()) {
            node.getNodeController().distributeJob(jobId, acgBytes);
        }
        callback.setValue(jobId);
    } catch (Exception e) {
        callback.setException(e);
    }
}
Also used : INodeManager(org.apache.hyracks.control.cc.cluster.INodeManager) JobFlag(org.apache.hyracks.api.job.JobFlag) ActivityClusterGraph(org.apache.hyracks.api.job.ActivityClusterGraph) IActivityClusterGraphGeneratorFactory(org.apache.hyracks.api.job.IActivityClusterGraphGeneratorFactory) IActivityClusterGraphGenerator(org.apache.hyracks.api.job.IActivityClusterGraphGenerator) NodeControllerState(org.apache.hyracks.control.cc.NodeControllerState) CCServiceContext(org.apache.hyracks.control.cc.application.CCServiceContext)

Aggregations

ActivityClusterGraph (org.apache.hyracks.api.job.ActivityClusterGraph)12 IConnectorDescriptor (org.apache.hyracks.api.dataflow.IConnectorDescriptor)7 HashMap (java.util.HashMap)6 ActivityId (org.apache.hyracks.api.dataflow.ActivityId)6 ArrayList (java.util.ArrayList)5 List (java.util.List)5 ActivityCluster (org.apache.hyracks.api.job.ActivityCluster)5 ConnectorDescriptorId (org.apache.hyracks.api.dataflow.ConnectorDescriptorId)4 IActivity (org.apache.hyracks.api.dataflow.IActivity)4 HyracksException (org.apache.hyracks.api.exceptions.HyracksException)4 Pair (org.apache.commons.lang3.tuple.Pair)3 TaskId (org.apache.hyracks.api.dataflow.TaskId)3 IConnectorPolicy (org.apache.hyracks.api.dataflow.connectors.IConnectorPolicy)3 INodeManager (org.apache.hyracks.control.cc.cluster.INodeManager)3 TaskAttemptDescriptor (org.apache.hyracks.control.common.job.TaskAttemptDescriptor)3 HashSet (java.util.HashSet)2 Map (java.util.Map)2 Constraint (org.apache.hyracks.api.constraints.Constraint)2 TaskAttemptId (org.apache.hyracks.api.dataflow.TaskAttemptId)2 IRecordDescriptorProvider (org.apache.hyracks.api.dataflow.value.IRecordDescriptorProvider)2