Examples with ServiceNode org.apache.gobblin.runtime.api.ServiceNode used on opensource projects

Example 6 with ServiceNode

use of org.apache.gobblin.runtime.api.ServiceNode in project incubator-gobblin by apache.

the class MultiHopsFlowToJobSpecCompiler method weightGraphGenerateHelper.

// Helper function for transform TopologySpecMap into a weightedDirectedGraph.
private void weightGraphGenerateHelper(TopologySpec topologySpec) {
    try {
        Map<ServiceNode, ServiceNode> capabilities = topologySpec.getSpecExecutor().getCapabilities().get();
        for (Map.Entry<ServiceNode, ServiceNode> capability : capabilities.entrySet()) {
            BaseServiceNodeImpl sourceNode = new BaseServiceNodeImpl(capability.getKey().getNodeName());
            BaseServiceNodeImpl targetNode = new BaseServiceNodeImpl(capability.getValue().getNodeName());
            if (!weightedGraph.containsVertex(sourceNode)) {
                weightedGraph.addVertex(sourceNode);
            }
            if (!weightedGraph.containsVertex(targetNode)) {
                weightedGraph.addVertex(targetNode);
            }
            FlowEdge flowEdge = new LoadBasedFlowEdgeImpl(sourceNode, targetNode, defaultFlowEdgeProps, topologySpec.getSpecExecutor());
            // In Multi-Graph if flowEdge existed, just skip it.
            if (!weightedGraph.containsEdge(flowEdge)) {
                weightedGraph.addEdge(sourceNode, targetNode, flowEdge);
            }
        }
    } catch (InterruptedException | ExecutionException e) {
        Instrumented.markMeter(this.flowCompilationFailedMeter);
        throw new RuntimeException("Cannot determine topology capabilities", e);
    }
}

Example 7 with ServiceNode

use of org.apache.gobblin.runtime.api.ServiceNode in project incubator-gobblin by apache.

the class MultiHopsFlowToJobSpecCompiler method jobSpecURIGenerator.

/**
 * A naive implementation of generating a jobSpec's URI within a multi-hop logical Flow.
 */
@Override
public URI jobSpecURIGenerator(Object... objects) {
    FlowSpec flowSpec = (FlowSpec) objects[0];
    ServiceNode sourceNode = (ServiceNode) objects[1];
    ServiceNode targetNode = (ServiceNode) objects[2];
    try {
        return new URI(JobSpec.Builder.DEFAULT_JOB_CATALOG_SCHEME, flowSpec.getUri().getAuthority(), StringUtils.appendIfMissing(StringUtils.prependIfMissing(flowSpec.getUri().getPath(), "/"), "/") + sourceNode.getNodeName() + "-" + targetNode.getNodeName(), null);
    } catch (URISyntaxException e) {
        log.error("URI construction failed when jobSpec from " + sourceNode.getNodeName() + " to " + targetNode.getNodeName());
        throw new RuntimeException();
    }
}

Example 8 with ServiceNode

use of org.apache.gobblin.runtime.api.ServiceNode in project incubator-gobblin by apache.

the class StaticServicePolicy method populateBlackListedEdges.

@Override
public void populateBlackListedEdges(DirectedWeightedMultigraph<ServiceNode, FlowEdge> graph) {
    for (ServiceNode node : serviceNodes) {
        if (graph.containsVertex(node)) {
            blacklistedEdges.addAll(graph.incomingEdgesOf(node));
            blacklistedEdges.addAll(graph.outgoingEdgesOf(node));
        } else {
            log.info("The graph " + graph + " doesn't contains node " + node.toString());
        }
    }
    for (FlowEdge flowEdge : flowEdges) {
        if (graph.containsEdge(flowEdge)) {
            blacklistedEdges.add(flowEdge);
        } else {
            log.info("The graph " + graph + "doesn't contains edge " + flowEdge.toString());
        }
    }
}