Search in sources :

Example 1 with WorkflowIdentifier

use of org.flyte.api.v1.WorkflowIdentifier in project flytekit-java by flyteorg.

the class ProjectClosureTest method testCollectSubWorkflows.

@Test
public void testCollectSubWorkflows() {
    TypedInterface emptyInterface = TypedInterface.builder().inputs(ImmutableMap.of()).outputs(ImmutableMap.of()).build();
    WorkflowMetadata emptyMetadata = WorkflowMetadata.builder().build();
    PartialWorkflowIdentifier rewrittenSubWorkflowRef = PartialWorkflowIdentifier.builder().project("project").name("name").version("version").domain("domain").build();
    WorkflowIdentifier subWorkflowRef = WorkflowIdentifier.builder().project("project").name("name").version("version").domain("domain").build();
    WorkflowIdentifier otherSubWorkflowRef = WorkflowIdentifier.builder().project("project").name("other-name").version("version").domain("domain").build();
    PartialWorkflowIdentifier rewrittenNestedSubWorkflowRef = PartialWorkflowIdentifier.builder().project("project").name("nested").version("version").domain("domain").build();
    WorkflowIdentifier nestedSubWorkflowRef = WorkflowIdentifier.builder().project("project").name("nested").version("version").domain("domain").build();
    PartialWorkflowIdentifier rewrittenNestedOtherSubWorkflowRef = PartialWorkflowIdentifier.builder().project("project").name("nested-other").version("version").domain("domain").build();
    WorkflowIdentifier nestedOtherSubWorkflowRef = WorkflowIdentifier.builder().project("project").name("nested-other").version("version").domain("domain").build();
    WorkflowNode workflowNode = WorkflowNode.builder().reference(WorkflowNode.Reference.ofSubWorkflowRef(rewrittenSubWorkflowRef)).build();
    WorkflowNode nestedWorkflowNode = WorkflowNode.builder().reference(WorkflowNode.Reference.ofSubWorkflowRef(rewrittenNestedSubWorkflowRef)).build();
    WorkflowNode nestedOtherWorkflowNode = WorkflowNode.builder().reference(WorkflowNode.Reference.ofSubWorkflowRef(rewrittenNestedOtherSubWorkflowRef)).build();
    WorkflowTemplate emptyWorkflowTemplate = WorkflowTemplate.builder().interface_(emptyInterface).metadata(emptyMetadata).nodes(ImmutableList.of()).outputs(ImmutableList.of()).build();
    WorkflowTemplate nestedWorkflowTemplate = WorkflowTemplate.builder().interface_(emptyInterface).metadata(emptyMetadata).nodes(ImmutableList.of(Node.builder().id("nested-node").inputs(ImmutableList.of()).upstreamNodeIds(ImmutableList.of()).workflowNode(nestedOtherWorkflowNode).build())).outputs(ImmutableList.of()).build();
    Operand opTrue = Operand.ofPrimitive(Primitive.ofBooleanValue(true));
    BooleanExpression exprTrue = BooleanExpression.ofComparison(ComparisonExpression.builder().leftValue(opTrue).rightValue(opTrue).operator(ComparisonExpression.Operator.EQ).build());
    List<Node> nodes = ImmutableList.of(Node.builder().id("node-1").inputs(ImmutableList.of()).upstreamNodeIds(ImmutableList.of()).workflowNode(workflowNode).build(), // Same sub-workflow
    Node.builder().id("node-2").inputs(ImmutableList.of()).upstreamNodeIds(ImmutableList.of()).workflowNode(workflowNode).build(), // Sub-workflow which has a nested sub-workflow in branch (nestedOtherWorkflowNode)
    Node.builder().id("node-3").inputs(ImmutableList.of()).upstreamNodeIds(ImmutableList.of()).branchNode(BranchNode.builder().ifElse(IfElseBlock.builder().case_(IfBlock.builder().condition(exprTrue).thenNode(Node.builder().id("node-4").inputs(ImmutableList.of()).upstreamNodeIds(ImmutableList.of()).workflowNode(nestedWorkflowNode).build()).build()).other(ImmutableList.of()).build()).build()).build());
    // nestedOtherWorkflowNode is not in the previous list because
    // that node belongs to the template of a sub-workflow
    Map<WorkflowIdentifier, WorkflowTemplate> allWorkflows = ImmutableMap.of(subWorkflowRef, emptyWorkflowTemplate, otherSubWorkflowRef, emptyWorkflowTemplate, nestedSubWorkflowRef, nestedWorkflowTemplate, nestedOtherSubWorkflowRef, emptyWorkflowTemplate);
    Map<WorkflowIdentifier, WorkflowTemplate> collectedSubWorkflows = ProjectClosure.collectSubWorkflows(nodes, allWorkflows);
    assertThat(collectedSubWorkflows, equalTo(ImmutableMap.of(subWorkflowRef, emptyWorkflowTemplate, nestedSubWorkflowRef, nestedWorkflowTemplate, nestedOtherSubWorkflowRef, emptyWorkflowTemplate)));
}
Also used : TypedInterface(org.flyte.api.v1.TypedInterface) WorkflowMetadata(org.flyte.api.v1.WorkflowMetadata) PartialWorkflowIdentifier(org.flyte.api.v1.PartialWorkflowIdentifier) WorkflowIdentifier(org.flyte.api.v1.WorkflowIdentifier) PartialWorkflowIdentifier(org.flyte.api.v1.PartialWorkflowIdentifier) BooleanExpression(org.flyte.api.v1.BooleanExpression) WorkflowTemplate(org.flyte.api.v1.WorkflowTemplate) Operand(org.flyte.api.v1.Operand) BranchNode(org.flyte.api.v1.BranchNode) WorkflowNode(org.flyte.api.v1.WorkflowNode) Node(org.flyte.api.v1.Node) WorkflowNode(org.flyte.api.v1.WorkflowNode) Test(org.junit.jupiter.api.Test)

Example 2 with WorkflowIdentifier

use of org.flyte.api.v1.WorkflowIdentifier in project flytekit-java by flyteorg.

the class ProtoUtilTest method shouldSerializeWorkflowIdentifiers.

@Test
void shouldSerializeWorkflowIdentifiers() {
    String name = "workflow-a";
    WorkflowIdentifier id = WorkflowIdentifier.builder().domain(DOMAIN).project(PROJECT).name(name).version(VERSION).build();
    IdentifierOuterClass.Identifier serializedId = ProtoUtil.serialize(id);
    assertThat(serializedId, equalTo(IdentifierOuterClass.Identifier.newBuilder().setResourceType(WORKFLOW).setDomain(DOMAIN).setProject(PROJECT).setName(name).setVersion(VERSION).build()));
}
Also used : PartialWorkflowIdentifier(org.flyte.api.v1.PartialWorkflowIdentifier) WorkflowIdentifier(org.flyte.api.v1.WorkflowIdentifier) Matchers.containsString(org.hamcrest.Matchers.containsString) IdentifierOuterClass(flyteidl.core.IdentifierOuterClass) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 3 with WorkflowIdentifier

use of org.flyte.api.v1.WorkflowIdentifier in project flytekit-java by flyteorg.

the class IdentifierRewrite method getLatestWorkflowVersion.

private String getLatestWorkflowVersion(String project, String domain, String name) {
    WorkflowIdentifier latestWorkflowId = adminClient().fetchLatestWorkflowId(NamedEntityIdentifier.builder().project(project).domain(domain).name(name).build());
    Verify.verifyNotNull(latestWorkflowId, "workflow not found domain=[%s], project=[%s], name=[%s]", domain, project, name);
    return latestWorkflowId.version();
}
Also used : WorkflowIdentifier(org.flyte.api.v1.WorkflowIdentifier) PartialWorkflowIdentifier(org.flyte.api.v1.PartialWorkflowIdentifier)

Example 4 with WorkflowIdentifier

use of org.flyte.api.v1.WorkflowIdentifier in project flytekit-java by flyteorg.

the class ProjectClosure method checkCycles.

static boolean checkCycles(WorkflowIdentifier workflowId, Map<WorkflowIdentifier, WorkflowTemplate> allWorkflows, Set<WorkflowIdentifier> beingVisited, Set<WorkflowIdentifier> visited) {
    beingVisited.add(workflowId);
    WorkflowTemplate workflow = allWorkflows.get(workflowId);
    List<Node> nodes = workflow.nodes().stream().flatMap(ProjectClosure::flatBranch).collect(toUnmodifiableList());
    for (Node node : nodes) {
        if (isSubWorkflowNode(node)) {
            PartialWorkflowIdentifier partialSubWorkflowId = Objects.requireNonNull(node.workflowNode()).reference().subWorkflowRef();
            WorkflowIdentifier subWorkflowId = WorkflowIdentifier.builder().project(partialSubWorkflowId.project()).name(partialSubWorkflowId.name()).domain(partialSubWorkflowId.domain()).version(partialSubWorkflowId.version()).build();
            if (// backward edge
            beingVisited.contains(subWorkflowId) || (!visited.contains(subWorkflowId) && checkCycles(subWorkflowId, allWorkflows, beingVisited, visited))) {
                return true;
            }
        }
    }
    beingVisited.remove(workflowId);
    visited.add(workflowId);
    return false;
}
Also used : PartialWorkflowIdentifier(org.flyte.api.v1.PartialWorkflowIdentifier) PartialWorkflowIdentifier(org.flyte.api.v1.PartialWorkflowIdentifier) WorkflowIdentifier(org.flyte.api.v1.WorkflowIdentifier) WorkflowTemplate(org.flyte.api.v1.WorkflowTemplate) WorkflowNode(org.flyte.api.v1.WorkflowNode) Node(org.flyte.api.v1.Node)

Example 5 with WorkflowIdentifier

use of org.flyte.api.v1.WorkflowIdentifier in project flytekit-java by flyteorg.

the class FlyteAdminClientTest method fetchLatestWorkflowIdShouldReturnNullWhenEmptyList.

@Test
public void fetchLatestWorkflowIdShouldReturnNullWhenEmptyList() {
    stubService.workflowLists = Collections.emptyList();
    WorkflowIdentifier workflowId = client.fetchLatestWorkflowId(NamedEntityIdentifier.builder().project(PROJECT).domain(DOMAIN).name(WF_NAME).build());
    assertThat(workflowId, nullValue());
}
Also used : PartialWorkflowIdentifier(org.flyte.api.v1.PartialWorkflowIdentifier) WorkflowIdentifier(org.flyte.api.v1.WorkflowIdentifier) Test(org.junit.Test)

Aggregations

WorkflowIdentifier (org.flyte.api.v1.WorkflowIdentifier)11 PartialWorkflowIdentifier (org.flyte.api.v1.PartialWorkflowIdentifier)10 WorkflowTemplate (org.flyte.api.v1.WorkflowTemplate)6 Test (org.junit.jupiter.api.Test)4 Node (org.flyte.api.v1.Node)3 TypedInterface (org.flyte.api.v1.TypedInterface)3 WorkflowNode (org.flyte.api.v1.WorkflowNode)3 Test (org.junit.Test)3 WorkflowMetadata (org.flyte.api.v1.WorkflowMetadata)2 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)2 IdentifierOuterClass (flyteidl.core.IdentifierOuterClass)1 HashMap (java.util.HashMap)1 BooleanExpression (org.flyte.api.v1.BooleanExpression)1 BranchNode (org.flyte.api.v1.BranchNode)1 ContainerError (org.flyte.api.v1.ContainerError)1 DynamicJobSpec (org.flyte.api.v1.DynamicJobSpec)1 DynamicWorkflowTask (org.flyte.api.v1.DynamicWorkflowTask)1 DynamicWorkflowTaskRegistrar (org.flyte.api.v1.DynamicWorkflowTaskRegistrar)1 Literal (org.flyte.api.v1.Literal)1 Operand (org.flyte.api.v1.Operand)1