Examples with CompileNode org.talend.dataprep.transformation.pipeline.node.CompileNode used on opensource projects

Example 1 with CompileNode

use of org.talend.dataprep.transformation.pipeline.node.CompileNode in project data-prep by Talend.

the class StepNodeTransformerTest method shouldCreateStepNode.

@Test
public void shouldCreateStepNode() throws Exception {
    // given
    Node node = // 
    NodeBuilder.from(// 
    new CompileNode(null, null)).to(// 
    new ActionNode(null, null)).build();
    // when
    final Node processed = StepNodeTransformer.transform(node, asList(ROOT, STEP), s -> null);
    // then
    final Class[] expectedClasses = { SourceNode.class, StepNode.class };
    final NodeClassVisitor visitor = new NodeClassVisitor();
    processed.accept(visitor);
    assertThat(visitor.traversedClasses, hasItems(expectedClasses));
}

Example 2 with CompileNode

use of org.talend.dataprep.transformation.pipeline.node.CompileNode in project data-prep by Talend.

the class StepNodeTransformerTest method shouldFailCreateStepNode.

@Test(expected = IllegalArgumentException.class)
public void shouldFailCreateStepNode() throws Exception {
    // given
    Node node = // 
    NodeBuilder.from(// 
    new CompileNode(null, null)).to(// 
    new ActionNode(null, null)).build();
    // then
    StepNodeTransformer.transform(node, emptyList(), s -> null);
}

Example 3 with CompileNode

use of org.talend.dataprep.transformation.pipeline.node.CompileNode in project data-prep by Talend.

the class ActionNodesBuilder method build.

/**
 * Build the actions pipeline
 */
public Node build() {
    final StatisticsNodesBuilder statisticsNodesBuilder = StatisticsNodesBuilder.builder().analyzerService(// 
    analyzerService).actionRegistry(// 
    actionRegistry).statisticsAdapter(// 
    statisticsAdapter).allowSchemaAnalysis(// 
    allowSchemaAnalysis).actions(// 
    actions).columns(initialMetadata.getColumns());
    final NodeBuilder builder = NodeBuilder.source();
    // unless we don't have initial metadata or we explicitly ask it
    if (needStatisticsBefore || initialMetadata.getColumns().isEmpty()) {
        LOGGER.debug("No initial metadata submitted for transformation, computing new one.");
        builder.to(statisticsNodesBuilder.buildPreStatistics());
    }
    // transformation context is the parent of every action context
    // it will hold all the action context
    // that makes it the perfect entry point to clean up all the contexts
    final TransformationContext context = new TransformationContext();
    // * an action node
    for (final RunnableAction nextAction : actions) {
        // some actions need fresh statistics
        // in those cases, we gather the rows in a reservoir node that triggers statistics computation
        // before dispatching each row to the next node
        final Node neededReservoir = statisticsNodesBuilder.buildIntermediateStatistics(nextAction);
        if (neededReservoir != null) {
            builder.to(neededReservoir);
        }
        final DataSetRowAction rowAction = nextAction.getRowAction();
        builder.to(new CompileNode(nextAction, context.create(rowAction, initialMetadata)));
        builder.to(new ActionNode(nextAction, context.in(rowAction)));
    }
    // when it is explicitly asked and the actions changes the columns
    if (needStatisticsAfter) {
        builder.to(statisticsNodesBuilder.buildPostStatistics());
    }
    // cleanup all contexts after all actions
    builder.to(new CleanUpNode(context));
    return builder.build();
}

Example 4 with CompileNode

use of org.talend.dataprep.transformation.pipeline.node.CompileNode in project data-prep by Talend.

the class StepNodeTransformerTest method shouldCreateStepNodeWhenSurrounded.

@Test
public void shouldCreateStepNodeWhenSurrounded() throws Exception {
    // given
    Node node = // 
    NodeBuilder.from(// 
    new TestNode()).to(// 
    new CompileNode(null, null)).to(// 
    new ActionNode(null, null)).to(// 
    new BasicNode()).build();
    // when
    final Node processed = StepNodeTransformer.transform(node, asList(ROOT, STEP), s -> null);
    // then
    final Class[] expectedClasses = { SourceNode.class, TestNode.class, StepNode.class, BasicNode.class };
    final NodeClassVisitor visitor = new NodeClassVisitor();
    processed.accept(visitor);
    assertThat(visitor.traversedClasses, hasItems(expectedClasses));
}

Example 5 with CompileNode

use of org.talend.dataprep.transformation.pipeline.node.CompileNode in project data-prep by Talend.

the class StepNodeTransformerTest method shouldCreateStepNodeWithTooManySteps.

@Test
public void shouldCreateStepNodeWithTooManySteps() throws Exception {
    // given
    Node node = // 
    NodeBuilder.from(// 
    new CompileNode(null, null)).to(// 
    new ActionNode(null, null)).build();
    // when
    final Node processed = StepNodeTransformer.transform(node, asList(ROOT, STEP, STEP), s -> null);
    // then
    final Class[] expectedClasses = { SourceNode.class, StepNode.class };
    final NodeClassVisitor visitor = new NodeClassVisitor();
    processed.accept(visitor);
    assertThat(visitor.traversedClasses, hasItems(expectedClasses));
}