Examples with TransformationContext org.talend.dataprep.transformation.api.action.context.TransformationContext used on opensource projects

Example 11 with TransformationContext

use of org.talend.dataprep.transformation.api.action.context.TransformationContext in project data-prep by Talend.

the class PipelineTest method testCleanUp.

@Test
public void testCleanUp() throws Exception {
    // Given
    final TransformationContext transformationContext = new TransformationContext();
    final ActionContext actionContext = transformationContext.create((r, ac) -> r, new RowMetadata());
    final AtomicInteger wasDestroyed = new AtomicInteger(0);
    Destroyable destroyable = new Destroyable() {

        @Override
        public void destroy() {
            wasDestroyed.incrementAndGet();
        }
    };
    actionContext.get("test1", p -> destroyable);
    actionContext.get("test2", p -> destroyable);
    final Node node = // 
    NodeBuilder.source().to(// 
    new BasicNode()).to(// 
    new CleanUpNode(transformationContext)).to(// 
    output).build();
    // when
    node.exec().signal(END_OF_STREAM);
    // then
    assertEquals(2, wasDestroyed.get());
}

Example 12 with TransformationContext

use of org.talend.dataprep.transformation.api.action.context.TransformationContext in project data-prep by Talend.

the class ActionNodesBuilder method build.

/**
 * Build the actions pipeline
 */
public Node build() {
    final StatisticsNodesBuilder statisticsNodesBuilder = StatisticsNodesBuilder.builder().analyzerService(// 
    analyzerService).actionRegistry(// 
    actionRegistry).statisticsAdapter(// 
    statisticsAdapter).allowSchemaAnalysis(// 
    allowSchemaAnalysis).actions(// 
    actions).columns(initialMetadata.getColumns());
    final NodeBuilder builder = NodeBuilder.source();
    // unless we don't have initial metadata or we explicitly ask it
    if (needStatisticsBefore || initialMetadata.getColumns().isEmpty()) {
        LOGGER.debug("No initial metadata submitted for transformation, computing new one.");
        builder.to(statisticsNodesBuilder.buildPreStatistics());
    }
    // transformation context is the parent of every action context
    // it will hold all the action context
    // that makes it the perfect entry point to clean up all the contexts
    final TransformationContext context = new TransformationContext();
    // * an action node
    for (final RunnableAction nextAction : actions) {
        // some actions need fresh statistics
        // in those cases, we gather the rows in a reservoir node that triggers statistics computation
        // before dispatching each row to the next node
        final Node neededReservoir = statisticsNodesBuilder.buildIntermediateStatistics(nextAction);
        if (neededReservoir != null) {
            builder.to(neededReservoir);
        }
        final DataSetRowAction rowAction = nextAction.getRowAction();
        builder.to(new CompileNode(nextAction, context.create(rowAction, initialMetadata)));
        builder.to(new ActionNode(nextAction, context.in(rowAction)));
    }
    // when it is explicitly asked and the actions changes the columns
    if (needStatisticsAfter) {
        builder.to(statisticsNodesBuilder.buildPostStatistics());
    }
    // cleanup all contexts after all actions
    builder.to(new CleanUpNode(context));
    return builder.build();
}

Example 13 with TransformationContext

use of org.talend.dataprep.transformation.api.action.context.TransformationContext in project data-prep by Talend.

the class UpdatedStepVisitorTest method setUp.

@Before
public void setUp() throws Exception {
    actionContext = new ActionContext(new TransformationContext());
    entryNode = new ActionNode(new RunnableAction((row, context) -> row), actionContext);
}

Example 14 with TransformationContext

use of org.talend.dataprep.transformation.api.action.context.TransformationContext in project data-prep by Talend.

the class FillWithNumericIfInvalidTest method should_not_fill_non_valid_integer.

@Test
public void should_not_fill_non_valid_integer() throws Exception {
    // given
    final Map<String, String> values = new HashMap<>();
    values.put("0000", "David Bowie");
    values.put("0001", "30");
    values.put("0002", "Something");
    final DataSetRow row = new DataSetRow(values);
    final RowMetadata rowMetadata = row.getRowMetadata();
    rowMetadata.getById("0001").setType(Type.INTEGER.getName());
    Map<String, String> parameters = ActionMetadataTestUtils.parseParameters(this.getClass().getResourceAsStream("fillInvalidIntegerAction.json"));
    // when
    final RunnableAction runnableAction = factory.create(action, parameters);
    final ActionContext context = new ActionContext(new TransformationContext(), rowMetadata);
    context.setParameters(parameters);
    runnableAction.getRowAction().apply(row, context);
    // then
    assertEquals("30", row.get("0001"));
    assertEquals("David Bowie", row.get("0000"));
}

Example 15 with TransformationContext

use of org.talend.dataprep.transformation.api.action.context.TransformationContext in project data-prep by Talend.

the class FillWithNumericIfInvalidTest method should_fill_non_valid_integer.

@Test
public void should_fill_non_valid_integer() throws Exception {
    // given
    final Map<String, String> values = new HashMap<>();
    values.put("0000", "David Bowie");
    values.put("0001", "N");
    values.put("0002", "Something");
    final DataSetRow row = new DataSetRow(values);
    row.setInvalid("0001");
    final RowMetadata rowMetadata = row.getRowMetadata();
    rowMetadata.getById("0001").setType(Type.INTEGER.getName());
    Map<String, String> parameters = ActionMetadataTestUtils.parseParameters(this.getClass().getResourceAsStream("fillInvalidIntegerAction.json"));
    // when
    final RunnableAction runnableAction = factory.create(action, parameters);
    final ActionContext context = new ActionContext(new TransformationContext(), rowMetadata);
    context.setParameters(parameters);
    runnableAction.getRowAction().apply(row, context);
    // then
    assertEquals("25", row.get("0001"));
    assertEquals("David Bowie", row.get("0000"));
}