Search in sources :

Example 16 with Step

use of org.talend.dataprep.api.preparation.Step in project data-prep by Talend.

the class UpdatedStepVisitorTest method testUpdatedStepsWithKO.

@Test
public void testUpdatedStepsWithKO() throws Exception {
    // Given
    final Step step = new Step(ROOT_STEP.id(), new PreparationActions().id(), "0.0");
    final RowMetadata stepRowMetadata = new RowMetadata();
    final Node stepNode = NodeBuilder.from(new StepNode(step, stepRowMetadata, entryNode, new BasicNode())).to(new BasicNode()).build();
    final UpdatedStepVisitor visitor = new UpdatedStepVisitor(stepMetadataRepository);
    // Canceled action!
    actionContext.setActionStatus(ActionContext.ActionStatus.CANCELED);
    // When
    stepNode.exec().receive(new DataSetRow(metadata), metadata);
    // Then
    stepNode.accept(visitor);
    verify(stepMetadataRepository).update(step.id(), stepRowMetadata);
}
Also used : StepNode(org.talend.dataprep.transformation.pipeline.node.StepNode) StepNode(org.talend.dataprep.transformation.pipeline.node.StepNode) ActionNode(org.talend.dataprep.transformation.pipeline.node.ActionNode) Node(org.talend.dataprep.transformation.pipeline.Node) BasicNode(org.talend.dataprep.transformation.pipeline.node.BasicNode) BasicNode(org.talend.dataprep.transformation.pipeline.node.BasicNode) PreparationActions(org.talend.dataprep.api.preparation.PreparationActions) RowMetadata(org.talend.dataprep.api.dataset.RowMetadata) Step(org.talend.dataprep.api.preparation.Step) DataSetRow(org.talend.dataprep.api.dataset.row.DataSetRow) Test(org.junit.Test)

Example 17 with Step

use of org.talend.dataprep.api.preparation.Step in project data-prep by Talend.

the class UpdatedStepVisitorTest method testUpdatedStepsWithOk.

@Test
public void testUpdatedStepsWithOk() throws Exception {
    // Given
    final Step step = new Step(ROOT_STEP.id(), new PreparationActions().id(), "0.0");
    final RowMetadata stepRowMetadata = new RowMetadata();
    final Node stepNode = NodeBuilder.from(new StepNode(step, stepRowMetadata, entryNode, new BasicNode())).to(new BasicNode()).build();
    final UpdatedStepVisitor visitor = new UpdatedStepVisitor(stepMetadataRepository);
    // OK action!
    actionContext.setActionStatus(ActionContext.ActionStatus.OK);
    // When
    stepNode.exec().receive(new DataSetRow(metadata), metadata);
    // Then
    stepNode.accept(visitor);
    verify(stepMetadataRepository).update(step.id(), stepRowMetadata);
}
Also used : StepNode(org.talend.dataprep.transformation.pipeline.node.StepNode) StepNode(org.talend.dataprep.transformation.pipeline.node.StepNode) ActionNode(org.talend.dataprep.transformation.pipeline.node.ActionNode) Node(org.talend.dataprep.transformation.pipeline.Node) BasicNode(org.talend.dataprep.transformation.pipeline.node.BasicNode) BasicNode(org.talend.dataprep.transformation.pipeline.node.BasicNode) PreparationActions(org.talend.dataprep.api.preparation.PreparationActions) RowMetadata(org.talend.dataprep.api.dataset.RowMetadata) Step(org.talend.dataprep.api.preparation.Step) DataSetRow(org.talend.dataprep.api.dataset.row.DataSetRow) Test(org.junit.Test)

Example 18 with Step

use of org.talend.dataprep.api.preparation.Step in project data-prep by Talend.

the class OptimizedExportStrategy method getMatchingSteps.

/**
 * Return the steps that are between the from and the to steps IDs.
 *
 * @param steps the steps to start from.
 * @param fromId the from step id.
 * @param toId the to step id.
 * @return the steps that are between the from and the to steps IDs.
 */
private List<Step> getMatchingSteps(List<Step> steps, String fromId, String toId) {
    List<Step> result = new ArrayList<>();
    boolean addStep = false;
    for (Step step : steps) {
        // skip steps before the from
        if (fromId.equals(step.id())) {
            addStep = true;
        } else if (addStep) {
            // fromId should not be added, hence the else !
            result.add(step);
        }
        // skip steps after
        if (addStep && toId.equals(step.getId())) {
            break;
        }
    }
    LOGGER.debug("Matching steps from {} to {} are {}", fromId, toId, steps);
    return result;
}
Also used : ArrayList(java.util.ArrayList) Step(org.talend.dataprep.api.preparation.Step)

Example 19 with Step

use of org.talend.dataprep.api.preparation.Step in project data-prep by Talend.

the class PreparationTest method initialStepWithAppend.

@Test
public void initialStepWithAppend() {
    final String version = versionService.version().getVersionId();
    final List<Action> actions = getSimpleAction("uppercase", "column_name", "lastname");
    final PreparationActions newContent = PreparationActions.ROOT_ACTIONS.append(actions);
    repository.add(newContent);
    final Step s = new Step(Step.ROOT_STEP.id(), newContent.id(), version);
    repository.add(s);
    final Preparation preparation = new Preparation("#5438743", "1234", s.id(), version);
    preparation.setCreationDate(0L);
    repository.add(preparation);
    assertThat(preparation.id(), Is.is("#5438743"));
}
Also used : Action(org.talend.dataprep.api.preparation.Action) Preparation(org.talend.dataprep.api.preparation.Preparation) PreparationActions(org.talend.dataprep.api.preparation.PreparationActions) Step(org.talend.dataprep.api.preparation.Step) Test(org.junit.Test) ServiceBaseTest(org.talend.ServiceBaseTest)

Example 20 with Step

use of org.talend.dataprep.api.preparation.Step in project data-prep by Talend.

the class APIPreparationConversionsTest method getPreparationMessage.

private PreparationMessage getPreparationMessage(String dataSetId) {
    final PreparationMessage preparation = new PreparationMessage();
    preparation.setDataSetId(dataSetId);
    List<Step> steps = new ArrayList<>(12);
    Step parentStep = preparation.getSteps().get(0);
    for (int i = 0; i < 12; i++) {
        final Step step = new Step(parentStep.id(), new PreparationActions().id(), "2.1");
        steps.add(step);
        parentStep = step;
    }
    preparation.setSteps(steps);
    return preparation;
}
Also used : ArrayList(java.util.ArrayList) PreparationMessage(org.talend.dataprep.api.preparation.PreparationMessage) PreparationActions(org.talend.dataprep.api.preparation.PreparationActions) Step(org.talend.dataprep.api.preparation.Step)

Aggregations

Step (org.talend.dataprep.api.preparation.Step)28 Test (org.junit.Test)19 Preparation (org.talend.dataprep.api.preparation.Preparation)19 PreparationActions (org.talend.dataprep.api.preparation.PreparationActions)16 ArrayList (java.util.ArrayList)11 Action (org.talend.dataprep.api.preparation.Action)11 List (java.util.List)9 Arrays (java.util.Arrays)8 PersistentStep (org.talend.dataprep.preparation.store.PersistentStep)8 PreparationRepository (org.talend.dataprep.preparation.store.PreparationRepository)8 BaseMaintenanceTest (org.talend.dataprep.maintenance.BaseMaintenanceTest)7 Collections (java.util.Collections)6 Stream (java.util.stream.Stream)6 InjectMocks (org.mockito.InjectMocks)6 Mock (org.mockito.Mock)6 Mockito (org.mockito.Mockito)6 TqlBuilder (org.talend.tql.api.TqlBuilder)6 OutputStream (java.io.OutputStream)4 Autowired (org.springframework.beans.factory.annotation.Autowired)4 ExportParameters (org.talend.dataprep.api.export.ExportParameters)4