Examples with RowMetadata org.talend.dataprep.api.dataset.RowMetadata used on opensource projects

Example 66 with RowMetadata

use of org.talend.dataprep.api.dataset.RowMetadata in project data-prep by Talend.

the class ZipLinkTest method should_emit_single_input_row_when_all_source_has_emitted_one.

@Test
public void should_emit_single_input_row_when_all_source_has_emitted_one() {
    // given
    final TestNode source1 = new TestNode();
    final TestNode source2 = new TestNode();
    final TestNode target = new TestNode();
    ZipLink.zip(new Node[] { source1, source2 }, target);
    final DataSetRow row = new DataSetRow(new HashMap<>());
    final RowMetadata metadata = new RowMetadata(new ArrayList<>());
    // when
    source1.receive(row, metadata);
    // then
    assertThat(target.getReceivedRows(), hasSize(0));
    assertThat(target.getReceivedMetadata(), hasSize(0));
    // when
    source2.receive(row, metadata);
    // then
    assertThat(target.getReceivedRows(), hasSize(2));
    assertThat(target.getReceivedMetadata(), hasSize(2));
}

Example 67 with RowMetadata

use of org.talend.dataprep.api.dataset.RowMetadata in project data-prep by Talend.

the class BasicNodeTest method should_emit_multi_input_row_to_its_link.

@Test
public void should_emit_multi_input_row_to_its_link() {
    // given
    final TestLink link = new TestLink(new BasicNode());
    final BasicNode node = new BasicNode();
    node.setLink(link);
    final DataSetRow row1 = new DataSetRow(new HashMap<>());
    final DataSetRow row2 = new DataSetRow(new HashMap<>());
    final RowMetadata metadata1 = new RowMetadata(new ArrayList<>());
    final RowMetadata metadata2 = new RowMetadata(new ArrayList<>());
    final DataSetRow[] rows = new DataSetRow[] { row1, row2 };
    final RowMetadata[] metadatas = new RowMetadata[] { metadata1, metadata2 };
    // when
    node.receive(rows, metadatas);
    // then
    assertThat(link.getEmittedRows(), hasSize(2));
    assertThat(link.getEmittedRows(), contains(rows));
    assertThat(link.getEmittedMetadata(), hasSize(2));
    assertThat(link.getEmittedMetadata(), contains(metadatas));
}

Example 68 with RowMetadata

use of org.talend.dataprep.api.dataset.RowMetadata in project data-prep by Talend.

the class ActionContextTest method setUp.

@Before
public void setUp() throws Exception {
    parent = new TransformationContext();
    row = new RowMetadata();
    context = new ActionContext(parent, row);
}

Example 69 with RowMetadata

use of org.talend.dataprep.api.dataset.RowMetadata in project data-prep by Talend.

the class Pipeline method execute.

public void execute(DataSet dataSet) {
    final RowMetadata rowMetadata = dataSet.getMetadata().getRowMetadata().clone();
    try (Stream<DataSetRow> records = dataSet.getRecords()) {
        // get the lock on isFinished to make the signal(STOP) method wait for the whole pipeline to finish
        synchronized (isFinished) {
            AtomicLong counter = new AtomicLong();
            // we use map/allMatch to stop the stream when isStopped = true
            // with only forEach((row) -> if(isStopped)) for ex we just stop the processed code
            // but we proceed all the rows of the stream
            // to replace when java introduce more useful functions to stream (ex: takeWhile)
            // 
            records.peek(row -> {
                // 
                node.exec().receive(row, rowMetadata);
                counter.addAndGet(1L);
            }).allMatch(row -> !isStopped.get());
            LOG.debug("{} rows sent in the pipeline", counter.get());
            node.exec().signal(Signal.END_OF_STREAM);
        }
    }
}

Example 70 with RowMetadata

use of org.talend.dataprep.api.dataset.RowMetadata in project data-prep by Talend.

the class StepNode method receive.

@Override
public void receive(DataSetRow row, RowMetadata metadata) {
    RowMetadata processingRowMetadata = metadata;
    if (previousStepRowMetadata != null) {
        // Step node has associated metadata, use it instead of supplied one.
        processingRowMetadata = previousStepRowMetadata;
    }
    // make sure the last node (ActionNode) link is set to after the StepNode
    if (lastNode.getLink() == null) {
        final RuntimeLink stepLink = getLink().exec();
        lastNode.setLink(new StepLink(stepLink));
    }
    lastRowMetadata = processingRowMetadata;
    entryNode.exec().receive(row, processingRowMetadata);
}