Search in sources :

Example 41 with ColumnMetadata

use of org.talend.dataprep.api.dataset.ColumnMetadata in project data-prep by Talend.

the class ReplaceOnValueTest method test_apply_in_newcolumn.

@Test
public void test_apply_in_newcolumn() {
    // given
    final String columnId = "0000";
    final Map<String, String> values = new HashMap<>();
    values.put(columnId, "James Hetfield");
    final DataSetRow row = new DataSetRow(values);
    final Map<String, String> parameters = new HashMap<>();
    parameters.put(CELL_VALUE_PARAMETER, generateJson("James", ReplaceOnValueHelper.STARTS_WITH_MODE));
    parameters.put(REPLACE_VALUE_PARAMETER, "Jimmy");
    parameters.put(REPLACE_ENTIRE_CELL_PARAMETER, "true");
    parameters.put(ImplicitParameters.SCOPE.getKey().toLowerCase(), "column");
    parameters.put(ImplicitParameters.COLUMN_ID.getKey().toLowerCase(), columnId);
    parameters.put(ActionsUtils.CREATE_NEW_COLUMN, "true");
    // when
    ActionTestWorkbench.test(row, actionRegistry, factory.create(action, parameters));
    // then
    assertThat(row.get(columnId), is("James Hetfield"));
    assertThat(row.get("0001"), is("Jimmy"));
    ColumnMetadata expected = ColumnMetadata.Builder.column().id(1).name("0000_replace").type(Type.STRING).build();
    ColumnMetadata actual = row.getRowMetadata().getById("0001");
    assertEquals(expected, actual);
}
Also used : ColumnMetadata(org.talend.dataprep.api.dataset.ColumnMetadata) HashMap(java.util.HashMap) DataSetRow(org.talend.dataprep.api.dataset.row.DataSetRow) AbstractMetadataBaseTest(org.talend.dataprep.transformation.actions.AbstractMetadataBaseTest) Test(org.junit.Test)

Example 42 with ColumnMetadata

use of org.talend.dataprep.api.dataset.ColumnMetadata in project data-prep by Talend.

the class SplitTest method should_update_metadata.

@Test
public void should_update_metadata() {
    // given
    final List<ColumnMetadata> input = new ArrayList<>();
    input.add(createMetadata("0000", "recipe"));
    input.add(createMetadata("0001", "steps"));
    input.add(createMetadata("0002", "last update"));
    final RowMetadata rowMetadata = new RowMetadata(input);
    final List<ColumnMetadata> expected = new ArrayList<>();
    expected.add(createMetadata("0000", "recipe"));
    expected.add(createMetadata("0001", "steps"));
    expected.add(createMetadata("0003", "steps_split_1"));
    expected.add(createMetadata("0004", "steps_split_2"));
    expected.add(createMetadata("0002", "last update"));
    // when
    ActionTestWorkbench.test(rowMetadata, actionRegistry, factory.create(action, parameters));
    // then
    assertEquals(expected, rowMetadata.getColumns());
}
Also used : ColumnMetadata(org.talend.dataprep.api.dataset.ColumnMetadata) ArrayList(java.util.ArrayList) RowMetadata(org.talend.dataprep.api.dataset.RowMetadata) AbstractMetadataBaseTest(org.talend.dataprep.transformation.actions.AbstractMetadataBaseTest) Test(org.junit.Test)

Example 43 with ColumnMetadata

use of org.talend.dataprep.api.dataset.ColumnMetadata in project data-prep by Talend.

the class SplitTest method should_update_metadata_twice.

@Test
public void should_update_metadata_twice() {
    // given
    final List<ColumnMetadata> input = new ArrayList<>();
    input.add(createMetadata("0000", "recipe"));
    input.add(createMetadata("0001", "steps"));
    input.add(createMetadata("0002", "last update"));
    final RowMetadata rowMetadata = new RowMetadata(input);
    final List<ColumnMetadata> expected = new ArrayList<>();
    expected.add(createMetadata("0000", "recipe"));
    expected.add(createMetadata("0001", "steps"));
    expected.add(createMetadata("0005", "steps_split_1"));
    expected.add(createMetadata("0006", "steps_split_2"));
    expected.add(createMetadata("0003", "steps_split_1"));
    expected.add(createMetadata("0004", "steps_split_2"));
    expected.add(createMetadata("0002", "last update"));
    // when
    ActionTestWorkbench.test(rowMetadata, actionRegistry, factory.create(action, parameters), factory.create(action, parameters));
    assertEquals(expected, rowMetadata.getColumns());
}
Also used : ColumnMetadata(org.talend.dataprep.api.dataset.ColumnMetadata) ArrayList(java.util.ArrayList) RowMetadata(org.talend.dataprep.api.dataset.RowMetadata) AbstractMetadataBaseTest(org.talend.dataprep.transformation.actions.AbstractMetadataBaseTest) Test(org.junit.Test)

Example 44 with ColumnMetadata

use of org.talend.dataprep.api.dataset.ColumnMetadata in project data-prep by Talend.

the class SubstringTest method should_update_metadata_twice.

@Test
public void should_update_metadata_twice() {
    // given
    final List<ColumnMetadata> input = new ArrayList<>();
    input.add(createMetadata("0000", "recipe"));
    input.add(createMetadata("0001", "steps"));
    input.add(createMetadata("0002", "last update"));
    final RowMetadata rowMetadata = new RowMetadata(input);
    final DataSetRow row = new DataSetRow(rowMetadata);
    final List<ColumnMetadata> expected = new ArrayList<>();
    expected.add(createMetadata("0000", "recipe"));
    expected.add(createMetadata("0001", "steps"));
    expected.add(createMetadata("0004", "steps_substring"));
    expected.add(createMetadata("0003", "steps_substring"));
    expected.add(createMetadata("0002", "last update"));
    // when
    ActionTestWorkbench.test(row, actionRegistry, factory.create(action, parameters), factory.create(action, parameters));
    // then
    assertEquals(expected, row.getRowMetadata().getColumns());
}
Also used : ColumnMetadata(org.talend.dataprep.api.dataset.ColumnMetadata) RowMetadata(org.talend.dataprep.api.dataset.RowMetadata) DataSetRow(org.talend.dataprep.api.dataset.row.DataSetRow) Test(org.junit.Test) AbstractMetadataBaseTest(org.talend.dataprep.transformation.actions.AbstractMetadataBaseTest)

Example 45 with ColumnMetadata

use of org.talend.dataprep.api.dataset.ColumnMetadata in project data-prep by Talend.

the class SubstringTest method should_update_metadata.

@Test
public void should_update_metadata() {
    // given
    final List<ColumnMetadata> input = new ArrayList<>();
    input.add(createMetadata("0000", "recipe"));
    input.add(createMetadata("0001", "steps"));
    input.add(createMetadata("0002", "last update"));
    final RowMetadata rowMetadata = new RowMetadata(input);
    final List<ColumnMetadata> expected = new ArrayList<>();
    expected.add(createMetadata("0000", "recipe"));
    expected.add(createMetadata("0001", "steps"));
    expected.add(createMetadata("0003", "steps_substring"));
    expected.add(createMetadata("0002", "last update"));
    // when
    ActionTestWorkbench.test(rowMetadata, actionRegistry, factory.create(action, parameters));
    // then
    assertEquals(expected, rowMetadata.getColumns());
}
Also used : ColumnMetadata(org.talend.dataprep.api.dataset.ColumnMetadata) RowMetadata(org.talend.dataprep.api.dataset.RowMetadata) Test(org.junit.Test) AbstractMetadataBaseTest(org.talend.dataprep.transformation.actions.AbstractMetadataBaseTest)

Aggregations

ColumnMetadata (org.talend.dataprep.api.dataset.ColumnMetadata)320 Test (org.junit.Test)217 AbstractMetadataBaseTest (org.talend.dataprep.transformation.actions.AbstractMetadataBaseTest)115 RowMetadata (org.talend.dataprep.api.dataset.RowMetadata)86 DataSetRow (org.talend.dataprep.api.dataset.row.DataSetRow)80 DataSetMetadata (org.talend.dataprep.api.dataset.DataSetMetadata)48 InputStream (java.io.InputStream)25 Type (org.talend.dataprep.api.type.Type)23 DataSetBaseTest (org.talend.dataprep.dataset.DataSetBaseTest)22 ArrayList (java.util.ArrayList)19 HashMap (java.util.HashMap)17 IOException (java.io.IOException)14 TDPException (org.talend.dataprep.exception.TDPException)14 Schema (org.talend.dataprep.schema.Schema)14 Autowired (org.springframework.beans.factory.annotation.Autowired)13 Logger (org.slf4j.Logger)12 LoggerFactory (org.slf4j.LoggerFactory)12 SemanticDomain (org.talend.dataprep.api.dataset.statistics.SemanticDomain)12 DataSetServiceTest (org.talend.dataprep.dataset.service.DataSetServiceTest)11 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)10