Search in sources :

Example 51 with StepField

use of org.pentaho.metaverse.api.StepField in project pentaho-kettle by pentaho.

the class GetXMLDataStepAnalyzer method getChangeRecords.

@Override
public Set<ComponentDerivationRecord> getChangeRecords(GetXMLDataMeta meta) throws MetaverseAnalyzerException {
    Set<ComponentDerivationRecord> changes = new HashSet<>();
    boolean isInFields = meta.isInFields();
    boolean isAFile = meta.getIsAFile();
    boolean isAUrl = meta.isReadUrl();
    // if we are getting xml from a field, we need to add the "derives" links from the xml to the output fields
    if (isInFields && !isAFile && !isAUrl) {
        GetXMLDataField[] fields = baseStepMeta.getInputFields();
        if (getInputs() != null) {
            Set<StepField> inputFields = getInputs().getFieldNames();
            for (StepField inputField : inputFields) {
                if (inputField.getFieldName().equals(meta.getXMLField())) {
                    // link this to all of the outputs that come from the xml
                    for (GetXMLDataField field : fields) {
                        ComponentDerivationRecord change = new ComponentDerivationRecord(meta.getXMLField(), field.getName());
                        changes.add(change);
                    }
                    break;
                }
            }
        }
    }
    return changes;
}
Also used : StepField(org.pentaho.metaverse.api.StepField) ComponentDerivationRecord(org.pentaho.metaverse.api.analyzer.kettle.ComponentDerivationRecord) HashSet(java.util.HashSet)

Example 52 with StepField

use of org.pentaho.metaverse.api.StepField in project pentaho-kettle by pentaho.

the class JsonInputAnalyzer method getUsedFields.

@Override
protected Set<StepField> getUsedFields(final JsonInputMeta meta) {
    Set<StepField> usedFields = new HashSet<>();
    if (meta.isAcceptingFilenames() && StringUtils.isNotEmpty(meta.getAcceptingField())) {
        final Set<String> inpusStepNames = getInputStepNames(meta, meta.getAcceptingField());
        for (final String inpusStepName : inpusStepNames) {
            final StepField stepField = new StepField(inpusStepName, meta.getAcceptingField());
            usedFields.add(stepField);
        }
    }
    return usedFields;
}
Also used : StepField(org.pentaho.metaverse.api.StepField) HashSet(java.util.HashSet)

Example 53 with StepField

use of org.pentaho.metaverse.api.StepField in project pdi-dataservice-server-plugin by pentaho.

the class DataServiceStepValidationTest method mockOriginSteps.

private void mockOriginSteps(String expectedStepName, Collection<String> fieldNames) throws MetaverseException {
    Map<String, Set<StepField>> stepFieldMap = Maps.newHashMap();
    int i = 0;
    for (String fieldName : fieldNames) {
        stepFieldMap.put(fieldName, ImmutableSet.of(new StepField("originStepName" + i, "originFieldName" + i++)));
    }
    when(lineageClient.getOriginSteps(eq(checkStepsExtension.getTransMeta()), eq(expectedStepName), any(Collection.class))).thenReturn(stepFieldMap);
}
Also used : ImmutableSet(com.google.common.collect.ImmutableSet) Set(java.util.Set) StepField(org.pentaho.metaverse.api.StepField) Collection(java.util.Collection) Matchers.containsString(org.hamcrest.Matchers.containsString)

Aggregations

StepField (org.pentaho.metaverse.api.StepField)53 Test (org.junit.Test)33 HashSet (java.util.HashSet)23 StepNodes (org.pentaho.metaverse.api.analyzer.kettle.step.StepNodes)11 IMetaverseNode (org.pentaho.metaverse.api.IMetaverseNode)10 ComponentDerivationRecord (org.pentaho.metaverse.api.analyzer.kettle.ComponentDerivationRecord)10 Matchers.anyString (org.mockito.Matchers.anyString)6 Before (org.junit.Before)4 SelectMetadataChange (org.pentaho.di.trans.steps.selectvalues.SelectMetadataChange)4 Set (java.util.Set)3 BaseStepMeta (org.pentaho.di.trans.step.BaseStepMeta)3 MetaverseObjectFactory (org.pentaho.metaverse.api.MetaverseObjectFactory)3 ArrayList (java.util.ArrayList)2 List (java.util.List)2 ValueMetaInterface (org.pentaho.di.core.row.ValueMetaInterface)2 IAnalysisContext (org.pentaho.metaverse.api.IAnalysisContext)2 IExternalResourceInfo (org.pentaho.metaverse.api.model.IExternalResourceInfo)2 ImmutableSet (com.google.common.collect.ImmutableSet)1 Graph (com.tinkerpop.blueprints.Graph)1 Vertex (com.tinkerpop.blueprints.Vertex)1