Search in sources :

Example 1 with SubstituteTransform

use of com.hortonworks.streamline.streams.layout.component.rule.action.transform.SubstituteTransform in project streamline by hortonworks.

the class TransformRuntimePipelineActionTest method testMergeSubstituteProject.

@Test
public void testMergeSubstituteProject() throws Exception {
    Map<String, Object> fieldsAndValues = new HashMap<>();
    fieldsAndValues.put("1", "one");
    fieldsAndValues.put("2", "${1} plus ${1}");
    Map<String, Object> defaults = new HashMap<>();
    defaults.put("2", "TWO");
    defaults.put("3", "THREE");
    defaults.put("4", "${2} plus ${2}");
    StreamlineEvent event = StreamlineEventImpl.builder().fieldsAndValues(fieldsAndValues).dataSourceId("dsrcid").build();
    MergeTransform merge = new MergeTransform(defaults);
    SubstituteTransform substitute = new SubstituteTransform();
    ProjectionTransform projection = new ProjectionTransform("test-projection", defaults.keySet());
    TransformAction transformAction = new TransformAction(ImmutableList.of(merge, substitute, projection));
    transformAction.setOutputStreams(ImmutableSet.of("streamid"));
    ActionRuntime actionRuntime = new TransformActionRuntime(transformAction);
    List<StreamlineEvent> resultEvents = new ArrayList<>();
    for (Result result : actionRuntime.execute(event)) {
        resultEvents.addAll(result.events);
    }
    assertEquals(1, resultEvents.size());
    assertEquals(3, resultEvents.get(0).size());
    assertEquals("THREE", resultEvents.get(0).get("3"));
    assertEquals("one plus one", resultEvents.get(0).get("2"));
    assertEquals("one plus one plus one plus one", resultEvents.get(0).get("4"));
}
Also used : ProjectionTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.ProjectionTransform) MergeTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.MergeTransform) SubstituteTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.SubstituteTransform) HashMap(java.util.HashMap) StreamlineEvent(com.hortonworks.streamline.streams.StreamlineEvent) TransformAction(com.hortonworks.streamline.streams.layout.component.rule.action.TransformAction) ArrayList(java.util.ArrayList) ActionRuntime(com.hortonworks.streamline.streams.runtime.rule.action.ActionRuntime) Result(com.hortonworks.streamline.streams.Result) Test(org.junit.Test)

Example 2 with SubstituteTransform

use of com.hortonworks.streamline.streams.layout.component.rule.action.transform.SubstituteTransform in project streamline by hortonworks.

the class SubstituteTransformRuntimeTest method testSubstituteSpecificVars.

@Test
public void testSubstituteSpecificVars() throws Exception {
    Map<String, Object> fieldsAndValues = new HashMap<>();
    fieldsAndValues.put("1", "one");
    fieldsAndValues.put("2", "${1} plus ${1}");
    fieldsAndValues.put("3", "${1} plus two");
    StreamlineEvent event = StreamlineEventImpl.builder().fieldsAndValues(fieldsAndValues).dataSourceId("dsrcid").build();
    TransformRuntime transformRuntime = new SubstituteTransformRuntime(new SubstituteTransform(Collections.singleton("3")));
    List<StreamlineEvent> result = transformRuntime.execute(event);
    assertEquals(1, result.size());
    assertEquals(3, result.get(0).size());
    assertEquals("one", result.get(0).get("1"));
    assertEquals("${1} plus ${1}", result.get(0).get("2"));
    assertEquals("one plus two", result.get(0).get("3"));
}
Also used : SubstituteTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.SubstituteTransform) HashMap(java.util.HashMap) StreamlineEvent(com.hortonworks.streamline.streams.StreamlineEvent) TransformRuntime(com.hortonworks.streamline.streams.runtime.TransformRuntime) Test(org.junit.Test)

Example 3 with SubstituteTransform

use of com.hortonworks.streamline.streams.layout.component.rule.action.transform.SubstituteTransform in project streamline by hortonworks.

the class NotifierActionRuntime method getNotificationTransforms.

/**
 * Returns the necessary transforms to perform based on the action.
 */
private List<Transform> getNotificationTransforms(NotifierAction action, Long ruleId) {
    List<Transform> transforms = new ArrayList<>();
    if (action.getOutputFieldsAndDefaults() != null && !action.getOutputFieldsAndDefaults().isEmpty()) {
        transforms.add(new MergeTransform(action.getOutputFieldsAndDefaults()));
        transforms.add(new SubstituteTransform(action.getOutputFieldsAndDefaults().keySet()));
        transforms.add(new ProjectionTransform("projection-" + ruleId, action.getOutputFieldsAndDefaults().keySet()));
    }
    Map<String, Object> headers = new HashMap<>();
    headers.put(AddHeaderTransformRuntime.HEADER_FIELD_NOTIFIER_NAME, action.getNotifierName());
    headers.put(AddHeaderTransformRuntime.HEADER_FIELD_RULE_ID, ruleId);
    transforms.add(new AddHeaderTransform(headers));
    return transforms;
}
Also used : ProjectionTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.ProjectionTransform) AddHeaderTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.AddHeaderTransform) MergeTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.MergeTransform) SubstituteTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.SubstituteTransform) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) Transform(com.hortonworks.streamline.streams.layout.Transform) ProjectionTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.ProjectionTransform) MergeTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.MergeTransform) AddHeaderTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.AddHeaderTransform) SubstituteTransform(com.hortonworks.streamline.streams.layout.component.rule.action.transform.SubstituteTransform)

Aggregations

SubstituteTransform (com.hortonworks.streamline.streams.layout.component.rule.action.transform.SubstituteTransform)3 HashMap (java.util.HashMap)3 StreamlineEvent (com.hortonworks.streamline.streams.StreamlineEvent)2 MergeTransform (com.hortonworks.streamline.streams.layout.component.rule.action.transform.MergeTransform)2 ProjectionTransform (com.hortonworks.streamline.streams.layout.component.rule.action.transform.ProjectionTransform)2 ArrayList (java.util.ArrayList)2 Test (org.junit.Test)2 Result (com.hortonworks.streamline.streams.Result)1 Transform (com.hortonworks.streamline.streams.layout.Transform)1 TransformAction (com.hortonworks.streamline.streams.layout.component.rule.action.TransformAction)1 AddHeaderTransform (com.hortonworks.streamline.streams.layout.component.rule.action.transform.AddHeaderTransform)1 TransformRuntime (com.hortonworks.streamline.streams.runtime.TransformRuntime)1 ActionRuntime (com.hortonworks.streamline.streams.runtime.rule.action.ActionRuntime)1