Search in sources :

Example 1 with TransformAction

use of alluxio.table.common.transform.action.TransformAction in project alluxio by Alluxio.

the class TransformDefinition method parse.

/**
 * @param definition the string definition
 * @return the {@link TransformDefinition} representation
 */
public static TransformDefinition parse(String definition) {
    definition = definition.trim();
    if (definition.isEmpty()) {
        return new TransformDefinition(definition, Collections.emptyList(), new Properties());
    }
    // accept semicolon as new lines for inline definitions
    definition = definition.replace(";", "\n");
    final Properties properties = new Properties();
    final StringReader reader = new StringReader(definition);
    try {
        properties.load(reader);
    } catch (IOException e) {
        // The only way this throws an IOException is if the definition is null which isn't possible.
        return new TransformDefinition(definition, Collections.emptyList(), properties);
    }
    final List<TransformAction> actions = TransformActionRegistry.create(properties);
    return new TransformDefinition(definition, actions, properties);
}
Also used : StringReader(java.io.StringReader) TransformAction(alluxio.table.common.transform.action.TransformAction) IOException(java.io.IOException) Properties(java.util.Properties)

Example 2 with TransformAction

use of alluxio.table.common.transform.action.TransformAction in project alluxio by Alluxio.

the class TransformPlan method computeJobConfigs.

private ArrayList<JobConfig> computeJobConfigs(TransformDefinition definition) {
    ArrayList<JobConfig> actions = new ArrayList<>();
    Layout baseLayout = mBaseLayout;
    boolean deleteSrc = false;
    for (TransformAction action : definition.getActions()) {
        actions.add(action.generateJobConfig(baseLayout, mTransformedLayout, deleteSrc));
        baseLayout = mTransformedLayout;
        deleteSrc = true;
    }
    if (actions.isEmpty()) {
        throw new IllegalArgumentException("At least one action should be defined for the transformation");
    }
    return actions;
}
Also used : Layout(alluxio.table.common.Layout) ArrayList(java.util.ArrayList) TransformAction(alluxio.table.common.transform.action.TransformAction) JobConfig(alluxio.job.JobConfig)

Aggregations

TransformAction (alluxio.table.common.transform.action.TransformAction)2 JobConfig (alluxio.job.JobConfig)1 Layout (alluxio.table.common.Layout)1 IOException (java.io.IOException)1 StringReader (java.io.StringReader)1 ArrayList (java.util.ArrayList)1 Properties (java.util.Properties)1