Search in sources :

Example 1 with BasicOutputFormatProvider

use of io.cdap.cdap.etl.batch.BasicOutputFormatProvider in project cdap by caskdata.

the class SparkBatchSinkContext method addOutput.

@Override
public void addOutput(Output output) {
    Output actualOutput = suffixOutput(getOutput(output));
    // Wrap the output provider with tracking counter for metrics collection via MR counter.
    if (actualOutput instanceof Output.OutputFormatProviderOutput) {
        OutputFormatProvider provider = ((Output.OutputFormatProviderOutput) actualOutput).getOutputFormatProvider();
        Map<String, String> conf = new HashMap<>(provider.getOutputFormatConfiguration());
        conf.put(TrackingOutputFormat.DELEGATE_CLASS_NAME, provider.getOutputFormatClassName());
        provider = new BasicOutputFormatProvider(TrackingOutputFormat.class.getName(), conf);
        actualOutput = Output.of(actualOutput.getName(), provider).alias(actualOutput.getAlias());
    }
    sinkFactory.addOutput(getStageName(), actualOutput);
}
Also used : HashMap(java.util.HashMap) Output(io.cdap.cdap.api.data.batch.Output) BasicOutputFormatProvider(io.cdap.cdap.etl.batch.BasicOutputFormatProvider) BasicOutputFormatProvider(io.cdap.cdap.etl.batch.BasicOutputFormatProvider) OutputFormatProvider(io.cdap.cdap.api.data.batch.OutputFormatProvider) NullOutputFormatProvider(io.cdap.cdap.etl.batch.preview.NullOutputFormatProvider)

Aggregations

Output (io.cdap.cdap.api.data.batch.Output)1 OutputFormatProvider (io.cdap.cdap.api.data.batch.OutputFormatProvider)1 BasicOutputFormatProvider (io.cdap.cdap.etl.batch.BasicOutputFormatProvider)1 NullOutputFormatProvider (io.cdap.cdap.etl.batch.preview.NullOutputFormatProvider)1 HashMap (java.util.HashMap)1