Search in sources :

Example 1 with BatchReadable

use of co.cask.cdap.api.data.batch.BatchReadable in project cdap by caskdata.

the class BasicMapReduceTaskContext method getBatchReadable.

/**
 * Returns a {@link BatchReadable} that reads data from the given dataset.
 */
<K, V> BatchReadable<K, V> getBatchReadable(@Nullable String datasetNamespace, String datasetName, Map<String, String> datasetArgs) {
    Dataset dataset;
    if (datasetNamespace == null) {
        dataset = getDataset(datasetName, datasetArgs, AccessType.READ);
    } else {
        dataset = getDataset(datasetNamespace, datasetName, datasetArgs, AccessType.READ);
    }
    // Must be BatchReadable.
    Preconditions.checkArgument(dataset instanceof BatchReadable, "Dataset '%s' is not a BatchReadable.", datasetName);
    @SuppressWarnings("unchecked") final BatchReadable<K, V> delegate = (BatchReadable<K, V>) dataset;
    return new BatchReadable<K, V>() {

        @Override
        public List<Split> getSplits() {
            try {
                try {
                    return delegate.getSplits();
                } finally {
                    flushOperations();
                }
            } catch (Exception e) {
                throw Throwables.propagate(e);
            }
        }

        @Override
        public SplitReader<K, V> createSplitReader(Split split) {
            return new ForwardingSplitReader<K, V>(delegate.createSplitReader(split)) {

                @Override
                public void close() {
                    try {
                        try {
                            super.close();
                        } finally {
                            flushOperations();
                        }
                    } catch (Exception e) {
                        throw Throwables.propagate(e);
                    }
                }
            };
        }
    };
}
Also used : ForwardingSplitReader(co.cask.cdap.internal.app.runtime.batch.dataset.ForwardingSplitReader) Dataset(co.cask.cdap.api.dataset.Dataset) BatchReadable(co.cask.cdap.api.data.batch.BatchReadable) Split(co.cask.cdap.api.data.batch.Split) DatasetInstantiationException(co.cask.cdap.api.data.DatasetInstantiationException) FileNotFoundException(java.io.FileNotFoundException) TopicNotFoundException(co.cask.cdap.api.messaging.TopicNotFoundException) IOException(java.io.IOException)

Aggregations

DatasetInstantiationException (co.cask.cdap.api.data.DatasetInstantiationException)1 BatchReadable (co.cask.cdap.api.data.batch.BatchReadable)1 Split (co.cask.cdap.api.data.batch.Split)1 Dataset (co.cask.cdap.api.dataset.Dataset)1 TopicNotFoundException (co.cask.cdap.api.messaging.TopicNotFoundException)1 ForwardingSplitReader (co.cask.cdap.internal.app.runtime.batch.dataset.ForwardingSplitReader)1 FileNotFoundException (java.io.FileNotFoundException)1 IOException (java.io.IOException)1