Search in sources :

Example 1 with GoraInputSplit

use of org.apache.gora.mapreduce.GoraInputSplit in project gora by apache.

the class PigGoraInputFormat method getSplits.

@Override
public List<InputSplit> getSplits(JobContext context) throws IOException, InterruptedException {
    List<PartitionQuery<K, T>> partitionsQueries = this.dataStore.getPartitions(query);
    List<InputSplit> splits = new ArrayList<>(partitionsQueries.size());
    for (PartitionQuery<K, T> queryForSplit : partitionsQueries) {
        splits.add(new GoraInputSplit(context.getConfiguration(), queryForSplit));
    }
    return splits;
}
Also used : GoraInputSplit(org.apache.gora.mapreduce.GoraInputSplit) ArrayList(java.util.ArrayList) PartitionQuery(org.apache.gora.query.PartitionQuery) InputSplit(org.apache.hadoop.mapreduce.InputSplit) GoraInputSplit(org.apache.gora.mapreduce.GoraInputSplit)

Aggregations

ArrayList (java.util.ArrayList)1 GoraInputSplit (org.apache.gora.mapreduce.GoraInputSplit)1 PartitionQuery (org.apache.gora.query.PartitionQuery)1 InputSplit (org.apache.hadoop.mapreduce.InputSplit)1