Search in sources :

Example 1 with FileSplitPartitionQuery

use of org.apache.gora.query.impl.FileSplitPartitionQuery in project gora by apache.

the class FileBackedDataStoreBase method getPartitions.

@Override
public List<PartitionQuery<K, T>> getPartitions(Query<K, T> query) {
    List<InputSplit> splits = null;
    List<PartitionQuery<K, T>> queries = null;
    try {
        splits = GoraMapReduceUtils.getSplits(getConf(), inputPath);
        queries = new ArrayList<>(splits.size());
        for (InputSplit split : splits) {
            queries.add(new FileSplitPartitionQuery<>(query, (FileSplit) split));
        }
    } catch (IOException ex) {
        LOG.error(ex.getMessage(), ex);
    }
    return queries;
}
Also used : IOException(java.io.IOException) FileSplit(org.apache.hadoop.mapreduce.lib.input.FileSplit) InputSplit(org.apache.hadoop.mapreduce.InputSplit) PartitionQuery(org.apache.gora.query.PartitionQuery) FileSplitPartitionQuery(org.apache.gora.query.impl.FileSplitPartitionQuery)

Example 2 with FileSplitPartitionQuery

use of org.apache.gora.query.impl.FileSplitPartitionQuery in project gora by apache.

the class GoraInputFormat method setInputPath.

@SuppressWarnings({ "rawtypes" })
private void setInputPath(PartitionQuery<K, T> partitionQuery) throws IOException {
    //if the data store is file based
    if (partitionQuery instanceof FileSplitPartitionQuery) {
        FileSplit split = ((FileSplitPartitionQuery<K, T>) partitionQuery).getSplit();
        //set the input path to FileSplit's path.
        ((FileBackedDataStore) partitionQuery.getDataStore()).setInputPath(split.getPath().toString());
    }
}
Also used : FileBackedDataStore(org.apache.gora.store.FileBackedDataStore) FileSplitPartitionQuery(org.apache.gora.query.impl.FileSplitPartitionQuery) FileSplit(org.apache.hadoop.mapreduce.lib.input.FileSplit)

Aggregations

FileSplitPartitionQuery (org.apache.gora.query.impl.FileSplitPartitionQuery)2 FileSplit (org.apache.hadoop.mapreduce.lib.input.FileSplit)2 IOException (java.io.IOException)1 PartitionQuery (org.apache.gora.query.PartitionQuery)1 FileBackedDataStore (org.apache.gora.store.FileBackedDataStore)1 InputSplit (org.apache.hadoop.mapreduce.InputSplit)1