Search in sources :

Example 1 with AccumuloKeyRangePartitioner

use of uk.gov.gchq.gaffer.sparkaccumulo.operation.utils.AccumuloKeyRangePartitioner in project Gaffer by gchq.

the class AbstractImportKeyValuePairRDDToAccumuloHandler method doOperation.

public void doOperation(final T operation, final Context context, final AccumuloStore store) throws OperationException {
    final String outputPath = getOutputPath(operation);
    if (null == outputPath || outputPath.isEmpty()) {
        throw new OperationException("Option outputPath must be set for this option to be run against the accumulostore");
    }
    final String failurePath = getFailurePath(operation);
    if (null == failurePath || failurePath.isEmpty()) {
        throw new OperationException("Option failurePath must be set for this option to be run against the accumulostore");
    }
    prepareKeyValues(operation, new AccumuloKeyRangePartitioner(store));
    final ImportAccumuloKeyValueFiles importAccumuloKeyValueFiles = new ImportAccumuloKeyValueFiles.Builder().inputPath(outputPath).failurePath(failurePath).build();
    store._execute(new OperationChain<>(importAccumuloKeyValueFiles), context);
}
Also used : ImportAccumuloKeyValueFiles(uk.gov.gchq.gaffer.accumulostore.operation.hdfs.operation.ImportAccumuloKeyValueFiles) AccumuloKeyRangePartitioner(uk.gov.gchq.gaffer.sparkaccumulo.operation.utils.AccumuloKeyRangePartitioner) OperationException(uk.gov.gchq.gaffer.operation.OperationException)

Example 2 with AccumuloKeyRangePartitioner

use of uk.gov.gchq.gaffer.sparkaccumulo.operation.utils.AccumuloKeyRangePartitioner in project Gaffer by gchq.

the class AbstractImportKeyValuePairRDDToAccumuloHandler method doOperation.

public void doOperation(final OP operation, final Context context, final AccumuloStore store) throws OperationException {
    final String outputPath = getOutputPath(operation);
    if (null == outputPath || outputPath.isEmpty()) {
        throw new OperationException("Option outputPath must be set for this option to be run against the accumulostore");
    }
    final String failurePath = getFailurePath(operation);
    if (null == failurePath || failurePath.isEmpty()) {
        throw new OperationException("Option failurePath must be set for this option to be run against the accumulostore");
    }
    prepareKeyValues(operation, new AccumuloKeyRangePartitioner(store));
    final ImportAccumuloKeyValueFiles importAccumuloKeyValueFiles = new ImportAccumuloKeyValueFiles.Builder().inputPath(outputPath).failurePath(failurePath).build();
    store.execute(new OperationChain<>(importAccumuloKeyValueFiles), context);
}
Also used : ImportAccumuloKeyValueFiles(uk.gov.gchq.gaffer.accumulostore.operation.hdfs.operation.ImportAccumuloKeyValueFiles) AccumuloKeyRangePartitioner(uk.gov.gchq.gaffer.sparkaccumulo.operation.utils.AccumuloKeyRangePartitioner) OperationException(uk.gov.gchq.gaffer.operation.OperationException)

Aggregations

ImportAccumuloKeyValueFiles (uk.gov.gchq.gaffer.accumulostore.operation.hdfs.operation.ImportAccumuloKeyValueFiles)2 OperationException (uk.gov.gchq.gaffer.operation.OperationException)2 AccumuloKeyRangePartitioner (uk.gov.gchq.gaffer.sparkaccumulo.operation.utils.AccumuloKeyRangePartitioner)2