Search in sources :

Example 11 with BatchWriterOpts

use of org.apache.accumulo.examples.cli.BatchWriterOpts in project accumulo-examples by apache.

the class FileDataIngest method main.

public static void main(String[] args) throws Exception {
    Opts opts = new Opts();
    BatchWriterOpts bwOpts = new BatchWriterOpts();
    opts.parseArgs(FileDataIngest.class.getName(), args, bwOpts);
    Connector conn = opts.getConnector();
    if (!conn.tableOperations().exists(opts.getTableName())) {
        conn.tableOperations().create(opts.getTableName());
        conn.tableOperations().attachIterator(opts.getTableName(), new IteratorSetting(1, ChunkCombiner.class));
    }
    BatchWriter bw = conn.createBatchWriter(opts.getTableName(), bwOpts.getBatchWriterConfig());
    FileDataIngest fdi = new FileDataIngest(opts.chunkSize, opts.visibility);
    for (String filename : opts.files) {
        fdi.insertFileData(filename, bw);
    }
    bw.close();
// TODO
// opts.stopTracing();
}
Also used : Connector(org.apache.accumulo.core.client.Connector) IteratorSetting(org.apache.accumulo.core.client.IteratorSetting) BatchWriterOpts(org.apache.accumulo.examples.cli.BatchWriterOpts) BatchWriterOpts(org.apache.accumulo.examples.cli.BatchWriterOpts) BatchWriter(org.apache.accumulo.core.client.BatchWriter)

Aggregations

BatchWriterOpts (org.apache.accumulo.examples.cli.BatchWriterOpts)11 BatchWriter (org.apache.accumulo.core.client.BatchWriter)8 Connector (org.apache.accumulo.core.client.Connector)7 Scanner (org.apache.accumulo.core.client.Scanner)4 Mutation (org.apache.accumulo.core.data.Mutation)4 ScannerOpts (org.apache.accumulo.examples.cli.ScannerOpts)4 Value (org.apache.accumulo.core.data.Value)3 ClientOpts (org.apache.accumulo.examples.cli.ClientOpts)3 Text (org.apache.hadoop.io.Text)3 File (java.io.File)2 IteratorSetting (org.apache.accumulo.core.client.IteratorSetting)2 Key (org.apache.accumulo.core.data.Key)2 ColumnVisibility (org.apache.accumulo.core.security.ColumnVisibility)2 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 HashSet (java.util.HashSet)1 Entry (java.util.Map.Entry)1 Random (java.util.Random)1 Set (java.util.Set)1 IsolatedScanner (org.apache.accumulo.core.client.IsolatedScanner)1