Search in sources :

Example 11 with VolumeChooserEnvironment

use of org.apache.accumulo.server.fs.VolumeChooserEnvironment in project accumulo by apache.

the class ImportTableTest method testTabletDir.

@Test
public void testTabletDir() {
    Master master = EasyMock.createMock(Master.class);
    VolumeManager volumeManager = EasyMock.createMock(VolumeManager.class);
    ImportedTableInfo iti = new ImportedTableInfo();
    iti.tableId = Table.ID.of("5");
    // Different volumes with different paths
    String[] tableDirs = new String[] { "hdfs://nn1:8020/apps/accumulo1/tables", "hdfs://nn2:8020/applications/accumulo/tables" };
    // This needs to be unique WRT the importtable command
    String tabletDir = "/c-00000001";
    EasyMock.expect(master.getFileSystem()).andReturn(volumeManager);
    // Choose the 2nd element
    VolumeChooserEnvironment chooserEnv = new VolumeChooserEnvironment(iti.tableId);
    EasyMock.expect(volumeManager.choose(EasyMock.eq(chooserEnv), EasyMock.eq(tableDirs))).andReturn(tableDirs[1]);
    EasyMock.replay(master, volumeManager);
    PopulateMetadataTable pmt = new PopulateMetadataTable(iti);
    Assert.assertEquals(tableDirs[1] + "/" + iti.tableId + "/" + tabletDir, pmt.getClonedTabletDir(master, tableDirs, tabletDir));
    EasyMock.verify(master, volumeManager);
}
Also used : Master(org.apache.accumulo.master.Master) VolumeManager(org.apache.accumulo.server.fs.VolumeManager) VolumeChooserEnvironment(org.apache.accumulo.server.fs.VolumeChooserEnvironment) Test(org.junit.Test)

Example 12 with VolumeChooserEnvironment

use of org.apache.accumulo.server.fs.VolumeChooserEnvironment in project accumulo by apache.

the class PopulateMetadataTable method getClonedTabletDir.

/**
 * Given options for tables (across multiple volumes), construct an absolute path using the unique name within the chosen volume
 *
 * @return An absolute, unique path for the imported table
 */
protected String getClonedTabletDir(Master master, String[] tableDirs, String tabletDir) {
    // We can try to spread out the tablet dirs across all volumes
    VolumeChooserEnvironment chooserEnv = new VolumeChooserEnvironment(tableInfo.tableId);
    String tableDir = master.getFileSystem().choose(chooserEnv, tableDirs);
    // Build up a full hdfs://localhost:8020/accumulo/tables/$id/c-XXXXXXX
    return tableDir + "/" + tableInfo.tableId + "/" + tabletDir;
}
Also used : VolumeChooserEnvironment(org.apache.accumulo.server.fs.VolumeChooserEnvironment)

Aggregations

VolumeChooserEnvironment (org.apache.accumulo.server.fs.VolumeChooserEnvironment)12 IOException (java.io.IOException)6 Path (org.apache.hadoop.fs.Path)5 Mutation (org.apache.accumulo.core.data.Mutation)4 Value (org.apache.accumulo.core.data.Value)4 BatchWriter (org.apache.accumulo.core.client.BatchWriter)3 Scanner (org.apache.accumulo.core.client.Scanner)3 Key (org.apache.accumulo.core.data.Key)3 VolumeManager (org.apache.accumulo.server.fs.VolumeManager)3 Text (org.apache.hadoop.io.Text)3 AccumuloSecurityException (org.apache.accumulo.core.client.AccumuloSecurityException)2 BatchWriterConfig (org.apache.accumulo.core.client.BatchWriterConfig)2 Connector (org.apache.accumulo.core.client.Connector)2 PartialKey (org.apache.accumulo.core.data.PartialKey)2 KeyExtent (org.apache.accumulo.core.data.impl.KeyExtent)2 DataFileValue (org.apache.accumulo.core.metadata.schema.DataFileValue)2 DataOutputStream (java.io.DataOutputStream)1 EOFException (java.io.EOFException)1 FileNotFoundException (java.io.FileNotFoundException)1 OutputStream (java.io.OutputStream)1