Search in sources :

Example 31 with Stoppable

use of org.apache.hadoop.hbase.Stoppable in project hbase by apache.

the class TestCleanerChore method testSavesFilesOnRequest.

@Test
public void testSavesFilesOnRequest() throws Exception {
    Stoppable stop = new StoppableImplementation();
    Configuration conf = UTIL.getConfiguration();
    Path testDir = UTIL.getDataTestDir();
    FileSystem fs = UTIL.getTestFileSystem();
    String confKey = "hbase.test.cleaner.delegates";
    conf.set(confKey, NeverDelete.class.getName());
    AllValidPaths chore = new AllValidPaths("test-file-cleaner", stop, conf, fs, testDir, confKey, POOL);
    // create the directory layout in the directory to clean
    Path parent = new Path(testDir, "parent");
    Path file = new Path(parent, "someFile");
    fs.mkdirs(parent);
    // touch a new file
    fs.create(file).close();
    assertTrue("Test file didn't get created.", fs.exists(file));
    // run the chore
    chore.chore();
    // verify all the files were preserved
    assertTrue("File shouldn't have been deleted", fs.exists(file));
    assertTrue("directory shouldn't have been deleted", fs.exists(parent));
}
Also used : Path(org.apache.hadoop.fs.Path) Configuration(org.apache.hadoop.conf.Configuration) StoppableImplementation(org.apache.hadoop.hbase.util.StoppableImplementation) FileSystem(org.apache.hadoop.fs.FileSystem) FilterFileSystem(org.apache.hadoop.fs.FilterFileSystem) Stoppable(org.apache.hadoop.hbase.Stoppable) Test(org.junit.Test)

Example 32 with Stoppable

use of org.apache.hadoop.hbase.Stoppable in project hbase by apache.

the class TestSnapshotCleanerChore method testSnapshotChoreWithTtlOutOfRange.

@Test
public void testSnapshotChoreWithTtlOutOfRange() throws IOException {
    snapshotManager = Mockito.mock(SnapshotManager.class);
    Stoppable stopper = new StoppableImplementation();
    Configuration conf = getSnapshotCleanerConf();
    List<SnapshotProtos.SnapshotDescription> snapshotDescriptionList = new ArrayList<>();
    snapshotDescriptionList.add(getSnapshotDescription(Long.MAX_VALUE, "snapshot01", "table01", 1));
    snapshotDescriptionList.add(getSnapshotDescription(5, "snapshot02", "table02", 2));
    Mockito.when(snapshotManager.getCompletedSnapshots()).thenReturn(snapshotDescriptionList);
    SnapshotCleanerChore snapshotCleanerChore = new SnapshotCleanerChore(stopper, conf, snapshotManager);
    try {
        LOG.info("Snapshot Chore is disabled. No cleanup performed for Expired Snapshots");
        snapshotCleanerChore.chore();
    } finally {
        stopper.stop("Stopping Test Stopper");
    }
    Mockito.verify(snapshotManager, Mockito.times(1)).getCompletedSnapshots();
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) ArrayList(java.util.ArrayList) Stoppable(org.apache.hadoop.hbase.Stoppable) SnapshotManager(org.apache.hadoop.hbase.master.snapshot.SnapshotManager) Test(org.junit.Test)

Example 33 with Stoppable

use of org.apache.hadoop.hbase.Stoppable in project hbase by apache.

the class TestSnapshotCleanerChore method testSnapshotCleanerWithSomeTtlExpired.

@Test
public void testSnapshotCleanerWithSomeTtlExpired() throws IOException {
    snapshotManager = Mockito.mock(SnapshotManager.class);
    Stoppable stopper = new StoppableImplementation();
    Configuration conf = getSnapshotCleanerConf();
    SnapshotCleanerChore snapshotCleanerChore = new SnapshotCleanerChore(stopper, conf, snapshotManager);
    List<SnapshotProtos.SnapshotDescription> snapshotDescriptionList = new ArrayList<>();
    snapshotDescriptionList.add(getSnapshotDescription(10, "snapshot01", "table01", 1));
    snapshotDescriptionList.add(getSnapshotDescription(5, "snapshot02", "table02", 2));
    snapshotDescriptionList.add(getSnapshotDescription(30, "snapshot01", "table01", EnvironmentEdgeManager.currentTime()));
    snapshotDescriptionList.add(getSnapshotDescription(0, "snapshot02", "table02", EnvironmentEdgeManager.currentTime()));
    snapshotDescriptionList.add(getSnapshotDescription(40, "snapshot03", "table03", EnvironmentEdgeManager.currentTime()));
    Mockito.when(snapshotManager.getCompletedSnapshots()).thenReturn(snapshotDescriptionList);
    try {
        LOG.info("5 Snapshots are completed. TTL is expired for 2 them. Going to delete them");
        snapshotCleanerChore.chore();
    } finally {
        stopper.stop("Stopping Test Stopper");
    }
    Mockito.verify(snapshotManager, Mockito.times(2)).deleteSnapshot(Mockito.any());
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) ArrayList(java.util.ArrayList) Stoppable(org.apache.hadoop.hbase.Stoppable) SnapshotManager(org.apache.hadoop.hbase.master.snapshot.SnapshotManager) Test(org.junit.Test)

Example 34 with Stoppable

use of org.apache.hadoop.hbase.Stoppable in project hbase by apache.

the class TestSnapshotCleanerChore method testSnapshotCleanerWithNoTtlExpired.

@Test
public void testSnapshotCleanerWithNoTtlExpired() throws IOException {
    snapshotManager = Mockito.mock(SnapshotManager.class);
    Stoppable stopper = new StoppableImplementation();
    Configuration conf = getSnapshotCleanerConf();
    SnapshotCleanerChore snapshotCleanerChore = new SnapshotCleanerChore(stopper, conf, snapshotManager);
    List<SnapshotProtos.SnapshotDescription> snapshotDescriptionList = new ArrayList<>();
    snapshotDescriptionList.add(getSnapshotDescription(-2, "snapshot01", "table01", EnvironmentEdgeManager.currentTime() - 100000));
    snapshotDescriptionList.add(getSnapshotDescription(10, "snapshot02", "table02", EnvironmentEdgeManager.currentTime()));
    Mockito.when(snapshotManager.getCompletedSnapshots()).thenReturn(snapshotDescriptionList);
    try {
        LOG.info("2 Snapshots are completed but TTL is not expired for any of them");
        snapshotCleanerChore.chore();
    } finally {
        stopper.stop("Stopping Test Stopper");
    }
    Mockito.verify(snapshotManager, Mockito.times(0)).deleteSnapshot(Mockito.any());
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) ArrayList(java.util.ArrayList) Stoppable(org.apache.hadoop.hbase.Stoppable) SnapshotManager(org.apache.hadoop.hbase.master.snapshot.SnapshotManager) Test(org.junit.Test)

Aggregations

Stoppable (org.apache.hadoop.hbase.Stoppable)34 Configuration (org.apache.hadoop.conf.Configuration)31 Test (org.junit.Test)31 FileSystem (org.apache.hadoop.fs.FileSystem)16 Path (org.apache.hadoop.fs.Path)16 StoppableImplementation (org.apache.hadoop.hbase.util.StoppableImplementation)15 FilterFileSystem (org.apache.hadoop.fs.FilterFileSystem)11 InvocationOnMock (org.mockito.invocation.InvocationOnMock)8 ArrayList (java.util.ArrayList)7 ExecutorService (java.util.concurrent.ExecutorService)7 Abortable (org.apache.hadoop.hbase.Abortable)7 RegionCoprocessorEnvironment (org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment)7 StubAbortable (org.apache.phoenix.hbase.index.StubAbortable)7 HTableInterface (org.apache.hadoop.hbase.client.HTableInterface)5 Mutation (org.apache.hadoop.hbase.client.Mutation)5 Put (org.apache.hadoop.hbase.client.Put)5 SnapshotManager (org.apache.hadoop.hbase.master.snapshot.SnapshotManager)5 ImmutableBytesPtr (org.apache.phoenix.hbase.index.util.ImmutableBytesPtr)5 CountDownLatch (java.util.concurrent.CountDownLatch)4 HTableDescriptor (org.apache.hadoop.hbase.HTableDescriptor)4