Search in sources :

Example 11 with CmdletManager

use of org.smartdata.server.engine.CmdletManager in project SSM by Intel-bigdata.

the class TestCompressDecompress method testCheckCompressAction.

@Test
public void testCheckCompressAction() throws Exception {
    int arraySize = 1024 * 1024 * 8;
    String fileDir = "/ssm/compression/";
    String fileName = "file5";
    String filePath = fileDir + fileName;
    prepareFile(filePath, arraySize);
    CmdletManager cmdletManager = ssm.getCmdletManager();
    long cmdId = cmdletManager.submitCmdlet("checkcompress -file " + filePath);
    waitTillActionDone(cmdId);
    // Test directory case.
    cmdId = cmdletManager.submitCmdlet("checkcompress -file " + fileDir);
    waitTillActionDone(cmdId);
}
Also used : CmdletManager(org.smartdata.server.engine.CmdletManager) Test(org.junit.Test)

Example 12 with CmdletManager

use of org.smartdata.server.engine.CmdletManager in project SSM by Intel-bigdata.

the class TestCompressDecompress method testCompressDecompressDir.

@Test
public void testCompressDecompressDir() throws Exception {
    String dir = "/ssm/compression";
    dfsClient.mkdirs(dir, null, true);
    CmdletManager cmdletManager = ssm.getCmdletManager();
    List<ActionScheduler> schedulers = cmdletManager.getSchedulers("decompress");
    Assert.assertTrue(schedulers.size() == 1);
    ActionScheduler scheduler = schedulers.get(0);
    Assert.assertTrue(scheduler instanceof CompressionScheduler);
    // Expect that a dir cannot be compressed.
    Assert.assertFalse(((CompressionScheduler) scheduler).supportCompression(dir));
    // Expect that a dir cannot be decompressed.
    Assert.assertFalse(((CompressionScheduler) scheduler).supportDecompression(dir));
}
Also used : CmdletManager(org.smartdata.server.engine.CmdletManager) ActionScheduler(org.smartdata.model.action.ActionScheduler) CompressionScheduler(org.smartdata.hdfs.scheduler.CompressionScheduler) Test(org.junit.Test)

Example 13 with CmdletManager

use of org.smartdata.server.engine.CmdletManager in project SSM by Intel-bigdata.

the class TestSmallFileScheduler method testScheduler.

@Test(timeout = 180000)
public void testScheduler() throws Exception {
    waitTillSSMExitSafeMode();
    Thread.sleep(2000);
    CmdletManager cmdletManager = ssm.getCmdletManager();
    long cmdId = cmdletManager.submitCmdlet("compact -file " + "['/test/small_files/file_0','/test/small_files/file_1'] " + "-containerFile /test/small_files/container_file_2");
    while (true) {
        Thread.sleep(3000);
        CmdletState state = cmdletManager.getCmdletInfo(cmdId).getState();
        if (state == CmdletState.DONE) {
            long containerFileLen = dfsClient.getFileInfo("/test/small_files/container_file_2").getLen();
            Assert.assertEquals(sumFileLen, containerFileLen);
            Assert.assertEquals(0, dfsClient.getFileInfo("/test/small_files/file_1").getLen());
            return;
        } else if (state == CmdletState.FAILED) {
            Assert.fail("Compact failed.");
        } else {
            System.out.println(state);
        }
    }
}
Also used : CmdletManager(org.smartdata.server.engine.CmdletManager) CmdletState(org.smartdata.model.CmdletState) Test(org.junit.Test)

Aggregations

CmdletManager (org.smartdata.server.engine.CmdletManager)13 Test (org.junit.Test)11 Path (org.apache.hadoop.fs.Path)5 IOException (java.io.IOException)3 CmdletState (org.smartdata.model.CmdletState)3 ActionScheduler (org.smartdata.model.action.ActionScheduler)3 DFSInputStream (org.apache.hadoop.hdfs.DFSInputStream)2 HdfsFileStatus (org.apache.hadoop.hdfs.protocol.HdfsFileStatus)2 SmartDFSClient (org.smartdata.hdfs.client.SmartDFSClient)2 CompressionScheduler (org.smartdata.hdfs.scheduler.CompressionScheduler)2 CompressionFileState (org.smartdata.model.CompressionFileState)2 FileState (org.smartdata.model.FileState)2 Random (java.util.Random)1 BlockLocation (org.apache.hadoop.fs.BlockLocation)1 FSDataOutputStream (org.apache.hadoop.fs.FSDataOutputStream)1 LocatedFileStatus (org.apache.hadoop.fs.LocatedFileStatus)1 DistributedFileSystem (org.apache.hadoop.hdfs.DistributedFileSystem)1 CachePoolEntry (org.apache.hadoop.hdfs.protocol.CachePoolEntry)1 AbstractService (org.smartdata.AbstractService)1 SmartFileSystem (org.smartdata.hadoop.filesystem.SmartFileSystem)1