Search in sources :

Example 6 with ActionStatus

use of org.smartdata.actions.ActionStatus in project SSM by Intel-bigdata.

the class TestCacheFile method testCacheFile.

@Test
public void testCacheFile() throws IOException {
    dfs.mkdirs(new Path("/fileTestA"));
    String[] args = { "/fileTestA" };
    CacheFileAction cacheAction = new CacheFileAction();
    cacheAction.setContext(smartContext);
    cacheAction.setDfsClient(dfsClient);
    cacheAction.init(args);
    ActionStatus actionStatus = cacheAction.getActionStatus();
    try {
        Assert.assertEquals(false, cacheAction.isCached(args[0]));
        cacheAction.run();
        Assert.assertEquals(true, cacheAction.isCached(args[0]));
        Assert.assertTrue(actionStatus.isFinished());
        Assert.assertTrue(actionStatus.isSuccessful());
        System.out.println("Cache action running time : " + StringUtils.formatTime(actionStatus.getRunningTime()));
        Assert.assertEquals(1.0f, actionStatus.getPercentage(), 0.00001f);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}
Also used : Path(org.apache.hadoop.fs.Path) IOException(java.io.IOException) ActionStatus(org.smartdata.actions.ActionStatus) Test(org.junit.Test)

Example 7 with ActionStatus

use of org.smartdata.actions.ActionStatus in project SSM by Intel-bigdata.

the class WriteFileAction method execute.

@Override
protected void execute() {
    ActionStatus actionStatus = getActionStatus();
    actionStatus.begin();
    try {
        final OutputStream out = dfsClient.create(filePath, true);
        // generate random data with given length
        byte[] buffer = new byte[bufferSize];
        new Random().nextBytes(buffer);
        // write to HDFS
        for (int pos = 0; pos < length; pos += bufferSize) {
            int writeLength = pos + bufferSize < length ? bufferSize : length - pos;
            out.write(buffer, 0, writeLength);
        }
        out.close();
        actionStatus.setSuccessful(true);
    } catch (IOException e) {
        actionStatus.setSuccessful(false);
        resultOut.println("WriteFile Action fails!\n" + e.getMessage());
    } finally {
        actionStatus.end();
    }
}
Also used : Random(java.util.Random) OutputStream(java.io.OutputStream) IOException(java.io.IOException) ActionStatus(org.smartdata.actions.ActionStatus)

Example 8 with ActionStatus

use of org.smartdata.actions.ActionStatus in project SSM by Intel-bigdata.

the class CheckStorageAction method execute.

@Override
protected void execute() {
    ActionStatus actionStatus = getActionStatus();
    actionStatus.begin();
    try {
        HdfsFileStatus fileStatus = dfsClient.getFileInfo(fileName);
        long length = fileStatus.getLen();
        BlockLocation[] blockLocations = dfsClient.getBlockLocations(fileName, 0, length);
        for (BlockLocation blockLocation : blockLocations) {
            StringBuilder hosts = new StringBuilder();
            hosts.append("{");
            for (String host : blockLocation.getHosts()) {
                hosts.append(host + " ");
            }
            hosts.append("}");
            this.resultOut.println(hosts);
        }
        actionStatus.setSuccessful(true);
    } catch (Exception e) {
        actionStatus.setSuccessful(false);
        throw new RuntimeException(e);
    } finally {
        actionStatus.end();
    }
}
Also used : HdfsFileStatus(org.apache.hadoop.hdfs.protocol.HdfsFileStatus) BlockLocation(org.apache.hadoop.fs.BlockLocation) ActionStatus(org.smartdata.actions.ActionStatus)

Example 9 with ActionStatus

use of org.smartdata.actions.ActionStatus in project SSM by Intel-bigdata.

the class CacheFileAction method execute.

protected void execute() {
    ActionStatus actionStatus = getActionStatus();
    actionStatus.begin();
    try {
        addActionEvent(fileName);
        executeCacheAction(fileName);
        actionStatus.setSuccessful(true);
    } catch (Exception e) {
        actionStatus.setSuccessful(false);
        throw new RuntimeException(e);
    } finally {
        actionStatus.end();
    }
}
Also used : ActionStatus(org.smartdata.actions.ActionStatus)

Example 10 with ActionStatus

use of org.smartdata.actions.ActionStatus in project SSM by Intel-bigdata.

the class ReadFileAction method execute.

@Override
protected void execute() {
    ActionStatus actionStatus = getActionStatus();
    actionStatus.begin();
    try {
        HdfsFileStatus fileStatus = dfsClient.getFileInfo(filePath);
        if (fileStatus == null) {
            resultOut.println("ReadFile Action fails, file doesn't exist!");
        }
        DFSInputStream dfsInputStream = dfsClient.open(filePath);
        byte[] buffer = new byte[bufferSize];
        // read from HDFS
        while (dfsInputStream.read(buffer, 0, bufferSize) != -1) {
        }
        dfsInputStream.close();
        actionStatus.setSuccessful(true);
    } catch (IOException e) {
        actionStatus.setSuccessful(false);
        resultOut.println("ReadFile Action fails!\n" + e.getMessage());
    } finally {
        actionStatus.end();
    }
}
Also used : HdfsFileStatus(org.apache.hadoop.hdfs.protocol.HdfsFileStatus) DFSInputStream(org.apache.hadoop.hdfs.DFSInputStream) IOException(java.io.IOException) ActionStatus(org.smartdata.actions.ActionStatus)

Aggregations

ActionStatus (org.smartdata.actions.ActionStatus)17 Test (org.junit.Test)7 Path (org.apache.hadoop.fs.Path)5 FSDataOutputStream (org.apache.hadoop.fs.FSDataOutputStream)4 IOException (java.io.IOException)3 StorageType (org.apache.hadoop.fs.StorageType)3 HdfsFileStatus (org.apache.hadoop.hdfs.protocol.HdfsFileStatus)3 LocatedBlock (org.apache.hadoop.hdfs.protocol.LocatedBlock)3 OutputStream (java.io.OutputStream)2 SmartAction (org.smartdata.actions.SmartAction)2 ActionInfo (org.smartdata.common.actions.ActionInfo)2 ArrayList (java.util.ArrayList)1 Random (java.util.Random)1 ByteArrayOutputStream (org.apache.commons.io.output.ByteArrayOutputStream)1 BlockLocation (org.apache.hadoop.fs.BlockLocation)1 DFSInputStream (org.apache.hadoop.hdfs.DFSInputStream)1 MoverStatus (org.smartdata.actions.hdfs.move.MoverStatus)1 ActionInfoComparator (org.smartdata.common.actions.ActionInfoComparator)1