Search in sources :

Example 6 with ShareLibService

use of org.apache.oozie.service.ShareLibService in project oozie by apache.

the class TestOozieSharelibCLI method testOozieSharelibCLICreateConcurrent.

/**
 * test parallel copy libraries
 */
public void testOozieSharelibCLICreateConcurrent() throws Exception {
    final int testFiles = 7;
    final int concurrency = 5;
    File libDirectory = tmpFolder.newFolder("lib");
    for (int i = 0; i < testFiles; i++) {
        writeFile(libDirectory, generateFileName(i), generateFileContent(i));
    }
    String[] argsCreate = { "create", "-fs", outPath, "-locallib", libDirectory.getParentFile().getAbsolutePath(), "-concurrency", String.valueOf(concurrency) };
    assertEquals(0, execOozieSharelibCLICommands(argsCreate));
    getTargetFileSysyem();
    ShareLibService sharelibService = getServices().get(ShareLibService.class);
    Path latestLibPath = sharelibService.getLatestLibPath(getDistPath(), ShareLibService.SHARE_LIB_PREFIX);
    for (int i = 0; i < testFiles; i++) {
        String fileName = generateFileName(i);
        String expectedFileContent = generateFileContent(i);
        InputStream in = null;
        try {
            in = fs.open(new Path(latestLibPath, fileName));
            String actualFileContent = IOUtils.toString(in);
            assertEquals(fileName, expectedFileContent, actualFileContent);
        } finally {
            IOUtils.closeQuietly(in);
        }
    }
}
Also used : Path(org.apache.hadoop.fs.Path) InputStream(java.io.InputStream) ShareLibService(org.apache.oozie.service.ShareLibService) File(java.io.File)

Aggregations

ShareLibService (org.apache.oozie.service.ShareLibService)6 Path (org.apache.hadoop.fs.Path)5 IOException (java.io.IOException)3 File (java.io.File)2 FileSystem (org.apache.hadoop.fs.FileSystem)2 ActionExecutorException (org.apache.oozie.action.ActionExecutorException)2 JSONObject (org.json.simple.JSONObject)2 InputStream (java.io.InputStream)1 URI (java.net.URI)1 URISyntaxException (java.net.URISyntaxException)1 HashSet (java.util.HashSet)1 List (java.util.List)1 Pattern (java.util.regex.Pattern)1 ServletException (javax.servlet.ServletException)1 Configuration (org.apache.hadoop.conf.Configuration)1 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)1 CommandException (org.apache.oozie.command.CommandException)1 AuthorizationException (org.apache.oozie.service.AuthorizationException)1 XConfiguration (org.apache.oozie.util.XConfiguration)1 JSONArray (org.json.simple.JSONArray)1