Search in sources :

Example 31 with ApplicationManager

use of co.cask.cdap.test.ApplicationManager in project cdap by caskdata.

the class ServiceArtifactTestRun method testServiceArtifact.

@Test
public void testServiceArtifact() throws Exception {
    ApplicationManager appManager = deployWithArtifact(ServiceArtifactApp.class, artifactJar);
    ServiceManager serviceManager = appManager.getServiceManager("artifact").start();
    URL serviceURL = serviceManager.getServiceURL(30, TimeUnit.SECONDS);
    Assert.assertNotNull(serviceURL);
    URL listURL = serviceURL.toURI().resolve("list").toURL();
    try (Reader reader = new InputStreamReader(listURL.openStream(), StandardCharsets.UTF_8)) {
        List<ArtifactInfo> artifacts = new Gson().fromJson(reader, new TypeToken<List<ArtifactInfo>>() {
        }.getType());
        // It should have the test app, and two plugin artifacts
        Assert.assertEquals(3, artifacts.size());
        Assert.assertTrue(artifacts.stream().anyMatch(info -> info.getName().equals(ServiceArtifactApp.class.getSimpleName())));
        Assert.assertTrue(artifacts.stream().anyMatch(info -> info.getName().equals("dummybase")));
        Assert.assertTrue(artifacts.stream().anyMatch(info -> info.getName().equals("dummy")));
    }
    URL loadURL = serviceURL.toURI().resolve("load?parent=dummybase&plugin=dummy&class=" + DummyPlugin.class.getName()).toURL();
    HttpURLConnection urlConn = (HttpURLConnection) loadURL.openConnection();
    Assert.assertEquals(200, urlConn.getResponseCode());
    try (Reader reader = new InputStreamReader(urlConn.getInputStream(), StandardCharsets.UTF_8)) {
        Assert.assertEquals(DummyPlugin.class.getName(), CharStreams.toString(reader));
    }
    serviceManager.stop();
    serviceManager.waitForStatus(false);
}
Also used : HttpURLConnection(java.net.HttpURLConnection) NamespaceId(co.cask.cdap.proto.id.NamespaceId) TypeToken(com.google.gson.reflect.TypeToken) BeforeClass(org.junit.BeforeClass) URL(java.net.URL) JarEntry(java.util.jar.JarEntry) CharStreams(com.google.common.io.CharStreams) Gson(com.google.gson.Gson) ClassRule(org.junit.ClassRule) JarOutputStream(java.util.jar.JarOutputStream) Constants(co.cask.cdap.common.conf.Constants) TestFrameworkTestBase(co.cask.cdap.test.base.TestFrameworkTestBase) Service(co.cask.cdap.api.service.Service) ArtifactInfo(co.cask.cdap.api.artifact.ArtifactInfo) ApplicationManager(co.cask.cdap.test.ApplicationManager) FileOutputStream(java.io.FileOutputStream) Test(org.junit.Test) IOException(java.io.IOException) Reader(java.io.Reader) TestConfiguration(co.cask.cdap.test.TestConfiguration) InputStreamReader(java.io.InputStreamReader) File(java.io.File) StandardCharsets(java.nio.charset.StandardCharsets) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) ByteStreams(com.google.common.io.ByteStreams) ArtifactManager(co.cask.cdap.api.artifact.ArtifactManager) ServiceManager(co.cask.cdap.test.ServiceManager) Assert(org.junit.Assert) ApplicationManager(co.cask.cdap.test.ApplicationManager) InputStreamReader(java.io.InputStreamReader) Reader(java.io.Reader) InputStreamReader(java.io.InputStreamReader) Gson(com.google.gson.Gson) URL(java.net.URL) HttpURLConnection(java.net.HttpURLConnection) ArtifactInfo(co.cask.cdap.api.artifact.ArtifactInfo) ServiceManager(co.cask.cdap.test.ServiceManager) TypeToken(com.google.gson.reflect.TypeToken) Test(org.junit.Test)

Example 32 with ApplicationManager

use of co.cask.cdap.test.ApplicationManager in project cdap by caskdata.

the class BatchStreamIntegrationTestRun method submitAndVerifyStreamBatchJob.

private void submitAndVerifyStreamBatchJob(Class<? extends AbstractApplication> appClass, String streamWriter, String mapReduceName, int timeout) throws Exception {
    ApplicationManager applicationManager = deployApplication(appClass);
    StreamManager streamManager = getStreamManager(streamWriter);
    verifyStreamBatchJob(streamManager, applicationManager, mapReduceName, timeout);
}
Also used : ApplicationManager(co.cask.cdap.test.ApplicationManager) StreamManager(co.cask.cdap.test.StreamManager)

Example 33 with ApplicationManager

use of co.cask.cdap.test.ApplicationManager in project cdap by caskdata.

the class MapReduceServiceIntegrationTestRun method test.

@Test
public void test() throws Exception {
    ApplicationManager applicationManager = deployApplication(TestMapReduceServiceIntegrationApp.class);
    ServiceManager serviceManager = applicationManager.getServiceManager(TestMapReduceServiceIntegrationApp.SERVICE_NAME).start();
    serviceManager.waitForStatus(true);
    DataSetManager<MyKeyValueTableDefinition.KeyValueTable> inDataSet = getDataset(TestMapReduceServiceIntegrationApp.INPUT_DATASET);
    inDataSet.get().write("key1", "Two words");
    inDataSet.get().write("key2", "Plus three words");
    inDataSet.flush();
    MapReduceManager mrManager = applicationManager.getMapReduceManager(TestMapReduceServiceIntegrationApp.MR_NAME).start();
    mrManager.waitForRun(ProgramRunStatus.COMPLETED, 180, TimeUnit.SECONDS);
    DataSetManager<MyKeyValueTableDefinition.KeyValueTable> outDataSet = getDataset(TestMapReduceServiceIntegrationApp.OUTPUT_DATASET);
    MyKeyValueTableDefinition.KeyValueTable results = outDataSet.get();
    String total = results.get(TestMapReduceServiceIntegrationApp.SQUARED_TOTAL_WORDS_COUNT);
    Assert.assertEquals(25, Integer.parseInt(total));
}
Also used : ApplicationManager(co.cask.cdap.test.ApplicationManager) MapReduceManager(co.cask.cdap.test.MapReduceManager) ServiceManager(co.cask.cdap.test.ServiceManager) MyKeyValueTableDefinition(co.cask.cdap.test.app.MyKeyValueTableDefinition) Test(org.junit.Test)

Example 34 with ApplicationManager

use of co.cask.cdap.test.ApplicationManager in project cdap by caskdata.

the class SparkMetricsIntegrationTestRun method testSparkMetrics.

@Test
public void testSparkMetrics() throws Exception {
    ApplicationManager applicationManager = deployApplication(TestSparkMetricsIntegrationApp.class);
    SparkManager sparkManager = applicationManager.getSparkManager(TestSparkMetricsIntegrationApp.APP_SPARK_NAME).start();
    sparkManager.waitForRun(ProgramRunStatus.COMPLETED, 120, TimeUnit.SECONDS);
    List<RunRecord> history = sparkManager.getHistory(ProgramRunStatus.COMPLETED);
    Assert.assertEquals(1, history.size());
    // Wait for the metrics to get updated
    Tasks.waitFor(true, new Callable<Boolean>() {

        @Override
        public Boolean call() throws Exception {
            return getSparkMetric(TestSparkMetricsIntegrationApp.APP_NAME, TestSparkMetricsIntegrationApp.APP_SPARK_NAME, "system.driver.BlockManager.memory.remainingMem_MB") > 0;
        }
    }, 10, TimeUnit.SECONDS, 100, TimeUnit.MILLISECONDS);
    Tasks.waitFor(2L, new Callable<Long>() {

        @Override
        public Long call() throws Exception {
            return getSparkMetric(TestSparkMetricsIntegrationApp.APP_NAME, TestSparkMetricsIntegrationApp.APP_SPARK_NAME, "user.more.than.30");
        }
    }, 10, TimeUnit.SECONDS, 100, TimeUnit.MILLISECONDS);
}
Also used : RunRecord(co.cask.cdap.proto.RunRecord) ApplicationManager(co.cask.cdap.test.ApplicationManager) SparkManager(co.cask.cdap.test.SparkManager) Test(org.junit.Test)

Example 35 with ApplicationManager

use of co.cask.cdap.test.ApplicationManager in project cdap by caskdata.

the class SparkStreamIntegrationTestRun method testSparkWithStream.

@Test
public void testSparkWithStream() throws Exception {
    ApplicationManager applicationManager = deployApplication(TestSparkStreamIntegrationApp.class);
    StreamManager streamManager = getStreamManager("testStream");
    for (int i = 0; i < 50; i++) {
        streamManager.send(String.valueOf(i));
    }
    SparkManager sparkManager = applicationManager.getSparkManager("SparkStreamProgram").start();
    sparkManager.waitForRun(ProgramRunStatus.COMPLETED, 120, TimeUnit.SECONDS);
    // The Spark job simply turns every stream event body into key/value pairs, with key==value.
    DataSetManager<KeyValueTable> datasetManager = getDataset("result");
    verifyDatasetResult(datasetManager);
}
Also used : ApplicationManager(co.cask.cdap.test.ApplicationManager) SparkManager(co.cask.cdap.test.SparkManager) StreamManager(co.cask.cdap.test.StreamManager) KeyValueTable(co.cask.cdap.api.dataset.lib.KeyValueTable) Test(org.junit.Test)

Aggregations

ApplicationManager (co.cask.cdap.test.ApplicationManager)188 Test (org.junit.Test)155 KeyValueTable (co.cask.cdap.api.dataset.lib.KeyValueTable)88 ApplicationId (co.cask.cdap.proto.id.ApplicationId)71 AppRequest (co.cask.cdap.proto.artifact.AppRequest)61 WorkflowManager (co.cask.cdap.test.WorkflowManager)59 ETLStage (co.cask.cdap.etl.proto.v2.ETLStage)58 SparkManager (co.cask.cdap.test.SparkManager)52 Table (co.cask.cdap.api.dataset.table.Table)50 ServiceManager (co.cask.cdap.test.ServiceManager)48 StructuredRecord (co.cask.cdap.api.data.format.StructuredRecord)47 Schema (co.cask.cdap.api.data.schema.Schema)47 ETLBatchConfig (co.cask.cdap.etl.proto.v2.ETLBatchConfig)45 StreamManager (co.cask.cdap.test.StreamManager)43 URL (java.net.URL)33 HashSet (java.util.HashSet)27 ArrayList (java.util.ArrayList)26 IOException (java.io.IOException)25 HashMap (java.util.HashMap)24 Set (java.util.Set)24