Search in sources :

Example 6 with SourceConfig

use of org.apache.pulsar.common.io.SourceConfig in project pulsar by apache.

the class SourcesImpl method getSourceAsync.

@Override
public CompletableFuture<SourceConfig> getSourceAsync(String tenant, String namespace, String sourceName) {
    WebTarget path = source.path(tenant).path(namespace).path(sourceName);
    final CompletableFuture<SourceConfig> future = new CompletableFuture<>();
    asyncGetRequest(path, new InvocationCallback<Response>() {

        @Override
        public void completed(Response response) {
            if (response.getStatus() != Response.Status.OK.getStatusCode()) {
                future.completeExceptionally(getApiException(response));
            } else {
                future.complete(response.readEntity(SourceConfig.class));
            }
        }

        @Override
        public void failed(Throwable throwable) {
            future.completeExceptionally(getApiException(throwable.getCause()));
        }
    });
    return future;
}
Also used : Response(javax.ws.rs.core.Response) CompletableFuture(java.util.concurrent.CompletableFuture) SourceConfig(org.apache.pulsar.common.io.SourceConfig) WebTarget(javax.ws.rs.client.WebTarget)

Example 7 with SourceConfig

use of org.apache.pulsar.common.io.SourceConfig in project pulsar by apache.

the class TestCmdSources method testMissingArchive.

@Test(expectedExceptions = ParameterException.class, expectedExceptionsMessageRegExp = "Source archive not specified")
public void testMissingArchive() throws Exception {
    SourceConfig sourceConfig = getSourceConfig();
    sourceConfig.setArchive(null);
    testCmdSourceCliMissingArgs(TENANT, NAMESPACE, NAME, TOPIC_NAME, SERDE_CLASS_NAME, PROCESSING_GUARANTEES, PARALLELISM, null, CPU, RAM, DISK, SINK_CONFIG_STRING, sourceConfig);
}
Also used : BatchSourceConfig(org.apache.pulsar.common.io.BatchSourceConfig) SourceConfig(org.apache.pulsar.common.io.SourceConfig) Test(org.testng.annotations.Test)

Example 8 with SourceConfig

use of org.apache.pulsar.common.io.SourceConfig in project pulsar by apache.

the class TestCmdSources method testCmdSourceConfigFileMissingConfig.

@Test
public void testCmdSourceConfigFileMissingConfig() throws Exception {
    SourceConfig testSourceConfig = getSourceConfig();
    testSourceConfig.setConfigs(null);
    SourceConfig expectedSourceConfig = getSourceConfig();
    expectedSourceConfig.setConfigs(null);
    testCmdSourceConfigFile(testSourceConfig, expectedSourceConfig);
}
Also used : BatchSourceConfig(org.apache.pulsar.common.io.BatchSourceConfig) SourceConfig(org.apache.pulsar.common.io.SourceConfig) Test(org.testng.annotations.Test)

Example 9 with SourceConfig

use of org.apache.pulsar.common.io.SourceConfig in project pulsar by apache.

the class TestCmdSources method testMissingProcessingGuarantees.

@Test
public void testMissingProcessingGuarantees() throws Exception {
    SourceConfig sourceConfig = getSourceConfig();
    sourceConfig.setProcessingGuarantees(null);
    testCmdSourceCliMissingArgs(TENANT, NAMESPACE, NAME, TOPIC_NAME, SERDE_CLASS_NAME, null, PARALLELISM, JAR_FILE_PATH, CPU, RAM, DISK, SINK_CONFIG_STRING, sourceConfig);
}
Also used : BatchSourceConfig(org.apache.pulsar.common.io.BatchSourceConfig) SourceConfig(org.apache.pulsar.common.io.SourceConfig) Test(org.testng.annotations.Test)

Example 10 with SourceConfig

use of org.apache.pulsar.common.io.SourceConfig in project pulsar by apache.

the class TestCmdSources method testCmdSourceConfigFile.

public void testCmdSourceConfigFile(SourceConfig testSourceConfig, SourceConfig expectedSourceConfig) throws Exception {
    File file = Files.createTempFile("", "").toFile();
    new YAMLMapper().writeValue(file, testSourceConfig);
    Assert.assertEquals(testSourceConfig, CmdUtils.loadConfig(file.getAbsolutePath(), SourceConfig.class));
    // test create source
    createSource.sourceConfigFile = file.getAbsolutePath();
    createSource.processArguments();
    createSource.runCmd();
    // test update source
    updateSource.sourceConfigFile = file.getAbsolutePath();
    updateSource.processArguments();
    updateSource.runCmd();
    // test local runner
    localSourceRunner.sourceConfigFile = file.getAbsolutePath();
    localSourceRunner.processArguments();
    localSourceRunner.runCmd();
    verify(createSource).validateSourceConfigs(eq(expectedSourceConfig));
    verify(updateSource).validateSourceConfigs(eq(expectedSourceConfig));
    verify(localSourceRunner).validateSourceConfigs(eq(expectedSourceConfig));
}
Also used : YAMLMapper(com.fasterxml.jackson.dataformat.yaml.YAMLMapper) BatchSourceConfig(org.apache.pulsar.common.io.BatchSourceConfig) SourceConfig(org.apache.pulsar.common.io.SourceConfig) File(java.io.File)

Aggregations

SourceConfig (org.apache.pulsar.common.io.SourceConfig)192 BatchSourceConfig (org.apache.pulsar.common.io.BatchSourceConfig)141 Test (org.testng.annotations.Test)129 Gson (com.google.gson.Gson)42 HashMap (java.util.HashMap)30 File (java.io.File)27 PulsarAdminException (org.apache.pulsar.client.admin.PulsarAdminException)27 Function (org.apache.pulsar.functions.proto.Function)24 IOException (java.io.IOException)21 Response (javax.ws.rs.core.Response)18 List (java.util.List)15 Map (java.util.Map)15 RestException (org.apache.pulsar.common.util.RestException)15 IdentityFunction (org.apache.pulsar.functions.api.utils.IdentityFunction)15 FunctionMetaDataManager (org.apache.pulsar.functions.worker.FunctionMetaDataManager)15 Lists (com.google.common.collect.Lists)12 FileInputStream (java.io.FileInputStream)12 InputStream (java.io.InputStream)12 Files (java.nio.file.Files)12 PulsarAdmin (org.apache.pulsar.client.admin.PulsarAdmin)12