Search in sources :

Example 16 with SinkContext

use of org.apache.pulsar.io.core.SinkContext in project pulsar by apache.

the class KinesisSinkConfigTests method loadFromMapCredentialFromSecretTest.

@Test
public final void loadFromMapCredentialFromSecretTest() throws IOException {
    Map<String, Object> map = new HashMap<String, Object>();
    map.put("awsEndpoint", "https://some.endpoint.aws");
    map.put("awsRegion", "us-east-1");
    map.put("awsKinesisStreamName", "my-stream");
    SinkContext sinkContext = Mockito.mock(SinkContext.class);
    Mockito.when(sinkContext.getSecret("awsCredentialPluginParam")).thenReturn("{\"accessKey\":\"myKey\",\"secretKey\":\"my-Secret\"}");
    KinesisSinkConfig config = IOConfigUtils.loadWithSecrets(map, KinesisSinkConfig.class, sinkContext);
    assertNotNull(config);
    assertEquals(config.getAwsEndpoint(), "https://some.endpoint.aws");
    assertEquals(config.getAwsRegion(), "us-east-1");
    assertEquals(config.getAwsKinesisStreamName(), "my-stream");
    assertEquals(config.getAwsCredentialPluginParam(), "{\"accessKey\":\"myKey\",\"secretKey\":\"my-Secret\"}");
}
Also used : HashMap(java.util.HashMap) SinkContext(org.apache.pulsar.io.core.SinkContext) Test(org.testng.annotations.Test)

Example 17 with SinkContext

use of org.apache.pulsar.io.core.SinkContext in project pulsar-io-cloud-storage by streamnative.

the class ConnectorConfigTest method loadFromMapCredentialFromSecretTest.

@Test
public final void loadFromMapCredentialFromSecretTest() throws IOException {
    Map<String, Object> config = new HashMap<>();
    config.put("provider", "aws-s3");
    config.put("bucket", "testbucket");
    config.put("region", "localhost");
    config.put("endpoint", "us-standard");
    config.put("formatType", "avro");
    config.put("partitionerType", "default");
    config.put("timePartitionPattern", "yyyy-MM-dd");
    config.put("timePartitionDuration", "2d");
    config.put("batchSize", 10);
    SinkContext sinkContext = Mockito.mock(SinkContext.class);
    Mockito.when(sinkContext.getSecret("accessKeyId")).thenReturn("myKeyId");
    Mockito.when(sinkContext.getSecret("secretAccessKey")).thenReturn("myAccessKey");
    CloudStorageSinkConfig sinkConfig = IOConfigUtils.loadWithSecrets(config, CloudStorageSinkConfig.class, sinkContext);
    Assert.assertNotNull(sinkConfig);
    Assert.assertEquals(sinkConfig.getProvider(), "aws-s3");
    Assert.assertEquals(sinkConfig.getBucket(), "testbucket");
    Assert.assertEquals(sinkConfig.getSecretAccessKey(), "myAccessKey");
    Assert.assertEquals(sinkConfig.getAccessKeyId(), "myKeyId");
}
Also used : CloudStorageSinkConfig(org.apache.pulsar.io.jcloud.sink.CloudStorageSinkConfig) HashMap(java.util.HashMap) SinkContext(org.apache.pulsar.io.core.SinkContext) Test(org.junit.Test)

Aggregations

SinkContext (org.apache.pulsar.io.core.SinkContext)17 Test (org.testng.annotations.Test)16 HashMap (java.util.HashMap)10 ByteBuffer (java.nio.ByteBuffer)3 KafkaAbstractSink (org.apache.pulsar.io.kafka.KafkaAbstractSink)3 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)2 AtomicLong (java.util.concurrent.atomic.AtomicLong)2 Message (org.apache.pulsar.client.api.Message)2 GenericObject (org.apache.pulsar.client.api.schema.GenericObject)2 SinkConfig (org.apache.pulsar.common.io.SinkConfig)2 Record (org.apache.pulsar.functions.api.Record)2 GetRecordsResponse (software.amazon.awssdk.services.kinesis.model.GetRecordsResponse)2 CloudStorageSinkConfig (org.apache.pulsar.io.jcloud.sink.CloudStorageSinkConfig)1 Test (org.junit.Test)1