Search in sources :

Example 1 with StreamsContext

use of com.ibm.streamsx.topology.context.StreamsContext in project streamsx.kafka by IBMStreams.

the class KafkaOperatorsBlobTypeTest method kafkaBlobTypeTest.

@Test
public void kafkaBlobTypeTest() throws Exception {
    Topology topo = getTopology();
    StreamSchema schema = KafkaSPLStreamsUtils.BLOB_SCHEMA;
    // create the producer (produces tuples after a short delay)
    TStream<Blob> srcStream = topo.strings(Constants.STRING_DATA).transform(s -> ValueFactory.newBlob(s.getBytes())).modify(new Delay<>(5000));
    SPLStream splSrcStream = SPLStreams.convertStream(srcStream, new Converter(), schema);
    SPL.invokeSink(Constants.KafkaProducerOp, splSrcStream, getKafkaParams());
    // create the consumer
    SPLStream consumerStream = SPL.invokeSource(topo, Constants.KafkaConsumerOp, getKafkaParams(), schema);
    SPLStream msgStream = SPLStreams.stringToSPLStream(consumerStream.convert(t -> new String(t.getBlob("message").getData())));
    // test the output of the consumer
    StreamsContext<?> context = StreamsContextFactory.getStreamsContext(Type.DISTRIBUTED_TESTER);
    Tester tester = topo.getTester();
    Condition<List<String>> condition = KafkaSPLStreamsUtils.stringContentsUnordered(tester, msgStream, Constants.STRING_DATA);
    tester.complete(context, new HashMap<>(), condition, 30, TimeUnit.SECONDS);
    // check the results
    Assert.assertTrue(condition.getResult().size() > 0);
    Assert.assertTrue(condition.getResult().toString(), condition.valid());
}
Also used : TStream(com.ibm.streamsx.topology.TStream) Tester(com.ibm.streamsx.topology.tester.Tester) Delay(com.ibm.streamsx.kafka.test.utils.Delay) BiFunction(com.ibm.streamsx.topology.function.BiFunction) StreamsContextFactory(com.ibm.streamsx.topology.context.StreamsContextFactory) SPLStream(com.ibm.streamsx.topology.spl.SPLStream) HashMap(java.util.HashMap) Test(org.junit.Test) StreamSchema(com.ibm.streams.operator.StreamSchema) OutputTuple(com.ibm.streams.operator.OutputTuple) KafkaSPLStreamsUtils(com.ibm.streamsx.kafka.test.utils.KafkaSPLStreamsUtils) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Topology(com.ibm.streamsx.topology.Topology) StreamsContext(com.ibm.streamsx.topology.context.StreamsContext) Constants(com.ibm.streamsx.kafka.test.utils.Constants) Map(java.util.Map) SPL(com.ibm.streamsx.topology.spl.SPL) Condition(com.ibm.streamsx.topology.tester.Condition) Type(com.ibm.streamsx.topology.context.StreamsContext.Type) SPLStreams(com.ibm.streamsx.topology.spl.SPLStreams) Blob(com.ibm.streams.operator.types.Blob) Assert(org.junit.Assert) ValueFactory(com.ibm.streams.operator.types.ValueFactory) Blob(com.ibm.streams.operator.types.Blob) Tester(com.ibm.streamsx.topology.tester.Tester) Topology(com.ibm.streamsx.topology.Topology) StreamSchema(com.ibm.streams.operator.StreamSchema) SPLStream(com.ibm.streamsx.topology.spl.SPLStream) List(java.util.List) Test(org.junit.Test)

Example 2 with StreamsContext

use of com.ibm.streamsx.topology.context.StreamsContext in project streamsx.kafka by IBMStreams.

the class KafkaOperatorsDoubleTypeTest method kafkaDoubleTypeTest.

@Test
public void kafkaDoubleTypeTest() throws Exception {
    Topology topo = getTopology();
    StreamSchema schema = KafkaSPLStreamsUtils.DOUBLE_SCHEMA;
    // create the producer (produces tuples after a short delay)
    TStream<Double> srcStream = topo.strings(DATA).transform(s -> Double.valueOf(s)).modify(new Delay<>(5000));
    SPLStream splSrcStream = SPLStreams.convertStream(srcStream, new Converter(), schema);
    SPL.invokeSink(Constants.KafkaProducerOp, splSrcStream, getKafkaParams());
    // create the consumer
    SPLStream consumerStream = SPL.invokeSource(topo, Constants.KafkaConsumerOp, getKafkaParams(), schema);
    SPLStream msgStream = SPLStreams.stringToSPLStream(consumerStream.convert(t -> String.valueOf(t.getDouble("message"))));
    // test the output of the consumer
    StreamsContext<?> context = StreamsContextFactory.getStreamsContext(Type.DISTRIBUTED_TESTER);
    Tester tester = topo.getTester();
    Condition<List<String>> condition = KafkaSPLStreamsUtils.stringContentsUnordered(tester, msgStream, DATA);
    tester.complete(context, new HashMap<>(), condition, 30, TimeUnit.SECONDS);
    // check the results
    Assert.assertTrue(condition.getResult().size() > 0);
    Assert.assertTrue(condition.getResult().toString(), condition.valid());
}
Also used : TStream(com.ibm.streamsx.topology.TStream) Tester(com.ibm.streamsx.topology.tester.Tester) Delay(com.ibm.streamsx.kafka.test.utils.Delay) BiFunction(com.ibm.streamsx.topology.function.BiFunction) StreamsContextFactory(com.ibm.streamsx.topology.context.StreamsContextFactory) SPLStream(com.ibm.streamsx.topology.spl.SPLStream) HashMap(java.util.HashMap) Test(org.junit.Test) StreamSchema(com.ibm.streams.operator.StreamSchema) OutputTuple(com.ibm.streams.operator.OutputTuple) KafkaSPLStreamsUtils(com.ibm.streamsx.kafka.test.utils.KafkaSPLStreamsUtils) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Topology(com.ibm.streamsx.topology.Topology) StreamsContext(com.ibm.streamsx.topology.context.StreamsContext) Constants(com.ibm.streamsx.kafka.test.utils.Constants) Map(java.util.Map) SPL(com.ibm.streamsx.topology.spl.SPL) Condition(com.ibm.streamsx.topology.tester.Condition) Type(com.ibm.streamsx.topology.context.StreamsContext.Type) SPLStreams(com.ibm.streamsx.topology.spl.SPLStreams) Assert(org.junit.Assert) Tester(com.ibm.streamsx.topology.tester.Tester) Topology(com.ibm.streamsx.topology.Topology) StreamSchema(com.ibm.streams.operator.StreamSchema) SPLStream(com.ibm.streamsx.topology.spl.SPLStream) List(java.util.List) Test(org.junit.Test)

Example 3 with StreamsContext

use of com.ibm.streamsx.topology.context.StreamsContext in project streamsx.kafka by IBMStreams.

the class KafkaOperatorsFloatTypeTest method kafkaFloatTypeTest.

@Test
public void kafkaFloatTypeTest() throws Exception {
    Topology topo = getTopology();
    StreamSchema schema = KafkaSPLStreamsUtils.FLOAT_SCHEMA;
    // create the producer (produces tuples after a short delay)
    TStream<Float> srcStream = topo.strings(DATA).transform(s -> Float.valueOf(s)).modify(new Delay<>(5000));
    SPLStream splSrcStream = SPLStreams.convertStream(srcStream, new Converter(), schema);
    SPL.invokeSink(Constants.KafkaProducerOp, splSrcStream, getKafkaParams());
    // create the consumer
    SPLStream consumerStream = SPL.invokeSource(topo, Constants.KafkaConsumerOp, getKafkaParams(), schema);
    SPLStream msgStream = SPLStreams.stringToSPLStream(consumerStream.convert(t -> String.valueOf(t.getFloat("message"))));
    // test the output of the consumer
    StreamsContext<?> context = StreamsContextFactory.getStreamsContext(Type.DISTRIBUTED_TESTER);
    Tester tester = topo.getTester();
    Condition<List<String>> condition = KafkaSPLStreamsUtils.stringContentsUnordered(tester, msgStream, DATA);
    tester.complete(context, new HashMap<>(), condition, 30, TimeUnit.SECONDS);
    // check the results
    Assert.assertTrue(condition.getResult().size() > 0);
    Assert.assertTrue(condition.getResult().toString(), condition.valid());
}
Also used : TStream(com.ibm.streamsx.topology.TStream) Tester(com.ibm.streamsx.topology.tester.Tester) Delay(com.ibm.streamsx.kafka.test.utils.Delay) BiFunction(com.ibm.streamsx.topology.function.BiFunction) StreamsContextFactory(com.ibm.streamsx.topology.context.StreamsContextFactory) SPLStream(com.ibm.streamsx.topology.spl.SPLStream) HashMap(java.util.HashMap) Test(org.junit.Test) StreamSchema(com.ibm.streams.operator.StreamSchema) OutputTuple(com.ibm.streams.operator.OutputTuple) KafkaSPLStreamsUtils(com.ibm.streamsx.kafka.test.utils.KafkaSPLStreamsUtils) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Topology(com.ibm.streamsx.topology.Topology) StreamsContext(com.ibm.streamsx.topology.context.StreamsContext) Constants(com.ibm.streamsx.kafka.test.utils.Constants) Map(java.util.Map) SPL(com.ibm.streamsx.topology.spl.SPL) Condition(com.ibm.streamsx.topology.tester.Condition) Type(com.ibm.streamsx.topology.context.StreamsContext.Type) SPLStreams(com.ibm.streamsx.topology.spl.SPLStreams) Assert(org.junit.Assert) Tester(com.ibm.streamsx.topology.tester.Tester) Topology(com.ibm.streamsx.topology.Topology) StreamSchema(com.ibm.streams.operator.StreamSchema) SPLStream(com.ibm.streamsx.topology.spl.SPLStream) List(java.util.List) Test(org.junit.Test)

Example 4 with StreamsContext

use of com.ibm.streamsx.topology.context.StreamsContext in project streamsx.kafka by IBMStreams.

the class KafkaOperatorsGreenThread method kafkaGreenThread.

@Test
public void kafkaGreenThread() throws Exception {
    Topology topo = getTopology();
    // create the producer (produces tuples after a short delay)
    TStream<String> stringSrcStream = topo.strings(Constants.STRING_DATA).modify(new Delay<>(5000));
    SPL.invokeSink(Constants.KafkaProducerOp, KafkaSPLStreamsUtils.convertStreamToKafkaTuple(stringSrcStream), getKafkaParams());
    // create the consumer
    SPLStream consumerStream = SPL.invokeSource(topo, Constants.KafkaConsumerOp, getKafkaParams(), KafkaSPLStreamsUtils.STRING_SCHEMA);
    SPLStream msgStream = SPLStreams.stringToSPLStream(consumerStream.convert(t -> t.getString("message")));
    // test the output of the consumer
    StreamsContext<?> context = StreamsContextFactory.getStreamsContext(Type.DISTRIBUTED_TESTER);
    Tester tester = topo.getTester();
    Condition<List<String>> condition = KafkaSPLStreamsUtils.stringContentsUnordered(tester, msgStream, Constants.STRING_DATA);
    tester.complete(context, new HashMap<>(), condition, 30, TimeUnit.SECONDS);
    // check the results
    Assert.assertTrue(condition.getResult().size() > 0);
    Assert.assertTrue(condition.getResult().toString(), condition.valid());
}
Also used : TStream(com.ibm.streamsx.topology.TStream) Tester(com.ibm.streamsx.topology.tester.Tester) Delay(com.ibm.streamsx.kafka.test.utils.Delay) StreamsContextFactory(com.ibm.streamsx.topology.context.StreamsContextFactory) SPLStream(com.ibm.streamsx.topology.spl.SPLStream) HashMap(java.util.HashMap) Test(org.junit.Test) KafkaSPLStreamsUtils(com.ibm.streamsx.kafka.test.utils.KafkaSPLStreamsUtils) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Topology(com.ibm.streamsx.topology.Topology) StreamsContext(com.ibm.streamsx.topology.context.StreamsContext) Constants(com.ibm.streamsx.kafka.test.utils.Constants) Map(java.util.Map) SPL(com.ibm.streamsx.topology.spl.SPL) Condition(com.ibm.streamsx.topology.tester.Condition) Type(com.ibm.streamsx.topology.context.StreamsContext.Type) SPLStreams(com.ibm.streamsx.topology.spl.SPLStreams) Assert(org.junit.Assert) Tester(com.ibm.streamsx.topology.tester.Tester) List(java.util.List) Topology(com.ibm.streamsx.topology.Topology) SPLStream(com.ibm.streamsx.topology.spl.SPLStream) Test(org.junit.Test)

Example 5 with StreamsContext

use of com.ibm.streamsx.topology.context.StreamsContext in project streamsx.kafka by IBMStreams.

the class KafkaOperatorsLongTypeTest method kafkaLongTypeTest.

@Test
public void kafkaLongTypeTest() throws Exception {
    Topology topo = getTopology();
    StreamSchema schema = KafkaSPLStreamsUtils.LONG_SCHEMA;
    // create the producer (produces tuples after a short delay)
    TStream<Long> srcStream = topo.strings(DATA).transform(s -> Long.valueOf(s)).modify(new Delay<>(5000));
    SPLStream splSrcStream = SPLStreams.convertStream(srcStream, new Converter(), schema);
    SPL.invokeSink(Constants.KafkaProducerOp, splSrcStream, getKafkaParams());
    // create the consumer
    SPLStream consumerStream = SPL.invokeSource(topo, Constants.KafkaConsumerOp, getKafkaParams(), schema);
    SPLStream msgStream = SPLStreams.stringToSPLStream(consumerStream.convert(t -> String.valueOf(t.getLong("message"))));
    // test the output of the consumer
    StreamsContext<?> context = StreamsContextFactory.getStreamsContext(Type.DISTRIBUTED_TESTER);
    Tester tester = topo.getTester();
    Condition<List<String>> condition = KafkaSPLStreamsUtils.stringContentsUnordered(tester, msgStream, DATA);
    tester.complete(context, new HashMap<>(), condition, 30, TimeUnit.SECONDS);
    // check the results
    Assert.assertTrue(condition.getResult().size() > 0);
    Assert.assertTrue(condition.getResult().toString(), condition.valid());
}
Also used : TStream(com.ibm.streamsx.topology.TStream) Tester(com.ibm.streamsx.topology.tester.Tester) Delay(com.ibm.streamsx.kafka.test.utils.Delay) BiFunction(com.ibm.streamsx.topology.function.BiFunction) StreamsContextFactory(com.ibm.streamsx.topology.context.StreamsContextFactory) SPLStream(com.ibm.streamsx.topology.spl.SPLStream) HashMap(java.util.HashMap) Test(org.junit.Test) StreamSchema(com.ibm.streams.operator.StreamSchema) OutputTuple(com.ibm.streams.operator.OutputTuple) KafkaSPLStreamsUtils(com.ibm.streamsx.kafka.test.utils.KafkaSPLStreamsUtils) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Topology(com.ibm.streamsx.topology.Topology) StreamsContext(com.ibm.streamsx.topology.context.StreamsContext) Constants(com.ibm.streamsx.kafka.test.utils.Constants) Map(java.util.Map) SPL(com.ibm.streamsx.topology.spl.SPL) Condition(com.ibm.streamsx.topology.tester.Condition) Type(com.ibm.streamsx.topology.context.StreamsContext.Type) SPLStreams(com.ibm.streamsx.topology.spl.SPLStreams) Assert(org.junit.Assert) Tester(com.ibm.streamsx.topology.tester.Tester) Topology(com.ibm.streamsx.topology.Topology) StreamSchema(com.ibm.streams.operator.StreamSchema) SPLStream(com.ibm.streamsx.topology.spl.SPLStream) List(java.util.List) Test(org.junit.Test)

Aggregations

StreamsContext (com.ibm.streamsx.topology.context.StreamsContext)21 Topology (com.ibm.streamsx.topology.Topology)17 HashMap (java.util.HashMap)17 Test (org.junit.Test)16 Constants (com.ibm.streamsx.kafka.test.utils.Constants)12 KafkaSPLStreamsUtils (com.ibm.streamsx.kafka.test.utils.KafkaSPLStreamsUtils)12 TStream (com.ibm.streamsx.topology.TStream)12 Type (com.ibm.streamsx.topology.context.StreamsContext.Type)12 StreamsContextFactory (com.ibm.streamsx.topology.context.StreamsContextFactory)12 SPL (com.ibm.streamsx.topology.spl.SPL)12 SPLStream (com.ibm.streamsx.topology.spl.SPLStream)12 SPLStreams (com.ibm.streamsx.topology.spl.SPLStreams)12 Condition (com.ibm.streamsx.topology.tester.Condition)12 Tester (com.ibm.streamsx.topology.tester.Tester)12 List (java.util.List)12 Map (java.util.Map)12 TimeUnit (java.util.concurrent.TimeUnit)12 Assert (org.junit.Assert)12 Delay (com.ibm.streamsx.kafka.test.utils.Delay)11 BigInteger (java.math.BigInteger)7