Search in sources :

Example 6 with Sink

use of com.hazelcast.jet.pipeline.Sink in project hazelcast by hazelcast.

the class CdcSinks method sink.

@Nonnull
private static <K, V> Sink<ChangeRecord> sink(@Nonnull String name, @Nonnull String map, @Nullable ClientConfig clientConfig, @Nonnull FunctionEx<? super ChangeRecord, ? extends K> keyFn, @Nonnull FunctionEx<? super ChangeRecord, ? extends V> valueFn) {
    FunctionEx<? super ChangeRecord, ? extends V> toValueFn = record -> DELETE.equals(record.operation()) ? null : valueFn.apply(record);
    String clientXml = asXmlString(clientConfig);
    ProcessorSupplier supplier = AbstractHazelcastConnectorSupplier.ofMap(clientXml, procFn(name, map, clientXml, keyFn, toValueFn));
    ProcessorMetaSupplier metaSupplier = ProcessorMetaSupplier.of(mapUpdatePermission(clientXml, name), supplier);
    return new SinkImpl<>(name, metaSupplier, DISTRIBUTED_PARTITIONED, keyFn);
}
Also used : HazelcastInstance(com.hazelcast.core.HazelcastInstance) FunctionEx(com.hazelcast.function.FunctionEx) DELETE(com.hazelcast.jet.cdc.Operation.DELETE) ProcessorMetaSupplier(com.hazelcast.jet.core.ProcessorMetaSupplier) Processor(com.hazelcast.jet.core.Processor) DISTRIBUTED_PARTITIONED(com.hazelcast.jet.impl.pipeline.SinkImpl.Type.DISTRIBUTED_PARTITIONED) Collections.singletonList(java.util.Collections.singletonList) PermissionsUtil.mapUpdatePermission(com.hazelcast.security.PermissionsUtil.mapUpdatePermission) List(java.util.List) Permission(java.security.Permission) AbstractHazelcastConnectorSupplier(com.hazelcast.jet.impl.connector.AbstractHazelcastConnectorSupplier) HazelcastProperty(com.hazelcast.spi.properties.HazelcastProperty) ClientConfig(com.hazelcast.client.config.ClientConfig) ImdgUtil.asXmlString(com.hazelcast.jet.impl.util.ImdgUtil.asXmlString) Nonnull(javax.annotation.Nonnull) ProcessorSupplier(com.hazelcast.jet.core.ProcessorSupplier) SinkImpl(com.hazelcast.jet.impl.pipeline.SinkImpl) Sink(com.hazelcast.jet.pipeline.Sink) Nullable(javax.annotation.Nullable) SECONDS(java.util.concurrent.TimeUnit.SECONDS) WriteCdcP(com.hazelcast.jet.cdc.impl.WriteCdcP) IMap(com.hazelcast.map.IMap) SinkImpl(com.hazelcast.jet.impl.pipeline.SinkImpl) ImdgUtil.asXmlString(com.hazelcast.jet.impl.util.ImdgUtil.asXmlString) ProcessorSupplier(com.hazelcast.jet.core.ProcessorSupplier) ProcessorMetaSupplier(com.hazelcast.jet.core.ProcessorMetaSupplier) Nonnull(javax.annotation.Nonnull)

Example 7 with Sink

use of com.hazelcast.jet.pipeline.Sink in project hazelcast by hazelcast.

the class ManagedContextTest method testSinks.

private void testSinks(SupplierEx<? extends AnotherSinkContext> sinkSupplier) {
    Sink<Object> sink = SinkBuilder.sinkBuilder("sink", c -> sinkSupplier.get()).receiveFn((c, i) -> assertEquals(INJECTED_VALUE, c.injectedValue)).build();
    Pipeline pipeline = Pipeline.create();
    pipeline.readFrom(TestSources.items(1)).writeTo(sink);
    hz.getJet().newJob(pipeline).join();
}
Also used : Config(com.hazelcast.config.Config) HazelcastInstance(com.hazelcast.core.HazelcastInstance) FunctionEx(com.hazelcast.function.FunctionEx) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) ManagedContext(com.hazelcast.core.ManagedContext) BatchSource(com.hazelcast.jet.pipeline.BatchSource) Pipeline(com.hazelcast.jet.pipeline.Pipeline) QuickTest(com.hazelcast.test.annotation.QuickTest) RunWith(org.junit.runner.RunWith) Test(org.junit.Test) Category(org.junit.experimental.categories.Category) SupplierEx(com.hazelcast.function.SupplierEx) Collections.singletonList(java.util.Collections.singletonList) Sources(com.hazelcast.jet.pipeline.Sources) AssertionSinks.assertAnyOrder(com.hazelcast.jet.pipeline.test.AssertionSinks.assertAnyOrder) TestSources(com.hazelcast.jet.pipeline.test.TestSources) HazelcastParallelClassRunner(com.hazelcast.test.HazelcastParallelClassRunner) ServiceFactory(com.hazelcast.jet.pipeline.ServiceFactory) SinkBuilder(com.hazelcast.jet.pipeline.SinkBuilder) SourceBuilder(com.hazelcast.jet.pipeline.SourceBuilder) ServiceFactories(com.hazelcast.jet.pipeline.ServiceFactories) Sink(com.hazelcast.jet.pipeline.Sink) Assert.assertEquals(org.junit.Assert.assertEquals) Before(org.junit.Before) Pipeline(com.hazelcast.jet.pipeline.Pipeline)

Aggregations

Sink (com.hazelcast.jet.pipeline.Sink)7 Pipeline (com.hazelcast.jet.pipeline.Pipeline)5 Nonnull (javax.annotation.Nonnull)5 HazelcastInstance (com.hazelcast.core.HazelcastInstance)4 FunctionEx (com.hazelcast.function.FunctionEx)3 SupplierEx (com.hazelcast.function.SupplierEx)3 BatchSource (com.hazelcast.jet.pipeline.BatchSource)3 List (java.util.List)3 IntStream (java.util.stream.IntStream)3 DAG (com.hazelcast.jet.core.DAG)2 ADAPT_TO_JET_EVENT (com.hazelcast.jet.impl.pipeline.ComputeStageImplBase.ADAPT_TO_JET_EVENT)2 BatchSourceTransform (com.hazelcast.jet.impl.pipeline.transform.BatchSourceTransform)2 SinkTransform (com.hazelcast.jet.impl.pipeline.transform.SinkTransform)2 StreamSourceTransform (com.hazelcast.jet.impl.pipeline.transform.StreamSourceTransform)2 Transform (com.hazelcast.jet.impl.pipeline.transform.Transform)2 BatchStage (com.hazelcast.jet.pipeline.BatchStage)2 GeneralStage (com.hazelcast.jet.pipeline.GeneralStage)2 SinkBuilder (com.hazelcast.jet.pipeline.SinkBuilder)2 SinkStage (com.hazelcast.jet.pipeline.SinkStage)2 SourceBuilder (com.hazelcast.jet.pipeline.SourceBuilder)2