Search in sources :

Example 26 with ProcessBundleDescriptor

use of org.apache.beam.model.fnexecution.v1.BeamFnApi.ProcessBundleDescriptor in project beam by apache.

the class SdkHarnessClientTest method testRegisterCachesBundleProcessors.

@Test
public void testRegisterCachesBundleProcessors() throws Exception {
    ProcessBundleDescriptor descriptor1 = ProcessBundleDescriptor.newBuilder().setId("descriptor1").build();
    ProcessBundleDescriptor descriptor2 = ProcessBundleDescriptor.newBuilder().setId("descriptor2").build();
    List<RemoteInputDestination> remoteInputs = Collections.singletonList(RemoteInputDestination.of((FullWindowedValueCoder) FullWindowedValueCoder.of(VarIntCoder.of(), GlobalWindow.Coder.INSTANCE), SDK_GRPC_READ_TRANSFORM));
    BundleProcessor processor1 = sdkHarnessClient.getProcessor(descriptor1, remoteInputs);
    BundleProcessor processor2 = sdkHarnessClient.getProcessor(descriptor2, remoteInputs);
    assertNotSame(processor1, processor2);
    // Ensure that caching works.
    assertSame(processor1, sdkHarnessClient.getProcessor(descriptor1, remoteInputs));
}
Also used : FullWindowedValueCoder(org.apache.beam.sdk.util.WindowedValue.FullWindowedValueCoder) RemoteInputDestination(org.apache.beam.runners.fnexecution.data.RemoteInputDestination) ProcessBundleDescriptor(org.apache.beam.model.fnexecution.v1.BeamFnApi.ProcessBundleDescriptor) BundleProcessor(org.apache.beam.runners.fnexecution.control.SdkHarnessClient.BundleProcessor) Test(org.junit.Test)

Aggregations

ProcessBundleDescriptor (org.apache.beam.model.fnexecution.v1.BeamFnApi.ProcessBundleDescriptor)21 Test (org.junit.Test)19 BeamFnApi (org.apache.beam.model.fnexecution.v1.BeamFnApi)18 ShortIdMap (org.apache.beam.runners.core.metrics.ShortIdMap)14 ByteString (org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.ByteString)14 PTransformRunnerFactory (org.apache.beam.fn.harness.PTransformRunnerFactory)13 BundleProcessorCache (org.apache.beam.fn.harness.control.ProcessBundleHandler.BundleProcessorCache)13 Map (java.util.Map)8 RunnerApi (org.apache.beam.model.pipeline.v1.RunnerApi)8 IOException (java.io.IOException)7 BeamFnStateClient (org.apache.beam.fn.harness.state.BeamFnStateClient)6 Elements (org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements)6 InstructionRequest (org.apache.beam.model.fnexecution.v1.BeamFnApi.InstructionRequest)6 InstructionResponse (org.apache.beam.model.fnexecution.v1.BeamFnApi.InstructionResponse)6 ImmutableMap (org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.ImmutableMap)6 Collections (java.util.Collections)5 List (java.util.List)5 Iterables (org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.Iterables)5 ArrayList (java.util.ArrayList)4 FunctionSpec (org.apache.beam.model.pipeline.v1.RunnerApi.FunctionSpec)4