Search in sources :

Example 46 with StateSpec

use of org.apache.beam.sdk.state.StateSpec in project beam by apache.

the class DoFnSignaturesSplittableDoFnTest method testSplittableProcessElementMustNotHaveUnsupportedParams.

@Test
// used via reflection
@SuppressWarnings("unused")
public void testSplittableProcessElementMustNotHaveUnsupportedParams() throws Exception {
    thrown.expect(IllegalArgumentException.class);
    thrown.expectMessage("Illegal parameter");
    thrown.expectMessage("ValueState");
    DoFn<Integer, String> doFn = new DoFn<Integer, String>() {

        @StateId("my-state-id")
        public final StateSpec<ValueState<String>> myStateSpec = StateSpecs.value(StringUtf8Coder.of());

        @ProcessElement
        public void method(DoFn<Integer, String>.ProcessContext context, SomeRestrictionTracker tracker, @StateId("my-state-id") ValueState<String> myState) {
        }
    };
    Method processElementMethod = null;
    for (Method method : doFn.getClass().getDeclaredMethods()) {
        if ("method".equals(method.getName())) {
            processElementMethod = method;
        }
    }
    checkState(processElementMethod != null);
    FnAnalysisContext context = FnAnalysisContext.create();
    context.addStateDeclaration(DoFnSignature.StateDeclaration.create("my-state-id", doFn.getClass().getField("myStateSpec"), new TypeDescriptor<ValueState<String>>() {
    }));
    DoFnSignatures.analyzeProcessElementMethod(errors(), new TypeDescriptor<DoFn<Integer, String>>() {
    }, processElementMethod, TypeDescriptor.of(Integer.class), TypeDescriptor.of(String.class), context);
}
Also used : DoFnSignaturesTestUtils.analyzeProcessElementMethod(org.apache.beam.sdk.transforms.reflect.DoFnSignaturesTestUtils.analyzeProcessElementMethod) AnonymousMethod(org.apache.beam.sdk.transforms.reflect.DoFnSignaturesTestUtils.AnonymousMethod) Method(java.lang.reflect.Method) StateSpec(org.apache.beam.sdk.state.StateSpec) DoFn(org.apache.beam.sdk.transforms.DoFn) FakeDoFn(org.apache.beam.sdk.transforms.reflect.DoFnSignaturesTestUtils.FakeDoFn) ValueState(org.apache.beam.sdk.state.ValueState) FnAnalysisContext(org.apache.beam.sdk.transforms.reflect.DoFnSignatures.FnAnalysisContext) TypeDescriptor(org.apache.beam.sdk.values.TypeDescriptor) Test(org.junit.Test)

Example 47 with StateSpec

use of org.apache.beam.sdk.state.StateSpec in project beam by apache.

the class DoFnSignaturesTest method testStateParameterAlwaysFetched.

@Test
public void testStateParameterAlwaysFetched() {
    thrown.expect(IllegalArgumentException.class);
    thrown.expectMessage("ReadableStates");
    DoFnSignature sig = DoFnSignatures.getSignature(new DoFn<KV<String, Integer>, Long>() {

        @StateId("my-id")
        private final StateSpec<MapState<Integer, Integer>> myfield = StateSpecs.map(VarIntCoder.of(), VarIntCoder.of());

        @ProcessElement
        public void myProcessElement(ProcessContext context, @AlwaysFetched @StateId("my-id") MapState<Integer, Integer> one) {
        }
    }.getClass());
    StateParameter stateParameter = (StateParameter) sig.processElement().extraParameters().get(1);
    assertTrue(stateParameter.alwaysFetched());
}
Also used : StateSpec(org.apache.beam.sdk.state.StateSpec) DoFn(org.apache.beam.sdk.transforms.DoFn) FakeDoFn(org.apache.beam.sdk.transforms.reflect.DoFnSignaturesTestUtils.FakeDoFn) MapState(org.apache.beam.sdk.state.MapState) StateParameter(org.apache.beam.sdk.transforms.reflect.DoFnSignature.Parameter.StateParameter) Matchers.containsString(org.hamcrest.Matchers.containsString) Test(org.junit.Test)

Aggregations

StateSpec (org.apache.beam.sdk.state.StateSpec)47 Test (org.junit.Test)38 KV (org.apache.beam.sdk.values.KV)35 Matchers.containsString (org.hamcrest.Matchers.containsString)24 Category (org.junit.experimental.categories.Category)24 StringUtils.byteArrayToJsonString (org.apache.beam.sdk.util.StringUtils.byteArrayToJsonString)23 ValueState (org.apache.beam.sdk.state.ValueState)21 DoFn (org.apache.beam.sdk.transforms.DoFn)19 ArrayList (java.util.ArrayList)10 CombiningState (org.apache.beam.sdk.state.CombiningState)10 Pipeline (org.apache.beam.sdk.Pipeline)9 List (java.util.List)8 TupleTag (org.apache.beam.sdk.values.TupleTag)8 Instant (org.joda.time.Instant)8 BagState (org.apache.beam.sdk.state.BagState)7 BoundedWindow (org.apache.beam.sdk.transforms.windowing.BoundedWindow)7 TupleTagList (org.apache.beam.sdk.values.TupleTagList)7 MapState (org.apache.beam.sdk.state.MapState)6 SetState (org.apache.beam.sdk.state.SetState)6 Timer (org.apache.beam.sdk.state.Timer)6