Search in sources :

Example 11 with ByteArray

use of org.apache.beam.runners.samza.runtime.SamzaStoreStateInternals.ByteArray in project beam by apache.

the class SamzaTimerInternalsFactoryTest method testEventTimeTimers.

@Test
public void testEventTimeTimers() {
    final SamzaPipelineOptions pipelineOptions = PipelineOptionsFactory.create().as(SamzaPipelineOptions.class);
    final KeyValueStore<ByteArray, StateValue<?>> store = createStore();
    final SamzaTimerInternalsFactory<String> timerInternalsFactory = createTimerInternalsFactory(null, "timer", pipelineOptions, store);
    final StateNamespace nameSpace = StateNamespaces.global();
    final TimerInternals timerInternals = timerInternalsFactory.timerInternalsForKey("testKey");
    final TimerInternals.TimerData timer1 = TimerInternals.TimerData.of("timer1", nameSpace, new Instant(10), new Instant(10), TimeDomain.EVENT_TIME);
    timerInternals.setTimer(timer1);
    final TimerInternals.TimerData timer2 = TimerInternals.TimerData.of("timer2", nameSpace, new Instant(100), new Instant(100), TimeDomain.EVENT_TIME);
    timerInternals.setTimer(timer2);
    timerInternalsFactory.setInputWatermark(new Instant(5));
    Collection<KeyedTimerData<String>> readyTimers = timerInternalsFactory.removeReadyTimers();
    assertTrue(readyTimers.isEmpty());
    timerInternalsFactory.setInputWatermark(new Instant(20));
    readyTimers = timerInternalsFactory.removeReadyTimers();
    assertEquals(1, readyTimers.size());
    assertEquals(timer1, readyTimers.iterator().next().getTimerData());
    timerInternalsFactory.setInputWatermark(new Instant(150));
    readyTimers = timerInternalsFactory.removeReadyTimers();
    assertEquals(1, readyTimers.size());
    assertEquals(timer2, readyTimers.iterator().next().getTimerData());
    store.close();
}
Also used : TimerInternals(org.apache.beam.runners.core.TimerInternals) Instant(org.joda.time.Instant) ByteArray(org.apache.beam.runners.samza.runtime.SamzaStoreStateInternals.ByteArray) StateValue(org.apache.beam.runners.samza.runtime.SamzaStoreStateInternals.StateValue) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) StateNamespace(org.apache.beam.runners.core.StateNamespace) SamzaPipelineOptions(org.apache.beam.runners.samza.SamzaPipelineOptions) Test(org.junit.Test)

Example 12 with ByteArray

use of org.apache.beam.runners.samza.runtime.SamzaStoreStateInternals.ByteArray in project beam by apache.

the class SamzaTimerInternalsFactoryTest method testOverride.

@Test
public void testOverride() {
    final SamzaPipelineOptions pipelineOptions = PipelineOptionsFactory.create().as(SamzaPipelineOptions.class);
    KeyValueStore<ByteArray, StateValue<?>> store = createStore();
    final SamzaTimerInternalsFactory<String> timerInternalsFactory = createTimerInternalsFactory(null, "timer", pipelineOptions, store);
    final StateNamespace nameSpace = StateNamespaces.global();
    final TimerInternals timerInternals = timerInternalsFactory.timerInternalsForKey("testKey");
    final TimerInternals.TimerData timer1 = TimerInternals.TimerData.of("timerId", nameSpace, new Instant(10), new Instant(10), TimeDomain.EVENT_TIME);
    timerInternals.setTimer(timer1);
    // this timer should override the first timer
    final TimerInternals.TimerData timer2 = TimerInternals.TimerData.of("timerId", nameSpace, new Instant(100), new Instant(100), TimeDomain.EVENT_TIME);
    timerInternals.setTimer(timer2);
    final TimerInternals.TimerData timer3 = TimerInternals.TimerData.of("timerId2", nameSpace, new Instant(200), new Instant(200), TimeDomain.EVENT_TIME);
    timerInternals.setTimer(timer3);
    // this timer shouldn't override since it has a different id
    timerInternalsFactory.setInputWatermark(new Instant(50));
    Collection<KeyedTimerData<String>> readyTimers = timerInternalsFactory.removeReadyTimers();
    assertEquals(0, readyTimers.size());
    timerInternalsFactory.setInputWatermark(new Instant(150));
    readyTimers = timerInternalsFactory.removeReadyTimers();
    assertEquals(1, readyTimers.size());
    timerInternalsFactory.setInputWatermark(new Instant(250));
    readyTimers = timerInternalsFactory.removeReadyTimers();
    assertEquals(1, readyTimers.size());
    store.close();
}
Also used : TimerInternals(org.apache.beam.runners.core.TimerInternals) Instant(org.joda.time.Instant) ByteArray(org.apache.beam.runners.samza.runtime.SamzaStoreStateInternals.ByteArray) StateValue(org.apache.beam.runners.samza.runtime.SamzaStoreStateInternals.StateValue) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) StateNamespace(org.apache.beam.runners.core.StateNamespace) SamzaPipelineOptions(org.apache.beam.runners.samza.SamzaPipelineOptions) Test(org.junit.Test)

Aggregations

ByteArray (org.apache.beam.runners.samza.runtime.SamzaStoreStateInternals.ByteArray)12 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)12 StateNamespace (org.apache.beam.runners.core.StateNamespace)11 TimerInternals (org.apache.beam.runners.core.TimerInternals)11 SamzaPipelineOptions (org.apache.beam.runners.samza.SamzaPipelineOptions)11 StateValue (org.apache.beam.runners.samza.runtime.SamzaStoreStateInternals.StateValue)11 Instant (org.joda.time.Instant)11 Test (org.junit.Test)11 ByteArrayOutputStream (java.io.ByteArrayOutputStream)3 HashMap (java.util.HashMap)1 ByteArraySerdeFactory (org.apache.beam.runners.samza.runtime.SamzaStoreStateInternals.ByteArraySerdeFactory)1