Search in sources :

Example 1 with ExecutionPlanner

use of org.apache.samza.execution.ExecutionPlanner in project samza by apache.

the class TestLocalApplicationRunner method testStreamCreation.

@Test
public void testStreamCreation() throws Exception {
    Map<String, String> config = new HashMap<>();
    LocalApplicationRunner runner = new LocalApplicationRunner(new MapConfig(config));
    StreamApplication app = mock(StreamApplication.class);
    doNothing().when(app).init(anyObject(), anyObject());
    ExecutionPlanner planner = mock(ExecutionPlanner.class);
    Field plannerField = runner.getClass().getSuperclass().getDeclaredField("planner");
    plannerField.setAccessible(true);
    plannerField.set(runner, planner);
    StreamManager streamManager = mock(StreamManager.class);
    Field streamManagerField = runner.getClass().getSuperclass().getDeclaredField("streamManager");
    streamManagerField.setAccessible(true);
    streamManagerField.set(runner, streamManager);
    ArgumentCaptor<List> captor = ArgumentCaptor.forClass(List.class);
    ExecutionPlan plan = new ExecutionPlan() {

        @Override
        public List<JobConfig> getJobConfigs() {
            return Collections.emptyList();
        }

        @Override
        public List<StreamSpec> getIntermediateStreams() {
            return Collections.singletonList(new StreamSpec("test-stream", "test-stream", "test-system"));
        }

        @Override
        public String getPlanAsJson() throws Exception {
            return "";
        }
    };
    when(planner.plan(anyObject())).thenReturn(plan);
    LocalApplicationRunner spy = spy(runner);
    try {
        spy.run(app);
    } catch (Throwable t) {
        //no jobs exception
        assertNotNull(t);
    }
    verify(streamManager).createStreams(captor.capture());
    List<StreamSpec> streamSpecs = captor.getValue();
    assertEquals(streamSpecs.size(), 1);
    assertEquals(streamSpecs.get(0).getId(), "test-stream");
}
Also used : StreamSpec(org.apache.samza.system.StreamSpec) HashMap(java.util.HashMap) StreamApplication(org.apache.samza.application.StreamApplication) ExecutionPlanner(org.apache.samza.execution.ExecutionPlanner) Matchers.anyString(org.mockito.Matchers.anyString) JobConfig(org.apache.samza.config.JobConfig) Field(java.lang.reflect.Field) ExecutionPlan(org.apache.samza.execution.ExecutionPlan) StreamManager(org.apache.samza.execution.StreamManager) List(java.util.List) MapConfig(org.apache.samza.config.MapConfig) Test(org.junit.Test)

Example 2 with ExecutionPlanner

use of org.apache.samza.execution.ExecutionPlanner in project samza by apache.

the class TestLocalApplicationRunner method testRunFailure.

@Test
public void testRunFailure() throws Exception {
    final Map<String, String> config = new HashMap<>();
    config.put(ApplicationConfig.PROCESSOR_ID, "0");
    LocalApplicationRunner runner = new LocalApplicationRunner(new MapConfig(config));
    StreamApplication app = mock(StreamApplication.class);
    doNothing().when(app).init(anyObject(), anyObject());
    ExecutionPlanner planner = mock(ExecutionPlanner.class);
    Field plannerField = runner.getClass().getSuperclass().getDeclaredField("planner");
    plannerField.setAccessible(true);
    plannerField.set(runner, planner);
    ExecutionPlan plan = new ExecutionPlan() {

        @Override
        public List<JobConfig> getJobConfigs() {
            return Collections.singletonList(new JobConfig(new MapConfig(config)));
        }

        @Override
        public List<StreamSpec> getIntermediateStreams() {
            return Collections.emptyList();
        }

        @Override
        public String getPlanAsJson() throws Exception {
            return "";
        }
    };
    when(planner.plan(anyObject())).thenReturn(plan);
    Throwable t = new Throwable("test failure");
    StreamProcessor sp = mock(StreamProcessor.class);
    ArgumentCaptor<StreamProcessorLifecycleListener> captor = ArgumentCaptor.forClass(StreamProcessorLifecycleListener.class);
    doAnswer(i -> {
        StreamProcessorLifecycleListener listener = captor.getValue();
        listener.onFailure(t);
        return null;
    }).when(sp).start();
    LocalApplicationRunner spy = spy(runner);
    doReturn(sp).when(spy).createStreamProcessor(anyObject(), anyObject(), captor.capture());
    try {
        spy.run(app);
    } catch (Throwable th) {
        assertNotNull(th);
    }
    assertEquals(spy.status(app), ApplicationStatus.UnsuccessfulFinish);
}
Also used : StreamSpec(org.apache.samza.system.StreamSpec) StreamProcessor(org.apache.samza.processor.StreamProcessor) HashMap(java.util.HashMap) StreamApplication(org.apache.samza.application.StreamApplication) ExecutionPlanner(org.apache.samza.execution.ExecutionPlanner) Matchers.anyString(org.mockito.Matchers.anyString) StreamProcessorLifecycleListener(org.apache.samza.processor.StreamProcessorLifecycleListener) JobConfig(org.apache.samza.config.JobConfig) Field(java.lang.reflect.Field) ExecutionPlan(org.apache.samza.execution.ExecutionPlan) MapConfig(org.apache.samza.config.MapConfig) Test(org.junit.Test)

Example 3 with ExecutionPlanner

use of org.apache.samza.execution.ExecutionPlanner in project samza by apache.

the class TestLocalApplicationRunner method testRunComplete.

@Test
public void testRunComplete() throws Exception {
    final Map<String, String> config = new HashMap<>();
    config.put(ApplicationConfig.APP_PROCESSOR_ID_GENERATOR_CLASS, UUIDGenerator.class.getName());
    LocalApplicationRunner runner = new LocalApplicationRunner(new MapConfig(config));
    StreamApplication app = mock(StreamApplication.class);
    doNothing().when(app).init(anyObject(), anyObject());
    ExecutionPlanner planner = mock(ExecutionPlanner.class);
    Field plannerField = runner.getClass().getSuperclass().getDeclaredField("planner");
    plannerField.setAccessible(true);
    plannerField.set(runner, planner);
    ExecutionPlan plan = new ExecutionPlan() {

        @Override
        public List<JobConfig> getJobConfigs() {
            return Collections.singletonList(new JobConfig(new MapConfig(config)));
        }

        @Override
        public List<StreamSpec> getIntermediateStreams() {
            return Collections.emptyList();
        }

        @Override
        public String getPlanAsJson() throws Exception {
            return "";
        }
    };
    when(planner.plan(anyObject())).thenReturn(plan);
    StreamProcessor sp = mock(StreamProcessor.class);
    ArgumentCaptor<StreamProcessorLifecycleListener> captor = ArgumentCaptor.forClass(StreamProcessorLifecycleListener.class);
    doAnswer(i -> {
        StreamProcessorLifecycleListener listener = captor.getValue();
        listener.onStart();
        listener.onShutdown();
        return null;
    }).when(sp).start();
    LocalApplicationRunner spy = spy(runner);
    doReturn(sp).when(spy).createStreamProcessor(anyObject(), anyObject(), captor.capture());
    spy.run(app);
    assertEquals(spy.status(app), ApplicationStatus.SuccessfulFinish);
}
Also used : StreamSpec(org.apache.samza.system.StreamSpec) StreamProcessor(org.apache.samza.processor.StreamProcessor) HashMap(java.util.HashMap) StreamApplication(org.apache.samza.application.StreamApplication) ExecutionPlanner(org.apache.samza.execution.ExecutionPlanner) Matchers.anyString(org.mockito.Matchers.anyString) StreamProcessorLifecycleListener(org.apache.samza.processor.StreamProcessorLifecycleListener) JobConfig(org.apache.samza.config.JobConfig) Field(java.lang.reflect.Field) ExecutionPlan(org.apache.samza.execution.ExecutionPlan) MapConfig(org.apache.samza.config.MapConfig) Test(org.junit.Test)

Example 4 with ExecutionPlanner

use of org.apache.samza.execution.ExecutionPlanner in project samza by apache.

the class TestLocalApplicationRunner method testStreamCreationWithCoordination.

@Test
public void testStreamCreationWithCoordination() throws Exception {
    Map<String, String> config = new HashMap<>();
    LocalApplicationRunner runner = new LocalApplicationRunner(new MapConfig(config));
    StreamApplication app = mock(StreamApplication.class);
    doNothing().when(app).init(anyObject(), anyObject());
    ExecutionPlanner planner = mock(ExecutionPlanner.class);
    Field plannerField = runner.getClass().getSuperclass().getDeclaredField("planner");
    plannerField.setAccessible(true);
    plannerField.set(runner, planner);
    StreamManager streamManager = mock(StreamManager.class);
    Field streamManagerField = runner.getClass().getSuperclass().getDeclaredField("streamManager");
    streamManagerField.setAccessible(true);
    streamManagerField.set(runner, streamManager);
    ArgumentCaptor<List> captor = ArgumentCaptor.forClass(List.class);
    ExecutionPlan plan = new ExecutionPlan() {

        @Override
        public List<JobConfig> getJobConfigs() {
            return Collections.emptyList();
        }

        @Override
        public List<StreamSpec> getIntermediateStreams() {
            return Collections.singletonList(new StreamSpec("test-stream", "test-stream", "test-system"));
        }

        @Override
        public String getPlanAsJson() throws Exception {
            return "";
        }
    };
    when(planner.plan(anyObject())).thenReturn(plan);
    LocalApplicationRunner spy = spy(runner);
    CoordinationUtils coordinationUtils = mock(CoordinationUtils.class);
    LeaderElector leaderElector = new LeaderElector() {

        private LeaderElectorListener leaderElectorListener;

        @Override
        public void setLeaderElectorListener(LeaderElectorListener listener) {
            this.leaderElectorListener = listener;
        }

        @Override
        public void tryBecomeLeader() {
            leaderElectorListener.onBecomingLeader();
        }

        @Override
        public void resignLeadership() {
        }

        @Override
        public boolean amILeader() {
            return false;
        }
    };
    Latch latch = new Latch() {

        boolean done = false;

        @Override
        public void await(long timeout, TimeUnit tu) throws TimeoutException {
            // in this test, latch is released before wait
            assertTrue(done);
        }

        @Override
        public void countDown() {
            done = true;
        }
    };
    when(coordinationUtils.getLeaderElector()).thenReturn(leaderElector);
    when(coordinationUtils.getLatch(anyInt(), anyString())).thenReturn(latch);
    doReturn(coordinationUtils).when(spy).createCoordinationUtils();
    try {
        spy.run(app);
    } catch (Throwable t) {
        //no jobs exception
        assertNotNull(t);
    }
    verify(streamManager).createStreams(captor.capture());
    List<StreamSpec> streamSpecs = captor.getValue();
    assertEquals(streamSpecs.size(), 1);
    assertEquals(streamSpecs.get(0).getId(), "test-stream");
}
Also used : StreamSpec(org.apache.samza.system.StreamSpec) HashMap(java.util.HashMap) StreamApplication(org.apache.samza.application.StreamApplication) ExecutionPlanner(org.apache.samza.execution.ExecutionPlanner) Matchers.anyString(org.mockito.Matchers.anyString) JobConfig(org.apache.samza.config.JobConfig) Field(java.lang.reflect.Field) ExecutionPlan(org.apache.samza.execution.ExecutionPlan) StreamManager(org.apache.samza.execution.StreamManager) LeaderElectorListener(org.apache.samza.coordinator.LeaderElectorListener) LeaderElector(org.apache.samza.coordinator.LeaderElector) Latch(org.apache.samza.coordinator.Latch) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) MapConfig(org.apache.samza.config.MapConfig) CoordinationUtils(org.apache.samza.coordinator.CoordinationUtils) Test(org.junit.Test)

Aggregations

Field (java.lang.reflect.Field)4 HashMap (java.util.HashMap)4 StreamApplication (org.apache.samza.application.StreamApplication)4 JobConfig (org.apache.samza.config.JobConfig)4 MapConfig (org.apache.samza.config.MapConfig)4 ExecutionPlan (org.apache.samza.execution.ExecutionPlan)4 ExecutionPlanner (org.apache.samza.execution.ExecutionPlanner)4 StreamSpec (org.apache.samza.system.StreamSpec)4 Test (org.junit.Test)4 Matchers.anyString (org.mockito.Matchers.anyString)4 List (java.util.List)2 StreamManager (org.apache.samza.execution.StreamManager)2 StreamProcessor (org.apache.samza.processor.StreamProcessor)2 StreamProcessorLifecycleListener (org.apache.samza.processor.StreamProcessorLifecycleListener)2 TimeUnit (java.util.concurrent.TimeUnit)1 CoordinationUtils (org.apache.samza.coordinator.CoordinationUtils)1 Latch (org.apache.samza.coordinator.Latch)1 LeaderElector (org.apache.samza.coordinator.LeaderElector)1 LeaderElectorListener (org.apache.samza.coordinator.LeaderElectorListener)1