Search in sources :

Example 11 with SystemAdmin

use of org.apache.samza.system.SystemAdmin in project samza by apache.

the class TestKafkaSystemAdminJava method testValidateStreamWrongName.

@Test(expected = StreamValidationException.class)
public void testValidateStreamWrongName() {
    SystemAdmin admin = this.basicSystemAdmin;
    StreamSpec spec1 = new StreamSpec("testId", "testStreamName1", "testSystem", 8);
    StreamSpec spec2 = new StreamSpec("testId", "testStreamName2", "testSystem", 8);
    assertTrue("createStream should return true if the stream does not exist and then is created.", admin.createStream(spec1));
    admin.validateStream(spec2);
}
Also used : StreamSpec(org.apache.samza.system.StreamSpec) SystemAdmin(org.apache.samza.system.SystemAdmin) Test(org.junit.Test)

Example 12 with SystemAdmin

use of org.apache.samza.system.SystemAdmin in project samza by apache.

the class TestKafkaSystemAdminJava method testValidateStreamWrongPartitionCount.

@Test(expected = StreamValidationException.class)
public void testValidateStreamWrongPartitionCount() {
    SystemAdmin admin = this.basicSystemAdmin;
    StreamSpec spec1 = new StreamSpec("testId", "testStreamPartition", "testSystem", 8);
    StreamSpec spec2 = new StreamSpec("testId", "testStreamPartition", "testSystem", 4);
    assertTrue("createStream should return true if the stream does not exist and then is created.", admin.createStream(spec1));
    admin.validateStream(spec2);
}
Also used : StreamSpec(org.apache.samza.system.StreamSpec) SystemAdmin(org.apache.samza.system.SystemAdmin) Test(org.junit.Test)

Example 13 with SystemAdmin

use of org.apache.samza.system.SystemAdmin in project samza by apache.

the class TestKafkaSystemAdminJava method testValidateChangelogStreamDelegatesToValidateStream.

@Test
public void testValidateChangelogStreamDelegatesToValidateStream() {
    final String STREAM = "testChangeLogValidate";
    Properties coordProps = new Properties();
    Map<String, ChangelogInfo> changeLogMap = new HashMap<>();
    changeLogMap.put(STREAM, new ChangelogInfo(3, new Properties()));
    KafkaSystemAdmin systemAdmin = createSystemAdmin(coordProps, 3, Util.javaMapAsScalaMap(changeLogMap));
    SystemAdmin admin = Mockito.spy(systemAdmin);
    StreamSpec spec = new StreamSpec("testId", STREAM, "testSystem", 12);
    admin.createChangelogStream(spec.getPhysicalName(), spec.getPartitionCount());
    admin.validateStream(spec);
    admin.validateChangelogStream(spec.getPhysicalName(), spec.getPartitionCount());
    Mockito.verify(admin).createStream(Mockito.any());
    Mockito.verify(admin, Mockito.times(3)).validateStream(Mockito.any());
}
Also used : StreamSpec(org.apache.samza.system.StreamSpec) HashMap(java.util.HashMap) SystemAdmin(org.apache.samza.system.SystemAdmin) Properties(java.util.Properties) Test(org.junit.Test)

Example 14 with SystemAdmin

use of org.apache.samza.system.SystemAdmin in project samza by apache.

the class TestKafkaSystemAdminJava method testCreateCoordinatorStreamDelegatesToCreateStream.

@Test
public void testCreateCoordinatorStreamDelegatesToCreateStream() {
    //coordProps, 3, new scala.collection.immutable.HashMap<>(), 1000);
    KafkaSystemAdmin systemAdmin = createSystemAdmin();
    SystemAdmin admin = Mockito.spy(systemAdmin);
    StreamSpec spec = new StreamSpec("testId", "testCoordinatorStream", "testSystem");
    admin.createCoordinatorStream(spec.getPhysicalName());
    admin.validateStream(spec);
    Mockito.verify(admin).createStream(Mockito.any());
}
Also used : StreamSpec(org.apache.samza.system.StreamSpec) SystemAdmin(org.apache.samza.system.SystemAdmin) Test(org.junit.Test)

Example 15 with SystemAdmin

use of org.apache.samza.system.SystemAdmin in project samza by apache.

the class StreamManager method createStreams.

public void createStreams(List<StreamSpec> streams) {
    Multimap<String, StreamSpec> streamsGroupedBySystem = HashMultimap.create();
    streams.forEach(streamSpec -> streamsGroupedBySystem.put(streamSpec.getSystemName(), streamSpec));
    for (Map.Entry<String, Collection<StreamSpec>> entry : streamsGroupedBySystem.asMap().entrySet()) {
        String systemName = entry.getKey();
        SystemAdmin systemAdmin = sysAdmins.get(systemName);
        for (StreamSpec stream : entry.getValue()) {
            LOGGER.info("Creating stream {} with partitions {} on system {}", new Object[] { stream.getPhysicalName(), stream.getPartitionCount(), systemName });
            systemAdmin.createStream(stream);
        }
    }
}
Also used : StreamSpec(org.apache.samza.system.StreamSpec) Collection(java.util.Collection) SystemAdmin(org.apache.samza.system.SystemAdmin) HashMap(java.util.HashMap) Map(java.util.Map)

Aggregations

SystemAdmin (org.apache.samza.system.SystemAdmin)16 StreamSpec (org.apache.samza.system.StreamSpec)12 HashMap (java.util.HashMap)10 Test (org.junit.Test)10 Properties (java.util.Properties)4 Map (java.util.Map)3 MapConfig (org.apache.samza.config.MapConfig)3 SamzaException (org.apache.samza.SamzaException)2 Config (org.apache.samza.config.Config)2 JobConfig (org.apache.samza.config.JobConfig)2 ApplicationRunner (org.apache.samza.runtime.ApplicationRunner)2 StreamMetadataCache (org.apache.samza.system.StreamMetadataCache)2 SystemStreamMetadata (org.apache.samza.system.SystemStreamMetadata)2 Duration (java.time.Duration)1 Collection (java.util.Collection)1 Set (java.util.Set)1 BiFunction (java.util.function.BiFunction)1 Function (java.util.function.Function)1 Partition (org.apache.samza.Partition)1 JavaSystemConfig (org.apache.samza.config.JavaSystemConfig)1