Examples with JavaSystemConfig org.apache.samza.config.JavaSystemConfig used on opensource projects

Example 1 with JavaSystemConfig

use of org.apache.samza.config.JavaSystemConfig in project samza by apache.

the class StorageRecovery method getSystemFactoriesAndAdmins.

/**
   * get the SystemFactories and SystemAdmins specified in the config file and
   * put them into the maps
   */
private void getSystemFactoriesAndAdmins() {
    JavaSystemConfig systemConfig = new JavaSystemConfig(jobConfig);
    List<String> systems = systemConfig.getSystemNames();
    for (String system : systems) {
        String systemFactory = systemConfig.getSystemFactory(system);
        if (systemFactory == null) {
            throw new SamzaException("A stream uses system " + system + " which is missing from the configuration.");
        }
        systemFactories.put(system, Util.<SystemFactory>getObj(systemFactory));
        systemAdmins.put(system, Util.<SystemFactory>getObj(systemFactory).getAdmin(system, jobConfig));
    }
    log.info("Got system factories: " + systemFactories.keySet().toString());
    log.info("Got system admins: " + systemAdmins.keySet().toString());
}

Example 2 with JavaSystemConfig

use of org.apache.samza.config.JavaSystemConfig in project samza by apache.

the class PassthroughJobCoordinator method getJobModel.

@Override
public JobModel getJobModel() {
    JavaSystemConfig systemConfig = new JavaSystemConfig(this.config);
    Map<String, SystemAdmin> systemAdmins = new HashMap<>();
    for (String systemName : systemConfig.getSystemNames()) {
        String systemFactoryClassName = systemConfig.getSystemFactory(systemName);
        if (systemFactoryClassName == null) {
            LOGGER.error(String.format("A stream uses system %s, which is missing from the configuration.", systemName));
            throw new SamzaException(String.format("A stream uses system %s, which is missing from the configuration.", systemName));
        }
        SystemFactory systemFactory = Util.<SystemFactory>getObj(systemFactoryClassName);
        systemAdmins.put(systemName, systemFactory.getAdmin(systemName, this.config));
    }
    StreamMetadataCache streamMetadataCache = new StreamMetadataCache(Util.<String, SystemAdmin>javaMapAsScalaMap(systemAdmins), 5000, SystemClock.instance());
    /** TODO:
     Locality Manager seems to be required in JC for reading locality info and grouping tasks intelligently and also,
     in SamzaContainer for writing locality info to the coordinator stream. This closely couples together
     TaskNameGrouper with the LocalityManager! Hence, groupers should be a property of the jobcoordinator
     (job.coordinator.task.grouper, instead of task.systemstreampartition.grouper)
     */
    return JobModelManager.readJobModel(this.config, Collections.emptyMap(), null, streamMetadataCache, null);
}