Search in sources :

Example 1 with LocalRedisModule

use of com.netflix.conductor.server.LocalRedisModule in project conductor by Netflix.

the class TestModule method configure.

@Override
protected void configure() {
    System.setProperty("workflow.system.task.worker.callback.seconds", "0");
    System.setProperty("workflow.system.task.worker.queue.size", "10000");
    System.setProperty("workflow.system.task.worker.thread.count", "10");
    configureExecutorService();
    MockConfiguration config = new MockConfiguration();
    bind(Configuration.class).toInstance(config);
    install(new LocalRedisModule());
    bind(ShardingStrategy.class).toProvider(RedisQueuesShardingStrategyProvider.class).asEagerSingleton();
    bind(RedisQueues.class).toProvider(RedisQueuesProvider.class);
    bind(MetadataDAO.class).to(RedisMetadataDAO.class);
    bind(ExecutionDAO.class).to(RedisExecutionDAO.class);
    bind(RateLimitingDAO.class).to(RedisRateLimitingDAO.class);
    bind(EventHandlerDAO.class).to(RedisEventHandlerDAO.class);
    bind(PollDataDAO.class).to(RedisPollDataDAO.class);
    bind(IndexDAO.class).to(MockIndexDAO.class);
    configureQueueDAO();
    bind(WorkflowStatusListener.class).to(WorkflowStatusListenerStub.class);
    bind(MetadataService.class).to(MetadataServiceImpl.class);
    install(new CoreModule());
    install(new EventModule());
    bind(UserTask.class).asEagerSingleton();
    bind(ObjectMapper.class).toProvider(JsonMapperProvider.class);
    bind(ExternalPayloadStorage.class).to(MockExternalPayloadStorage.class);
    install(new LocalOnlyLockModule());
}
Also used : PollDataDAO(com.netflix.conductor.dao.PollDataDAO) RedisPollDataDAO(com.netflix.conductor.dao.dynomite.RedisPollDataDAO) Configuration(com.netflix.conductor.core.config.Configuration) RedisQueuesShardingStrategyProvider(com.netflix.conductor.dyno.RedisQueuesShardingStrategyProvider) ExecutionDAO(com.netflix.conductor.dao.ExecutionDAO) RedisExecutionDAO(com.netflix.conductor.dao.dynomite.RedisExecutionDAO) EventModule(com.netflix.conductor.core.config.EventModule) LocalRedisModule(com.netflix.conductor.server.LocalRedisModule) RedisQueues(com.netflix.dyno.queues.redis.RedisQueues) RedisRateLimitingDAO(com.netflix.conductor.dao.dynomite.RedisRateLimitingDAO) RateLimitingDAO(com.netflix.conductor.dao.RateLimitingDAO) IndexDAO(com.netflix.conductor.dao.IndexDAO) MetadataService(com.netflix.conductor.service.MetadataService) LocalOnlyLockModule(com.netflix.conductor.core.utils.LocalOnlyLockModule) WorkflowStatusListener(com.netflix.conductor.core.execution.WorkflowStatusListener) ExternalPayloadStorage(com.netflix.conductor.common.utils.ExternalPayloadStorage) CoreModule(com.netflix.conductor.core.config.CoreModule) RedisEventHandlerDAO(com.netflix.conductor.dao.dynomite.RedisEventHandlerDAO) EventHandlerDAO(com.netflix.conductor.dao.EventHandlerDAO) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) MetadataDAO(com.netflix.conductor.dao.MetadataDAO) RedisMetadataDAO(com.netflix.conductor.dao.dynomite.RedisMetadataDAO)

Example 2 with LocalRedisModule

use of com.netflix.conductor.server.LocalRedisModule in project conductor by Netflix.

the class ModulesProvider method selectModulesToLoad.

private List<AbstractModule> selectModulesToLoad() {
    Configuration.DB database;
    List<AbstractModule> modules = new ArrayList<>();
    // Load Jackson module early to make ObjectMapper provider available across all the usages.
    modules.add(new JacksonModule());
    try {
        database = configuration.getDB();
    } catch (IllegalArgumentException ie) {
        final String message = "Invalid db name: " + configuration.getDBString() + ", supported values are: " + Arrays.toString(Configuration.DB.values());
        logger.error(message);
        throw new ProvisionException(message, ie);
    }
    switch(database) {
        case REDIS:
        case DYNOMITE:
            modules.add(new DynomiteClusterModule());
            modules.add(new RedisWorkflowModule());
            logger.info("Starting conductor server using dynomite/redis cluster.");
            break;
        case MYSQL:
            modules.add(new MySQLWorkflowModule());
            logger.info("Starting conductor server using MySQL data store.");
            break;
        case POSTGRES:
            modules.add(new PostgresWorkflowModule());
            logger.info("Starting conductor server using Postgres data store.");
            break;
        case MEMORY:
            modules.add(new LocalRedisModule());
            modules.add(new RedisWorkflowModule());
            logger.info("Starting conductor server using in memory data store.");
            break;
        case REDIS_CLUSTER:
            modules.add(new RedisClusterModule());
            modules.add(new RedisWorkflowModule());
            logger.info("Starting conductor server using redis_cluster.");
            break;
        case CASSANDRA:
            modules.add(new CassandraModule());
            logger.info("Starting conductor server using cassandra.");
            break;
        case REDIS_SENTINEL:
            modules.add(new RedisSentinelModule());
            modules.add(new RedisWorkflowModule());
            logger.info("Starting conductor server using redis_sentinel.");
            break;
    }
    if (configuration.isIndexingPersistenceEnabled())
        modules.add(new ElasticSearchModule());
    else
        modules.add(new NoopIndexModule());
    modules.add(new WorkflowExecutorModule());
    if (configuration.getJerseyEnabled()) {
        modules.add(new JerseyModule());
        modules.add(new SwaggerModule());
    }
    if (configuration.enableWorkflowExecutionLock()) {
        Configuration.LOCKING_SERVER lockingServer;
        try {
            lockingServer = configuration.getLockingServer();
        } catch (IllegalArgumentException ie) {
            final String message = "Invalid locking server name: " + configuration.getLockingServerString() + ", supported values are: " + Arrays.toString(Configuration.LOCKING_SERVER.values());
            logger.error(message);
            throw new ProvisionException(message, ie);
        }
        switch(lockingServer) {
            case REDIS:
                modules.add(new RedisLockModule());
                logger.info("Starting locking module using Redis cluster.");
                break;
            case ZOOKEEPER:
                modules.add(new ZookeeperModule());
                logger.info("Starting locking module using Zookeeper cluster.");
                break;
            case LOCAL_ONLY:
                modules.add(new LocalOnlyLockModule());
                logger.info("Starting locking module using local only JVM locking.");
                break;
            default:
                break;
        }
    } else {
        modules.add(new NoopLockModule());
        logger.warn("Starting locking module using Noop Lock.");
    }
    ExternalPayloadStorageType externalPayloadStorageType = null;
    String externalPayloadStorageString = configuration.getProperty("workflow.external.payload.storage", "DUMMY");
    try {
        externalPayloadStorageType = ExternalPayloadStorageType.valueOf(externalPayloadStorageString);
    } catch (IllegalArgumentException e) {
        logger.info("External payload storage is not configured, provided: {}, supported values are: {}", externalPayloadStorageString, Arrays.toString(ExternalPayloadStorageType.values()), e);
    }
    if (externalPayloadStorageType == ExternalPayloadStorageType.S3) {
        modules.add(new AbstractModule() {

            @Override
            protected void configure() {
                bind(ExternalPayloadStorage.class).to(S3PayloadStorage.class);
            }
        });
    } else {
        modules.add(new AbstractModule() {

            @Override
            protected void configure() {
                bind(ExternalPayloadStorage.class).to(DummyPayloadStorage.class);
            }
        });
    }
    new HttpTask(new RestClientManager(configuration), configuration, new JsonMapperProvider().get());
    new KafkaPublishTask(configuration, new KafkaProducerManager(configuration), new JsonMapperProvider().get());
    new JsonJqTransform(new JsonMapperProvider().get());
    modules.add(new ServerModule());
    return modules;
}
Also used : RedisClusterModule(com.netflix.conductor.server.RedisClusterModule) JerseyModule(com.netflix.conductor.server.JerseyModule) Configuration(com.netflix.conductor.core.config.Configuration) LocalRedisModule(com.netflix.conductor.server.LocalRedisModule) ArrayList(java.util.ArrayList) RedisSentinelModule(com.netflix.conductor.server.RedisSentinelModule) KafkaProducerManager(com.netflix.conductor.contribs.kafka.KafkaProducerManager) ServerModule(com.netflix.conductor.server.ServerModule) PostgresWorkflowModule(com.netflix.conductor.postgres.PostgresWorkflowModule) ElasticSearchModule(com.netflix.conductor.elasticsearch.ElasticSearchModule) ProvisionException(com.google.inject.ProvisionException) JsonJqTransform(com.netflix.conductor.contribs.json.JsonJqTransform) KafkaPublishTask(com.netflix.conductor.contribs.kafka.KafkaPublishTask) MySQLWorkflowModule(com.netflix.conductor.mysql.MySQLWorkflowModule) S3PayloadStorage(com.netflix.conductor.core.utils.S3PayloadStorage) WorkflowExecutorModule(com.netflix.conductor.core.execution.WorkflowExecutorModule) HttpTask(com.netflix.conductor.contribs.http.HttpTask) ZookeeperModule(com.netflix.conductor.zookeeper.config.ZookeeperModule) NoopLockModule(com.netflix.conductor.core.utils.NoopLockModule) DummyPayloadStorage(com.netflix.conductor.core.utils.DummyPayloadStorage) JacksonModule(com.netflix.conductor.core.config.JacksonModule) JsonMapperProvider(com.netflix.conductor.common.utils.JsonMapperProvider) AbstractModule(com.google.inject.AbstractModule) SwaggerModule(com.netflix.conductor.server.SwaggerModule) CassandraModule(com.netflix.conductor.cassandra.CassandraModule) LocalOnlyLockModule(com.netflix.conductor.core.utils.LocalOnlyLockModule) RestClientManager(com.netflix.conductor.contribs.http.RestClientManager) DynomiteClusterModule(com.netflix.conductor.server.DynomiteClusterModule) RedisWorkflowModule(com.netflix.conductor.dao.RedisWorkflowModule) RedisLockModule(com.netflix.conductor.locking.redis.config.RedisLockModule) NoopIndexModule(com.netflix.conductor.noopindex.NoopIndexModule)

Aggregations

Configuration (com.netflix.conductor.core.config.Configuration)2 LocalOnlyLockModule (com.netflix.conductor.core.utils.LocalOnlyLockModule)2 LocalRedisModule (com.netflix.conductor.server.LocalRedisModule)2 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 AbstractModule (com.google.inject.AbstractModule)1 ProvisionException (com.google.inject.ProvisionException)1 CassandraModule (com.netflix.conductor.cassandra.CassandraModule)1 ExternalPayloadStorage (com.netflix.conductor.common.utils.ExternalPayloadStorage)1 JsonMapperProvider (com.netflix.conductor.common.utils.JsonMapperProvider)1 HttpTask (com.netflix.conductor.contribs.http.HttpTask)1 RestClientManager (com.netflix.conductor.contribs.http.RestClientManager)1 JsonJqTransform (com.netflix.conductor.contribs.json.JsonJqTransform)1 KafkaProducerManager (com.netflix.conductor.contribs.kafka.KafkaProducerManager)1 KafkaPublishTask (com.netflix.conductor.contribs.kafka.KafkaPublishTask)1 CoreModule (com.netflix.conductor.core.config.CoreModule)1 EventModule (com.netflix.conductor.core.config.EventModule)1 JacksonModule (com.netflix.conductor.core.config.JacksonModule)1 WorkflowExecutorModule (com.netflix.conductor.core.execution.WorkflowExecutorModule)1 WorkflowStatusListener (com.netflix.conductor.core.execution.WorkflowStatusListener)1 DummyPayloadStorage (com.netflix.conductor.core.utils.DummyPayloadStorage)1