Search in sources :

Example 11 with InvalidConfigException

use of com.linkedin.databus.core.util.InvalidConfigException in project databus by linkedin.

the class OpenReplicatorEventProducerServiceProvider method buildPartitionFunction.

public PartitionFunction buildPartitionFunction(LogicalSourceStaticConfig sourceConfig) throws InvalidConfigException {
    String partitionFunction = sourceConfig.getPartitionFunction();
    if (partitionFunction.startsWith("constant:")) {
        try {
            String numberPart = partitionFunction.substring("constant:".length()).trim();
            short constantPartitionNumber = Short.valueOf(numberPart);
            return new ConstantPartitionFunction(constantPartitionNumber);
        } catch (Exception ex) {
            // Could be a NumberFormatException, IndexOutOfBoundsException or other exception when trying to parse the partition number.
            throw new InvalidConfigException("Invalid partition configuration (" + partitionFunction + "). " + "Could not parse the constant partition number.");
        }
    } else {
        throw new InvalidConfigException("Invalid partition configuration (" + partitionFunction + ").");
    }
}
Also used : InvalidConfigException(com.linkedin.databus.core.util.InvalidConfigException) DatabusException(com.linkedin.databus2.core.DatabusException) InvalidConfigException(com.linkedin.databus.core.util.InvalidConfigException) UnsupportedKeyException(com.linkedin.databus.core.UnsupportedKeyException) NoSuchSchemaException(com.linkedin.databus2.schemas.NoSuchSchemaException)

Example 12 with InvalidConfigException

use of com.linkedin.databus.core.util.InvalidConfigException in project databus by linkedin.

the class OpenReplicatorEventProducerServiceProvider method buildEventFactory.

public OpenReplicatorAvroEventFactory buildEventFactory(LogicalSourceStaticConfig sourceConfig, PhysicalSourceStaticConfig pConfig, SchemaRegistryService schemaRegistryService) throws DatabusException, EventCreationException, UnsupportedKeyException, InvalidConfigException {
    String schema = null;
    try {
        schema = schemaRegistryService.fetchLatestSchemaBySourceName(sourceConfig.getName());
    } catch (NoSuchSchemaException e) {
        throw new InvalidConfigException("Unable to load the schema for source (" + sourceConfig.getName() + ").");
    }
    if (schema == null) {
        throw new InvalidConfigException("Unable to load the schema for source (" + sourceConfig.getName() + ").");
    }
    LOG.info("Loading schema for source id " + sourceConfig.getId() + ": " + schema);
    String eventViewSchema;
    String eventView;
    if (sourceConfig.getUri().indexOf('.') != -1) {
        String[] parts = sourceConfig.getUri().split("\\.");
        eventViewSchema = parts[0];
        eventView = parts[1];
    } else {
        eventViewSchema = null;
        eventView = sourceConfig.getUri();
    }
    PartitionFunction partitionFunction = buildPartitionFunction(sourceConfig);
    OpenReplicatorAvroEventFactory factory = createEventFactory(eventViewSchema, eventView, sourceConfig, pConfig, schema, partitionFunction);
    return factory;
}
Also used : PartitionFunction(com.linkedin.databus2.producers.PartitionFunction) NoSuchSchemaException(com.linkedin.databus2.schemas.NoSuchSchemaException) InvalidConfigException(com.linkedin.databus.core.util.InvalidConfigException)

Example 13 with InvalidConfigException

use of com.linkedin.databus.core.util.InvalidConfigException in project databus by linkedin.

the class OpenReplicatorEventProducerServiceProvider method createProducer.

@Override
public EventProducer createProducer(PhysicalSourceStaticConfig physicalSourceConfig, SchemaRegistryService schemaRegistryService, DbusEventBufferAppendable eventBuffer, DbusEventsStatisticsCollector statsCollector, MaxSCNReaderWriter checkpointWriter) throws InvalidConfigException {
    // Parse each one of the logical sources
    List<OpenReplicatorAvroEventFactory> eventFactories = new ArrayList<OpenReplicatorAvroEventFactory>();
    for (LogicalSourceStaticConfig sourceConfig : physicalSourceConfig.getSources()) {
        OpenReplicatorAvroEventFactory factory = null;
        try {
            factory = buildEventFactory(sourceConfig, physicalSourceConfig, schemaRegistryService);
        } catch (Exception ex) {
            LOG.error("Got exception while building monitored sources for config :" + sourceConfig, ex);
            throw new InvalidConfigException(ex);
        }
        eventFactories.add(factory);
    }
    EventProducer producer = null;
    try {
        producer = new OpenReplicatorEventProducer(eventFactories, eventBuffer, checkpointWriter, physicalSourceConfig, statsCollector, null, null, schemaRegistryService, JMX_DOMAIN);
    } catch (DatabusException e) {
        LOG.error("Got databus exception when instantiating Open Replicator event producer for source : " + physicalSourceConfig.getName(), e);
        throw new InvalidConfigException(e);
    }
    return producer;
}
Also used : DatabusException(com.linkedin.databus2.core.DatabusException) ArrayList(java.util.ArrayList) LogicalSourceStaticConfig(com.linkedin.databus2.relay.config.LogicalSourceStaticConfig) InvalidConfigException(com.linkedin.databus.core.util.InvalidConfigException) DatabusException(com.linkedin.databus2.core.DatabusException) InvalidConfigException(com.linkedin.databus.core.util.InvalidConfigException) UnsupportedKeyException(com.linkedin.databus.core.UnsupportedKeyException) NoSuchSchemaException(com.linkedin.databus2.schemas.NoSuchSchemaException)

Example 14 with InvalidConfigException

use of com.linkedin.databus.core.util.InvalidConfigException in project databus by linkedin.

the class OracleEventProducerFactory method buildEventProducer.

public EventProducer buildEventProducer(PhysicalSourceStaticConfig physicalSourceConfig, SchemaRegistryService schemaRegistryService, DbusEventBufferAppendable dbusEventBuffer, MBeanServer mbeanServer, DbusEventsStatisticsCollector dbusEventsStatisticsCollector, MaxSCNReaderWriter _maxScnReaderWriter) throws DatabusException, EventCreationException, UnsupportedKeyException, SQLException, InvalidConfigException {
    // Make sure the URI from the configuration file identifies an Oracle JDBC source.
    String uri = physicalSourceConfig.getUri();
    if (!uri.startsWith("jdbc:oracle")) {
        throw new InvalidConfigException("Invalid source URI (" + physicalSourceConfig.getUri() + "). Only jdbc:oracle: URIs are supported.");
    }
    // Parse each one of the logical sources
    List<OracleTriggerMonitoredSourceInfo> sources = new ArrayList<OracleTriggerMonitoredSourceInfo>();
    for (LogicalSourceStaticConfig sourceConfig : physicalSourceConfig.getSources()) {
        OracleTriggerMonitoredSourceInfo source = buildOracleMonitoredSourceInfo(sourceConfig, physicalSourceConfig, schemaRegistryService);
        sources.add(source);
    }
    DataSource ds = null;
    try {
        ds = OracleJarUtils.createOracleDataSource(uri);
    } catch (Exception e) {
        String errMsg = "Oracle URI likely not supported. Trouble creating OracleDataSource";
        _log.error(errMsg);
        throw new InvalidConfigException(errMsg + e.getMessage());
    }
    // Create the event producer
    EventProducer eventProducer = new OracleEventProducer(sources, ds, dbusEventBuffer, true, dbusEventsStatisticsCollector, _maxScnReaderWriter, physicalSourceConfig, ManagementFactory.getPlatformMBeanServer());
    _log.info("Created OracleEventProducer for config:  " + physicalSourceConfig + " with slowSourceQueryThreshold = " + physicalSourceConfig.getSlowSourceQueryThreshold());
    return eventProducer;
}
Also used : ArrayList(java.util.ArrayList) OracleEventProducer(com.linkedin.databus2.producers.db.OracleEventProducer) EventProducer(com.linkedin.databus2.producers.EventProducer) OracleEventProducer(com.linkedin.databus2.producers.db.OracleEventProducer) LogicalSourceStaticConfig(com.linkedin.databus2.relay.config.LogicalSourceStaticConfig) InvalidConfigException(com.linkedin.databus.core.util.InvalidConfigException) UnsupportedKeyException(com.linkedin.databus.core.UnsupportedKeyException) SQLException(java.sql.SQLException) NoSuchSchemaException(com.linkedin.databus2.schemas.NoSuchSchemaException) EventCreationException(com.linkedin.databus2.producers.EventCreationException) DatabusException(com.linkedin.databus2.core.DatabusException) InvalidConfigException(com.linkedin.databus.core.util.InvalidConfigException) OracleTriggerMonitoredSourceInfo(com.linkedin.databus2.producers.db.OracleTriggerMonitoredSourceInfo) DataSource(javax.sql.DataSource)

Example 15 with InvalidConfigException

use of com.linkedin.databus.core.util.InvalidConfigException in project databus by linkedin.

the class OracleEventProducerFactory method buildOracleMonitoredSourceInfo.

public OracleTriggerMonitoredSourceInfo buildOracleMonitoredSourceInfo(LogicalSourceStaticConfig sourceConfig, PhysicalSourceStaticConfig pConfig, SchemaRegistryService schemaRegistryService) throws DatabusException, EventCreationException, UnsupportedKeyException, InvalidConfigException {
    String schema = null;
    try {
        schema = schemaRegistryService.fetchLatestSchemaBySourceName(sourceConfig.getName());
    } catch (NoSuchSchemaException e) {
        throw new InvalidConfigException("Unable to load the schema for source (" + sourceConfig.getName() + ").");
    }
    if (schema == null) {
        throw new InvalidConfigException("Unable to load the schema for source (" + sourceConfig.getName() + ").");
    }
    _log.info("Loading schema for source id " + sourceConfig.getId() + ": " + schema);
    String eventViewSchema;
    String eventView;
    if (sourceConfig.getUri().indexOf('.') != -1) {
        String[] parts = sourceConfig.getUri().split("\\.");
        eventViewSchema = parts[0];
        eventView = parts[1];
    } else {
        eventViewSchema = null;
        eventView = sourceConfig.getUri();
    }
    if (eventView.toLowerCase().startsWith("sy$")) {
        eventView = eventView.substring(3);
    }
    PartitionFunction partitionFunction = buildPartitionFunction(sourceConfig);
    EventFactory factory = createEventFactory(eventViewSchema, eventView, sourceConfig, pConfig, schema, partitionFunction);
    EventSourceStatistics statisticsBean = new EventSourceStatistics(sourceConfig.getName());
    OracleTriggerMonitoredSourceInfo sourceInfo = new OracleTriggerMonitoredSourceInfo(sourceConfig.getId(), sourceConfig.getName(), eventViewSchema, eventView, factory, statisticsBean, sourceConfig.getRegularQueryHints(), sourceConfig.getChunkedTxnQueryHints(), sourceConfig.getChunkedScnQueryHints(), sourceConfig.isSkipInfinityScn());
    return sourceInfo;
}
Also used : ConstantPartitionFunction(com.linkedin.databus2.producers.ConstantPartitionFunction) PartitionFunction(com.linkedin.databus2.producers.PartitionFunction) NoSuchSchemaException(com.linkedin.databus2.schemas.NoSuchSchemaException) OracleAvroGenericEventFactory(com.linkedin.databus2.producers.db.OracleAvroGenericEventFactory) EventFactory(com.linkedin.databus2.producers.db.EventFactory) InvalidConfigException(com.linkedin.databus.core.util.InvalidConfigException) EventSourceStatistics(com.linkedin.databus.monitoring.mbean.EventSourceStatistics) OracleTriggerMonitoredSourceInfo(com.linkedin.databus2.producers.db.OracleTriggerMonitoredSourceInfo)

Aggregations

InvalidConfigException (com.linkedin.databus.core.util.InvalidConfigException)29 DatabusException (com.linkedin.databus2.core.DatabusException)7 IOException (java.io.IOException)7 DbusEventBuffer (com.linkedin.databus.core.DbusEventBuffer)6 NoSuchSchemaException (com.linkedin.databus2.schemas.NoSuchSchemaException)6 ArrayList (java.util.ArrayList)6 DbusEventV2Factory (com.linkedin.databus.core.DbusEventV2Factory)5 SimpleObjectCaptureHandler (com.linkedin.databus2.test.container.SimpleObjectCaptureHandler)5 SimpleTestServerConnection (com.linkedin.databus2.test.container.SimpleTestServerConnection)5 ChannelPipeline (org.jboss.netty.channel.ChannelPipeline)5 ChannelPipelineFactory (org.jboss.netty.channel.ChannelPipelineFactory)5 HttpServerCodec (org.jboss.netty.handler.codec.http.HttpServerCodec)5 LoggingHandler (org.jboss.netty.handler.logging.LoggingHandler)5 Log4JLoggerFactory (org.jboss.netty.logging.Log4JLoggerFactory)5 BeforeClass (org.testng.annotations.BeforeClass)5 DatabusHttpClientImpl (com.linkedin.databus.client.DatabusHttpClientImpl)4 UnsupportedKeyException (com.linkedin.databus.core.UnsupportedKeyException)4 OracleTriggerMonitoredSourceInfo (com.linkedin.databus2.producers.db.OracleTriggerMonitoredSourceInfo)4 Test (org.testng.annotations.Test)4 Checkpoint (com.linkedin.databus.core.Checkpoint)3