Search in sources :

Example 1 with InfobrightNamedPipeLoader

use of com.infobright.io.InfobrightNamedPipeLoader in project pentaho-kettle by pentaho.

the class InfobrightLoaderData method databaseSetup.

void databaseSetup(InfobrightLoaderMeta meta, InfobrightLoader step) throws KettleException {
    db = new Database(step, meta.getDatabaseMeta());
    db.connect();
    // FIXME: This will fail if the first row of the table contains a value that
    // cannot be read by Java. For example, a DATE field that contains the value
    // '0000-00-00'. In this case, the Kettle error message will misleadingly say
    // that the table doesn't exist. There doesn't seem to be any workaround.
    // See Pentaho JIRA: PDI-2117.
    // 
    requiredRowMeta = meta.getRequiredFields(step);
    requiredFields = requiredRowMeta.getFieldNames();
    try {
        // the loader is using it and any other uses of the connection will block.
        if (meta.getInfobrightProductType() == null) {
            // default for ICE
            meta.setDataFormat(DataFormat.TXT_VARIABLE);
        }
        DataFormat dataFormat = DataFormat.valueForDisplayName(meta.getInfobrightProductType());
        int agentPort = meta.getAgentPort();
        Charset charset = meta.getCharset();
        Connection conn = db.getConnection();
        String tableName = meta.getDatabaseMeta().getQuotedSchemaTableCombination(step.environmentSubstitute(meta.getSchemaName()), step.environmentSubstitute(meta.getTableName()));
        EtlLogger logger = new KettleEtlLogger(step);
        loader = new InfobrightNamedPipeLoader(tableName, conn, logger, dataFormat, charset, agentPort);
        loader.setTimeout(30);
        String debugFile = meta.getDebugFile();
        if (debugFile != null) {
            OutputStream debugOutputStream = new FileOutputStream(debugFile);
            loader.setDebugOutputStream(debugOutputStream);
        }
        // TODO set to true to support error path
        record = loader.createRecord(false);
        loader.start();
    } catch (Exception e) {
        db.disconnect();
        db = null;
        if (loader != null) {
            try {
                loader.killQuery();
            } catch (SQLException e1) {
                throw new KettleDatabaseException(e1);
            }
        }
        throw new KettleDatabaseException(e);
    }
}
Also used : InfobrightNamedPipeLoader(com.infobright.io.InfobrightNamedPipeLoader) SQLException(java.sql.SQLException) KettleDatabaseException(org.pentaho.di.core.exception.KettleDatabaseException) OutputStream(java.io.OutputStream) FileOutputStream(java.io.FileOutputStream) Connection(java.sql.Connection) Charset(java.nio.charset.Charset) KettleException(org.pentaho.di.core.exception.KettleException) KettleDatabaseException(org.pentaho.di.core.exception.KettleDatabaseException) SQLException(java.sql.SQLException) EtlLogger(com.infobright.logging.EtlLogger) FileOutputStream(java.io.FileOutputStream) Database(org.pentaho.di.core.database.Database) DataFormat(com.infobright.etl.model.DataFormat)

Aggregations

DataFormat (com.infobright.etl.model.DataFormat)1 InfobrightNamedPipeLoader (com.infobright.io.InfobrightNamedPipeLoader)1 EtlLogger (com.infobright.logging.EtlLogger)1 FileOutputStream (java.io.FileOutputStream)1 OutputStream (java.io.OutputStream)1 Charset (java.nio.charset.Charset)1 Connection (java.sql.Connection)1 SQLException (java.sql.SQLException)1 Database (org.pentaho.di.core.database.Database)1 KettleDatabaseException (org.pentaho.di.core.exception.KettleDatabaseException)1 KettleException (org.pentaho.di.core.exception.KettleException)1