Search in sources :

Example 1 with HiveInsertTableHandle

use of io.prestosql.plugin.hive.HiveInsertTableHandle in project carbondata by apache.

the class CarbonDataMetaData method beginInsert.

@Override
public CarbonDataInsertTableHandle beginInsert(ConnectorSession session, ConnectorTableHandle tableHandle) {
    HiveInsertTableHandle hiveInsertTableHandle = super.beginInsert(session, tableHandle);
    SchemaTableName tableName = hiveInsertTableHandle.getSchemaTableName();
    Optional<Table> table = this.metastore.getTable(new HiveIdentity(session), tableName.getSchemaName(), tableName.getTableName());
    Path outputPath = new Path(hiveInsertTableHandle.getLocationHandle().getJsonSerializableTargetPath());
    JobConf jobConf = ConfigurationUtils.toJobConf(this.hdfsEnvironment.getConfiguration(new HdfsEnvironment.HdfsContext(session, hiveInsertTableHandle.getSchemaName(), hiveInsertTableHandle.getTableName()), new Path(hiveInsertTableHandle.getLocationHandle().getJsonSerializableWritePath())));
    jobConf.set("location", outputPath.toString());
    Properties hiveSchema = MetastoreUtil.getHiveSchema(table.get());
    try {
        CarbonLoadModel carbonLoadModel = HiveCarbonUtil.getCarbonLoadModel(hiveSchema, jobConf);
        CarbonTableOutputFormat.setLoadModel(jobConf, carbonLoadModel);
    } catch (IOException ex) {
        LOG.error("Error while creating carbon load model", ex);
        throw new RuntimeException(ex);
    }
    try {
        carbonOutputCommitter = new MapredCarbonOutputCommitter();
        jobContext = new JobContextImpl(jobConf, new JobID());
        carbonOutputCommitter.setupJob(jobContext);
        ThreadLocalSessionInfo.setConfigurationToCurrentThread(jobConf);
    } catch (IOException e) {
        LOG.error("error setting the output committer", e);
        throw new RuntimeException("error setting the output committer");
    }
    return new CarbonDataInsertTableHandle(hiveInsertTableHandle.getSchemaTableName().getSchemaName(), hiveInsertTableHandle.getTableName(), hiveInsertTableHandle.getInputColumns(), hiveInsertTableHandle.getPageSinkMetadata(), hiveInsertTableHandle.getLocationHandle(), hiveInsertTableHandle.getBucketProperty(), hiveInsertTableHandle.getTableStorageFormat(), hiveInsertTableHandle.getPartitionStorageFormat(), ImmutableMap.of(CarbonTableConfig.CARBON_PRESTO_LOAD_MODEL, jobContext.getConfiguration().get(CarbonTableOutputFormat.LOAD_MODEL)));
}
Also used : Path(org.apache.hadoop.fs.Path) JobContextImpl(org.apache.hadoop.mapred.JobContextImpl) Table(io.prestosql.plugin.hive.metastore.Table) IOException(java.io.IOException) Properties(java.util.Properties) MapredCarbonOutputCommitter(org.apache.carbondata.hive.MapredCarbonOutputCommitter) SchemaTableName(io.prestosql.spi.connector.SchemaTableName) HiveIdentity(io.prestosql.plugin.hive.authentication.HiveIdentity) CarbonLoadModel(org.apache.carbondata.processing.loading.model.CarbonLoadModel) HiveInsertTableHandle(io.prestosql.plugin.hive.HiveInsertTableHandle) JobConf(org.apache.hadoop.mapred.JobConf) JobID(org.apache.hadoop.mapred.JobID)

Aggregations

HiveInsertTableHandle (io.prestosql.plugin.hive.HiveInsertTableHandle)1 HiveIdentity (io.prestosql.plugin.hive.authentication.HiveIdentity)1 Table (io.prestosql.plugin.hive.metastore.Table)1 SchemaTableName (io.prestosql.spi.connector.SchemaTableName)1 IOException (java.io.IOException)1 Properties (java.util.Properties)1 MapredCarbonOutputCommitter (org.apache.carbondata.hive.MapredCarbonOutputCommitter)1 CarbonLoadModel (org.apache.carbondata.processing.loading.model.CarbonLoadModel)1 Path (org.apache.hadoop.fs.Path)1 JobConf (org.apache.hadoop.mapred.JobConf)1 JobContextImpl (org.apache.hadoop.mapred.JobContextImpl)1 JobID (org.apache.hadoop.mapred.JobID)1