Search in sources :

Example 1 with StorageStrategy

use of org.apache.drill.exec.store.StorageStrategy in project drill by apache.

the class TestCTAS method createTableWithCustomUmask.

@Test
public void createTableWithCustomUmask() throws Exception {
    test("use %s", TEMP_SCHEMA);
    String tableName = "with_custom_permission";
    StorageStrategy storageStrategy = new StorageStrategy("000", false);
    try (FileSystem fs = FileSystem.get(new Configuration())) {
        test("alter session set `%s` = '%s'", ExecConstants.PERSISTENT_TABLE_UMASK, storageStrategy.getUmask());
        test("create table %s as select 'A' from (values(1))", tableName);
        Path tableLocation = new Path(getDfsTestTmpSchemaLocation(), tableName);
        assertEquals("Directory permission should match", storageStrategy.getFolderPermission(), fs.getFileStatus(tableLocation).getPermission());
        assertEquals("File permission should match", storageStrategy.getFilePermission(), fs.listLocatedStatus(tableLocation).next().getPermission());
    } finally {
        test("alter session reset `%s`", ExecConstants.PERSISTENT_TABLE_UMASK);
        test("drop table if exists %s", tableName);
    }
}
Also used : Path(org.apache.hadoop.fs.Path) StorageStrategy(org.apache.drill.exec.store.StorageStrategy) Configuration(org.apache.hadoop.conf.Configuration) FileSystem(org.apache.hadoop.fs.FileSystem) Test(org.junit.Test)

Example 2 with StorageStrategy

use of org.apache.drill.exec.store.StorageStrategy in project drill by apache.

the class CreateTableHandler method getPlan.

@Override
public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException, RelConversionException, IOException, ForemanSetupException {
    SqlCreateTable sqlCreateTable = unwrap(sqlNode, SqlCreateTable.class);
    String originalTableName = sqlCreateTable.getName();
    final ConvertedRelNode convertedRelNode = validateAndConvert(sqlCreateTable.getQuery());
    final RelDataType validatedRowType = convertedRelNode.getValidatedRowType();
    final RelNode queryRelNode = convertedRelNode.getConvertedNode();
    final RelNode newTblRelNode = SqlHandlerUtil.resolveNewTableRel(false, sqlCreateTable.getFieldNames(), validatedRowType, queryRelNode);
    final DrillConfig drillConfig = context.getConfig();
    final AbstractSchema drillSchema = resolveSchema(sqlCreateTable, config.getConverter().getDefaultSchema(), drillConfig);
    checkDuplicatedObjectExistence(drillSchema, originalTableName, drillConfig, context.getSession());
    final RelNode newTblRelNodeWithPCol = SqlHandlerUtil.qualifyPartitionCol(newTblRelNode, sqlCreateTable.getPartitionColumns());
    log("Calcite", newTblRelNodeWithPCol, logger, null);
    // Convert the query to Drill Logical plan and insert a writer operator on top.
    StorageStrategy storageStrategy = sqlCreateTable.isTemporary() ? StorageStrategy.TEMPORARY : new StorageStrategy(context.getOption(ExecConstants.PERSISTENT_TABLE_UMASK).string_val, false);
    // If we are creating temporary table, initial table name will be replaced with generated table name.
    // Generated table name is unique, UUID.randomUUID() is used for its generation.
    // Original table name is stored in temporary tables cache, so it can be substituted to generated one during querying.
    String newTableName = sqlCreateTable.isTemporary() ? context.getSession().registerTemporaryTable(drillSchema, originalTableName, drillConfig) : originalTableName;
    DrillRel drel = convertToDrel(newTblRelNodeWithPCol, drillSchema, newTableName, sqlCreateTable.getPartitionColumns(), newTblRelNode.getRowType(), storageStrategy);
    Prel prel = convertToPrel(drel, newTblRelNode.getRowType(), sqlCreateTable.getPartitionColumns());
    logAndSetTextPlan("Drill Physical", prel, logger);
    PhysicalOperator pop = convertToPop(prel);
    PhysicalPlan plan = convertToPlan(pop);
    log("Drill Plan", plan, logger);
    String message = String.format("Creating %s table [%s].", sqlCreateTable.isTemporary() ? "temporary" : "persistent", originalTableName);
    logger.info(message);
    return plan;
}
Also used : StorageStrategy(org.apache.drill.exec.store.StorageStrategy) PhysicalPlan(org.apache.drill.exec.physical.PhysicalPlan) RelNode(org.apache.calcite.rel.RelNode) DrillConfig(org.apache.drill.common.config.DrillConfig) AbstractSchema(org.apache.drill.exec.store.AbstractSchema) PhysicalOperator(org.apache.drill.exec.physical.base.PhysicalOperator) DrillRel(org.apache.drill.exec.planner.logical.DrillRel) RelDataType(org.apache.calcite.rel.type.RelDataType) SqlCreateTable(org.apache.drill.exec.planner.sql.parser.SqlCreateTable) WriterPrel(org.apache.drill.exec.planner.physical.WriterPrel) Prel(org.apache.drill.exec.planner.physical.Prel) ProjectAllowDupPrel(org.apache.drill.exec.planner.physical.ProjectAllowDupPrel) ProjectPrel(org.apache.drill.exec.planner.physical.ProjectPrel)

Aggregations

StorageStrategy (org.apache.drill.exec.store.StorageStrategy)2 RelNode (org.apache.calcite.rel.RelNode)1 RelDataType (org.apache.calcite.rel.type.RelDataType)1 DrillConfig (org.apache.drill.common.config.DrillConfig)1 PhysicalPlan (org.apache.drill.exec.physical.PhysicalPlan)1 PhysicalOperator (org.apache.drill.exec.physical.base.PhysicalOperator)1 DrillRel (org.apache.drill.exec.planner.logical.DrillRel)1 Prel (org.apache.drill.exec.planner.physical.Prel)1 ProjectAllowDupPrel (org.apache.drill.exec.planner.physical.ProjectAllowDupPrel)1 ProjectPrel (org.apache.drill.exec.planner.physical.ProjectPrel)1 WriterPrel (org.apache.drill.exec.planner.physical.WriterPrel)1 SqlCreateTable (org.apache.drill.exec.planner.sql.parser.SqlCreateTable)1 AbstractSchema (org.apache.drill.exec.store.AbstractSchema)1 Configuration (org.apache.hadoop.conf.Configuration)1 FileSystem (org.apache.hadoop.fs.FileSystem)1 Path (org.apache.hadoop.fs.Path)1 Test (org.junit.Test)1