Search in sources :

Example 1 with RelOptTableImpl

use of org.apache.calcite.prepare.RelOptTableImpl in project drill by apache.

the class FileSystemPartitionDescriptor method createTableScan.

@Override
public TableScan createTableScan(List<PartitionLocation> newPartitionLocation, Path cacheFileRoot, boolean wasAllPartitionsPruned, MetadataContext metaContext) throws Exception {
    List<Path> newFiles = new ArrayList<>();
    for (PartitionLocation location : newPartitionLocation) {
        if (!location.isCompositePartition()) {
            newFiles.add(location.getEntirePartitionLocation());
        } else {
            final Collection<SimplePartitionLocation> subPartitions = location.getPartitionLocationRecursive();
            for (PartitionLocation subPart : subPartitions) {
                newFiles.add(subPart.getEntirePartitionLocation());
            }
        }
    }
    FormatSelection formatSelection = (FormatSelection) table.getSelection();
    FileSelection newFileSelection = new FileSelection(null, newFiles, getBaseTableLocation(), cacheFileRoot, wasAllPartitionsPruned, formatSelection.getSelection().getDirStatus());
    newFileSelection.setMetaContext(metaContext);
    RelOptTable relOptTable = scanRel.getTable();
    if (scanRel instanceof DrillScanRel) {
        FileGroupScan newGroupScan = ((FileGroupScan) ((DrillScanRel) scanRel).getGroupScan()).clone(newFileSelection);
        return new DrillScanRel(scanRel.getCluster(), scanRel.getTraitSet().plus(DrillRel.DRILL_LOGICAL), relOptTable, newGroupScan, scanRel.getRowType(), ((DrillScanRel) scanRel).getColumns(), true);
    } else if (scanRel instanceof EnumerableTableScan) {
        FormatSelection newFormatSelection = new FormatSelection(formatSelection.getFormat(), newFileSelection);
        DynamicDrillTable dynamicDrillTable = new DynamicDrillTable(table.getPlugin(), table.getStorageEngineName(), table.getUserName(), newFormatSelection, table.getMetadataProviderManager());
        /* Copy statistics from the original relOptTable */
        DrillTranslatableTable newTable = new DrillTranslatableTable(dynamicDrillTable);
        RelOptTableImpl newOptTableImpl = RelOptTableImpl.create(relOptTable.getRelOptSchema(), relOptTable.getRowType(), newTable, GuavaUtils.convertToUnshadedImmutableList(ImmutableList.of()));
        // return an EnumerableTableScan with fileSelection being part of digest of TableScan node.
        return DirPrunedEnumerableTableScan.create(scanRel.getCluster(), newOptTableImpl, newFileSelection.toString());
    } else {
        throw new UnsupportedOperationException("Only DrillScanRel and EnumerableTableScan is allowed!");
    }
}
Also used : Path(org.apache.hadoop.fs.Path) SchemaPath(org.apache.drill.common.expression.SchemaPath) FileSelection(org.apache.drill.exec.store.dfs.FileSelection) DrillScanRel(org.apache.drill.exec.planner.logical.DrillScanRel) ArrayList(java.util.ArrayList) RelOptTableImpl(org.apache.calcite.prepare.RelOptTableImpl) FormatSelection(org.apache.drill.exec.store.dfs.FormatSelection) FileGroupScan(org.apache.drill.exec.physical.base.FileGroupScan) DrillTranslatableTable(org.apache.drill.exec.planner.logical.DrillTranslatableTable) EnumerableTableScan(org.apache.calcite.adapter.enumerable.EnumerableTableScan) DirPrunedEnumerableTableScan(org.apache.drill.exec.planner.logical.DirPrunedEnumerableTableScan) DynamicDrillTable(org.apache.drill.exec.planner.logical.DynamicDrillTable) RelOptTable(org.apache.calcite.plan.RelOptTable)

Example 2 with RelOptTableImpl

use of org.apache.calcite.prepare.RelOptTableImpl in project drill by apache.

the class FileSystemPartitionDescriptor method createNewTableScanFromSelection.

private TableScan createNewTableScanFromSelection(EnumerableTableScan oldScan, List<String> newFiles, String cacheFileRoot, boolean wasAllPartitionsPruned, MetadataContext metaContext) {
    final RelOptTableImpl t = (RelOptTableImpl) oldScan.getTable();
    final FormatSelection formatSelection = (FormatSelection) table.getSelection();
    final FileSelection newFileSelection = new FileSelection(null, newFiles, getBaseTableLocation(), cacheFileRoot, wasAllPartitionsPruned, formatSelection.getSelection().getDirStatus());
    newFileSelection.setMetaContext(metaContext);
    final FormatSelection newFormatSelection = new FormatSelection(formatSelection.getFormat(), newFileSelection);
    final DrillTranslatableTable newTable = new DrillTranslatableTable(new DynamicDrillTable(table.getPlugin(), table.getStorageEngineName(), table.getUserName(), newFormatSelection));
    final RelOptTableImpl newOptTableImpl = RelOptTableImpl.create(t.getRelOptSchema(), t.getRowType(), newTable);
    // return an EnumerableTableScan with fileSelection being part of digest of TableScan node.
    return DirPrunedEnumerableTableScan.create(oldScan.getCluster(), newOptTableImpl, newFileSelection.toString());
}
Also used : FileSelection(org.apache.drill.exec.store.dfs.FileSelection) DrillTranslatableTable(org.apache.drill.exec.planner.logical.DrillTranslatableTable) DynamicDrillTable(org.apache.drill.exec.planner.logical.DynamicDrillTable) RelOptTableImpl(org.apache.calcite.prepare.RelOptTableImpl) FormatSelection(org.apache.drill.exec.store.dfs.FormatSelection)

Example 3 with RelOptTableImpl

use of org.apache.calcite.prepare.RelOptTableImpl in project drill by axbaretto.

the class FileSystemPartitionDescriptor method createNewTableScanFromSelection.

private TableScan createNewTableScanFromSelection(EnumerableTableScan oldScan, List<String> newFiles, String cacheFileRoot, boolean wasAllPartitionsPruned, MetadataContext metaContext) {
    final RelOptTableImpl t = (RelOptTableImpl) oldScan.getTable();
    final FormatSelection formatSelection = (FormatSelection) table.getSelection();
    final FileSelection newFileSelection = new FileSelection(null, newFiles, getBaseTableLocation(), cacheFileRoot, wasAllPartitionsPruned, formatSelection.getSelection().getDirStatus());
    newFileSelection.setMetaContext(metaContext);
    final FormatSelection newFormatSelection = new FormatSelection(formatSelection.getFormat(), newFileSelection);
    final DrillTranslatableTable newTable = new DrillTranslatableTable(new DynamicDrillTable(table.getPlugin(), table.getStorageEngineName(), table.getUserName(), newFormatSelection));
    final RelOptTableImpl newOptTableImpl = RelOptTableImpl.create(t.getRelOptSchema(), t.getRowType(), newTable, ImmutableList.<String>of());
    // return an EnumerableTableScan with fileSelection being part of digest of TableScan node.
    return DirPrunedEnumerableTableScan.create(oldScan.getCluster(), newOptTableImpl, newFileSelection.toString());
}
Also used : FileSelection(org.apache.drill.exec.store.dfs.FileSelection) DrillTranslatableTable(org.apache.drill.exec.planner.logical.DrillTranslatableTable) DynamicDrillTable(org.apache.drill.exec.planner.logical.DynamicDrillTable) RelOptTableImpl(org.apache.calcite.prepare.RelOptTableImpl) FormatSelection(org.apache.drill.exec.store.dfs.FormatSelection)

Example 4 with RelOptTableImpl

use of org.apache.calcite.prepare.RelOptTableImpl in project hazelcast by hazelcast.

the class OptUtils method createRelTable.

public static HazelcastRelOptTable createRelTable(List<String> names, HazelcastTable hazelcastTable, RelDataTypeFactory typeFactory) {
    RelDataType rowType = hazelcastTable.getRowType(typeFactory);
    RelOptTableImpl relTable = RelOptTableImpl.create(null, rowType, names, hazelcastTable, null);
    return new HazelcastRelOptTable(relTable);
}
Also used : HazelcastRelOptTable(com.hazelcast.jet.sql.impl.schema.HazelcastRelOptTable) RelOptTableImpl(org.apache.calcite.prepare.RelOptTableImpl) RelDataType(org.apache.calcite.rel.type.RelDataType)

Example 5 with RelOptTableImpl

use of org.apache.calcite.prepare.RelOptTableImpl in project drill by apache.

the class ElasticPlanTransformer method visit.

/**
 * Replaces rowType of RelOptTable by rowType obtained from ElasticsearchTable.
 */
@Override
public RelNode visit(TableScan other) {
    RelOptTableImpl table = (RelOptTableImpl) other.getTable();
    ElasticsearchTable elasticsearchTable = Objects.requireNonNull(table.unwrap(ElasticsearchTable.class), "ElasticSearch table cannot be null");
    RelDataType rowType = elasticsearchTable.getRowType(other.getCluster().getTypeFactory());
    mapField = rowType.getFieldList().get(0);
    return new DrillElasticsearchTableScan(other.getCluster(), other.getTraitSet(), table.copy(rowType), elasticsearchTable, rowType);
}
Also used : ElasticsearchTable(org.apache.calcite.adapter.elasticsearch.ElasticsearchTable) RelOptTableImpl(org.apache.calcite.prepare.RelOptTableImpl) RelDataType(org.apache.calcite.rel.type.RelDataType) DrillElasticsearchTableScan(org.apache.calcite.adapter.elasticsearch.DrillElasticsearchTableScan)

Aggregations

RelOptTableImpl (org.apache.calcite.prepare.RelOptTableImpl)5 DrillTranslatableTable (org.apache.drill.exec.planner.logical.DrillTranslatableTable)3 DynamicDrillTable (org.apache.drill.exec.planner.logical.DynamicDrillTable)3 FileSelection (org.apache.drill.exec.store.dfs.FileSelection)3 FormatSelection (org.apache.drill.exec.store.dfs.FormatSelection)3 RelDataType (org.apache.calcite.rel.type.RelDataType)2 HazelcastRelOptTable (com.hazelcast.jet.sql.impl.schema.HazelcastRelOptTable)1 ArrayList (java.util.ArrayList)1 Properties (java.util.Properties)1 CalciteConnectionConfigImpl (org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.config.CalciteConnectionConfigImpl)1 RelOptTable (org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelOptTable)1 CalciteCatalogReader (org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.prepare.CalciteCatalogReader)1 RelOptTableImpl (org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.prepare.RelOptTableImpl)1 SchemaPlus (org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.schema.SchemaPlus)1 Table (org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.schema.Table)1 TranslatableTable (org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.schema.TranslatableTable)1 DrillElasticsearchTableScan (org.apache.calcite.adapter.elasticsearch.DrillElasticsearchTableScan)1 ElasticsearchTable (org.apache.calcite.adapter.elasticsearch.ElasticsearchTable)1 EnumerableTableScan (org.apache.calcite.adapter.enumerable.EnumerableTableScan)1 RelOptTable (org.apache.calcite.plan.RelOptTable)1