Search in sources :

Example 11 with DruidTable

use of org.apache.druid.sql.calcite.table.DruidTable in project druid by druid-io.

the class DruidRelsTest method mockDruidRel.

public static <T extends DruidRel<?>> T mockDruidRel(final Class<T> clazz, final Consumer<T> additionalExpectationsFunction, final PartialDruidQuery.Stage stage, @Nullable DruidTable druidTable, @Nullable Project selectProject, @Nullable Filter whereFilter) {
    // DruidQueryRels rely on a ton of Calcite stuff like RelOptCluster, RelOptTable, etc, which is quite verbose to
    // create real instances of. So, tragically, we'll use EasyMock.
    final PartialDruidQuery mockPartialQuery = EasyMock.mock(PartialDruidQuery.class);
    EasyMock.expect(mockPartialQuery.stage()).andReturn(stage).anyTimes();
    EasyMock.expect(mockPartialQuery.getSelectProject()).andReturn(selectProject).anyTimes();
    EasyMock.expect(mockPartialQuery.getWhereFilter()).andReturn(whereFilter).anyTimes();
    final RelOptTable mockRelOptTable = EasyMock.mock(RelOptTable.class);
    EasyMock.expect(mockRelOptTable.unwrap(DruidTable.class)).andReturn(druidTable).anyTimes();
    final T mockRel = EasyMock.mock(clazz);
    EasyMock.expect(mockRel.getPartialDruidQuery()).andReturn(mockPartialQuery).anyTimes();
    EasyMock.expect(mockRel.getTable()).andReturn(mockRelOptTable).anyTimes();
    if (clazz == DruidQueryRel.class) {
        EasyMock.expect(((DruidQueryRel) mockRel).getDruidTable()).andReturn(druidTable).anyTimes();
    }
    additionalExpectationsFunction.accept(mockRel);
    EasyMock.replay(mockRel, mockPartialQuery, mockRelOptTable);
    return mockRel;
}
Also used : DruidTable(org.apache.druid.sql.calcite.table.DruidTable) RelOptTable(org.apache.calcite.plan.RelOptTable)

Example 12 with DruidTable

use of org.apache.druid.sql.calcite.table.DruidTable in project druid by druid-io.

the class DruidSchema method refresh.

@VisibleForTesting
void refresh(final Set<SegmentId> segmentsToRefresh, final Set<String> dataSourcesToRebuild) throws IOException {
    // Refresh the segments.
    final Set<SegmentId> refreshed = refreshSegments(segmentsToRefresh);
    synchronized (lock) {
        // Add missing segments back to the refresh list.
        segmentsNeedingRefresh.addAll(Sets.difference(segmentsToRefresh, refreshed));
        // Compute the list of dataSources to rebuild tables for.
        dataSourcesToRebuild.addAll(dataSourcesNeedingRebuild);
        refreshed.forEach(segment -> dataSourcesToRebuild.add(segment.getDataSource()));
        dataSourcesNeedingRebuild.clear();
    }
    // Rebuild the dataSources.
    for (String dataSource : dataSourcesToRebuild) {
        final DruidTable druidTable = buildDruidTable(dataSource);
        final DruidTable oldTable = tables.put(dataSource, druidTable);
        final String description = druidTable.getDataSource().isGlobal() ? "global dataSource" : "dataSource";
        if (oldTable == null || !oldTable.getRowSignature().equals(druidTable.getRowSignature())) {
            log.info("%s [%s] has new signature: %s.", description, dataSource, druidTable.getRowSignature());
        } else {
            log.debug("%s [%s] signature is unchanged.", description, dataSource);
        }
    }
}
Also used : SegmentId(org.apache.druid.timeline.SegmentId) DruidTable(org.apache.druid.sql.calcite.table.DruidTable) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 13 with DruidTable

use of org.apache.druid.sql.calcite.table.DruidTable in project druid by druid-io.

the class DruidLogicalValuesRule method onMatch.

@Override
public void onMatch(RelOptRuleCall call) {
    final LogicalValues values = call.rel(0);
    final List<ImmutableList<RexLiteral>> tuples = values.getTuples();
    final List<Object[]> objectTuples = tuples.stream().map(tuple -> tuple.stream().map(v -> getValueFromLiteral(v, plannerContext)).collect(Collectors.toList()).toArray(new Object[0])).collect(Collectors.toList());
    final RowSignature rowSignature = RowSignatures.fromRelDataType(values.getRowType().getFieldNames(), values.getRowType());
    final DruidTable druidTable = new DruidTable(InlineDataSource.fromIterable(objectTuples, rowSignature), rowSignature, null, true, false);
    call.transformTo(DruidQueryRel.scanValues(values, druidTable, plannerContext));
}
Also used : InlineDataSource(org.apache.druid.query.InlineDataSource) RexLiteral(org.apache.calcite.rex.RexLiteral) RowSignatures(org.apache.druid.sql.calcite.table.RowSignatures) Collectors(java.util.stream.Collectors) RelOptRuleCall(org.apache.calcite.plan.RelOptRuleCall) UnsupportedSQLQueryException(org.apache.druid.sql.calcite.planner.UnsupportedSQLQueryException) RelOptRule(org.apache.calcite.plan.RelOptRule) List(java.util.List) ImmutableList(com.google.common.collect.ImmutableList) DruidTable(org.apache.druid.sql.calcite.table.DruidTable) RowSignature(org.apache.druid.segment.column.RowSignature) DruidQueryRel(org.apache.druid.sql.calcite.rel.DruidQueryRel) LogicalValues(org.apache.calcite.rel.logical.LogicalValues) PlannerContext(org.apache.druid.sql.calcite.planner.PlannerContext) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Calcites(org.apache.druid.sql.calcite.planner.Calcites) Nullable(javax.annotation.Nullable) ImmutableList(com.google.common.collect.ImmutableList) DruidTable(org.apache.druid.sql.calcite.table.DruidTable) RowSignature(org.apache.druid.segment.column.RowSignature) LogicalValues(org.apache.calcite.rel.logical.LogicalValues)

Example 14 with DruidTable

use of org.apache.druid.sql.calcite.table.DruidTable in project druid by druid-io.

the class DruidTableScanRule method onMatch.

@Override
public void onMatch(final RelOptRuleCall call) {
    final LogicalTableScan scan = call.rel(0);
    final RelOptTable table = scan.getTable();
    final DruidTable druidTable = table.unwrap(DruidTable.class);
    if (druidTable != null) {
        call.transformTo(DruidQueryRel.scanTable(scan, table, druidTable, plannerContext));
    }
}
Also used : DruidTable(org.apache.druid.sql.calcite.table.DruidTable) RelOptTable(org.apache.calcite.plan.RelOptTable) LogicalTableScan(org.apache.calcite.rel.logical.LogicalTableScan)

Aggregations

DruidTable (org.apache.druid.sql.calcite.table.DruidTable)14 Test (org.junit.Test)6 TableDataSource (org.apache.druid.query.TableDataSource)5 DataSegment (org.apache.druid.timeline.DataSegment)5 CountDownLatch (java.util.concurrent.CountDownLatch)4 GlobalTableDataSource (org.apache.druid.query.GlobalTableDataSource)4 RowSignature (org.apache.druid.segment.column.RowSignature)4 SegmentId (org.apache.druid.timeline.SegmentId)4 VisibleForTesting (com.google.common.annotations.VisibleForTesting)3 BrokerInternalQueryConfig (org.apache.druid.client.BrokerInternalQueryConfig)3 MapJoinableFactory (org.apache.druid.segment.join.MapJoinableFactory)3 NoopEscalator (org.apache.druid.server.security.NoopEscalator)3 NumberedShardSpec (org.apache.druid.timeline.partition.NumberedShardSpec)3 Future (java.util.concurrent.Future)2 JavaTypeFactoryImpl (org.apache.calcite.jdbc.JavaTypeFactoryImpl)2 RelOptTable (org.apache.calcite.plan.RelOptTable)2 RelDataType (org.apache.calcite.rel.type.RelDataType)2 RelDataTypeField (org.apache.calcite.rel.type.RelDataTypeField)2 CallbackAction (org.apache.druid.client.ServerView.CallbackAction)2 TimelineCallback (org.apache.druid.client.TimelineServerView.TimelineCallback)2