Search in sources :

Example 51 with DistributedQueryRunner

use of com.facebook.presto.tests.DistributedQueryRunner in project presto by prestodb.

the class TestHiveDistributedJoinQueriesWithDynamicFiltering method searchScanFilterAndProjectOperatorStats.

private OperatorStats searchScanFilterAndProjectOperatorStats(QueryId queryId, String tableName) {
    DistributedQueryRunner runner = (DistributedQueryRunner) getQueryRunner();
    Plan plan = runner.getQueryPlan(queryId);
    PlanNodeId nodeId = PlanNodeSearcher.searchFrom(plan.getRoot()).where(node -> {
        if (!(node instanceof ProjectNode)) {
            return false;
        }
        ProjectNode projectNode = (ProjectNode) node;
        FilterNode filterNode = (FilterNode) projectNode.getSource();
        TableScanNode tableScanNode = (TableScanNode) filterNode.getSource();
        return tableName.equals(((HiveTableHandle) (tableScanNode.getTable().getConnectorHandle())).getTableName());
    }).findOnlyElement().getId();
    return runner.getCoordinator().getQueryManager().getFullQueryInfo(queryId).getQueryStats().getOperatorSummaries().stream().filter(summary -> nodeId.equals(summary.getPlanNodeId())).collect(MoreCollectors.onlyElement());
}
Also used : PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) AbstractTestJoinQueries(com.facebook.presto.tests.AbstractTestJoinQueries) Assert.assertEquals(org.testng.Assert.assertEquals) QueryRunner(com.facebook.presto.testing.QueryRunner) Test(org.testng.annotations.Test) DistributedQueryRunner(com.facebook.presto.tests.DistributedQueryRunner) OperatorStats(com.facebook.presto.operator.OperatorStats) ENABLE_DYNAMIC_FILTERING(com.facebook.presto.SystemSessionProperties.ENABLE_DYNAMIC_FILTERING) FilterNode(com.facebook.presto.spi.plan.FilterNode) ResultWithQueryId(com.facebook.presto.tests.ResultWithQueryId) Plan(com.facebook.presto.sql.planner.Plan) PUSHDOWN_SUBFIELDS_ENABLED(com.facebook.presto.SystemSessionProperties.PUSHDOWN_SUBFIELDS_ENABLED) HIVE_CATALOG(com.facebook.presto.hive.HiveQueryRunner.HIVE_CATALOG) Session(com.facebook.presto.Session) MoreCollectors(com.google.common.collect.MoreCollectors) PlanNodeSearcher(com.facebook.presto.sql.planner.optimizations.PlanNodeSearcher) JOIN_DISTRIBUTION_TYPE(com.facebook.presto.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE) FeaturesConfig(com.facebook.presto.sql.analyzer.FeaturesConfig) TpchTable.getTables(io.airlift.tpch.TpchTable.getTables) MaterializedResult(com.facebook.presto.testing.MaterializedResult) ProjectNode(com.facebook.presto.spi.plan.ProjectNode) JOIN_REORDERING_STRATEGY(com.facebook.presto.SystemSessionProperties.JOIN_REORDERING_STRATEGY) PUSHDOWN_FILTER_ENABLED(com.facebook.presto.hive.HiveSessionProperties.PUSHDOWN_FILTER_ENABLED) TableScanNode(com.facebook.presto.spi.plan.TableScanNode) Assertions.assertLessThanOrEqual(com.facebook.airlift.testing.Assertions.assertLessThanOrEqual) QueryId(com.facebook.presto.spi.QueryId) Assertions.assertGreaterThan(com.facebook.airlift.testing.Assertions.assertGreaterThan) BROADCAST(com.facebook.presto.sql.analyzer.FeaturesConfig.JoinDistributionType.BROADCAST) DistributedQueryRunner(com.facebook.presto.tests.DistributedQueryRunner) TableScanNode(com.facebook.presto.spi.plan.TableScanNode) FilterNode(com.facebook.presto.spi.plan.FilterNode) ProjectNode(com.facebook.presto.spi.plan.ProjectNode) Plan(com.facebook.presto.sql.planner.Plan)

Example 52 with DistributedQueryRunner

use of com.facebook.presto.tests.DistributedQueryRunner in project presto by prestodb.

the class KafkaQueryRunner method createKafkaQueryRunner.

public static DistributedQueryRunner createKafkaQueryRunner(EmbeddedKafka embeddedKafka, Iterable<TpchTable<?>> tables) throws Exception {
    DistributedQueryRunner queryRunner = null;
    try {
        queryRunner = new DistributedQueryRunner(createSession(), 2);
        queryRunner.installPlugin(new TpchPlugin());
        queryRunner.createCatalog("tpch", "tpch");
        embeddedKafka.start();
        for (TpchTable<?> table : tables) {
            embeddedKafka.createTopics(kafkaTopicName(table));
        }
        Map<SchemaTableName, KafkaTopicDescription> topicDescriptions = createTpchTopicDescriptions(queryRunner.getCoordinator().getMetadata(), tables, embeddedKafka);
        installKafkaPlugin(embeddedKafka, queryRunner, topicDescriptions);
        TestingPrestoClient prestoClient = queryRunner.getRandomClient();
        log.info("Loading data...");
        long startTime = System.nanoTime();
        for (TpchTable<?> table : tables) {
            loadTpchTopic(embeddedKafka, prestoClient, table);
        }
        log.info("Loading complete in %s", nanosSince(startTime).toString(SECONDS));
        return queryRunner;
    } catch (Throwable e) {
        closeAllSuppress(e, queryRunner, embeddedKafka);
        throw e;
    }
}
Also used : TestingPrestoClient(com.facebook.presto.tests.TestingPrestoClient) DistributedQueryRunner(com.facebook.presto.tests.DistributedQueryRunner) TpchPlugin(com.facebook.presto.tpch.TpchPlugin) SchemaTableName(com.facebook.presto.spi.SchemaTableName)

Example 53 with DistributedQueryRunner

use of com.facebook.presto.tests.DistributedQueryRunner in project presto by prestodb.

the class KafkaQueryRunner method main.

public static void main(String[] args) throws Exception {
    Logging.initialize();
    DistributedQueryRunner queryRunner = createKafkaQueryRunner(EmbeddedKafka.createEmbeddedKafka(), TpchTable.getTables());
    Thread.sleep(10);
    Logger log = Logger.get(KafkaQueryRunner.class);
    log.info("======== SERVER STARTED ========");
    log.info("\n====\n%s\n====", queryRunner.getCoordinator().getBaseUrl());
}
Also used : DistributedQueryRunner(com.facebook.presto.tests.DistributedQueryRunner) Logger(com.facebook.airlift.log.Logger)

Example 54 with DistributedQueryRunner

use of com.facebook.presto.tests.DistributedQueryRunner in project presto by prestodb.

the class TestQueryInterceptor method setupServer.

@BeforeMethod
public void setupServer() throws Exception {
    DistributedQueryRunner distributedQueryRunner = createTpchQueryRunner();
    server = distributedQueryRunner.getCoordinator();
    qm = server.getQueryManager();
}
Also used : DistributedQueryRunner(com.facebook.presto.tests.DistributedQueryRunner) BeforeMethod(org.testng.annotations.BeforeMethod)

Example 55 with DistributedQueryRunner

use of com.facebook.presto.tests.DistributedQueryRunner in project presto by prestodb.

the class JmxQueryRunner method createJmxQueryRunner.

@SuppressWarnings("ThrowableResultOfMethodCallIgnored")
public static DistributedQueryRunner createJmxQueryRunner() throws Exception {
    DistributedQueryRunner queryRunner = null;
    try {
        queryRunner = DistributedQueryRunner.builder(createSession()).setNodeCount(3).build();
        queryRunner.installPlugin(new JmxPlugin());
        queryRunner.createCatalog("jmx", "jmx");
        return queryRunner;
    } catch (Throwable e) {
        closeAllSuppress(e, queryRunner);
        throw e;
    }
}
Also used : DistributedQueryRunner(com.facebook.presto.tests.DistributedQueryRunner)

Aggregations

DistributedQueryRunner (com.facebook.presto.tests.DistributedQueryRunner)111 Test (org.testng.annotations.Test)36 TpchPlugin (com.facebook.presto.tpch.TpchPlugin)33 Session (com.facebook.presto.Session)26 QueryId (com.facebook.presto.spi.QueryId)19 Logger (com.facebook.airlift.log.Logger)15 TestingPrestoServer (com.facebook.presto.server.testing.TestingPrestoServer)14 MaterializedResult (com.facebook.presto.testing.MaterializedResult)13 QueryRunner (com.facebook.presto.testing.QueryRunner)13 ImmutableMap (com.google.common.collect.ImmutableMap)10 ResourceGroupManagerPlugin (com.facebook.presto.resourceGroups.ResourceGroupManagerPlugin)9 ArrayList (java.util.ArrayList)8 File (java.io.File)7 BeforeClass (org.testng.annotations.BeforeClass)7 JettyHttpClient (com.facebook.airlift.http.client.jetty.JettyHttpClient)6 FileResourceGroupConfigurationManagerFactory (com.facebook.presto.resourceGroups.FileResourceGroupConfigurationManagerFactory)6 H2ResourceGroupsDao (com.facebook.presto.resourceGroups.db.H2ResourceGroupsDao)6 Path (java.nio.file.Path)6 Future (java.util.concurrent.Future)6 MetastoreContext (com.facebook.presto.hive.metastore.MetastoreContext)5