Search in sources :

Example 41 with TestingPrestoServer

use of com.facebook.presto.server.testing.TestingPrestoServer in project presto by prestodb.

the class TestMemoryManager method testOutOfMemoryKiller.

@Test(timeOut = 240_000, expectedExceptions = ExecutionException.class, expectedExceptionsMessageRegExp = ".*Query killed because the cluster is out of memory. Please try again in a few minutes.")
public void testOutOfMemoryKiller() throws Exception {
    Map<String, String> properties = ImmutableMap.<String, String>builder().put("task.verbose-stats", "true").put("query.low-memory-killer.delay", "5s").put("query.low-memory-killer.policy", "total-reservation").build();
    try (DistributedQueryRunner queryRunner = createQueryRunner(properties)) {
        // Reserve all the memory
        QueryId fakeQueryId = new QueryId("fake");
        for (TestingPrestoServer server : queryRunner.getServers()) {
            for (MemoryPool pool : server.getLocalMemoryManager().getPools()) {
                assertTrue(pool.tryReserve(fakeQueryId, "test", pool.getMaxBytes()));
            }
        }
        List<Future<?>> queryFutures = new ArrayList<>();
        for (int i = 0; i < 2; i++) {
            queryFutures.add(executor.submit(() -> queryRunner.execute("SELECT COUNT(*), clerk FROM orders GROUP BY clerk")));
        }
        // Wait for one of the queries to die
        waitForQueryToBeKilled(queryRunner);
        // Release the memory in the reserved pool
        for (TestingPrestoServer server : queryRunner.getServers()) {
            Optional<MemoryPool> reserved = server.getLocalMemoryManager().getReservedPool();
            assertTrue(reserved.isPresent());
            // Free up the entire pool
            reserved.get().free(fakeQueryId, "test", reserved.get().getMaxBytes());
            assertTrue(reserved.get().getFreeBytes() > 0);
        }
        for (Future<?> query : queryFutures) {
            query.get();
        }
    }
}
Also used : DistributedQueryRunner(com.facebook.presto.tests.DistributedQueryRunner) QueryId(com.facebook.presto.spi.QueryId) TestingPrestoServer(com.facebook.presto.server.testing.TestingPrestoServer) ArrayList(java.util.ArrayList) Future(java.util.concurrent.Future) Test(org.testng.annotations.Test)

Example 42 with TestingPrestoServer

use of com.facebook.presto.server.testing.TestingPrestoServer in project presto by prestodb.

the class TestDistributedQueryInfoResource method setup.

@BeforeClass
public void setup() throws Exception {
    client = new JettyHttpClient();
    DistributedQueryRunner runner = createQueryRunner(ImmutableMap.of("query.client.timeout", "20s"), COORDINATOR_COUNT);
    coordinator1 = runner.getCoordinator(0);
    coordinator2 = runner.getCoordinator(1);
    Optional<TestingPrestoServer> resourceManager = runner.getResourceManager();
    checkState(resourceManager.isPresent(), "resource manager not present");
    this.resourceManager = resourceManager.get();
    coordinator1.getResourceGroupManager().get().addConfigurationManagerFactory(new FileResourceGroupConfigurationManagerFactory());
    coordinator1.getResourceGroupManager().get().setConfigurationManager("file", ImmutableMap.of("resource-groups.config-file", getResourceFilePath("resource_groups_config_simple.json")));
    coordinator2.getResourceGroupManager().get().addConfigurationManagerFactory(new FileResourceGroupConfigurationManagerFactory());
    coordinator2.getResourceGroupManager().get().setConfigurationManager("file", ImmutableMap.of("resource-groups.config-file", getResourceFilePath("resource_groups_config_simple.json")));
}
Also used : DistributedQueryRunner(com.facebook.presto.tests.DistributedQueryRunner) JettyHttpClient(com.facebook.airlift.http.client.jetty.JettyHttpClient) TestingPrestoServer(com.facebook.presto.server.testing.TestingPrestoServer) FileResourceGroupConfigurationManagerFactory(com.facebook.presto.resourceGroups.FileResourceGroupConfigurationManagerFactory) BeforeClass(org.testng.annotations.BeforeClass)

Example 43 with TestingPrestoServer

use of com.facebook.presto.server.testing.TestingPrestoServer in project presto by prestodb.

the class TestDistributedResourceGroupInfoResource method setup.

@BeforeClass
public void setup() throws Exception {
    client = new JettyHttpClient();
    DistributedQueryRunner runner = createQueryRunner(ImmutableMap.of("query.client.timeout", "20s"), COORDINATOR_COUNT);
    coordinator1 = runner.getCoordinator(0);
    coordinator2 = runner.getCoordinator(1);
    Optional<TestingPrestoServer> resourceManager = runner.getResourceManager();
    checkState(resourceManager.isPresent(), "resource manager not present");
    this.resourceManager = resourceManager.get();
    coordinator1.getResourceGroupManager().get().addConfigurationManagerFactory(new FileResourceGroupConfigurationManagerFactory());
    coordinator1.getResourceGroupManager().get().setConfigurationManager("file", ImmutableMap.of("resource-groups.config-file", getResourceFilePath("resource_groups_config_simple.json")));
    coordinator2.getResourceGroupManager().get().addConfigurationManagerFactory(new FileResourceGroupConfigurationManagerFactory());
    coordinator2.getResourceGroupManager().get().setConfigurationManager("file", ImmutableMap.of("resource-groups.config-file", getResourceFilePath("resource_groups_config_simple.json")));
}
Also used : DistributedQueryRunner(com.facebook.presto.tests.DistributedQueryRunner) JettyHttpClient(com.facebook.airlift.http.client.jetty.JettyHttpClient) TestingPrestoServer(com.facebook.presto.server.testing.TestingPrestoServer) FileResourceGroupConfigurationManagerFactory(com.facebook.presto.resourceGroups.FileResourceGroupConfigurationManagerFactory) BeforeClass(org.testng.annotations.BeforeClass)

Example 44 with TestingPrestoServer

use of com.facebook.presto.server.testing.TestingPrestoServer in project presto by prestodb.

the class TestServerInfoResource method testGetServerStateWithoutRequiredCoordinators.

@Test(timeOut = 30_000, groups = { "getServerStateWithoutRequiredCoordinators" })
public void testGetServerStateWithoutRequiredCoordinators() {
    TestingPrestoServer server = queryRunner.getCoordinator(0);
    URI uri = uriBuilderFrom(server.getBaseUrl().resolve("/v1/info/state")).build();
    Request request = prepareGet().setHeader(PRESTO_USER, "user").setUri(uri).build();
    NodeState state = client.execute(request, createJsonResponseHandler(jsonCodec(NodeState.class)));
    assertEquals(state, NodeState.INACTIVE);
}
Also used : NodeState(com.facebook.presto.spi.NodeState) TestingPrestoServer(com.facebook.presto.server.testing.TestingPrestoServer) Request(com.facebook.airlift.http.client.Request) URI(java.net.URI) Test(org.testng.annotations.Test)

Example 45 with TestingPrestoServer

use of com.facebook.presto.server.testing.TestingPrestoServer in project presto by prestodb.

the class TestServerInfoResource method testGetServerStateWithoutRequiredResourceManagers.

@Test(timeOut = 30_000, groups = { "getServerStateWithoutRequiredResourceManagers" })
public void testGetServerStateWithoutRequiredResourceManagers() {
    TestingPrestoServer server = queryRunner.getCoordinator(0);
    URI uri = uriBuilderFrom(server.getBaseUrl().resolve("/v1/info/state")).build();
    Request request = prepareGet().setHeader(PRESTO_USER, "user").setUri(uri).build();
    NodeState state = client.execute(request, createJsonResponseHandler(jsonCodec(NodeState.class)));
    assertEquals(state, NodeState.INACTIVE);
}
Also used : NodeState(com.facebook.presto.spi.NodeState) TestingPrestoServer(com.facebook.presto.server.testing.TestingPrestoServer) Request(com.facebook.airlift.http.client.Request) URI(java.net.URI) Test(org.testng.annotations.Test)

Aggregations

TestingPrestoServer (com.facebook.presto.server.testing.TestingPrestoServer)47 Test (org.testng.annotations.Test)19 BeforeClass (org.testng.annotations.BeforeClass)16 DistributedQueryRunner (com.facebook.presto.tests.DistributedQueryRunner)13 ArrayList (java.util.ArrayList)9 JettyHttpClient (com.facebook.airlift.http.client.jetty.JettyHttpClient)7 FileResourceGroupConfigurationManagerFactory (com.facebook.presto.resourceGroups.FileResourceGroupConfigurationManagerFactory)6 NodeState (com.facebook.presto.spi.NodeState)6 QueryId (com.facebook.presto.spi.QueryId)6 Future (java.util.concurrent.Future)6 ServerInfoResource (com.facebook.presto.server.ServerInfoResource)5 BasicQueryInfo (com.facebook.presto.server.BasicQueryInfo)4 TpchPlugin (com.facebook.presto.tpch.TpchPlugin)4 ImmutableMap (com.google.common.collect.ImmutableMap)4 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)4 Response (javax.ws.rs.core.Response)4 Request (com.facebook.airlift.http.client.Request)3 TaskManager (com.facebook.presto.execution.TaskManager)3 BlackHolePlugin (com.facebook.presto.plugin.blackhole.BlackHolePlugin)3 TestShutdownAction (com.facebook.presto.server.testing.TestingPrestoServer.TestShutdownAction)3