Search in sources :

Example 26 with InMemoryNodeManager

use of io.trino.metadata.InMemoryNodeManager in project trino by trinodb.

the class TestFullNodeCapableNodeAllocator method testAllocateSharedReleaseBeforeAcquired.

@Test(timeOut = TEST_TIMEOUT)
public void testAllocateSharedReleaseBeforeAcquired() throws Exception {
    InMemoryNodeManager nodeManager = testingNodeManager(basicNodesMap(NODE_1));
    setupNodeAllocatorService(nodeManager, 1);
    try (NodeAllocator nodeAllocator = nodeAllocatorService.getNodeAllocator(Q1_SESSION)) {
        // first two allocation should not block
        NodeAllocator.NodeLease acquire1 = nodeAllocator.acquire(NO_REQUIREMENTS);
        assertAcquired(acquire1, NODE_1);
        NodeAllocator.NodeLease acquire2 = nodeAllocator.acquire(NO_REQUIREMENTS);
        assertAcquired(acquire2, NODE_1);
        // another two should block
        NodeAllocator.NodeLease acquire3 = nodeAllocator.acquire(NO_REQUIREMENTS);
        assertNotAcquired(acquire3);
        NodeAllocator.NodeLease acquire4 = nodeAllocator.acquire(NO_REQUIREMENTS);
        assertNotAcquired(acquire4);
        // releasing a blocked one should not unblock anything
        acquire3.release();
        assertNotAcquired(acquire4);
        // releasing an acquired one should unblock one which is still blocked
        acquire2.release();
        assertEventually(() -> assertAcquired(acquire4, NODE_1));
    }
}
Also used : InMemoryNodeManager(io.trino.metadata.InMemoryNodeManager) Test(org.testng.annotations.Test)

Example 27 with InMemoryNodeManager

use of io.trino.metadata.InMemoryNodeManager in project trino by trinodb.

the class TestDetermineTableScanNodePartitioning method setUp.

@BeforeClass
public void setUp() {
    tester = defaultRuleTester();
    tester.getQueryRunner().createCatalog(MOCK_CATALOG, createMockFactory(), ImmutableMap.of());
    tester.getQueryRunner().getNodePartitioningManager().addPartitioningProvider(new CatalogName(MOCK_CATALOG), new TestPartitioningProvider(new InMemoryNodeManager()));
}
Also used : CatalogName(io.trino.connector.CatalogName) TestPartitioningProvider(io.trino.sql.planner.TestTableScanNodePartitioning.TestPartitioningProvider) InMemoryNodeManager(io.trino.metadata.InMemoryNodeManager) BeforeClass(org.testng.annotations.BeforeClass)

Example 28 with InMemoryNodeManager

use of io.trino.metadata.InMemoryNodeManager in project trino by trinodb.

the class TestHashJoinOperator method setUp.

@BeforeMethod
public void setUp() {
    // Before/AfterMethod is chosen here because the executor needs to be shutdown
    // after every single test case to terminate outstanding threads, if any.
    // The line below is the same as newCachedThreadPool(daemonThreadsNamed(...)) except RejectionExecutionHandler.
    // RejectionExecutionHandler is set to DiscardPolicy (instead of the default AbortPolicy) here.
    // Otherwise, a large number of RejectedExecutionException will flood logging, resulting in Travis failure.
    executor = new ThreadPoolExecutor(0, Integer.MAX_VALUE, 60L, SECONDS, new SynchronousQueue<>(), daemonThreadsNamed("test-executor-%s"), new ThreadPoolExecutor.DiscardPolicy());
    scheduledExecutor = newScheduledThreadPool(2, daemonThreadsNamed(getClass().getSimpleName() + "-scheduledExecutor-%s"));
    NodeScheduler nodeScheduler = new NodeScheduler(new UniformNodeSelectorFactory(new InMemoryNodeManager(), new NodeSchedulerConfig().setIncludeCoordinator(true), new NodeTaskMap(new FinalizerService())));
    nodePartitioningManager = new NodePartitioningManager(nodeScheduler, new BlockTypeOperators(new TypeOperators()));
}
Also used : BlockTypeOperators(io.trino.type.BlockTypeOperators) NodeTaskMap(io.trino.execution.NodeTaskMap) FinalizerService(io.trino.util.FinalizerService) UniformNodeSelectorFactory(io.trino.execution.scheduler.UniformNodeSelectorFactory) SynchronousQueue(java.util.concurrent.SynchronousQueue) NodeScheduler(io.trino.execution.scheduler.NodeScheduler) NodeSchedulerConfig(io.trino.execution.scheduler.NodeSchedulerConfig) ThreadPoolExecutor(java.util.concurrent.ThreadPoolExecutor) NodePartitioningManager(io.trino.sql.planner.NodePartitioningManager) InMemoryNodeManager(io.trino.metadata.InMemoryNodeManager) TypeOperators(io.trino.spi.type.TypeOperators) BlockTypeOperators(io.trino.type.BlockTypeOperators) BeforeMethod(org.testng.annotations.BeforeMethod)

Aggregations

InMemoryNodeManager (io.trino.metadata.InMemoryNodeManager)28 Test (org.testng.annotations.Test)20 InternalNode (io.trino.metadata.InternalNode)7 NodeTaskMap (io.trino.execution.NodeTaskMap)6 NodeScheduler (io.trino.execution.scheduler.NodeScheduler)5 NodeSchedulerConfig (io.trino.execution.scheduler.NodeSchedulerConfig)5 UniformNodeSelectorFactory (io.trino.execution.scheduler.UniformNodeSelectorFactory)5 PlanFragment (io.trino.sql.planner.PlanFragment)5 CatalogName (io.trino.connector.CatalogName)4 MockRemoteTask (io.trino.execution.MockRemoteTaskFactory.MockRemoteTask)4 PartitionedSplitsInfo (io.trino.execution.PartitionedSplitsInfo)4 RemoteTask (io.trino.execution.RemoteTask)4 PipelinedStageExecution.createPipelinedStageExecution (io.trino.execution.scheduler.PipelinedStageExecution.createPipelinedStageExecution)4 SourcePartitionedScheduler.newSourcePartitionedSchedulerAsStageScheduler (io.trino.execution.scheduler.SourcePartitionedScheduler.newSourcePartitionedSchedulerAsStageScheduler)4 TestingSplit (io.trino.testing.TestingSplit)4 FinalizerService (io.trino.util.FinalizerService)4 ImmutableList (com.google.common.collect.ImmutableList)3 NodePartitioningManager (io.trino.sql.planner.NodePartitioningManager)3 BlockTypeOperators (io.trino.type.BlockTypeOperators)3 ImmutableMap (com.google.common.collect.ImmutableMap)2