Search in sources :

Example 6 with QueryContext

use of io.prestosql.memory.QueryContext in project hetu-core by openlookeng.

the class AbstractOperatorBenchmark method runOnce.

@Override
protected Map<String, Long> runOnce() {
    Session setSession = testSessionBuilder().setSystemProperty("optimizer.optimize-hash-generation", "true").build();
    MemoryPool memoryPool = new MemoryPool(new MemoryPoolId("test"), new DataSize(1, GIGABYTE));
    SpillSpaceTracker spillSpaceTracker = new SpillSpaceTracker(new DataSize(1, GIGABYTE));
    TaskContext taskContext = new QueryContext(new QueryId("test"), new DataSize(256, MEGABYTE), new DataSize(512, MEGABYTE), memoryPool, new TestingGcMonitor(), localQueryRunner.getExecutor(), localQueryRunner.getScheduler(), new DataSize(256, MEGABYTE), spillSpaceTracker, NOOP_SNAPSHOT_UTILS).addTaskContext(new TaskStateMachine(new TaskId("query", 0, 0), localQueryRunner.getExecutor()), setSession, false, false, OptionalInt.empty(), Optional.empty(), TESTING_SERDE_FACTORY);
    CpuTimer cpuTimer = new CpuTimer();
    Map<String, Long> executionStats = execute(taskContext);
    CpuDuration executionTime = cpuTimer.elapsedTime();
    TaskStats taskStats = taskContext.getTaskStats();
    long inputRows = taskStats.getRawInputPositions();
    long inputBytes = taskStats.getRawInputDataSize().toBytes();
    long outputRows = taskStats.getOutputPositions();
    long outputBytes = taskStats.getOutputDataSize().toBytes();
    double inputMegaBytes = new DataSize(inputBytes, BYTE).getValue(MEGABYTE);
    return ImmutableMap.<String, Long>builder().putAll(executionStats).put("elapsed_millis", executionTime.getWall().toMillis()).put("input_rows_per_second", (long) (inputRows / executionTime.getWall().getValue(SECONDS))).put("output_rows_per_second", (long) (outputRows / executionTime.getWall().getValue(SECONDS))).put("input_megabytes", (long) inputMegaBytes).put("input_megabytes_per_second", (long) (inputMegaBytes / executionTime.getWall().getValue(SECONDS))).put("wall_nanos", executionTime.getWall().roundTo(NANOSECONDS)).put("cpu_nanos", executionTime.getCpu().roundTo(NANOSECONDS)).put("user_nanos", executionTime.getUser().roundTo(NANOSECONDS)).put("input_rows", inputRows).put("input_bytes", inputBytes).put("output_rows", outputRows).put("output_bytes", outputBytes).build();
}
Also used : SpillSpaceTracker(io.prestosql.spiller.SpillSpaceTracker) TaskContext(io.prestosql.operator.TaskContext) TaskId(io.prestosql.execution.TaskId) QueryId(io.prestosql.spi.QueryId) QueryContext(io.prestosql.memory.QueryContext) TaskStats(io.prestosql.operator.TaskStats) TaskStateMachine(io.prestosql.execution.TaskStateMachine) DataSize(io.airlift.units.DataSize) TestingGcMonitor(io.airlift.stats.TestingGcMonitor) CpuTimer(io.airlift.stats.CpuTimer) CpuDuration(io.airlift.stats.CpuTimer.CpuDuration) MemoryPoolId(io.prestosql.spi.memory.MemoryPoolId) Session(io.prestosql.Session) MemoryPool(io.prestosql.memory.MemoryPool)

Example 7 with QueryContext

use of io.prestosql.memory.QueryContext in project hetu-core by openlookeng.

the class TestMemoryRevokingScheduler method newSqlTask.

private SqlTask newSqlTask() {
    TaskId taskId = new TaskId("query", 0, idGeneator.incrementAndGet());
    String instanceId = "Testing instance id";
    URI location = URI.create("fake://task/" + taskId);
    return createSqlTask(taskId, instanceId, location, "fake", new QueryContext(new QueryId("query"), new DataSize(1, MEGABYTE), new DataSize(2, MEGABYTE), memoryPool, new TestingGcMonitor(), executor, scheduledExecutor, new DataSize(1, GIGABYTE), spillSpaceTracker, NOOP_SNAPSHOT_UTILS), sqlTaskExecutionFactory, executor, Functions.identity(), new DataSize(32, MEGABYTE), new CounterStat(), createTestMetadataManager());
}
Also used : CounterStat(io.airlift.stats.CounterStat) QueryId(io.prestosql.spi.QueryId) DataSize(io.airlift.units.DataSize) TestingGcMonitor(io.airlift.stats.TestingGcMonitor) QueryContext(io.prestosql.memory.QueryContext) URI(java.net.URI)

Aggregations

QueryContext (io.prestosql.memory.QueryContext)7 TestingGcMonitor (io.airlift.stats.TestingGcMonitor)6 DataSize (io.airlift.units.DataSize)6 MemoryPool (io.prestosql.memory.MemoryPool)6 QueryId (io.prestosql.spi.QueryId)6 MemoryPoolId (io.prestosql.spi.memory.MemoryPoolId)5 SpillSpaceTracker (io.prestosql.spiller.SpillSpaceTracker)5 TaskId (io.prestosql.execution.TaskId)3 TaskStateMachine (io.prestosql.execution.TaskStateMachine)3 TaskContext (io.prestosql.operator.TaskContext)3 ImmutableList (com.google.common.collect.ImmutableList)2 CounterStat (io.airlift.stats.CounterStat)2 Session (io.prestosql.Session)2 Page (io.prestosql.spi.Page)2 URI (java.net.URI)2 List (java.util.List)2 Map (java.util.Map)2 ScheduledExecutorService (java.util.concurrent.ScheduledExecutorService)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 Preconditions.checkArgument (com.google.common.base.Preconditions.checkArgument)1