Search in sources :

Example 46 with PhysicalPlan

use of org.apache.drill.exec.physical.PhysicalPlan in project drill by apache.

the class Foreman method parseAndRunPhysicalPlan.

private void parseAndRunPhysicalPlan(final String json) throws ExecutionSetupException {
    try {
        final PhysicalPlan plan = drillbitContext.getPlanReader().readPhysicalPlan(json);
        runPhysicalPlan(plan);
    } catch (final IOException e) {
        throw new ForemanSetupException("Failure while parsing physical plan.", e);
    }
}
Also used : PhysicalPlan(org.apache.drill.exec.physical.PhysicalPlan) IOException(java.io.IOException)

Example 47 with PhysicalPlan

use of org.apache.drill.exec.physical.PhysicalPlan in project drill by apache.

the class TestHashJoin method testHJMockScanCommon.

private void testHJMockScanCommon(String physicalPlan, int expectedRows) throws Throwable {
    final DrillbitContext bitContext = mockDrillbitContext();
    final UserClientConnection connection = Mockito.mock(UserClientConnection.class);
    final PhysicalPlanReader reader = PhysicalPlanReaderTestFactory.defaultPhysicalPlanReader(c);
    final PhysicalPlan plan = reader.readPhysicalPlan(Files.asCharSource(DrillFileUtils.getResourceAsFile(physicalPlan), Charsets.UTF_8).read());
    final FunctionImplementationRegistry registry = new FunctionImplementationRegistry(c);
    final FragmentContextImpl context = new FragmentContextImpl(bitContext, PlanFragment.getDefaultInstance(), connection, registry);
    final SimpleRootExec exec = new SimpleRootExec(ImplCreator.getExec(context, (FragmentRoot) plan.getSortedOperators(false).iterator().next()));
    int totalRecordCount = 0;
    while (exec.next()) {
        totalRecordCount += exec.getRecordCount();
    }
    exec.close();
    assertEquals(expectedRows, totalRecordCount);
    if (context.getExecutorState().getFailureCause() != null) {
        throw context.getExecutorState().getFailureCause();
    }
    assertTrue(!context.getExecutorState().isFailed());
}
Also used : DrillbitContext(org.apache.drill.exec.server.DrillbitContext) SimpleRootExec(org.apache.drill.exec.physical.impl.SimpleRootExec) PhysicalPlan(org.apache.drill.exec.physical.PhysicalPlan) PhysicalPlanReader(org.apache.drill.exec.planner.PhysicalPlanReader) UserClientConnection(org.apache.drill.exec.rpc.UserClientConnection) FragmentContextImpl(org.apache.drill.exec.ops.FragmentContextImpl) FragmentRoot(org.apache.drill.exec.physical.base.FragmentRoot) FunctionImplementationRegistry(org.apache.drill.exec.expr.fn.FunctionImplementationRegistry)

Example 48 with PhysicalPlan

use of org.apache.drill.exec.physical.PhysicalPlan in project drill by apache.

the class TestMergeJoin method simpleEqualityJoin.

@Test
// this doesn't have a sort.  it also causes an infinite loop.  these may or may not be related.
@Ignore
public void simpleEqualityJoin() throws Throwable {
    final DrillbitContext bitContext = mockDrillbitContext();
    final UserClientConnection connection = Mockito.mock(UserClientConnection.class);
    final PhysicalPlanReader reader = PhysicalPlanReaderTestFactory.defaultPhysicalPlanReader(c);
    final PhysicalPlan plan = reader.readPhysicalPlan(Files.asCharSource(DrillFileUtils.getResourceAsFile("/join/merge_join.json"), Charsets.UTF_8).read());
    final FunctionImplementationRegistry registry = new FunctionImplementationRegistry(c);
    final FragmentContextImpl context = new FragmentContextImpl(bitContext, PlanFragment.getDefaultInstance(), connection, registry);
    final SimpleRootExec exec = new SimpleRootExec(ImplCreator.getExec(context, (FragmentRoot) plan.getSortedOperators(false).iterator().next()));
    int totalRecordCount = 0;
    final StringBuilder sb = new StringBuilder();
    while (exec.next()) {
        totalRecordCount += exec.getRecordCount();
        for (final ValueVector v : exec) {
            sb.append("[" + v.getField().getName() + "]        ");
        }
        sb.append("\n\n");
        for (int valueIdx = 0; valueIdx < exec.getRecordCount(); valueIdx++) {
            final List<Object> row = new ArrayList<>();
            for (final ValueVector v : exec) {
                row.add(v.getAccessor().getObject(valueIdx));
            }
            for (final Object cell : row) {
                if (cell == null) {
                    sb.append("<null>          ");
                    continue;
                }
                final int len = cell.toString().length();
                sb.append(cell);
                for (int i = 0; i < (14 - len); ++i) {
                    sb.append(" ");
                }
            }
            sb.append("\n");
        }
        sb.append("\n");
    }
    logger.info(sb.toString());
    assertEquals(100, totalRecordCount);
    if (context.getExecutorState().getFailureCause() != null) {
        throw context.getExecutorState().getFailureCause();
    }
    assertTrue(!context.getExecutorState().isFailed());
}
Also used : DrillbitContext(org.apache.drill.exec.server.DrillbitContext) PhysicalPlan(org.apache.drill.exec.physical.PhysicalPlan) PhysicalPlanReader(org.apache.drill.exec.planner.PhysicalPlanReader) ArrayList(java.util.ArrayList) FragmentContextImpl(org.apache.drill.exec.ops.FragmentContextImpl) FragmentRoot(org.apache.drill.exec.physical.base.FragmentRoot) ValueVector(org.apache.drill.exec.vector.ValueVector) SimpleRootExec(org.apache.drill.exec.physical.impl.SimpleRootExec) UserClientConnection(org.apache.drill.exec.rpc.UserClientConnection) FunctionImplementationRegistry(org.apache.drill.exec.expr.fn.FunctionImplementationRegistry) Ignore(org.junit.Ignore) SlowTest(org.apache.drill.categories.SlowTest) OperatorTest(org.apache.drill.categories.OperatorTest) Test(org.junit.Test)

Example 49 with PhysicalPlan

use of org.apache.drill.exec.physical.PhysicalPlan in project drill by apache.

the class TestMergeJoin method testJoinBatchSize.

@Test
public void testJoinBatchSize() throws Throwable {
    final DrillbitContext bitContext = mockDrillbitContext();
    final UserClientConnection connection = Mockito.mock(UserClientConnection.class);
    final PhysicalPlanReader reader = PhysicalPlanReaderTestFactory.defaultPhysicalPlanReader(c);
    final PhysicalPlan plan = reader.readPhysicalPlan(Files.asCharSource(DrillFileUtils.getResourceAsFile("/join/join_batchsize.json"), Charsets.UTF_8).read());
    final FunctionImplementationRegistry registry = new FunctionImplementationRegistry(c);
    final FragmentContextImpl context = new FragmentContextImpl(bitContext, PlanFragment.getDefaultInstance(), connection, registry);
    final SimpleRootExec exec = new SimpleRootExec(ImplCreator.getExec(context, (FragmentRoot) plan.getSortedOperators(false).iterator().next()));
    // skip schema batch
    exec.next();
    while (exec.next()) {
        assertEquals(100, exec.getRecordCount());
    }
    if (context.getExecutorState().getFailureCause() != null) {
        throw context.getExecutorState().getFailureCause();
    }
    assertTrue(!context.getExecutorState().isFailed());
}
Also used : DrillbitContext(org.apache.drill.exec.server.DrillbitContext) SimpleRootExec(org.apache.drill.exec.physical.impl.SimpleRootExec) PhysicalPlan(org.apache.drill.exec.physical.PhysicalPlan) PhysicalPlanReader(org.apache.drill.exec.planner.PhysicalPlanReader) UserClientConnection(org.apache.drill.exec.rpc.UserClientConnection) FragmentContextImpl(org.apache.drill.exec.ops.FragmentContextImpl) FragmentRoot(org.apache.drill.exec.physical.base.FragmentRoot) FunctionImplementationRegistry(org.apache.drill.exec.expr.fn.FunctionImplementationRegistry) SlowTest(org.apache.drill.categories.SlowTest) OperatorTest(org.apache.drill.categories.OperatorTest) Test(org.junit.Test)

Example 50 with PhysicalPlan

use of org.apache.drill.exec.physical.PhysicalPlan in project drill by apache.

the class TestMergeJoin method orderedEqualityInnerJoin.

@Test
@Ignore
public void orderedEqualityInnerJoin() throws Throwable {
    final DrillbitContext bitContext = mockDrillbitContext();
    final UserClientConnection connection = Mockito.mock(UserClientConnection.class);
    final PhysicalPlanReader reader = PhysicalPlanReaderTestFactory.defaultPhysicalPlanReader(c, new StoragePluginRegistryImpl(bitContext));
    final PhysicalPlan plan = reader.readPhysicalPlan(Files.asCharSource(DrillFileUtils.getResourceAsFile("/join/merge_inner_single_batch.json"), Charsets.UTF_8).read().replace("#{LEFT_FILE}", DrillFileUtils.getResourceAsFile("/join/merge_single_batch.left.json").toURI().toString()).replace("#{RIGHT_FILE}", DrillFileUtils.getResourceAsFile("/join/merge_single_batch.right.json").toURI().toString()));
    final FunctionImplementationRegistry registry = new FunctionImplementationRegistry(c);
    final FragmentContextImpl context = new FragmentContextImpl(bitContext, PlanFragment.getDefaultInstance(), connection, registry);
    final SimpleRootExec exec = new SimpleRootExec(ImplCreator.getExec(context, (FragmentRoot) plan.getSortedOperators(false).iterator().next()));
    int totalRecordCount = 0;
    final StringBuilder sb = new StringBuilder();
    while (exec.next()) {
        totalRecordCount += exec.getRecordCount();
        sb.append(String.format("got next with record count: %d (total: %d):\n", exec.getRecordCount(), totalRecordCount));
        sb.append("       t1                 t2\n");
        for (int valueIdx = 0; valueIdx < exec.getRecordCount(); valueIdx++) {
            final List<Object> row = Lists.newArrayList();
            for (final ValueVector v : exec) {
                row.add(v.getField().getName() + ":" + v.getAccessor().getObject(valueIdx));
            }
            for (final Object cell : row) {
                if (cell == null) {
                    sb.append("<null>    ");
                    continue;
                }
                final int len = cell.toString().length();
                sb.append(cell + " ");
                for (int i = 0; i < (10 - len); ++i) {
                    sb.append(" ");
                }
            }
            sb.append('\n');
        }
    }
    assertEquals(23, totalRecordCount);
    if (context.getExecutorState().getFailureCause() != null) {
        throw context.getExecutorState().getFailureCause();
    }
    assertTrue(!context.getExecutorState().isFailed());
}
Also used : DrillbitContext(org.apache.drill.exec.server.DrillbitContext) PhysicalPlan(org.apache.drill.exec.physical.PhysicalPlan) PhysicalPlanReader(org.apache.drill.exec.planner.PhysicalPlanReader) FragmentContextImpl(org.apache.drill.exec.ops.FragmentContextImpl) FragmentRoot(org.apache.drill.exec.physical.base.FragmentRoot) ValueVector(org.apache.drill.exec.vector.ValueVector) SimpleRootExec(org.apache.drill.exec.physical.impl.SimpleRootExec) StoragePluginRegistryImpl(org.apache.drill.exec.store.StoragePluginRegistryImpl) UserClientConnection(org.apache.drill.exec.rpc.UserClientConnection) FunctionImplementationRegistry(org.apache.drill.exec.expr.fn.FunctionImplementationRegistry) Ignore(org.junit.Ignore) SlowTest(org.apache.drill.categories.SlowTest) OperatorTest(org.apache.drill.categories.OperatorTest) Test(org.junit.Test)

Aggregations

PhysicalPlan (org.apache.drill.exec.physical.PhysicalPlan)155 FunctionImplementationRegistry (org.apache.drill.exec.expr.fn.FunctionImplementationRegistry)121 FragmentRoot (org.apache.drill.exec.physical.base.FragmentRoot)113 PhysicalPlanReader (org.apache.drill.exec.planner.PhysicalPlanReader)111 Test (org.junit.Test)97 DrillbitContext (org.apache.drill.exec.server.DrillbitContext)80 FragmentContextImpl (org.apache.drill.exec.ops.FragmentContextImpl)78 UserClientConnection (org.apache.drill.exec.rpc.UserClientConnection)70 SimpleRootExec (org.apache.drill.exec.physical.impl.SimpleRootExec)69 SchemaPath (org.apache.drill.common.expression.SchemaPath)46 ExecTest (org.apache.drill.exec.ExecTest)46 FragmentContext (org.apache.drill.exec.ops.FragmentContext)39 OperatorTest (org.apache.drill.categories.OperatorTest)32 PhysicalOperator (org.apache.drill.exec.physical.base.PhysicalOperator)30 ValueVector (org.apache.drill.exec.vector.ValueVector)23 IntVector (org.apache.drill.exec.vector.IntVector)18 BigIntVector (org.apache.drill.exec.vector.BigIntVector)17 StoragePluginRegistryImpl (org.apache.drill.exec.store.StoragePluginRegistryImpl)15 Ignore (org.junit.Ignore)15 SlowTest (org.apache.drill.categories.SlowTest)10