Search in sources :

Example 6 with BlockEncodingManager

use of com.facebook.presto.block.BlockEncodingManager in project presto by prestodb.

the class TestFunctionRegistry method testDuplicateFunctions.

@Test(expectedExceptions = IllegalArgumentException.class, expectedExceptionsMessageRegExp = "\\QFunction already registered: custom_add(bigint,bigint):bigint\\E")
public void testDuplicateFunctions() {
    List<SqlFunction> functions = new FunctionListBuilder().scalars(CustomFunctions.class).getFunctions().stream().filter(input -> input.getSignature().getName().equals("custom_add")).collect(toImmutableList());
    TypeRegistry typeManager = new TypeRegistry();
    FunctionRegistry registry = new FunctionRegistry(typeManager, new BlockEncodingManager(typeManager), new FeaturesConfig());
    registry.addFunctions(functions);
    registry.addFunctions(functions);
}
Also used : TypeSignature(com.facebook.presto.spi.type.TypeSignature) QualifiedName(com.facebook.presto.sql.tree.QualifiedName) TypeManager(com.facebook.presto.spi.type.TypeManager) TIMESTAMP_WITH_TIME_ZONE(com.facebook.presto.spi.type.TimestampWithTimeZoneType.TIMESTAMP_WITH_TIME_ZONE) TypeRegistry(com.facebook.presto.type.TypeRegistry) Assert.assertEquals(org.testng.Assert.assertEquals) ScalarFunctionImplementation(com.facebook.presto.operator.scalar.ScalarFunctionImplementation) Test(org.testng.annotations.Test) FunctionRegistry.mangleOperatorName(com.facebook.presto.metadata.FunctionRegistry.mangleOperatorName) Lists.transform(com.google.common.collect.Lists.transform) BlockEncodingSerde(com.facebook.presto.spi.block.BlockEncodingSerde) SCALAR(com.facebook.presto.metadata.FunctionKind.SCALAR) ImmutableList(com.google.common.collect.ImmutableList) FunctionRegistry.getMagicLiteralFunctionSignature(com.facebook.presto.metadata.FunctionRegistry.getMagicLiteralFunctionSignature) TypeSignatureProvider.fromTypeSignatures(com.facebook.presto.sql.analyzer.TypeSignatureProvider.fromTypeSignatures) Signature.typeVariable(com.facebook.presto.metadata.Signature.typeVariable) StandardTypes(com.facebook.presto.spi.type.StandardTypes) ImmutableCollectors.toImmutableList(com.facebook.presto.util.ImmutableCollectors.toImmutableList) Assert.assertFalse(org.testng.Assert.assertFalse) Functions(com.google.common.base.Functions) ImmutableSet(com.google.common.collect.ImmutableSet) FunctionRegistry.unmangleOperator(com.facebook.presto.metadata.FunctionRegistry.unmangleOperator) HYPER_LOG_LOG(com.facebook.presto.spi.type.HyperLogLogType.HYPER_LOG_LOG) MethodHandles(java.lang.invoke.MethodHandles) Assert.fail(org.testng.Assert.fail) String.format(java.lang.String.format) CustomFunctions(com.facebook.presto.operator.scalar.CustomFunctions) TypeUtils.resolveTypes(com.facebook.presto.type.TypeUtils.resolveTypes) FeaturesConfig(com.facebook.presto.sql.analyzer.FeaturesConfig) List(java.util.List) Collectors.toList(java.util.stream.Collectors.toList) OperatorType(com.facebook.presto.spi.function.OperatorType) Assert.assertTrue(org.testng.Assert.assertTrue) TypeSignature.parseTypeSignature(com.facebook.presto.spi.type.TypeSignature.parseTypeSignature) ScalarFunction(com.facebook.presto.spi.function.ScalarFunction) Collections(java.util.Collections) SqlType(com.facebook.presto.spi.function.SqlType) BlockEncodingManager(com.facebook.presto.block.BlockEncodingManager) BlockEncodingManager(com.facebook.presto.block.BlockEncodingManager) FeaturesConfig(com.facebook.presto.sql.analyzer.FeaturesConfig) TypeRegistry(com.facebook.presto.type.TypeRegistry) Test(org.testng.annotations.Test)

Example 7 with BlockEncodingManager

use of com.facebook.presto.block.BlockEncodingManager in project presto by prestodb.

the class TestFunctionRegistry method testListingHiddenFunctions.

@Test
public void testListingHiddenFunctions() throws Exception {
    TypeRegistry typeManager = new TypeRegistry();
    FunctionRegistry registry = new FunctionRegistry(typeManager, new BlockEncodingManager(typeManager), new FeaturesConfig());
    List<SqlFunction> functions = registry.list();
    List<String> names = transform(functions, input -> input.getSignature().getName());
    assertTrue(names.contains("length"), "Expected function names " + names + " to contain 'length'");
    assertTrue(names.contains("stddev"), "Expected function names " + names + " to contain 'stddev'");
    assertTrue(names.contains("rank"), "Expected function names " + names + " to contain 'rank'");
    assertFalse(names.contains("like"), "Expected function names " + names + " not to contain 'like'");
}
Also used : BlockEncodingManager(com.facebook.presto.block.BlockEncodingManager) FeaturesConfig(com.facebook.presto.sql.analyzer.FeaturesConfig) TypeRegistry(com.facebook.presto.type.TypeRegistry) Test(org.testng.annotations.Test)

Example 8 with BlockEncodingManager

use of com.facebook.presto.block.BlockEncodingManager in project presto by prestodb.

the class TestFunctionRegistry method testMagicLiteralFunction.

@Test
public void testMagicLiteralFunction() {
    Signature signature = getMagicLiteralFunctionSignature(TIMESTAMP_WITH_TIME_ZONE);
    assertEquals(signature.getName(), "$literal$timestamp with time zone");
    assertEquals(signature.getArgumentTypes(), ImmutableList.of(parseTypeSignature(StandardTypes.BIGINT)));
    assertEquals(signature.getReturnType().getBase(), StandardTypes.TIMESTAMP_WITH_TIME_ZONE);
    TypeRegistry typeManager = new TypeRegistry();
    FunctionRegistry registry = new FunctionRegistry(typeManager, new BlockEncodingManager(typeManager), new FeaturesConfig());
    Signature function = registry.resolveFunction(QualifiedName.of(signature.getName()), fromTypeSignatures(signature.getArgumentTypes()));
    assertEquals(function.getArgumentTypes(), ImmutableList.of(parseTypeSignature(StandardTypes.BIGINT)));
    assertEquals(signature.getReturnType().getBase(), StandardTypes.TIMESTAMP_WITH_TIME_ZONE);
}
Also used : BlockEncodingManager(com.facebook.presto.block.BlockEncodingManager) FeaturesConfig(com.facebook.presto.sql.analyzer.FeaturesConfig) TypeSignature(com.facebook.presto.spi.type.TypeSignature) FunctionRegistry.getMagicLiteralFunctionSignature(com.facebook.presto.metadata.FunctionRegistry.getMagicLiteralFunctionSignature) TypeSignature.parseTypeSignature(com.facebook.presto.spi.type.TypeSignature.parseTypeSignature) TypeRegistry(com.facebook.presto.type.TypeRegistry) Test(org.testng.annotations.Test)

Example 9 with BlockEncodingManager

use of com.facebook.presto.block.BlockEncodingManager in project presto by prestodb.

the class TestAnalyzer method setup.

@BeforeMethod(alwaysRun = true)
public void setup() throws Exception {
    TypeManager typeManager = new TypeRegistry();
    CatalogManager catalogManager = new CatalogManager();
    transactionManager = createTestTransactionManager(catalogManager);
    accessControl = new AccessControlManager(transactionManager);
    metadata = new MetadataManager(new FeaturesConfig(), typeManager, new BlockEncodingManager(typeManager), new SessionPropertyManager(), new SchemaPropertyManager(), new TablePropertyManager(), transactionManager);
    metadata.getFunctionRegistry().addFunctions(ImmutableList.of(APPLY_FUNCTION));
    catalogManager.registerCatalog(createTestingCatalog(TPCH_CATALOG, TPCH_CONNECTOR_ID));
    catalogManager.registerCatalog(createTestingCatalog(SECOND_CATALOG, SECOND_CONNECTOR_ID));
    catalogManager.registerCatalog(createTestingCatalog(THIRD_CATALOG, THIRD_CONNECTOR_ID));
    SchemaTableName table1 = new SchemaTableName("s1", "t1");
    inSetupTransaction(session -> metadata.createTable(session, TPCH_CATALOG, new ConnectorTableMetadata(table1, ImmutableList.of(new ColumnMetadata("a", BIGINT), new ColumnMetadata("b", BIGINT), new ColumnMetadata("c", BIGINT), new ColumnMetadata("d", BIGINT)))));
    SchemaTableName table2 = new SchemaTableName("s1", "t2");
    inSetupTransaction(session -> metadata.createTable(session, TPCH_CATALOG, new ConnectorTableMetadata(table2, ImmutableList.of(new ColumnMetadata("a", BIGINT), new ColumnMetadata("b", BIGINT)))));
    SchemaTableName table3 = new SchemaTableName("s1", "t3");
    inSetupTransaction(session -> metadata.createTable(session, TPCH_CATALOG, new ConnectorTableMetadata(table3, ImmutableList.of(new ColumnMetadata("a", BIGINT), new ColumnMetadata("b", BIGINT), new ColumnMetadata("x", BIGINT, null, true)))));
    // table in different catalog
    SchemaTableName table4 = new SchemaTableName("s2", "t4");
    inSetupTransaction(session -> metadata.createTable(session, SECOND_CATALOG, new ConnectorTableMetadata(table4, ImmutableList.of(new ColumnMetadata("a", BIGINT)))));
    // table with a hidden column
    SchemaTableName table5 = new SchemaTableName("s1", "t5");
    inSetupTransaction(session -> metadata.createTable(session, TPCH_CATALOG, new ConnectorTableMetadata(table5, ImmutableList.of(new ColumnMetadata("a", BIGINT), new ColumnMetadata("b", BIGINT, null, true)))));
    // table with a varchar column
    SchemaTableName table6 = new SchemaTableName("s1", "t6");
    inSetupTransaction(session -> metadata.createTable(session, TPCH_CATALOG, new ConnectorTableMetadata(table6, ImmutableList.of(new ColumnMetadata("a", BIGINT), new ColumnMetadata("b", VARCHAR), new ColumnMetadata("c", BIGINT), new ColumnMetadata("d", BIGINT)))));
    // table with bigint, double, array of bigints and array of doubles column
    SchemaTableName table7 = new SchemaTableName("s1", "t7");
    inSetupTransaction(session -> metadata.createTable(session, TPCH_CATALOG, new ConnectorTableMetadata(table7, ImmutableList.of(new ColumnMetadata("a", BIGINT), new ColumnMetadata("b", DOUBLE), new ColumnMetadata("c", new ArrayType(BIGINT)), new ColumnMetadata("d", new ArrayType(DOUBLE))))));
    // valid view referencing table in same schema
    String viewData1 = JsonCodec.jsonCodec(ViewDefinition.class).toJson(new ViewDefinition("select a from t1", Optional.of(TPCH_CATALOG), Optional.of("s1"), ImmutableList.of(new ViewColumn("a", BIGINT)), Optional.of("user")));
    inSetupTransaction(session -> metadata.createView(session, new QualifiedObjectName(TPCH_CATALOG, "s1", "v1"), viewData1, false));
    // stale view (different column type)
    String viewData2 = JsonCodec.jsonCodec(ViewDefinition.class).toJson(new ViewDefinition("select a from t1", Optional.of(TPCH_CATALOG), Optional.of("s1"), ImmutableList.of(new ViewColumn("a", VARCHAR)), Optional.of("user")));
    inSetupTransaction(session -> metadata.createView(session, new QualifiedObjectName(TPCH_CATALOG, "s1", "v2"), viewData2, false));
    // view referencing table in different schema from itself and session
    String viewData3 = JsonCodec.jsonCodec(ViewDefinition.class).toJson(new ViewDefinition("select a from t4", Optional.of(SECOND_CATALOG), Optional.of("s2"), ImmutableList.of(new ViewColumn("a", BIGINT)), Optional.of("owner")));
    inSetupTransaction(session -> metadata.createView(session, new QualifiedObjectName(THIRD_CATALOG, "s3", "v3"), viewData3, false));
    // valid view with uppercase column name
    String viewData4 = JsonCodec.jsonCodec(ViewDefinition.class).toJson(new ViewDefinition("select A from t1", Optional.of("tpch"), Optional.of("s1"), ImmutableList.of(new ViewColumn("a", BIGINT)), Optional.of("user")));
    inSetupTransaction(session -> metadata.createView(session, new QualifiedObjectName("tpch", "s1", "v4"), viewData4, false));
    // recursive view referencing to itself
    String viewData5 = JsonCodec.jsonCodec(ViewDefinition.class).toJson(new ViewDefinition("select * from v5", Optional.of(TPCH_CATALOG), Optional.of("s1"), ImmutableList.of(new ViewColumn("a", BIGINT)), Optional.of("user")));
    inSetupTransaction(session -> metadata.createView(session, new QualifiedObjectName(TPCH_CATALOG, "s1", "v5"), viewData5, false));
    this.metadata = metadata;
}
Also used : AccessControlManager(com.facebook.presto.security.AccessControlManager) ColumnMetadata(com.facebook.presto.spi.ColumnMetadata) ViewColumn(com.facebook.presto.metadata.ViewDefinition.ViewColumn) ViewDefinition(com.facebook.presto.metadata.ViewDefinition) TypeRegistry(com.facebook.presto.type.TypeRegistry) SchemaTableName(com.facebook.presto.spi.SchemaTableName) CatalogManager(com.facebook.presto.metadata.CatalogManager) QualifiedObjectName(com.facebook.presto.metadata.QualifiedObjectName) ArrayType(com.facebook.presto.type.ArrayType) MetadataManager(com.facebook.presto.metadata.MetadataManager) BlockEncodingManager(com.facebook.presto.block.BlockEncodingManager) SessionPropertyManager(com.facebook.presto.metadata.SessionPropertyManager) TypeManager(com.facebook.presto.spi.type.TypeManager) TablePropertyManager(com.facebook.presto.metadata.TablePropertyManager) SchemaPropertyManager(com.facebook.presto.metadata.SchemaPropertyManager) ConnectorTableMetadata(com.facebook.presto.spi.ConnectorTableMetadata) BeforeMethod(org.testng.annotations.BeforeMethod)

Example 10 with BlockEncodingManager

use of com.facebook.presto.block.BlockEncodingManager in project presto by prestodb.

the class TestExpressionOptimizer method testPossibleExponentialOptimizationTime.

@Test(timeOut = 10_000)
public void testPossibleExponentialOptimizationTime() {
    TypeRegistry typeManager = new TypeRegistry();
    ExpressionOptimizer optimizer = new ExpressionOptimizer(new FunctionRegistry(typeManager, new BlockEncodingManager(typeManager), new FeaturesConfig()), typeManager, TEST_SESSION);
    RowExpression expression = new ConstantExpression(1L, BIGINT);
    for (int i = 0; i < 100; i++) {
        Signature signature = internalOperator(OperatorType.ADD.name(), parseTypeSignature(StandardTypes.BIGINT), parseTypeSignature(StandardTypes.BIGINT), parseTypeSignature(StandardTypes.BIGINT));
        expression = new CallExpression(signature, BIGINT, ImmutableList.of(expression, new ConstantExpression(1L, BIGINT)));
    }
    optimizer.optimize(expression);
}
Also used : FunctionRegistry(com.facebook.presto.metadata.FunctionRegistry) BlockEncodingManager(com.facebook.presto.block.BlockEncodingManager) FeaturesConfig(com.facebook.presto.sql.analyzer.FeaturesConfig) ExpressionOptimizer(com.facebook.presto.sql.relational.optimizer.ExpressionOptimizer) ConstantExpression(com.facebook.presto.sql.relational.ConstantExpression) Signature(com.facebook.presto.metadata.Signature) TypeSignature.parseTypeSignature(com.facebook.presto.spi.type.TypeSignature.parseTypeSignature) RowExpression(com.facebook.presto.sql.relational.RowExpression) TypeRegistry(com.facebook.presto.type.TypeRegistry) CallExpression(com.facebook.presto.sql.relational.CallExpression) Test(org.testng.annotations.Test)

Aggregations

BlockEncodingManager (com.facebook.presto.block.BlockEncodingManager)12 FeaturesConfig (com.facebook.presto.sql.analyzer.FeaturesConfig)11 TypeRegistry (com.facebook.presto.type.TypeRegistry)10 Test (org.testng.annotations.Test)10 TypeSignature.parseTypeSignature (com.facebook.presto.spi.type.TypeSignature.parseTypeSignature)7 FunctionRegistry (com.facebook.presto.metadata.FunctionRegistry)4 FunctionRegistry.getMagicLiteralFunctionSignature (com.facebook.presto.metadata.FunctionRegistry.getMagicLiteralFunctionSignature)4 TypeSignature (com.facebook.presto.spi.type.TypeSignature)4 Signature (com.facebook.presto.metadata.Signature)3 OperatorType (com.facebook.presto.spi.function.OperatorType)3 CallExpression (com.facebook.presto.sql.relational.CallExpression)3 ConstantExpression (com.facebook.presto.sql.relational.ConstantExpression)3 RowExpression (com.facebook.presto.sql.relational.RowExpression)3 ExpressionOptimizer (com.facebook.presto.sql.relational.optimizer.ExpressionOptimizer)3 MetadataManager (com.facebook.presto.metadata.MetadataManager)2 TypeManager (com.facebook.presto.spi.type.TypeManager)2 MockExchangeClientSupplier (com.facebook.presto.execution.TestSqlTaskManager.MockExchangeClientSupplier)1 LegacyNetworkTopology (com.facebook.presto.execution.scheduler.LegacyNetworkTopology)1 NodeScheduler (com.facebook.presto.execution.scheduler.NodeScheduler)1 NodeSchedulerConfig (com.facebook.presto.execution.scheduler.NodeSchedulerConfig)1