Search in sources :

Example 26 with NodeContext

use of io.crate.metadata.NodeContext in project crate by crate.

the class CreateRepositoryPlan method createRequest.

@VisibleForTesting
public static PutRepositoryRequest createRequest(AnalyzedCreateRepository createRepository, CoordinatorTxnCtx txnCtx, NodeContext nodeCtx, Row parameters, SubQueryResults subQueryResults, RepositoryParamValidator repositoryParamValidator) {
    Function<? super Symbol, Object> eval = x -> SymbolEvaluator.evaluate(txnCtx, nodeCtx, x, parameters, subQueryResults);
    var genericProperties = createRepository.properties().map(eval);
    var settings = genericPropertiesToSettings(genericProperties, // supported settings for the repository type
    repositoryParamValidator.settingsForType(createRepository.type()).all());
    repositoryParamValidator.validate(createRepository.type(), createRepository.properties(), settings);
    PutRepositoryRequest request = new PutRepositoryRequest(createRepository.name());
    request.type(createRepository.type());
    request.settings(settings);
    return request;
}
Also used : NodeContext(io.crate.metadata.NodeContext) GenericPropertiesConverter.genericPropertiesToSettings(io.crate.analyze.GenericPropertiesConverter.genericPropertiesToSettings) RepositoryParamValidator(io.crate.analyze.repositories.RepositoryParamValidator) Function(java.util.function.Function) RowConsumer(io.crate.data.RowConsumer) DependencyCarrier(io.crate.planner.DependencyCarrier) PutRepositoryRequest(org.elasticsearch.action.admin.cluster.repositories.put.PutRepositoryRequest) SymbolEvaluator(io.crate.analyze.SymbolEvaluator) Row(io.crate.data.Row) AnalyzedCreateRepository(io.crate.analyze.AnalyzedCreateRepository) Symbol(io.crate.expression.symbol.Symbol) PlannerContext(io.crate.planner.PlannerContext) Plan(io.crate.planner.Plan) SubQueryResults(io.crate.planner.operators.SubQueryResults) VisibleForTesting(io.crate.common.annotations.VisibleForTesting) OneRowActionListener(io.crate.execution.support.OneRowActionListener) Row1(io.crate.data.Row1) CoordinatorTxnCtx(io.crate.metadata.CoordinatorTxnCtx) PutRepositoryRequest(org.elasticsearch.action.admin.cluster.repositories.put.PutRepositoryRequest) VisibleForTesting(io.crate.common.annotations.VisibleForTesting)

Example 27 with NodeContext

use of io.crate.metadata.NodeContext in project crate by crate.

the class CreateTablePlan method bind.

@VisibleForTesting
public static BoundCreateTable bind(AnalyzedCreateTable createTable, CoordinatorTxnCtx txnCtx, NodeContext nodeCtx, Row params, SubQueryResults subQueryResults, NumberOfShards numberOfShards, Schemas schemas, FulltextAnalyzerResolver fulltextAnalyzerResolver) {
    Function<? super Symbol, Object> eval = x -> SymbolEvaluator.evaluate(txnCtx, nodeCtx, x, params, subQueryResults);
    CreateTable<Symbol> table = createTable.createTable();
    RelationName relationName = createTable.relationName();
    GenericProperties<Object> properties = table.properties().map(eval);
    AnalyzedTableElements<Object> tableElements = createTable.analyzedTableElements().map(eval);
    TableParameter tableParameter = new TableParameter();
    Optional<ClusteredBy<Object>> mappedClusteredBy = table.clusteredBy().map(x -> x.map(eval));
    Integer numShards = mappedClusteredBy.flatMap(ClusteredBy::numberOfShards).map(numberOfShards::fromNumberOfShards).orElseGet(numberOfShards::defaultNumberOfShards);
    tableParameter.settingsBuilder().put(IndexMetadata.SETTING_NUMBER_OF_SHARDS, numShards);
    // apply default in case it is not specified in the properties,
    // if it is it will get overwritten afterwards.
    TablePropertiesAnalyzer.analyzeWithBoundValues(tableParameter, TableParameters.TABLE_CREATE_PARAMETER_INFO, properties, true);
    AnalyzedTableElements<Symbol> tableElementsWithExpressions = createTable.analyzedTableElementsWithExpressions().map(x -> SubQueryAndParamBinder.convert(x, params, subQueryResults));
    // validate table elements
    AnalyzedTableElements.finalizeAndValidate(relationName, tableElementsWithExpressions, tableElements);
    // update table settings
    Settings tableSettings = AnalyzedTableElements.validateAndBuildSettings(tableElements, fulltextAnalyzerResolver);
    tableParameter.settingsBuilder().put(tableSettings);
    ColumnIdent routingColumn = mappedClusteredBy.map(clusteredBy -> resolveRoutingFromClusteredBy(clusteredBy, tableElements)).orElse(null);
    Optional<PartitionedBy<Object>> partitionedByOptional = table.partitionedBy().map(x -> x.map(eval));
    partitionedByOptional.ifPresent(partitionedBy -> processPartitionedBy(partitionedByOptional.get(), tableElements, relationName, routingColumn));
    return new BoundCreateTable(relationName, tableElements, tableParameter, routingColumn, table.ifNotExists(), schemas);
}
Also used : AnalyzedColumnDefinition(io.crate.analyze.AnalyzedColumnDefinition) RelationName(io.crate.metadata.RelationName) IndexMetadata(org.elasticsearch.cluster.metadata.IndexMetadata) SENTINEL(io.crate.data.SentinelRow.SENTINEL) GenericProperties(io.crate.sql.tree.GenericProperties) Function(java.util.function.Function) DependencyCarrier(io.crate.planner.DependencyCarrier) SymbolEvaluator(io.crate.analyze.SymbolEvaluator) Settings(org.elasticsearch.common.settings.Settings) Locale(java.util.Locale) OneRowActionListener(io.crate.execution.support.OneRowActionListener) BoundCreateTable(io.crate.analyze.BoundCreateTable) FulltextAnalyzerResolver(io.crate.metadata.FulltextAnalyzerResolver) Nullable(javax.annotation.Nullable) TableParameter(io.crate.analyze.TableParameter) CreateTable(io.crate.sql.tree.CreateTable) AnalyzedCreateTable(io.crate.analyze.AnalyzedCreateTable) NodeContext(io.crate.metadata.NodeContext) PartitionedBy(io.crate.sql.tree.PartitionedBy) TableCreator(io.crate.execution.ddl.tables.TableCreator) TablePropertiesAnalyzer(io.crate.analyze.TablePropertiesAnalyzer) InMemoryBatchIterator(io.crate.data.InMemoryBatchIterator) ColumnIdent(io.crate.metadata.ColumnIdent) SubQueryAndParamBinder(io.crate.planner.operators.SubQueryAndParamBinder) TableParameters(io.crate.analyze.TableParameters) RowConsumer(io.crate.data.RowConsumer) AnalyzedTableElements(io.crate.analyze.AnalyzedTableElements) NumberOfShards(io.crate.analyze.NumberOfShards) ClusteredBy(io.crate.sql.tree.ClusteredBy) Row(io.crate.data.Row) Symbol(io.crate.expression.symbol.Symbol) PlannerContext(io.crate.planner.PlannerContext) Plan(io.crate.planner.Plan) SubQueryResults(io.crate.planner.operators.SubQueryResults) Optional(java.util.Optional) Schemas(io.crate.metadata.Schemas) VisibleForTesting(io.crate.common.annotations.VisibleForTesting) Row1(io.crate.data.Row1) CoordinatorTxnCtx(io.crate.metadata.CoordinatorTxnCtx) Symbol(io.crate.expression.symbol.Symbol) TableParameter(io.crate.analyze.TableParameter) PartitionedBy(io.crate.sql.tree.PartitionedBy) ColumnIdent(io.crate.metadata.ColumnIdent) BoundCreateTable(io.crate.analyze.BoundCreateTable) ClusteredBy(io.crate.sql.tree.ClusteredBy) RelationName(io.crate.metadata.RelationName) Settings(org.elasticsearch.common.settings.Settings) VisibleForTesting(io.crate.common.annotations.VisibleForTesting)

Example 28 with NodeContext

use of io.crate.metadata.NodeContext in project crate by crate.

the class CreateBlobTablePlan method buildSettings.

@VisibleForTesting
public static Settings buildSettings(CreateBlobTable<Symbol> createBlobTable, CoordinatorTxnCtx txnCtx, NodeContext nodeCtx, Row params, SubQueryResults subQueryResults, NumberOfShards numberOfShards) {
    Function<? super Symbol, Object> eval = x -> SymbolEvaluator.evaluate(txnCtx, nodeCtx, x, params, subQueryResults);
    CreateBlobTable<Object> blobTable = createBlobTable.map(eval);
    GenericProperties<Object> properties = blobTable.genericProperties();
    // apply default in case it is not specified in the properties,
    // if it is it will get overwritten afterwards.
    TableParameter tableParameter = new TableParameter();
    TablePropertiesAnalyzer.analyzeWithBoundValues(tableParameter, TableParameters.CREATE_BLOB_TABLE_PARAMETERS, properties, true);
    Settings.Builder builder = Settings.builder();
    builder.put(tableParameter.settings());
    builder.put(SETTING_INDEX_BLOBS_ENABLED.getKey(), true);
    int numShards;
    ClusteredBy<Object> clusteredBy = blobTable.clusteredBy();
    if (clusteredBy != null) {
        numShards = numberOfShards.fromClusteredByClause(clusteredBy);
    } else {
        numShards = numberOfShards.defaultNumberOfShards();
    }
    builder.put(IndexMetadata.SETTING_NUMBER_OF_SHARDS, numShards);
    return builder.build();
}
Also used : SETTING_INDEX_BLOBS_ENABLED(io.crate.blob.v2.BlobIndicesService.SETTING_INDEX_BLOBS_ENABLED) RelationName(io.crate.metadata.RelationName) IndexMetadata(org.elasticsearch.cluster.metadata.IndexMetadata) GenericProperties(io.crate.sql.tree.GenericProperties) Function(java.util.function.Function) DependencyCarrier(io.crate.planner.DependencyCarrier) SymbolEvaluator(io.crate.analyze.SymbolEvaluator) Settings(org.elasticsearch.common.settings.Settings) CreateIndexRequest(org.elasticsearch.action.admin.indices.create.CreateIndexRequest) OneRowActionListener(io.crate.execution.support.OneRowActionListener) BlobIndex.fullIndexName(io.crate.blob.v2.BlobIndex.fullIndexName) CreateBlobTable(io.crate.sql.tree.CreateBlobTable) TableParameter(io.crate.analyze.TableParameter) NodeContext(io.crate.metadata.NodeContext) TablePropertiesAnalyzer(io.crate.analyze.TablePropertiesAnalyzer) TableParameters(io.crate.analyze.TableParameters) CreateIndexResponse(org.elasticsearch.action.admin.indices.create.CreateIndexResponse) RowConsumer(io.crate.data.RowConsumer) NumberOfShards(io.crate.analyze.NumberOfShards) ClusteredBy(io.crate.sql.tree.ClusteredBy) AnalyzedCreateBlobTable(io.crate.analyze.AnalyzedCreateBlobTable) Row(io.crate.data.Row) Symbol(io.crate.expression.symbol.Symbol) PlannerContext(io.crate.planner.PlannerContext) Plan(io.crate.planner.Plan) SubQueryResults(io.crate.planner.operators.SubQueryResults) VisibleForTesting(io.crate.common.annotations.VisibleForTesting) Row1(io.crate.data.Row1) CoordinatorTxnCtx(io.crate.metadata.CoordinatorTxnCtx) TableParameter(io.crate.analyze.TableParameter) Settings(org.elasticsearch.common.settings.Settings) VisibleForTesting(io.crate.common.annotations.VisibleForTesting)

Example 29 with NodeContext

use of io.crate.metadata.NodeContext in project crate by crate.

the class DocSchemaInfoTest method setup.

@Before
public void setup() throws Exception {
    nodeCtx = createNodeContext();
    udfService = new UserDefinedFunctionService(clusterService, nodeCtx);
    udfService.registerLanguage(new UDFLanguage() {

        @Override
        public Scalar createFunctionImplementation(UserDefinedFunctionMetadata metadata, Signature signature) throws ScriptException {
            String error = validate(metadata);
            if (error != null) {
                throw new ScriptException("this is not Burlesque");
            }
            return new Scalar<>() {

                @Override
                public Object evaluate(TransactionContext txnCtx, NodeContext nodeCtx, Input[] args) {
                    return null;
                }

                @Override
                public Signature signature() {
                    return signature;
                }

                @Override
                public Signature boundSignature() {
                    return signature;
                }
            };
        }

        @Override
        @Nullable
        public String validate(UserDefinedFunctionMetadata metadata) {
            if (!metadata.definition().equals("\"Hello, World!\"Q")) {
                return "this is not Burlesque";
            }
            return null;
        }

        @Override
        public String name() {
            return "burlesque";
        }
    });
    docSchemaInfo = new DocSchemaInfo("doc", clusterService, nodeCtx, udfService, (ident, state) -> null, new TestingDocTableInfoFactory(Map.of()));
}
Also used : UserDefinedFunctionService(io.crate.expression.udf.UserDefinedFunctionService) Input(io.crate.data.Input) UDFLanguage(io.crate.expression.udf.UDFLanguage) TransactionContext(io.crate.metadata.TransactionContext) NodeContext(io.crate.metadata.NodeContext) UserDefinedFunctionService(io.crate.expression.udf.UserDefinedFunctionService) Matchers(org.hamcrest.Matchers) Test(org.junit.Test) Signature(io.crate.metadata.functions.Signature) CrateDummyClusterServiceUnitTest(io.crate.test.integration.CrateDummyClusterServiceUnitTest) Index(org.elasticsearch.index.Index) UserDefinedFunctionsMetadata(io.crate.expression.udf.UserDefinedFunctionsMetadata) UserDefinedFunctionMetadata(io.crate.expression.udf.UserDefinedFunctionMetadata) Metadata(org.elasticsearch.cluster.metadata.Metadata) List(java.util.List) DataTypes(io.crate.types.DataTypes) Map(java.util.Map) Scalar(io.crate.metadata.Scalar) TestingHelpers.createNodeContext(io.crate.testing.TestingHelpers.createNodeContext) Nullable(javax.annotation.Nullable) SearchPath.pathWithPGCatalogAndDoc(io.crate.metadata.SearchPath.pathWithPGCatalogAndDoc) Before(org.junit.Before) ScriptException(javax.script.ScriptException) NodeContext(io.crate.metadata.NodeContext) TestingHelpers.createNodeContext(io.crate.testing.TestingHelpers.createNodeContext) UserDefinedFunctionMetadata(io.crate.expression.udf.UserDefinedFunctionMetadata) Scalar(io.crate.metadata.Scalar) ScriptException(javax.script.ScriptException) Input(io.crate.data.Input) TransactionContext(io.crate.metadata.TransactionContext) Signature(io.crate.metadata.functions.Signature) UDFLanguage(io.crate.expression.udf.UDFLanguage) Nullable(javax.annotation.Nullable) Before(org.junit.Before)

Aggregations

NodeContext (io.crate.metadata.NodeContext)29 Symbol (io.crate.expression.symbol.Symbol)16 Row (io.crate.data.Row)14 CoordinatorTxnCtx (io.crate.metadata.CoordinatorTxnCtx)12 List (java.util.List)12 SymbolEvaluator (io.crate.analyze.SymbolEvaluator)10 VisibleForTesting (io.crate.common.annotations.VisibleForTesting)10 Plan (io.crate.planner.Plan)10 PlannerContext (io.crate.planner.PlannerContext)10 SubQueryResults (io.crate.planner.operators.SubQueryResults)10 Function (java.util.function.Function)10 Nullable (javax.annotation.Nullable)10 RowConsumer (io.crate.data.RowConsumer)9 TransactionContext (io.crate.metadata.TransactionContext)9 DependencyCarrier (io.crate.planner.DependencyCarrier)9 ArrayList (java.util.ArrayList)9 Settings (org.elasticsearch.common.settings.Settings)9 Row1 (io.crate.data.Row1)8 OneRowActionListener (io.crate.execution.support.OneRowActionListener)8 Map (java.util.Map)7