Search in sources :

Example 1 with UserDefinedFunctionsMetadata

use of io.crate.expression.udf.UserDefinedFunctionsMetadata in project crate by crate.

the class DocSchemaInfo method update.

@Override
public void update(ClusterChangedEvent event) {
    assert event.metadataChanged() : "metadataChanged must be true if update is called";
    // search for aliases of deleted and created indices, they must be invalidated also
    Metadata prevMetadata = event.previousState().metadata();
    for (Index index : event.indicesDeleted()) {
        invalidateFromIndex(index, prevMetadata);
    }
    Metadata newMetadata = event.state().metadata();
    for (String index : event.indicesCreated()) {
        invalidateAliases(newMetadata.index(index).getAliases());
    }
    // search for templates with changed meta data => invalidate template aliases
    ImmutableOpenMap<String, IndexTemplateMetadata> newTemplates = newMetadata.templates();
    ImmutableOpenMap<String, IndexTemplateMetadata> prevTemplates = prevMetadata.templates();
    if (!newTemplates.equals(prevTemplates)) {
        for (ObjectCursor<IndexTemplateMetadata> cursor : newTemplates.values()) {
            invalidateAliases(cursor.value.aliases());
        }
        for (ObjectCursor<IndexTemplateMetadata> cursor : prevTemplates.values()) {
            invalidateAliases(cursor.value.aliases());
        }
    }
    // search indices with changed meta data
    Iterator<String> currentTablesIt = docTableByName.keySet().iterator();
    ObjectLookupContainer<String> templates = newTemplates.keys();
    ImmutableOpenMap<String, IndexMetadata> indices = newMetadata.indices();
    while (currentTablesIt.hasNext()) {
        String tableName = currentTablesIt.next();
        String indexName = getIndexName(tableName);
        IndexMetadata newIndexMetadata = newMetadata.index(indexName);
        if (newIndexMetadata == null) {
            docTableByName.remove(tableName);
        } else {
            IndexMetadata oldIndexMetadata = prevMetadata.index(indexName);
            if (oldIndexMetadata != null && ClusterChangedEvent.indexMetadataChanged(oldIndexMetadata, newIndexMetadata)) {
                docTableByName.remove(tableName);
                // invalidate aliases of changed indices
                invalidateAliases(newIndexMetadata.getAliases());
                invalidateAliases(oldIndexMetadata.getAliases());
            } else {
                // this is the case if a single partition has been modified using alter table <t> partition (...)
                String possibleTemplateName = PartitionName.templateName(name(), tableName);
                if (templates.contains(possibleTemplateName)) {
                    for (ObjectObjectCursor<String, IndexMetadata> indexEntry : indices) {
                        if (IndexParts.isPartitioned(indexEntry.key)) {
                            docTableByName.remove(tableName);
                            break;
                        }
                    }
                }
            }
        }
    }
    // re register UDFs for this schema
    UserDefinedFunctionsMetadata udfMetadata = newMetadata.custom(UserDefinedFunctionsMetadata.TYPE);
    if (udfMetadata != null) {
        udfService.updateImplementations(schemaName, udfMetadata.functionsMetadata().stream().filter(f -> schemaName.equals(f.schema())));
    }
}
Also used : BlobIndex(io.crate.blob.v2.BlobIndex) IndexParts(io.crate.metadata.IndexParts) ImmutableOpenMap(org.elasticsearch.common.collect.ImmutableOpenMap) RelationName(io.crate.metadata.RelationName) IndexMetadata(org.elasticsearch.cluster.metadata.IndexMetadata) ViewInfoFactory(io.crate.metadata.view.ViewInfoFactory) ClusterService(org.elasticsearch.cluster.service.ClusterService) UserDefinedFunctionService(io.crate.expression.udf.UserDefinedFunctionService) Index(org.elasticsearch.index.Index) UserDefinedFunctionsMetadata(io.crate.expression.udf.UserDefinedFunctionsMetadata) PartitionName(io.crate.metadata.PartitionName) HashSet(java.util.HashSet) ObjectObjectCursor(com.carrotsearch.hppc.cursors.ObjectObjectCursor) Metadata(org.elasticsearch.cluster.metadata.Metadata) AliasMetadata(org.elasticsearch.cluster.metadata.AliasMetadata) ViewInfo(io.crate.metadata.view.ViewInfo) StreamSupport(java.util.stream.StreamSupport) Nullable(javax.annotation.Nullable) SchemaInfo(io.crate.metadata.table.SchemaInfo) TableInfo(io.crate.metadata.table.TableInfo) NodeContext(io.crate.metadata.NodeContext) Iterator(java.util.Iterator) Predicate(java.util.function.Predicate) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) ViewsMetadata(io.crate.metadata.view.ViewsMetadata) Set(java.util.Set) ResourceUnknownException(io.crate.exceptions.ResourceUnknownException) IndexTemplateMetadata(org.elasticsearch.cluster.metadata.IndexTemplateMetadata) ObjectCursor(com.carrotsearch.hppc.cursors.ObjectCursor) ClusterChangedEvent(org.elasticsearch.cluster.ClusterChangedEvent) Objects(java.util.Objects) Stream(java.util.stream.Stream) Schemas(io.crate.metadata.Schemas) VisibleForTesting(io.crate.common.annotations.VisibleForTesting) Collections(java.util.Collections) ObjectLookupContainer(com.carrotsearch.hppc.ObjectLookupContainer) IndexTemplateMetadata(org.elasticsearch.cluster.metadata.IndexTemplateMetadata) IndexMetadata(org.elasticsearch.cluster.metadata.IndexMetadata) UserDefinedFunctionsMetadata(io.crate.expression.udf.UserDefinedFunctionsMetadata) Metadata(org.elasticsearch.cluster.metadata.Metadata) AliasMetadata(org.elasticsearch.cluster.metadata.AliasMetadata) ViewsMetadata(io.crate.metadata.view.ViewsMetadata) IndexTemplateMetadata(org.elasticsearch.cluster.metadata.IndexTemplateMetadata) BlobIndex(io.crate.blob.v2.BlobIndex) Index(org.elasticsearch.index.Index) UserDefinedFunctionsMetadata(io.crate.expression.udf.UserDefinedFunctionsMetadata) IndexMetadata(org.elasticsearch.cluster.metadata.IndexMetadata)

Example 2 with UserDefinedFunctionsMetadata

use of io.crate.expression.udf.UserDefinedFunctionsMetadata in project crate by crate.

the class PgCatalogSchemaInfo method update.

@Override
public void update(ClusterChangedEvent event) {
    assert event.metadataChanged() : "metadataChanged must be true if update is called";
    Metadata newMetadata = event.state().metadata();
    // re register UDFs for this schema
    UserDefinedFunctionsMetadata udfMetadata = newMetadata.custom(UserDefinedFunctionsMetadata.TYPE);
    if (udfMetadata != null) {
        udfService.updateImplementations(NAME, udfMetadata.functionsMetadata().stream().filter(f -> NAME.equals(f.schema())));
    }
}
Also used : SchemaInfo(io.crate.metadata.table.SchemaInfo) TableInfo(io.crate.metadata.table.TableInfo) UserDefinedFunctionService(io.crate.expression.udf.UserDefinedFunctionService) UserDefinedFunctionsMetadata(io.crate.expression.udf.UserDefinedFunctionsMetadata) ClusterChangedEvent(org.elasticsearch.cluster.ClusterChangedEvent) Inject(org.elasticsearch.common.inject.Inject) Metadata(org.elasticsearch.cluster.metadata.Metadata) SystemTable(io.crate.metadata.SystemTable) TableStats(io.crate.statistics.TableStats) Singleton(org.elasticsearch.common.inject.Singleton) Map(java.util.Map) ViewInfo(io.crate.metadata.view.ViewInfo) Collections(java.util.Collections) Nullable(javax.annotation.Nullable) UserDefinedFunctionsMetadata(io.crate.expression.udf.UserDefinedFunctionsMetadata) Metadata(org.elasticsearch.cluster.metadata.Metadata) UserDefinedFunctionsMetadata(io.crate.expression.udf.UserDefinedFunctionsMetadata)

Example 3 with UserDefinedFunctionsMetadata

use of io.crate.expression.udf.UserDefinedFunctionsMetadata in project crate by crate.

the class Schemas method getNewCurrentSchemas.

@VisibleForTesting
static Set<String> getNewCurrentSchemas(Metadata metadata) {
    Set<String> schemas = new HashSet<>();
    // 'doc' schema is always available and has the special property that its indices
    // don't have to be prefixed with the schema name
    schemas.add(DOC_SCHEMA_NAME);
    for (String index : metadata.getConcreteAllIndices()) {
        addIfSchema(schemas, index);
    }
    for (ObjectCursor<String> cursor : metadata.templates().keys()) {
        addIfSchema(schemas, cursor.value);
    }
    UserDefinedFunctionsMetadata udfMetadata = metadata.custom(UserDefinedFunctionsMetadata.TYPE);
    if (udfMetadata != null) {
        udfMetadata.functionsMetadata().stream().map(UserDefinedFunctionMetadata::schema).forEach(schemas::add);
    }
    ViewsMetadata viewsMetadata = metadata.custom(ViewsMetadata.TYPE);
    if (viewsMetadata != null) {
        StreamSupport.stream(viewsMetadata.names().spliterator(), false).map(IndexParts::new).map(IndexParts::getSchema).forEach(schemas::add);
    }
    return schemas;
}
Also used : UserDefinedFunctionsMetadata(io.crate.expression.udf.UserDefinedFunctionsMetadata) ViewsMetadata(io.crate.metadata.view.ViewsMetadata) HashSet(java.util.HashSet) VisibleForTesting(io.crate.common.annotations.VisibleForTesting)

Example 4 with UserDefinedFunctionsMetadata

use of io.crate.expression.udf.UserDefinedFunctionsMetadata in project crate by crate.

the class DocSchemaInfoTest method testInvalidFunction.

@Test
public void testInvalidFunction() throws Exception {
    UserDefinedFunctionMetadata invalid = new UserDefinedFunctionMetadata("my_schema", "invalid", List.of(), DataTypes.INTEGER, "burlesque", "this is not valid burlesque code");
    UserDefinedFunctionMetadata valid = new UserDefinedFunctionMetadata("my_schema", "valid", List.of(), DataTypes.INTEGER, "burlesque", "\"Hello, World!\"Q");
    UserDefinedFunctionsMetadata metadata = UserDefinedFunctionsMetadata.of(invalid, valid);
    // if a functionImpl can't be created, it won't be registered
    udfService.updateImplementations("my_schema", metadata.functionsMetadata().stream());
    assertThat(nodeCtx.functions().get("my_schema", "valid", List.of(), pathWithPGCatalogAndDoc()), Matchers.notNullValue());
    expectedException.expectMessage("Unknown function: my_schema.invalid()");
    nodeCtx.functions().get("my_schema", "invalid", List.of(), pathWithPGCatalogAndDoc());
}
Also used : UserDefinedFunctionMetadata(io.crate.expression.udf.UserDefinedFunctionMetadata) UserDefinedFunctionsMetadata(io.crate.expression.udf.UserDefinedFunctionsMetadata) Test(org.junit.Test) CrateDummyClusterServiceUnitTest(io.crate.test.integration.CrateDummyClusterServiceUnitTest)

Aggregations

UserDefinedFunctionsMetadata (io.crate.expression.udf.UserDefinedFunctionsMetadata)4 VisibleForTesting (io.crate.common.annotations.VisibleForTesting)2 UserDefinedFunctionService (io.crate.expression.udf.UserDefinedFunctionService)2 SchemaInfo (io.crate.metadata.table.SchemaInfo)2 TableInfo (io.crate.metadata.table.TableInfo)2 ViewInfo (io.crate.metadata.view.ViewInfo)2 ViewsMetadata (io.crate.metadata.view.ViewsMetadata)2 Collections (java.util.Collections)2 HashSet (java.util.HashSet)2 Nullable (javax.annotation.Nullable)2 ClusterChangedEvent (org.elasticsearch.cluster.ClusterChangedEvent)2 Metadata (org.elasticsearch.cluster.metadata.Metadata)2 ObjectLookupContainer (com.carrotsearch.hppc.ObjectLookupContainer)1 ObjectCursor (com.carrotsearch.hppc.cursors.ObjectCursor)1 ObjectObjectCursor (com.carrotsearch.hppc.cursors.ObjectObjectCursor)1 BlobIndex (io.crate.blob.v2.BlobIndex)1 ResourceUnknownException (io.crate.exceptions.ResourceUnknownException)1 UserDefinedFunctionMetadata (io.crate.expression.udf.UserDefinedFunctionMetadata)1 IndexParts (io.crate.metadata.IndexParts)1 NodeContext (io.crate.metadata.NodeContext)1