Search in sources :

Example 11 with CompiledScript

use of org.elasticsearch.script.CompiledScript in project elasticsearch by elastic.

the class BucketScriptPipelineAggregator method reduce.

@Override
public InternalAggregation reduce(InternalAggregation aggregation, ReduceContext reduceContext) {
    InternalMultiBucketAggregation<InternalMultiBucketAggregation, InternalMultiBucketAggregation.InternalBucket> originalAgg = (InternalMultiBucketAggregation<InternalMultiBucketAggregation, InternalMultiBucketAggregation.InternalBucket>) aggregation;
    List<? extends Bucket> buckets = originalAgg.getBuckets();
    CompiledScript compiledScript = reduceContext.scriptService().compile(script, ScriptContext.Standard.AGGS);
    List newBuckets = new ArrayList<>();
    for (Bucket bucket : buckets) {
        Map<String, Object> vars = new HashMap<>();
        if (script.getParams() != null) {
            vars.putAll(script.getParams());
        }
        boolean skipBucket = false;
        for (Map.Entry<String, String> entry : bucketsPathsMap.entrySet()) {
            String varName = entry.getKey();
            String bucketsPath = entry.getValue();
            Double value = resolveBucketValue(originalAgg, bucket, bucketsPath, gapPolicy);
            if (GapPolicy.SKIP == gapPolicy && (value == null || Double.isNaN(value))) {
                skipBucket = true;
                break;
            }
            vars.put(varName, value);
        }
        if (skipBucket) {
            newBuckets.add(bucket);
        } else {
            ExecutableScript executableScript = reduceContext.scriptService().executable(compiledScript, vars);
            Object returned = executableScript.run();
            if (returned == null) {
                newBuckets.add(bucket);
            } else {
                if (!(returned instanceof Number)) {
                    throw new AggregationExecutionException("series_arithmetic script for reducer [" + name() + "] must return a Number");
                }
                final List<InternalAggregation> aggs = StreamSupport.stream(bucket.getAggregations().spliterator(), false).map((p) -> {
                    return (InternalAggregation) p;
                }).collect(Collectors.toList());
                aggs.add(new InternalSimpleValue(name(), ((Number) returned).doubleValue(), formatter, new ArrayList<>(), metaData()));
                InternalMultiBucketAggregation.InternalBucket newBucket = originalAgg.createBucket(new InternalAggregations(aggs), (InternalMultiBucketAggregation.InternalBucket) bucket);
                newBuckets.add(newBucket);
            }
        }
    }
    return originalAgg.create(newBuckets);
}
Also used : CompiledScript(org.elasticsearch.script.CompiledScript) StreamOutput(org.elasticsearch.common.io.stream.StreamOutput) HashMap(java.util.HashMap) DocValueFormat(org.elasticsearch.search.DocValueFormat) ScriptContext(org.elasticsearch.script.ScriptContext) ArrayList(java.util.ArrayList) InternalMultiBucketAggregation(org.elasticsearch.search.aggregations.InternalMultiBucketAggregation) BucketHelpers.resolveBucketValue(org.elasticsearch.search.aggregations.pipeline.BucketHelpers.resolveBucketValue) InternalAggregations(org.elasticsearch.search.aggregations.InternalAggregations) Bucket(org.elasticsearch.search.aggregations.bucket.MultiBucketsAggregation.Bucket) Map(java.util.Map) GapPolicy(org.elasticsearch.search.aggregations.pipeline.BucketHelpers.GapPolicy) StreamSupport(java.util.stream.StreamSupport) AggregationExecutionException(org.elasticsearch.search.aggregations.AggregationExecutionException) Script(org.elasticsearch.script.Script) PipelineAggregator(org.elasticsearch.search.aggregations.pipeline.PipelineAggregator) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) InternalAggregation(org.elasticsearch.search.aggregations.InternalAggregation) ReduceContext(org.elasticsearch.search.aggregations.InternalAggregation.ReduceContext) List(java.util.List) CompiledScript(org.elasticsearch.script.CompiledScript) InternalSimpleValue(org.elasticsearch.search.aggregations.pipeline.InternalSimpleValue) StreamInput(org.elasticsearch.common.io.stream.StreamInput) ExecutableScript(org.elasticsearch.script.ExecutableScript) Collections(java.util.Collections) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) ExecutableScript(org.elasticsearch.script.ExecutableScript) ArrayList(java.util.ArrayList) List(java.util.List) InternalMultiBucketAggregation(org.elasticsearch.search.aggregations.InternalMultiBucketAggregation) AggregationExecutionException(org.elasticsearch.search.aggregations.AggregationExecutionException) InternalAggregation(org.elasticsearch.search.aggregations.InternalAggregation) InternalSimpleValue(org.elasticsearch.search.aggregations.pipeline.InternalSimpleValue) InternalAggregations(org.elasticsearch.search.aggregations.InternalAggregations) Bucket(org.elasticsearch.search.aggregations.bucket.MultiBucketsAggregation.Bucket) HashMap(java.util.HashMap) Map(java.util.Map)

Example 12 with CompiledScript

use of org.elasticsearch.script.CompiledScript in project elasticsearch by elastic.

the class InternalScriptedMetric method doReduce.

@Override
public InternalAggregation doReduce(List<InternalAggregation> aggregations, ReduceContext reduceContext) {
    List<Object> aggregationObjects = new ArrayList<>();
    for (InternalAggregation aggregation : aggregations) {
        InternalScriptedMetric mapReduceAggregation = (InternalScriptedMetric) aggregation;
        aggregationObjects.addAll(mapReduceAggregation.aggregation);
    }
    InternalScriptedMetric firstAggregation = ((InternalScriptedMetric) aggregations.get(0));
    List<Object> aggregation;
    if (firstAggregation.reduceScript != null && reduceContext.isFinalReduce()) {
        Map<String, Object> vars = new HashMap<>();
        vars.put("_aggs", aggregationObjects);
        if (firstAggregation.reduceScript.getParams() != null) {
            vars.putAll(firstAggregation.reduceScript.getParams());
        }
        CompiledScript compiledScript = reduceContext.scriptService().compile(firstAggregation.reduceScript, ScriptContext.Standard.AGGS);
        ExecutableScript script = reduceContext.scriptService().executable(compiledScript, vars);
        aggregation = Collections.singletonList(script.run());
    } else if (reduceContext.isFinalReduce()) {
        aggregation = Collections.singletonList(aggregationObjects);
    } else {
        // if we are not an final reduce we have to maintain all the aggs from all the incoming one
        // until we hit the final reduce phase.
        aggregation = aggregationObjects;
    }
    return new InternalScriptedMetric(firstAggregation.getName(), aggregation, firstAggregation.reduceScript, pipelineAggregators(), getMetaData());
}
Also used : InternalAggregation(org.elasticsearch.search.aggregations.InternalAggregation) CompiledScript(org.elasticsearch.script.CompiledScript) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) ExecutableScript(org.elasticsearch.script.ExecutableScript)

Example 13 with CompiledScript

use of org.elasticsearch.script.CompiledScript in project elasticsearch by elastic.

the class BucketSelectorPipelineAggregator method reduce.

@Override
public InternalAggregation reduce(InternalAggregation aggregation, ReduceContext reduceContext) {
    InternalMultiBucketAggregation<InternalMultiBucketAggregation, InternalMultiBucketAggregation.InternalBucket> originalAgg = (InternalMultiBucketAggregation<InternalMultiBucketAggregation, InternalMultiBucketAggregation.InternalBucket>) aggregation;
    List<? extends Bucket> buckets = originalAgg.getBuckets();
    CompiledScript compiledScript = reduceContext.scriptService().compile(script, ScriptContext.Standard.AGGS);
    List newBuckets = new ArrayList<>();
    for (Bucket bucket : buckets) {
        Map<String, Object> vars = new HashMap<>();
        if (script.getParams() != null) {
            vars.putAll(script.getParams());
        }
        for (Map.Entry<String, String> entry : bucketsPathsMap.entrySet()) {
            String varName = entry.getKey();
            String bucketsPath = entry.getValue();
            Double value = resolveBucketValue(originalAgg, bucket, bucketsPath, gapPolicy);
            vars.put(varName, value);
        }
        ExecutableScript executableScript = reduceContext.scriptService().executable(compiledScript, vars);
        Object scriptReturnValue = executableScript.run();
        final boolean keepBucket;
        // TODO: WTF!!!!!
        if ("expression".equals(script.getLang())) {
            double scriptDoubleValue = (double) scriptReturnValue;
            keepBucket = scriptDoubleValue == 1.0;
        } else {
            keepBucket = (boolean) scriptReturnValue;
        }
        if (keepBucket) {
            newBuckets.add(bucket);
        }
    }
    return originalAgg.create(newBuckets);
}
Also used : CompiledScript(org.elasticsearch.script.CompiledScript) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) Bucket(org.elasticsearch.search.aggregations.bucket.MultiBucketsAggregation.Bucket) ExecutableScript(org.elasticsearch.script.ExecutableScript) ArrayList(java.util.ArrayList) List(java.util.List) InternalMultiBucketAggregation(org.elasticsearch.search.aggregations.InternalMultiBucketAggregation) HashMap(java.util.HashMap) Map(java.util.Map)

Example 14 with CompiledScript

use of org.elasticsearch.script.CompiledScript in project elasticsearch by elastic.

the class QueryShardContext method getLazyExecutableScript.

/**
     * Returns a lazily created {@link ExecutableScript} that is compiled immediately but can be pulled later once all
     * parameters are available.
     */
public final Function<Map<String, Object>, ExecutableScript> getLazyExecutableScript(Script script, ScriptContext context) {
    failIfFrozen();
    CompiledScript executable = scriptService.compile(script, context);
    return (p) -> scriptService.executable(executable, p);
}
Also used : CompiledScript(org.elasticsearch.script.CompiledScript) Query(org.apache.lucene.search.Query) BitsetFilterCache(org.elasticsearch.index.cache.bitset.BitsetFilterCache) Arrays(java.util.Arrays) LongSupplier(java.util.function.LongSupplier) ParsingException(org.elasticsearch.common.ParsingException) IndexFieldDataService(org.elasticsearch.index.fielddata.IndexFieldDataService) IndexAnalyzers(org.elasticsearch.index.analysis.IndexAnalyzers) HashMap(java.util.HashMap) Index(org.elasticsearch.index.Index) MapperQueryParser(org.apache.lucene.queryparser.classic.MapperQueryParser) Function(java.util.function.Function) ScriptContext(org.elasticsearch.script.ScriptContext) Strings(org.elasticsearch.common.Strings) NestedScope(org.elasticsearch.index.query.support.NestedScope) BitSetProducer(org.apache.lucene.search.join.BitSetProducer) MappedFieldType(org.elasticsearch.index.mapper.MappedFieldType) Similarity(org.apache.lucene.search.similarities.Similarity) Map(java.util.Map) IndexSettings(org.elasticsearch.index.IndexSettings) QueryParserSettings(org.apache.lucene.queryparser.classic.QueryParserSettings) NamedXContentRegistry(org.elasticsearch.common.xcontent.NamedXContentRegistry) DocumentMapper(org.elasticsearch.index.mapper.DocumentMapper) SearchScript(org.elasticsearch.script.SearchScript) SearchLookup(org.elasticsearch.search.lookup.SearchLookup) ObjectMapper(org.elasticsearch.index.mapper.ObjectMapper) Script(org.elasticsearch.script.Script) SetOnce(org.apache.lucene.util.SetOnce) Mapper(org.elasticsearch.index.mapper.Mapper) Analyzer(org.apache.lucene.analysis.Analyzer) Client(org.elasticsearch.client.Client) Collection(java.util.Collection) IOException(java.io.IOException) TextFieldMapper(org.elasticsearch.index.mapper.TextFieldMapper) BytesReference(org.elasticsearch.common.bytes.BytesReference) SimilarityService(org.elasticsearch.index.similarity.SimilarityService) CheckedFunction(org.elasticsearch.common.CheckedFunction) MapperService(org.elasticsearch.index.mapper.MapperService) Version(org.elasticsearch.Version) CompiledScript(org.elasticsearch.script.CompiledScript) IndexFieldData(org.elasticsearch.index.fielddata.IndexFieldData) Queries(org.elasticsearch.common.lucene.search.Queries) Collections.unmodifiableMap(java.util.Collections.unmodifiableMap) ExecutableScript(org.elasticsearch.script.ExecutableScript) ContentPath(org.elasticsearch.index.mapper.ContentPath) ScriptService(org.elasticsearch.script.ScriptService) IndexReader(org.apache.lucene.index.IndexReader)

Example 15 with CompiledScript

use of org.elasticsearch.script.CompiledScript in project elasticsearch by elastic.

the class ScriptEngineTests method testChangingVarsCrossExecution1.

public void testChangingVarsCrossExecution1() {
    Map<String, Object> vars = new HashMap<>();
    Map<String, Object> ctx = new HashMap<>();
    vars.put("ctx", ctx);
    Object compiledScript = scriptEngine.compile(null, "return ctx.value;", Collections.emptyMap());
    ExecutableScript script = scriptEngine.executable(new CompiledScript(ScriptType.INLINE, "testChangingVarsCrossExecution1", "painless", compiledScript), vars);
    ctx.put("value", 1);
    Object o = script.run();
    assertEquals(1, ((Number) o).intValue());
    ctx.put("value", 2);
    o = script.run();
    assertEquals(2, ((Number) o).intValue());
}
Also used : CompiledScript(org.elasticsearch.script.CompiledScript) HashMap(java.util.HashMap) ExecutableScript(org.elasticsearch.script.ExecutableScript)

Aggregations

CompiledScript (org.elasticsearch.script.CompiledScript)22 ExecutableScript (org.elasticsearch.script.ExecutableScript)15 HashMap (java.util.HashMap)12 BytesReference (org.elasticsearch.common.bytes.BytesReference)10 Script (org.elasticsearch.script.Script)7 Map (java.util.Map)5 Matchers.containsString (org.hamcrest.Matchers.containsString)5 Matchers.isEmptyOrNullString (org.hamcrest.Matchers.isEmptyOrNullString)5 Mustache (com.github.mustachejava.Mustache)4 IOException (java.io.IOException)3 ArrayList (java.util.ArrayList)3 NamedXContentRegistry (org.elasticsearch.common.xcontent.NamedXContentRegistry)3 IndexSettings (org.elasticsearch.index.IndexSettings)3 ScriptContext (org.elasticsearch.script.ScriptContext)3 Arrays (java.util.Arrays)2 Collection (java.util.Collection)2 Collections.unmodifiableMap (java.util.Collections.unmodifiableMap)2 HashSet (java.util.HashSet)2 List (java.util.List)2 Function (java.util.function.Function)2