Search in sources :

Example 1 with StellarProcessor

use of org.apache.metron.stellar.common.StellarProcessor in project metron by apache.

the class HdfsWriter method init.

@Override
public void init(Map stormConfig, TopologyContext topologyContext, WriterConfiguration configurations) {
    this.stormConfig = stormConfig;
    this.stellarProcessor = new StellarProcessor();
    this.fileNameFormat.prepare(stormConfig, topologyContext);
    if (syncPolicy != null) {
        // if the user has specified the sync policy, we don't want to override their wishes.
        syncPolicyCreator = new ClonedSyncPolicyCreator(syncPolicy);
    } else {
        // if the user has not, then we want to have the sync policy depend on the batch size.
        syncPolicyCreator = (source, config) -> new CountSyncPolicy(config == null ? 1 : config.getBatchSize(source));
    }
}
Also used : StellarProcessor(org.apache.metron.stellar.common.StellarProcessor) CountSyncPolicy(org.apache.storm.hdfs.bolt.sync.CountSyncPolicy)

Example 2 with StellarProcessor

use of org.apache.metron.stellar.common.StellarProcessor in project metron by apache.

the class StellarServiceImpl method validateRules.

@Override
public Map<String, Boolean> validateRules(List<String> rules) {
    Map<String, Boolean> results = new HashMap<>();
    StellarProcessor stellarProcessor = new StellarProcessor();
    for (String rule : rules) {
        try {
            boolean result = stellarProcessor.validate(rule, Context.EMPTY_CONTEXT());
            results.put(rule, result);
        } catch (ParseException e) {
            results.put(rule, false);
        }
    }
    return results;
}
Also used : StellarProcessor(org.apache.metron.stellar.common.StellarProcessor) HashMap(java.util.HashMap) ParseException(org.apache.metron.stellar.dsl.ParseException)

Example 3 with StellarProcessor

use of org.apache.metron.stellar.common.StellarProcessor in project metron by apache.

the class GeoEnrichmentFunctionsTest method run.

public Object run(String rule, Map<String, Object> variables) {
    StellarProcessor processor = new StellarProcessor();
    assertTrue(processor.validate(rule, context), rule + " not valid.");
    return processor.parse(rule, new DefaultVariableResolver(x -> variables.get(x), x -> variables.containsKey(x)), StellarFunctions.FUNCTION_RESOLVER(), context);
}
Also used : StellarProcessor(org.apache.metron.stellar.common.StellarProcessor) BeforeEach(org.junit.jupiter.api.BeforeEach) ImmutableMap(com.google.common.collect.ImmutableMap) JSONParser(org.json.simple.parser.JSONParser) UnitTestHelper(org.apache.metron.test.utils.UnitTestHelper) StellarProcessor(org.apache.metron.stellar.common.StellarProcessor) HashMap(java.util.HashMap) DefaultVariableResolver(org.apache.metron.stellar.dsl.DefaultVariableResolver) File(java.io.File) Test(org.junit.jupiter.api.Test) JSONObject(org.json.simple.JSONObject) ParseException(org.json.simple.parser.ParseException) BeforeAll(org.junit.jupiter.api.BeforeAll) Multiline(org.adrianwalker.multilinestring.Multiline) Map(java.util.Map) StellarFunctions(org.apache.metron.stellar.dsl.StellarFunctions) Assertions(org.junit.jupiter.api.Assertions) GeoLiteCityDatabase(org.apache.metron.enrichment.adapters.maxmind.geo.GeoLiteCityDatabase) Collections(java.util.Collections) Context(org.apache.metron.stellar.dsl.Context) DefaultVariableResolver(org.apache.metron.stellar.dsl.DefaultVariableResolver)

Example 4 with StellarProcessor

use of org.apache.metron.stellar.common.StellarProcessor in project metron by apache.

the class SimpleHBaseEnrichmentFunctionsTest method run.

public Object run(String rule, Map<String, Object> variables) throws Exception {
    StellarProcessor processor = new StellarProcessor();
    assertTrue(processor.validate(rule, context), rule + " not valid.");
    return processor.parse(rule, new DefaultVariableResolver(x -> variables.get(x), x -> variables.containsKey(x)), StellarFunctions.FUNCTION_RESOLVER(), context);
}
Also used : StellarProcessor(org.apache.metron.stellar.common.StellarProcessor) EnrichmentHelper(org.apache.metron.enrichment.converter.EnrichmentHelper) BeforeEach(org.junit.jupiter.api.BeforeEach) ImmutableMap(com.google.common.collect.ImmutableMap) StellarProcessor(org.apache.metron.stellar.common.StellarProcessor) HashMap(java.util.HashMap) DefaultVariableResolver(org.apache.metron.stellar.dsl.DefaultVariableResolver) MockHBaseTableProvider(org.apache.metron.hbase.mock.MockHBaseTableProvider) ArrayList(java.util.ArrayList) Test(org.junit.jupiter.api.Test) LookupKV(org.apache.metron.enrichment.lookup.LookupKV) MockHTable(org.apache.metron.hbase.mock.MockHTable) List(java.util.List) EnrichmentValue(org.apache.metron.enrichment.converter.EnrichmentValue) Map(java.util.Map) StellarFunctions(org.apache.metron.stellar.dsl.StellarFunctions) Assertions(org.junit.jupiter.api.Assertions) EnrichmentKey(org.apache.metron.enrichment.converter.EnrichmentKey) ParseException(org.apache.metron.stellar.dsl.ParseException) Context(org.apache.metron.stellar.dsl.Context) DefaultVariableResolver(org.apache.metron.stellar.dsl.DefaultVariableResolver)

Example 5 with StellarProcessor

use of org.apache.metron.stellar.common.StellarProcessor in project metron by apache.

the class KafkaFunctionsIntegrationTest method run.

/**
 * Runs a Stellar expression.
 * @param expression The expression to run.
 */
private Object run(String expression) {
    // make the global properties available to the function
    Context context = new Context.Builder().with(Context.Capabilities.GLOBAL_CONFIG, () -> global).build();
    // execute the expression
    StellarProcessor processor = new StellarProcessor();
    return processor.parse(expression, new DefaultVariableResolver(x -> variables.get(x), x -> variables.containsKey(x)), functionResolver, context);
}
Also used : Context(org.apache.metron.stellar.dsl.Context) StellarProcessor(org.apache.metron.stellar.common.StellarProcessor) ComponentRunner(org.apache.metron.integration.ComponentRunner) FunctionResolver(org.apache.metron.stellar.dsl.functions.resolver.FunctionResolver) java.util(java.util) StellarProcessor(org.apache.metron.stellar.common.StellarProcessor) DefaultVariableResolver(org.apache.metron.stellar.dsl.DefaultVariableResolver) KafkaComponent(org.apache.metron.integration.components.KafkaComponent) Executors(java.util.concurrent.Executors) TimeUnit(java.util.concurrent.TimeUnit) Future(java.util.concurrent.Future) BaseIntegrationTest(org.apache.metron.integration.BaseIntegrationTest) org.junit.jupiter.api(org.junit.jupiter.api) ZKServerComponent(org.apache.metron.integration.components.ZKServerComponent) Assertions(org.junit.jupiter.api.Assertions) MapFunctions(org.apache.metron.stellar.dsl.functions.MapFunctions) SimpleFunctionResolver(org.apache.metron.stellar.dsl.functions.resolver.SimpleFunctionResolver) ExecutorService(java.util.concurrent.ExecutorService) Context(org.apache.metron.stellar.dsl.Context) DefaultVariableResolver(org.apache.metron.stellar.dsl.DefaultVariableResolver)

Aggregations

StellarProcessor (org.apache.metron.stellar.common.StellarProcessor)25 DefaultVariableResolver (org.apache.metron.stellar.dsl.DefaultVariableResolver)12 Context (org.apache.metron.stellar.dsl.Context)10 Map (java.util.Map)8 StellarFunctions (org.apache.metron.stellar.dsl.StellarFunctions)8 Assertions (org.junit.jupiter.api.Assertions)8 Test (org.junit.jupiter.api.Test)7 HashMap (java.util.HashMap)6 ImmutableMap (com.google.common.collect.ImmutableMap)5 ParseException (org.apache.metron.stellar.dsl.ParseException)5 ImmutableList (com.google.common.collect.ImmutableList)4 ArrayList (java.util.ArrayList)4 MapVariableResolver (org.apache.metron.stellar.dsl.MapVariableResolver)4 BeforeEach (org.junit.jupiter.api.BeforeEach)4 List (java.util.List)3 StellarPredicateProcessor (org.apache.metron.stellar.common.StellarPredicateProcessor)3 VariableResolver (org.apache.metron.stellar.dsl.VariableResolver)3 JSONObject (org.json.simple.JSONObject)3 java.util (java.util)2 GaussianRandomGenerator (org.apache.commons.math3.random.GaussianRandomGenerator)2