Search in sources :

Example 6 with DefaultStellarStatefulExecutor

use of org.apache.metron.stellar.common.DefaultStellarStatefulExecutor in project metron by apache.

the class GetProfileTest method setup2.

/**
 * This method is similar to setup(), in that it sets up profiler configuration context,
 * but only for the client.  Additionally, it uses periodDuration2, periodUnits2
 * and saltDivisor2, instead of periodDuration, periodUnits and saltDivisor respectively.
 *
 * This is used in the unit tests that test the config_overrides feature of PROFILE_GET.
 * In these tests, the context from @BeforeEach setup() is used to write the data, then the global
 * context is changed to context2 (from this method).  Each test validates that a default read
 * using global context2 then gets no valid results (as expected), and that a read using
 * original context values in the PROFILE_GET config_overrides argument gets all expected results.
 *
 * @return context2 - The profiler client configuration context created by this method.
 *    The context2 values are also set in the configuration of the StellarStatefulExecutor
 *    stored in the global variable 'executor'.  However, there is no API for querying the
 *    context values from a StellarStatefulExecutor, so we output the context2 Context object itself,
 *    for validation purposes (so that its values can be validated as being significantly
 *    different from the setup() settings).
 */
private Context setup2() {
    state = new HashMap<>();
    // global properties
    Map<String, Object> global = new HashMap<String, Object>() {

        {
            put(PROFILER_HBASE_TABLE.getKey(), tableName);
            put(PROFILER_COLUMN_FAMILY.getKey(), columnFamily);
            put(PROFILER_HBASE_TABLE_PROVIDER.getKey(), MockHBaseTableProvider.class.getName());
            put(PROFILER_PERIOD.getKey(), Long.toString(periodDuration2));
            put(PROFILER_PERIOD_UNITS.getKey(), periodUnits2.toString());
            put(PROFILER_SALT_DIVISOR.getKey(), Integer.toString(saltDivisor2));
        }
    };
    // create the modified context
    Context context2 = new Context.Builder().with(Context.Capabilities.GLOBAL_CONFIG, () -> global).build();
    // create the stellar execution environment
    executor = new DefaultStellarStatefulExecutor(new SimpleFunctionResolver().withClass(GetProfile.class).withClass(FixedLookback.class), context2);
    // because there is no executor.getContext() method
    return context2;
}
Also used : Context(org.apache.metron.stellar.dsl.Context) MockHBaseTableProvider(org.apache.metron.hbase.mock.MockHBaseTableProvider) SaltyRowKeyBuilder(org.apache.metron.profiler.hbase.SaltyRowKeyBuilder) ColumnBuilder(org.apache.metron.profiler.hbase.ColumnBuilder) ValueOnlyColumnBuilder(org.apache.metron.profiler.hbase.ValueOnlyColumnBuilder) RowKeyBuilder(org.apache.metron.profiler.hbase.RowKeyBuilder) DefaultStellarStatefulExecutor(org.apache.metron.stellar.common.DefaultStellarStatefulExecutor) SimpleFunctionResolver(org.apache.metron.stellar.dsl.functions.resolver.SimpleFunctionResolver)

Example 7 with DefaultStellarStatefulExecutor

use of org.apache.metron.stellar.common.DefaultStellarStatefulExecutor in project metron by apache.

the class VerboseProfileTest method setup.

@BeforeEach
public void setup() {
    state = new HashMap<>();
    final Table table = MockHBaseTableProvider.addToCache(tableName, columnFamily);
    TableProvider provider = new MockHBaseTableProvider();
    // used to write values to be read during testing
    long periodDurationMillis = TimeUnit.MINUTES.toMillis(15);
    RowKeyBuilder rowKeyBuilder = new SaltyRowKeyBuilder();
    ColumnBuilder columnBuilder = new ValueOnlyColumnBuilder(columnFamily);
    profileWriter = new ProfileWriter(rowKeyBuilder, columnBuilder, provider, periodDurationMillis, tableName, null);
    // global properties
    globals = new HashMap<String, Object>() {

        {
            put(PROFILER_HBASE_TABLE.getKey(), tableName);
            put(PROFILER_COLUMN_FAMILY.getKey(), columnFamily);
            put(PROFILER_HBASE_TABLE_PROVIDER.getKey(), MockHBaseTableProvider.class.getName());
            put(PROFILER_PERIOD.getKey(), Long.toString(periodDuration));
            put(PROFILER_PERIOD_UNITS.getKey(), periodUnits.toString());
            put(PROFILER_SALT_DIVISOR.getKey(), Integer.toString(saltDivisor));
        }
    };
    // create the stellar execution environment
    executor = new DefaultStellarStatefulExecutor(new SimpleFunctionResolver().withClass(VerboseProfile.class).withClass(FixedLookback.class), new Context.Builder().with(Context.Capabilities.GLOBAL_CONFIG, () -> globals).build());
}
Also used : Table(org.apache.hadoop.hbase.client.Table) SaltyRowKeyBuilder(org.apache.metron.profiler.hbase.SaltyRowKeyBuilder) MockHBaseTableProvider(org.apache.metron.hbase.mock.MockHBaseTableProvider) SaltyRowKeyBuilder(org.apache.metron.profiler.hbase.SaltyRowKeyBuilder) ColumnBuilder(org.apache.metron.profiler.hbase.ColumnBuilder) ValueOnlyColumnBuilder(org.apache.metron.profiler.hbase.ValueOnlyColumnBuilder) RowKeyBuilder(org.apache.metron.profiler.hbase.RowKeyBuilder) VerboseProfile(org.apache.metron.profiler.client.stellar.VerboseProfile) SaltyRowKeyBuilder(org.apache.metron.profiler.hbase.SaltyRowKeyBuilder) RowKeyBuilder(org.apache.metron.profiler.hbase.RowKeyBuilder) TableProvider(org.apache.metron.hbase.TableProvider) MockHBaseTableProvider(org.apache.metron.hbase.mock.MockHBaseTableProvider) ProfileWriter(org.apache.metron.profiler.client.ProfileWriter) ValueOnlyColumnBuilder(org.apache.metron.profiler.hbase.ValueOnlyColumnBuilder) DefaultStellarStatefulExecutor(org.apache.metron.stellar.common.DefaultStellarStatefulExecutor) ColumnBuilder(org.apache.metron.profiler.hbase.ColumnBuilder) ValueOnlyColumnBuilder(org.apache.metron.profiler.hbase.ValueOnlyColumnBuilder) SimpleFunctionResolver(org.apache.metron.stellar.dsl.functions.resolver.SimpleFunctionResolver) BeforeEach(org.junit.jupiter.api.BeforeEach)

Example 8 with DefaultStellarStatefulExecutor

use of org.apache.metron.stellar.common.DefaultStellarStatefulExecutor in project metron by apache.

the class ProfilerFunctionsTest method testProfilerInitWithNoGlobalConfig.

@Test
public void testProfilerInitWithNoGlobalConfig() {
    state.put("config", helloWorldProfilerDef);
    String expression = "PROFILER_INIT(config)";
    // use an executor with no GLOBAL_CONFIG defined in the context
    StellarStatefulExecutor executor = new DefaultStellarStatefulExecutor(new SimpleFunctionResolver().withClass(ProfilerFunctions.ProfilerInit.class).withClass(ProfilerFunctions.ProfilerApply.class).withClass(ProfilerFunctions.ProfilerFlush.class), Context.EMPTY_CONTEXT());
    StandAloneProfiler profiler = executor.execute(expression, state, StandAloneProfiler.class);
    assertNotNull(profiler);
    assertEquals(1, profiler.getProfileCount());
    assertEquals(0, profiler.getMessageCount());
    assertEquals(0, profiler.getRouteCount());
}
Also used : StellarStatefulExecutor(org.apache.metron.stellar.common.StellarStatefulExecutor) DefaultStellarStatefulExecutor(org.apache.metron.stellar.common.DefaultStellarStatefulExecutor) DefaultStellarStatefulExecutor(org.apache.metron.stellar.common.DefaultStellarStatefulExecutor) SimpleFunctionResolver(org.apache.metron.stellar.dsl.functions.resolver.SimpleFunctionResolver) Test(org.junit.jupiter.api.Test)

Example 9 with DefaultStellarStatefulExecutor

use of org.apache.metron.stellar.common.DefaultStellarStatefulExecutor in project metron by apache.

the class ParserFunctionsTest method setup.

@BeforeEach
public void setup() {
    variables = new HashMap<>();
    functionResolver = new SimpleFunctionResolver().withClass(ParserFunctions.ParseFunction.class).withClass(ParserFunctions.InitializeFunction.class).withClass(ParserFunctions.ConfigFunction.class);
    context = new Context.Builder().build();
    executor = new DefaultStellarStatefulExecutor(functionResolver, context);
}
Also used : GetDataBuilder(org.apache.curator.framework.api.GetDataBuilder) DefaultStellarStatefulExecutor(org.apache.metron.stellar.common.DefaultStellarStatefulExecutor) SimpleFunctionResolver(org.apache.metron.stellar.dsl.functions.resolver.SimpleFunctionResolver) BeforeEach(org.junit.jupiter.api.BeforeEach)

Example 10 with DefaultStellarStatefulExecutor

use of org.apache.metron.stellar.common.DefaultStellarStatefulExecutor in project metron by apache.

the class ProfilerIntegrationTest method setup.

@BeforeEach
public void setup() {
    // create the mock table
    profilerTable = (MockHTable) MockHBaseTableProvider.addToCache(tableName, columnFamily);
    // global properties
    Map<String, Object> global = new HashMap<String, Object>() {

        {
            put(PROFILER_HBASE_TABLE.getKey(), tableName);
            put(PROFILER_COLUMN_FAMILY.getKey(), columnFamily);
            put(PROFILER_HBASE_TABLE_PROVIDER.getKey(), MockHBaseTableProvider.class.getName());
            // client needs to use the same period duration
            put(PROFILER_PERIOD.getKey(), Long.toString(periodDurationMillis));
            put(PROFILER_PERIOD_UNITS.getKey(), "MILLISECONDS");
            // client needs to use the same salt divisor
            put(PROFILER_SALT_DIVISOR.getKey(), saltDivisor);
        }
    };
    // create the stellar execution environment
    executor = new DefaultStellarStatefulExecutor(new SimpleFunctionResolver().withClass(GetProfile.class).withClass(FixedLookback.class).withClass(WindowLookback.class), new Context.Builder().with(Context.Capabilities.GLOBAL_CONFIG, () -> global).build());
}
Also used : MockHBaseTableProvider(org.apache.metron.hbase.mock.MockHBaseTableProvider) FixedLookback(org.apache.metron.profiler.client.stellar.FixedLookback) JSONObject(org.json.simple.JSONObject) DefaultStellarStatefulExecutor(org.apache.metron.stellar.common.DefaultStellarStatefulExecutor) SimpleFunctionResolver(org.apache.metron.stellar.dsl.functions.resolver.SimpleFunctionResolver)

Aggregations

DefaultStellarStatefulExecutor (org.apache.metron.stellar.common.DefaultStellarStatefulExecutor)11 SimpleFunctionResolver (org.apache.metron.stellar.dsl.functions.resolver.SimpleFunctionResolver)10 MockHBaseTableProvider (org.apache.metron.hbase.mock.MockHBaseTableProvider)6 BeforeEach (org.junit.jupiter.api.BeforeEach)6 ColumnBuilder (org.apache.metron.profiler.hbase.ColumnBuilder)4 RowKeyBuilder (org.apache.metron.profiler.hbase.RowKeyBuilder)4 SaltyRowKeyBuilder (org.apache.metron.profiler.hbase.SaltyRowKeyBuilder)4 ValueOnlyColumnBuilder (org.apache.metron.profiler.hbase.ValueOnlyColumnBuilder)4 HashMap (java.util.HashMap)3 JSONObject (org.json.simple.JSONObject)3 Table (org.apache.hadoop.hbase.client.Table)2 ProfileWriter (org.apache.metron.profiler.client.ProfileWriter)2 FixedLookback (org.apache.metron.profiler.client.stellar.FixedLookback)2 StellarStatefulExecutor (org.apache.metron.stellar.common.StellarStatefulExecutor)2 Properties (java.util.Properties)1 GetDataBuilder (org.apache.curator.framework.api.GetDataBuilder)1 TableProvider (org.apache.metron.hbase.TableProvider)1 StandAloneProfiler (org.apache.metron.profiler.StandAloneProfiler)1 VerboseProfile (org.apache.metron.profiler.client.stellar.VerboseProfile)1 StellarStatisticsFunctions (org.apache.metron.statistics.StellarStatisticsFunctions)1