Search in sources :

Example 96 with Output

use of org.kie.kogito.explainability.model.Output in project kogito-apps by kiegroup.

the class CounterfactualScoreCalculatorTest method objectDistanceDifferentValue.

@ParameterizedTest
@ValueSource(ints = { 0, 1, 2, 3, 4 })
void objectDistanceDifferentValue(int seed) {
    Random random = new Random(seed);
    Feature x = FeatureFactory.newObjectFeature("x", "test");
    Feature y = FeatureFactory.newObjectFeature("y", 20);
    Output ox = outputFromFeature(x);
    Output oy = outputFromFeature(y);
    double distance = CounterFactualScoreCalculator.outputDistance(ox, oy);
    assertEquals(Type.UNDEFINED, ox.getType());
    assertEquals(Type.UNDEFINED, oy.getType());
    assertEquals(1.0, distance);
    // Use a random threshold, mustn't make a difference
    distance = CounterFactualScoreCalculator.outputDistance(ox, oy, random.nextDouble());
    assertEquals(1.0, distance);
}
Also used : Random(java.util.Random) PredictionOutput(org.kie.kogito.explainability.model.PredictionOutput) Output(org.kie.kogito.explainability.model.Output) Feature(org.kie.kogito.explainability.model.Feature) ValueSource(org.junit.jupiter.params.provider.ValueSource) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 97 with Output

use of org.kie.kogito.explainability.model.Output in project kogito-apps by kiegroup.

the class CounterfactualScoreCalculatorTest method testPrimarySoftScore.

/**
 * Test precision errors for primary soft score.
 * When the primary soft score is calculated between features with the same numerical
 * value a similarity of 1 is expected. For a large number of features, due to floating point errors this distance may be
 * in some cases slightly larger than 1, which will cause the distance (Math.sqrt(1.0-similarity)) to cause an exception.
 * The score calculation method should not let this should not occur.
 */
@ParameterizedTest
@ValueSource(ints = { 0, 1, 2, 3, 4 })
void testPrimarySoftScore(int seed) {
    final Random random = new Random(seed);
    final List<Feature> features = new ArrayList<>();
    final List<FeatureDomain> featureDomains = new ArrayList<>();
    final List<Boolean> constraints = new ArrayList<>();
    final int nFeatures = 1000;
    // Create a large number of identical features
    for (int n = 0; n < nFeatures; n++) {
        features.add(FeatureFactory.newNumericalFeature("f-" + n, random.nextDouble() * 1e-100));
        featureDomains.add(NumericalFeatureDomain.create(0.0, 10.0));
        constraints.add(false);
    }
    final PredictionInput input = new PredictionInput(features);
    final PredictionFeatureDomain domain = new PredictionFeatureDomain(featureDomains);
    final List<CounterfactualEntity> entities = CounterfactualEntityFactory.createEntities(input);
    // Create score calculator and model
    final CounterFactualScoreCalculator scoreCalculator = new CounterFactualScoreCalculator();
    PredictionProvider model = TestUtils.getFeatureSkipModel(0);
    // Create goal
    final List<Output> goal = new ArrayList<>();
    for (int n = 1; n < nFeatures; n++) {
        goal.add(new Output("f-" + n, Type.NUMBER, features.get(n).getValue(), 1.0));
    }
    final CounterfactualSolution solution = new CounterfactualSolution(entities, features, model, goal, UUID.randomUUID(), UUID.randomUUID(), 0.0);
    final BendableBigDecimalScore score = scoreCalculator.calculateScore(solution);
    assertEquals(0.0, score.getSoftScore(0).doubleValue(), 1e-5);
}
Also used : PredictionInput(org.kie.kogito.explainability.model.PredictionInput) ArrayList(java.util.ArrayList) BendableBigDecimalScore(org.optaplanner.core.api.score.buildin.bendablebigdecimal.BendableBigDecimalScore) EmptyFeatureDomain(org.kie.kogito.explainability.model.domain.EmptyFeatureDomain) PredictionFeatureDomain(org.kie.kogito.explainability.model.PredictionFeatureDomain) NumericalFeatureDomain(org.kie.kogito.explainability.model.domain.NumericalFeatureDomain) FeatureDomain(org.kie.kogito.explainability.model.domain.FeatureDomain) PredictionProvider(org.kie.kogito.explainability.model.PredictionProvider) Feature(org.kie.kogito.explainability.model.Feature) CounterfactualEntity(org.kie.kogito.explainability.local.counterfactual.entities.CounterfactualEntity) PredictionFeatureDomain(org.kie.kogito.explainability.model.PredictionFeatureDomain) Random(java.util.Random) PredictionOutput(org.kie.kogito.explainability.model.PredictionOutput) Output(org.kie.kogito.explainability.model.Output) ValueSource(org.junit.jupiter.params.provider.ValueSource) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 98 with Output

use of org.kie.kogito.explainability.model.Output in project kogito-apps by kiegroup.

the class CounterfactualScoreCalculatorTest method timeDistanceDifferentValue.

@Test
void timeDistanceDifferentValue() {
    final LocalTime value = LocalTime.now();
    Feature x = FeatureFactory.newTimeFeature("x", LocalTime.of(15, 59));
    Feature y = FeatureFactory.newTimeFeature("y", LocalTime.of(10, 1));
    Output ox = outputFromFeature(x);
    Output oy = outputFromFeature(y);
    double distance = CounterFactualScoreCalculator.outputDistance(ox, oy);
    assertEquals(Type.TIME, ox.getType());
    assertEquals(Type.TIME, oy.getType());
    assertEquals(0.248, distance, 0.01);
    x = FeatureFactory.newTimeFeature("x", LocalTime.of(12, 0));
    y = FeatureFactory.newTimeFeature("y", LocalTime.of(12, 57));
    ox = outputFromFeature(x);
    oy = outputFromFeature(y);
    distance = CounterFactualScoreCalculator.outputDistance(ox, oy);
    assertEquals(0.039, distance, 0.01);
    x = FeatureFactory.newTimeFeature("x", LocalTime.of(0, 0));
    y = FeatureFactory.newTimeFeature("y", LocalTime.of(15, 17));
    ox = outputFromFeature(x);
    oy = outputFromFeature(y);
    distance = CounterFactualScoreCalculator.outputDistance(ox, oy);
    assertEquals(0.636, distance, 0.01);
}
Also used : LocalTime(java.time.LocalTime) PredictionOutput(org.kie.kogito.explainability.model.PredictionOutput) Output(org.kie.kogito.explainability.model.Output) Feature(org.kie.kogito.explainability.model.Feature) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 99 with Output

use of org.kie.kogito.explainability.model.Output in project kogito-apps by kiegroup.

the class CounterfactualScoreCalculatorTest method IntegerDistanceDifferentValueThreshold.

@ParameterizedTest
@ValueSource(ints = { 0, 1, 2, 3, 4 })
void IntegerDistanceDifferentValueThreshold(int seed) {
    final Random random = new Random(seed);
    int value = random.nextInt(1000);
    Feature x = FeatureFactory.newNumericalFeature("x", value);
    Feature y = FeatureFactory.newNumericalFeature("y", value + 100);
    Output ox = outputFromFeature(x);
    Output oy = outputFromFeature(y);
    double distance = CounterFactualScoreCalculator.outputDistance(ox, oy, 0.05);
    assertEquals(Type.NUMBER, ox.getType());
    assertEquals(Type.NUMBER, oy.getType());
    assertTrue(distance * distance > 0);
    y = FeatureFactory.newNumericalFeature("y", value - 100);
    oy = outputFromFeature(y);
    distance = CounterFactualScoreCalculator.outputDistance(ox, oy, 0.05);
    assertTrue(distance * distance > 0);
}
Also used : Random(java.util.Random) PredictionOutput(org.kie.kogito.explainability.model.PredictionOutput) Output(org.kie.kogito.explainability.model.Output) Feature(org.kie.kogito.explainability.model.Feature) ValueSource(org.junit.jupiter.params.provider.ValueSource) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 100 with Output

use of org.kie.kogito.explainability.model.Output in project kogito-apps by kiegroup.

the class CounterfactualScoreCalculatorTest method durationDistanceSameValue.

@ParameterizedTest
@ValueSource(ints = { 0, 1, 2, 3, 4 })
void durationDistanceSameValue(int seed) {
    final Random random = new Random(seed);
    final Duration value = Duration.ofSeconds(random.nextLong());
    Feature x = FeatureFactory.newDurationFeature("x", value);
    Feature y = FeatureFactory.newDurationFeature("y", value);
    Output ox = outputFromFeature(x);
    Output oy = outputFromFeature(y);
    double distance = CounterFactualScoreCalculator.outputDistance(ox, oy);
    assertEquals(Type.DURATION, ox.getType());
    assertEquals(0.0, Math.abs(distance));
    // Use a random threshold, mustn't make a difference
    distance = CounterFactualScoreCalculator.outputDistance(ox, oy, random.nextDouble());
    assertEquals(0.0, Math.abs(distance));
}
Also used : Random(java.util.Random) PredictionOutput(org.kie.kogito.explainability.model.PredictionOutput) Output(org.kie.kogito.explainability.model.Output) Duration(java.time.Duration) Feature(org.kie.kogito.explainability.model.Feature) ValueSource(org.junit.jupiter.params.provider.ValueSource) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Aggregations

Output (org.kie.kogito.explainability.model.Output)120 PredictionOutput (org.kie.kogito.explainability.model.PredictionOutput)109 Feature (org.kie.kogito.explainability.model.Feature)102 Value (org.kie.kogito.explainability.model.Value)63 Random (java.util.Random)61 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)59 PredictionInput (org.kie.kogito.explainability.model.PredictionInput)57 PredictionProvider (org.kie.kogito.explainability.model.PredictionProvider)52 ArrayList (java.util.ArrayList)47 ValueSource (org.junit.jupiter.params.provider.ValueSource)47 Prediction (org.kie.kogito.explainability.model.Prediction)46 Test (org.junit.jupiter.api.Test)42 List (java.util.List)39 Type (org.kie.kogito.explainability.model.Type)36 LinkedList (java.util.LinkedList)35 CounterfactualEntity (org.kie.kogito.explainability.local.counterfactual.entities.CounterfactualEntity)23 Mockito.mock (org.mockito.Mockito.mock)20 Optional (java.util.Optional)19 ExecutionException (java.util.concurrent.ExecutionException)19 Collectors (java.util.stream.Collectors)18