Search in sources :

Example 76 with SimplePrediction

use of org.kie.kogito.explainability.model.SimplePrediction in project kogito-apps by kiegroup.

the class PrequalificationDmnLimeExplainerTest method testExplanationStabilityWithOptimization.

@Test
void testExplanationStabilityWithOptimization() throws ExecutionException, InterruptedException, TimeoutException {
    PredictionProvider model = getModel();
    List<PredictionInput> samples = DmnTestUtils.randomPrequalificationInputs();
    List<PredictionOutput> predictionOutputs = model.predictAsync(samples.subList(0, 10)).get();
    List<Prediction> predictions = DataUtils.getPredictions(samples, predictionOutputs);
    long seed = 0;
    LimeConfigOptimizer limeConfigOptimizer = new LimeConfigOptimizer().withDeterministicExecution(true).withSampling(false).withStepCountLimit(30);
    Random random = new Random();
    LimeConfig initialConfig = new LimeConfig().withSamples(10).withPerturbationContext(new PerturbationContext(seed, random, 1));
    LimeConfig optimizedConfig = limeConfigOptimizer.optimize(initialConfig, predictions, model);
    assertThat(optimizedConfig).isNotSameAs(initialConfig);
    LimeExplainer limeExplainer = new LimeExplainer(optimizedConfig);
    PredictionInput testPredictionInput = getTestInput();
    List<PredictionOutput> testPredictionOutputs = model.predictAsync(List.of(testPredictionInput)).get(Config.INSTANCE.getAsyncTimeout(), Config.INSTANCE.getAsyncTimeUnit());
    Prediction instance = new SimplePrediction(testPredictionInput, testPredictionOutputs.get(0));
    assertDoesNotThrow(() -> ValidationUtils.validateLocalSaliencyStability(model, instance, limeExplainer, 1, 0.4, 0.4));
}
Also used : SimplePrediction(org.kie.kogito.explainability.model.SimplePrediction) PerturbationContext(org.kie.kogito.explainability.model.PerturbationContext) PredictionInput(org.kie.kogito.explainability.model.PredictionInput) LimeExplainer(org.kie.kogito.explainability.local.lime.LimeExplainer) Prediction(org.kie.kogito.explainability.model.Prediction) SimplePrediction(org.kie.kogito.explainability.model.SimplePrediction) PredictionProvider(org.kie.kogito.explainability.model.PredictionProvider) LimeConfig(org.kie.kogito.explainability.local.lime.LimeConfig) Random(java.util.Random) PredictionOutput(org.kie.kogito.explainability.model.PredictionOutput) LimeConfigOptimizer(org.kie.kogito.explainability.local.lime.optim.LimeConfigOptimizer) Test(org.junit.jupiter.api.Test)

Example 77 with SimplePrediction

use of org.kie.kogito.explainability.model.SimplePrediction in project kogito-apps by kiegroup.

the class PrequalificationDmnPDPExplainerTest method testPrequalificationDMNExplanation.

@Test
void testPrequalificationDMNExplanation() throws ExecutionException, InterruptedException, TimeoutException {
    DMNRuntime dmnRuntime = DMNKogito.createGenericDMNRuntime(new InputStreamReader(getClass().getResourceAsStream("/dmn/Prequalification-1.dmn")));
    assertEquals(1, dmnRuntime.getModels().size());
    final String NS = "http://www.trisotech.com/definitions/_f31e1f8e-d4ce-4a3a-ac3b-747efa6b3401";
    final String NAME = "Prequalification";
    DecisionModel decisionModel = new DmnDecisionModel(dmnRuntime, NS, NAME);
    PredictionProvider model = new DecisionModelWrapper(decisionModel);
    List<PredictionInput> inputs = DmnTestUtils.randomPrequalificationInputs();
    List<PredictionOutput> predictionOutputs = model.predictAsync(inputs).get(Config.INSTANCE.getAsyncTimeout(), Config.INSTANCE.getAsyncTimeUnit());
    List<Prediction> predictions = new ArrayList<>();
    for (int i = 0; i < predictionOutputs.size(); i++) {
        predictions.add(new SimplePrediction(inputs.get(i), predictionOutputs.get(i)));
    }
    PartialDependencePlotExplainer partialDependencePlotExplainer = new PartialDependencePlotExplainer();
    List<PartialDependenceGraph> pdps = partialDependencePlotExplainer.explainFromPredictions(model, predictions);
    AssertionsForClassTypes.assertThat(pdps).isNotNull();
    Assertions.assertThat(pdps).hasSize(25);
}
Also used : SimplePrediction(org.kie.kogito.explainability.model.SimplePrediction) InputStreamReader(java.io.InputStreamReader) PredictionInput(org.kie.kogito.explainability.model.PredictionInput) SimplePrediction(org.kie.kogito.explainability.model.SimplePrediction) Prediction(org.kie.kogito.explainability.model.Prediction) ArrayList(java.util.ArrayList) DecisionModel(org.kie.kogito.decision.DecisionModel) DmnDecisionModel(org.kie.kogito.dmn.DmnDecisionModel) PredictionProvider(org.kie.kogito.explainability.model.PredictionProvider) DMNRuntime(org.kie.dmn.api.core.DMNRuntime) PartialDependencePlotExplainer(org.kie.kogito.explainability.global.pdp.PartialDependencePlotExplainer) PredictionOutput(org.kie.kogito.explainability.model.PredictionOutput) DmnDecisionModel(org.kie.kogito.dmn.DmnDecisionModel) PartialDependenceGraph(org.kie.kogito.explainability.model.PartialDependenceGraph) Test(org.junit.jupiter.api.Test)

Aggregations

SimplePrediction (org.kie.kogito.explainability.model.SimplePrediction)77 Prediction (org.kie.kogito.explainability.model.Prediction)76 PredictionInput (org.kie.kogito.explainability.model.PredictionInput)75 PredictionOutput (org.kie.kogito.explainability.model.PredictionOutput)74 PredictionProvider (org.kie.kogito.explainability.model.PredictionProvider)72 Test (org.junit.jupiter.api.Test)56 Random (java.util.Random)49 PerturbationContext (org.kie.kogito.explainability.model.PerturbationContext)48 Saliency (org.kie.kogito.explainability.model.Saliency)40 LimeConfig (org.kie.kogito.explainability.local.lime.LimeConfig)39 ArrayList (java.util.ArrayList)38 LimeExplainer (org.kie.kogito.explainability.local.lime.LimeExplainer)36 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)32 Feature (org.kie.kogito.explainability.model.Feature)29 LimeConfigOptimizer (org.kie.kogito.explainability.local.lime.optim.LimeConfigOptimizer)19 DataDistribution (org.kie.kogito.explainability.model.DataDistribution)19 PredictionInputsDataDistribution (org.kie.kogito.explainability.model.PredictionInputsDataDistribution)19 ValueSource (org.junit.jupiter.params.provider.ValueSource)17 LinkedList (java.util.LinkedList)14 FeatureImportance (org.kie.kogito.explainability.model.FeatureImportance)14