Search in sources :

Example 6 with CounterfactualSearchDomain

use of org.kie.kogito.explainability.api.CounterfactualSearchDomain in project kogito-apps by kiegroup.

the class TrustyServiceTest method doGivenStoredExecutionWhenCounterfactualRequestIsMadeThenRequestIsStoredTest.

@SuppressWarnings("unchecked")
void doGivenStoredExecutionWhenCounterfactualRequestIsMadeThenRequestIsStoredTest(CounterfactualDomain domain) {
    Storage<String, Decision> decisionStorage = mock(Storage.class);
    Storage<String, CounterfactualExplainabilityRequest> counterfactualStorage = mock(Storage.class);
    ArgumentCaptor<CounterfactualExplainabilityRequest> counterfactualArgumentCaptor = ArgumentCaptor.forClass(CounterfactualExplainabilityRequest.class);
    when(decisionStorage.containsKey(eq(TEST_EXECUTION_ID))).thenReturn(true);
    when(trustyStorageServiceMock.getDecisionsStorage()).thenReturn(decisionStorage);
    when(trustyStorageServiceMock.getCounterfactualRequestStorage()).thenReturn(counterfactualStorage);
    when(decisionStorage.get(eq(TEST_EXECUTION_ID))).thenReturn(TrustyServiceTestUtils.buildCorrectDecision(TEST_EXECUTION_ID));
    // The Goals structures must be comparable to the original decisions outcomes.
    // The Search Domain structures must be identical those of the original decision inputs.
    trustyService.requestCounterfactuals(TEST_EXECUTION_ID, List.of(new NamedTypedValue("Fine", new StructureValue("tFine", Map.of("Amount", new UnitValue("number", "number", new IntNode(0)), "Points", new UnitValue("number", "number", new IntNode(0))))), new NamedTypedValue("Should the driver be suspended?", new UnitValue("string", "string", new TextNode("No")))), List.of(new CounterfactualSearchDomain("Violation", new CounterfactualSearchDomainStructureValue("tViolation", Map.of("Type", new CounterfactualSearchDomainUnitValue("string", "string", true, domain), "Actual Speed", new CounterfactualSearchDomainUnitValue("number", "number", true, domain), "Speed Limit", new CounterfactualSearchDomainUnitValue("number", "number", true, domain)))), new CounterfactualSearchDomain("Driver", new CounterfactualSearchDomainStructureValue("tDriver", Map.of("Age", new CounterfactualSearchDomainUnitValue("number", "number", true, domain), "Points", new CounterfactualSearchDomainUnitValue("number", "number", true, domain))))));
    verify(counterfactualStorage).put(anyString(), counterfactualArgumentCaptor.capture());
    CounterfactualExplainabilityRequest counterfactual = counterfactualArgumentCaptor.getValue();
    assertNotNull(counterfactual);
    assertEquals(TEST_EXECUTION_ID, counterfactual.getExecutionId());
}
Also used : CounterfactualExplainabilityRequest(org.kie.kogito.explainability.api.CounterfactualExplainabilityRequest) UnitValue(org.kie.kogito.tracing.typedvalue.UnitValue) CounterfactualSearchDomainUnitValue(org.kie.kogito.explainability.api.CounterfactualSearchDomainUnitValue) TextNode(com.fasterxml.jackson.databind.node.TextNode) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Decision(org.kie.kogito.trusty.storage.api.model.decision.Decision) NamedTypedValue(org.kie.kogito.explainability.api.NamedTypedValue) IntNode(com.fasterxml.jackson.databind.node.IntNode) StructureValue(org.kie.kogito.tracing.typedvalue.StructureValue) CounterfactualSearchDomainStructureValue(org.kie.kogito.explainability.api.CounterfactualSearchDomainStructureValue) CounterfactualSearchDomainUnitValue(org.kie.kogito.explainability.api.CounterfactualSearchDomainUnitValue) CounterfactualSearchDomain(org.kie.kogito.explainability.api.CounterfactualSearchDomain) CounterfactualSearchDomainStructureValue(org.kie.kogito.explainability.api.CounterfactualSearchDomainStructureValue)

Example 7 with CounterfactualSearchDomain

use of org.kie.kogito.explainability.api.CounterfactualSearchDomain in project kogito-apps by kiegroup.

the class AbstractTrustyServiceIT method testCounterfactuals_StoreSingleAndRetrieveSingleWithEmptyDefinition.

@Test
public void testCounterfactuals_StoreSingleAndRetrieveSingleWithEmptyDefinition() {
    String executionId = "myCFExecution1";
    storeExecution(executionId, 0L);
    // The Goals structures must be comparable to the original decisions outcomes.
    // The Search Domain structures must be identical those of the original decision inputs.
    CounterfactualSearchDomain searchDomain = buildSearchDomainUnit("test", "number", new CounterfactualDomainRange(new IntNode(1), new IntNode(2)));
    CounterfactualExplainabilityRequest request = trustyService.requestCounterfactuals(executionId, Collections.emptyList(), Collections.singletonList(searchDomain));
    assertNotNull(request);
    assertEquals(request.getExecutionId(), executionId);
    assertNotNull(request.getCounterfactualId());
    CounterfactualExplainabilityRequest result = trustyService.getCounterfactualRequest(executionId, request.getCounterfactualId());
    assertNotNull(result);
    assertEquals(request.getExecutionId(), result.getExecutionId());
    assertEquals(request.getCounterfactualId(), result.getCounterfactualId());
}
Also used : CounterfactualExplainabilityRequest(org.kie.kogito.explainability.api.CounterfactualExplainabilityRequest) IntNode(com.fasterxml.jackson.databind.node.IntNode) CounterfactualDomainRange(org.kie.kogito.explainability.api.CounterfactualDomainRange) CounterfactualSearchDomain(org.kie.kogito.explainability.api.CounterfactualSearchDomain) Test(org.junit.jupiter.api.Test)

Example 8 with CounterfactualSearchDomain

use of org.kie.kogito.explainability.api.CounterfactualSearchDomain in project kogito-apps by kiegroup.

the class AbstractTrustyServiceIT method testCounterfactuals_StoreSingleAndRetrieveSingleWithSearchDomainRange.

@Test
public void testCounterfactuals_StoreSingleAndRetrieveSingleWithSearchDomainRange() {
    String executionId = "myCFExecution1";
    storeExecution(executionId, 0L);
    // The Goals structures must be comparable to the original decisions outcomes.
    // The Search Domain structures must be identical those of the original decision inputs.
    CounterfactualSearchDomain searchDomain = buildSearchDomainUnit("test", "number", new CounterfactualDomainRange(new IntNode(1), new IntNode(2)));
    CounterfactualExplainabilityRequest request = trustyService.requestCounterfactuals(executionId, Collections.emptyList(), Collections.singletonList(searchDomain));
    assertNotNull(request);
    assertEquals(request.getExecutionId(), executionId);
    assertNotNull(request.getCounterfactualId());
    assertEquals(1, request.getSearchDomains().size());
    List<CounterfactualSearchDomain> requestSearchDomains = new ArrayList<>(request.getSearchDomains());
    assertCounterfactualSearchDomainRange(searchDomain, requestSearchDomains.get(0));
    CounterfactualExplainabilityRequest result = trustyService.getCounterfactualRequest(executionId, request.getCounterfactualId());
    assertNotNull(result);
    assertEquals(request.getExecutionId(), result.getExecutionId());
    assertEquals(request.getCounterfactualId(), result.getCounterfactualId());
    assertEquals(1, result.getSearchDomains().size());
    List<CounterfactualSearchDomain> resultSearchDomains = new ArrayList<>(result.getSearchDomains());
    assertCounterfactualSearchDomainRange(searchDomain, resultSearchDomains.get(0));
}
Also used : CounterfactualExplainabilityRequest(org.kie.kogito.explainability.api.CounterfactualExplainabilityRequest) IntNode(com.fasterxml.jackson.databind.node.IntNode) CounterfactualDomainRange(org.kie.kogito.explainability.api.CounterfactualDomainRange) ArrayList(java.util.ArrayList) CounterfactualSearchDomain(org.kie.kogito.explainability.api.CounterfactualSearchDomain) Test(org.junit.jupiter.api.Test)

Example 9 with CounterfactualSearchDomain

use of org.kie.kogito.explainability.api.CounterfactualSearchDomain in project kogito-apps by kiegroup.

the class AbstractTrustyServiceIT method testCounterfactuals_StoreMultipleAndRetrieveAllWithEmptyDefinition.

@Test
public void testCounterfactuals_StoreMultipleAndRetrieveAllWithEmptyDefinition() {
    String executionId = "myCFExecution2";
    storeExecution(executionId, 0L);
    // The Goals structures must be comparable to the original decisions outcomes.
    // The Search Domain structures must be identical those of the original decision inputs.
    CounterfactualSearchDomain searchDomain = buildSearchDomainUnit("test", "number", new CounterfactualDomainRange(new IntNode(1), new IntNode(2)));
    CounterfactualExplainabilityRequest request1 = trustyService.requestCounterfactuals(executionId, Collections.emptyList(), Collections.singletonList(searchDomain));
    CounterfactualExplainabilityRequest request2 = trustyService.requestCounterfactuals(executionId, Collections.emptyList(), Collections.singletonList(searchDomain));
    List<CounterfactualExplainabilityRequest> result = trustyService.getCounterfactualRequests(executionId);
    assertNotNull(result);
    assertEquals(2, result.size());
    assertTrue(result.stream().anyMatch(c -> c.getCounterfactualId().equals(request1.getCounterfactualId())));
    assertTrue(result.stream().anyMatch(c -> c.getCounterfactualId().equals(request2.getCounterfactualId())));
}
Also used : CounterfactualExplainabilityRequest(org.kie.kogito.explainability.api.CounterfactualExplainabilityRequest) Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) BeforeEach(org.junit.jupiter.api.BeforeEach) LIMEExplainabilityResult(org.kie.kogito.explainability.api.LIMEExplainabilityResult) Assertions.assertNotNull(org.junit.jupiter.api.Assertions.assertNotNull) FeatureImportanceModel(org.kie.kogito.explainability.api.FeatureImportanceModel) Decision(org.kie.kogito.trusty.storage.api.model.decision.Decision) DecisionInput(org.kie.kogito.trusty.storage.api.model.decision.DecisionInput) IntNode(com.fasterxml.jackson.databind.node.IntNode) CounterfactualDomainRange(org.kie.kogito.explainability.api.CounterfactualDomainRange) ArrayList(java.util.ArrayList) MatchedExecutionHeaders(org.kie.kogito.trusty.service.common.models.MatchedExecutionHeaders) Inject(javax.inject.Inject) JsonNode(com.fasterxml.jackson.databind.JsonNode) TypedValueTestUtils.buildSearchDomainUnit(org.kie.kogito.trusty.service.common.TypedValueTestUtils.buildSearchDomainUnit) ZoneOffset(java.time.ZoneOffset) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) CounterfactualDomainCategorical(org.kie.kogito.explainability.api.CounterfactualDomainCategorical) TypedValueTestUtils.buildGoalUnit(org.kie.kogito.trusty.service.common.TypedValueTestUtils.buildGoalUnit) CounterfactualExplainabilityRequest(org.kie.kogito.explainability.api.CounterfactualExplainabilityRequest) NamedTypedValue(org.kie.kogito.explainability.api.NamedTypedValue) TrustyStorageService(org.kie.kogito.trusty.storage.common.TrustyStorageService) Collection(java.util.Collection) ExplainabilityStatus(org.kie.kogito.explainability.api.ExplainabilityStatus) Instant(java.time.Instant) TextNode(com.fasterxml.jackson.databind.node.TextNode) UnitValue(org.kie.kogito.tracing.typedvalue.UnitValue) Test(org.junit.jupiter.api.Test) DMNModelMetadata(org.kie.kogito.trusty.storage.api.model.decision.DMNModelMetadata) List(java.util.List) OffsetDateTime(java.time.OffsetDateTime) CounterfactualDomain(org.kie.kogito.explainability.api.CounterfactualDomain) JsonNodeFactory(com.fasterxml.jackson.databind.node.JsonNodeFactory) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) Assertions(org.junit.jupiter.api.Assertions) CounterfactualExplainabilityResult(org.kie.kogito.explainability.api.CounterfactualExplainabilityResult) CounterfactualSearchDomain(org.kie.kogito.explainability.api.CounterfactualSearchDomain) SaliencyModel(org.kie.kogito.explainability.api.SaliencyModel) Collections(java.util.Collections) DMNModelWithMetadata(org.kie.kogito.trusty.storage.api.model.decision.DMNModelWithMetadata) DecisionOutcome(org.kie.kogito.trusty.storage.api.model.decision.DecisionOutcome) IntNode(com.fasterxml.jackson.databind.node.IntNode) CounterfactualDomainRange(org.kie.kogito.explainability.api.CounterfactualDomainRange) CounterfactualSearchDomain(org.kie.kogito.explainability.api.CounterfactualSearchDomain) Test(org.junit.jupiter.api.Test)

Example 10 with CounterfactualSearchDomain

use of org.kie.kogito.explainability.api.CounterfactualSearchDomain in project kogito-apps by kiegroup.

the class TrustyServiceImpl method makeCounterfactualRequest.

protected CounterfactualExplainabilityRequest makeCounterfactualRequest(String executionId, List<NamedTypedValue> goals, List<CounterfactualSearchDomain> searchDomains, Long maxRunningTimeSeconds) {
    Decision decision = getDecisionById(executionId);
    // This is returned as null under Redis, so play safe
    Collection<DecisionInput> decisionInputs = Objects.nonNull(decision.getInputs()) ? decision.getInputs() : Collections.emptyList();
    if (!isStructureIdentical(decisionInputs, searchDomains)) {
        String error = buildCounterfactualErrorMessage(String.format("The structure of the Search Domains do not match the structure of the original Inputs for decision with ID %s.", executionId), "Decision inputs:-", decisionInputs, "Search domains:-", searchDomains);
        LOG.error(error);
        throw new IllegalArgumentException(error);
    }
    // This is returned as null under Redis, so play safe
    Collection<DecisionOutcome> decisionOutcomes = Objects.nonNull(decision.getOutcomes()) ? decision.getOutcomes() : Collections.emptyList();
    if (!isStructureSubset(decisionOutcomes, goals)) {
        String error = buildCounterfactualErrorMessage(String.format("The structure of the Goals is not comparable to the structure of the original Outcomes for decision with ID %s.", executionId), "Decision outcomes:-", decisionOutcomes, "Goals:-", goals);
        LOG.error(error);
        throw new IllegalArgumentException(error);
    }
    List<NamedTypedValue> cfInputs = decision.getInputs() != null ? decision.getInputs().stream().map(input -> new NamedTypedValue(input.getName(), input.getValue())).collect(Collectors.toList()) : Collections.emptyList();
    List<NamedTypedValue> cfGoals = goals != null ? goals : Collections.emptyList();
    List<CounterfactualSearchDomain> cfSearchDomains = searchDomains != null ? searchDomains : Collections.emptyList();
    return new CounterfactualExplainabilityRequest(executionId, decision.getServiceUrl(), createDecisionModelIdentifier(decision), UUID.randomUUID().toString(), cfInputs, cfGoals, cfSearchDomains, maxRunningTimeSeconds);
}
Also used : DecisionInput(org.kie.kogito.trusty.storage.api.model.decision.DecisionInput) CounterfactualExplainabilityRequest(org.kie.kogito.explainability.api.CounterfactualExplainabilityRequest) NamedTypedValue(org.kie.kogito.explainability.api.NamedTypedValue) DecisionOutcome(org.kie.kogito.trusty.storage.api.model.decision.DecisionOutcome) Decision(org.kie.kogito.trusty.storage.api.model.decision.Decision) CounterfactualSearchDomain(org.kie.kogito.explainability.api.CounterfactualSearchDomain)

Aggregations

CounterfactualSearchDomain (org.kie.kogito.explainability.api.CounterfactualSearchDomain)28 Test (org.junit.jupiter.api.Test)22 CounterfactualDomainRange (org.kie.kogito.explainability.api.CounterfactualDomainRange)19 CounterfactualExplainabilityRequest (org.kie.kogito.explainability.api.CounterfactualExplainabilityRequest)19 NamedTypedValue (org.kie.kogito.explainability.api.NamedTypedValue)17 IntNode (com.fasterxml.jackson.databind.node.IntNode)16 CounterfactualSearchDomainUnitValue (org.kie.kogito.explainability.api.CounterfactualSearchDomainUnitValue)12 TextNode (com.fasterxml.jackson.databind.node.TextNode)11 ArrayList (java.util.ArrayList)9 CounterfactualDomainCategorical (org.kie.kogito.explainability.api.CounterfactualDomainCategorical)9 UnitValue (org.kie.kogito.tracing.typedvalue.UnitValue)9 List (java.util.List)6 JsonNode (com.fasterxml.jackson.databind.JsonNode)5 Map (java.util.Map)4 CounterfactualDomain (org.kie.kogito.explainability.api.CounterfactualDomain)4 Feature (org.kie.kogito.explainability.model.Feature)4 StructureValue (org.kie.kogito.tracing.typedvalue.StructureValue)4 CounterfactualRequestResponse (org.kie.kogito.trusty.service.common.responses.CounterfactualRequestResponse)4 Decision (org.kie.kogito.trusty.storage.api.model.decision.Decision)4 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)4