Search in sources :

Example 1 with CollectionValue

use of org.kie.kogito.tracing.typedvalue.CollectionValue in project kogito-apps by kiegroup.

the class CounterfactualExplainerServiceHandlerTest method testGetPredictionWithCollectionOutputModel.

@Test
public void testGetPredictionWithCollectionOutputModel() {
    CounterfactualExplainabilityRequest request = new CounterfactualExplainabilityRequest(EXECUTION_ID, SERVICE_URL, MODEL_IDENTIFIER, COUNTERFACTUAL_ID, Collections.emptyList(), List.of(new NamedTypedValue("input1", new CollectionValue("number", List.of(new UnitValue("number", new IntNode(100)))))), Collections.emptyList(), MAX_RUNNING_TIME_SECONDS);
    assertThrows(IllegalArgumentException.class, () -> handler.getPrediction(request));
}
Also used : CounterfactualExplainabilityRequest(org.kie.kogito.explainability.api.CounterfactualExplainabilityRequest) NamedTypedValue(org.kie.kogito.explainability.api.NamedTypedValue) CollectionValue(org.kie.kogito.tracing.typedvalue.CollectionValue) CounterfactualSearchDomainCollectionValue(org.kie.kogito.explainability.api.CounterfactualSearchDomainCollectionValue) IntNode(com.fasterxml.jackson.databind.node.IntNode) UnitValue(org.kie.kogito.tracing.typedvalue.UnitValue) CounterfactualSearchDomainUnitValue(org.kie.kogito.explainability.api.CounterfactualSearchDomainUnitValue) Test(org.junit.jupiter.api.Test)

Example 2 with CollectionValue

use of org.kie.kogito.tracing.typedvalue.CollectionValue in project kogito-apps by kiegroup.

the class ConversionUtilsTest method toOutputTypedValue.

@Test
void toOutputTypedValue() {
    Output name = ConversionUtils.toOutput("name", new UnitValue("number", new DoubleNode(10d)));
    assertNotNull(name);
    assertEquals("name", name.getName());
    assertEquals(Type.NUMBER, name.getType());
    assertEquals(10d, name.getValue().getUnderlyingObject());
    Output name1 = ConversionUtils.toOutput("name1", new StructureValue("complex", singletonMap("key", new UnitValue("string1", new TextNode("stringValue")))));
    assertNotNull(name1);
    assertEquals("name1", name1.getName());
    assertEquals(Type.COMPOSITE, name1.getType());
    assertTrue(name1.getValue().getUnderlyingObject() instanceof List);
    @SuppressWarnings("unchecked") List<Output> outputs = (List<Output>) name1.getValue().getUnderlyingObject();
    assertEquals(1, outputs.size());
    assertEquals(Type.TEXT, outputs.get(0).getType());
    assertEquals("stringValue", outputs.get(0).getValue().getUnderlyingObject());
    List<TypedValue> values = List.of(new UnitValue("number", new DoubleNode(0d)), new UnitValue("number", new DoubleNode(1d)));
    assertNotNull(ConversionUtils.toOutput("name", new CollectionValue("list", values)));
}
Also used : CollectionValue(org.kie.kogito.tracing.typedvalue.CollectionValue) Output(org.kie.kogito.explainability.model.Output) UnitValue(org.kie.kogito.tracing.typedvalue.UnitValue) CounterfactualSearchDomainUnitValue(org.kie.kogito.explainability.api.CounterfactualSearchDomainUnitValue) DoubleNode(com.fasterxml.jackson.databind.node.DoubleNode) StructureValue(org.kie.kogito.tracing.typedvalue.StructureValue) TextNode(com.fasterxml.jackson.databind.node.TextNode) ArrayList(java.util.ArrayList) Collections.emptyList(java.util.Collections.emptyList) List(java.util.List) NamedTypedValue(org.kie.kogito.explainability.api.NamedTypedValue) TypedValue(org.kie.kogito.tracing.typedvalue.TypedValue) Test(org.junit.jupiter.api.Test)

Example 3 with CollectionValue

use of org.kie.kogito.tracing.typedvalue.CollectionValue in project kogito-apps by kiegroup.

the class TrustyServiceTest method givenADecisionToProcessWhenExplainabilityIsEnabledThenRequestIsSent.

@Test
@SuppressWarnings("unchecked")
void givenADecisionToProcessWhenExplainabilityIsEnabledThenRequestIsSent() throws JsonProcessingException {
    trustyService.enableExplainability();
    Decision decision = new Decision(TEST_EXECUTION_ID, TEST_SOURCE_URL, TEST_SERVICE_URL, 1591692950000L, true, null, "model", "modelNamespace", List.of(new DecisionInput("1", "Input1", new CollectionValue("string", List.of(new UnitValue("string", "string", toJsonNode("\"ONE\"")), new UnitValue("string", "string", toJsonNode("\"TWO\""))))), new DecisionInput("2", "Input2", new StructureValue("Person", Map.of("Name", new UnitValue("string", "string", toJsonNode("\"George Orwell\"")), "Age", new UnitValue("number", "number", toJsonNode("45")))))), List.of(new DecisionOutcome("OUT1", "Result", "SUCCEEDED", new UnitValue("string", "string", toJsonNode("\"YES\"")), Collections.emptyList(), Collections.emptyList())));
    Storage<String, Decision> decisionStorageMock = mock(Storage.class);
    when(decisionStorageMock.containsKey(eq(TEST_EXECUTION_ID))).thenReturn(false);
    when(trustyStorageServiceMock.getDecisionsStorage()).thenReturn(decisionStorageMock);
    trustyService.processDecision(TEST_EXECUTION_ID, decision);
    verify(explainabilityRequestProducerMock).sendEvent(any());
}
Also used : DecisionInput(org.kie.kogito.trusty.storage.api.model.decision.DecisionInput) CollectionValue(org.kie.kogito.tracing.typedvalue.CollectionValue) DecisionOutcome(org.kie.kogito.trusty.storage.api.model.decision.DecisionOutcome) UnitValue(org.kie.kogito.tracing.typedvalue.UnitValue) CounterfactualSearchDomainUnitValue(org.kie.kogito.explainability.api.CounterfactualSearchDomainUnitValue) StructureValue(org.kie.kogito.tracing.typedvalue.StructureValue) CounterfactualSearchDomainStructureValue(org.kie.kogito.explainability.api.CounterfactualSearchDomainStructureValue) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Decision(org.kie.kogito.trusty.storage.api.model.decision.Decision) Test(org.junit.jupiter.api.Test)

Example 4 with CollectionValue

use of org.kie.kogito.tracing.typedvalue.CollectionValue in project kogito-apps by kiegroup.

the class CounterfactualExplainerServiceHandlerTest method testGetPredictionWithCollectionInputModel.

@Test
public void testGetPredictionWithCollectionInputModel() {
    CounterfactualExplainabilityRequest request = new CounterfactualExplainabilityRequest(EXECUTION_ID, SERVICE_URL, MODEL_IDENTIFIER, COUNTERFACTUAL_ID, List.of(new NamedTypedValue("input1", new CollectionValue("number", List.of(new UnitValue("number", new IntNode(100)))))), Collections.emptyList(), Collections.emptyList(), MAX_RUNNING_TIME_SECONDS);
    assertThrows(IllegalArgumentException.class, () -> handler.getPrediction(request));
}
Also used : CounterfactualExplainabilityRequest(org.kie.kogito.explainability.api.CounterfactualExplainabilityRequest) NamedTypedValue(org.kie.kogito.explainability.api.NamedTypedValue) CollectionValue(org.kie.kogito.tracing.typedvalue.CollectionValue) CounterfactualSearchDomainCollectionValue(org.kie.kogito.explainability.api.CounterfactualSearchDomainCollectionValue) IntNode(com.fasterxml.jackson.databind.node.IntNode) UnitValue(org.kie.kogito.tracing.typedvalue.UnitValue) CounterfactualSearchDomainUnitValue(org.kie.kogito.explainability.api.CounterfactualSearchDomainUnitValue) Test(org.junit.jupiter.api.Test)

Example 5 with CollectionValue

use of org.kie.kogito.tracing.typedvalue.CollectionValue in project kogito-apps by kiegroup.

the class LimeExplainerServiceHandlerTest method testGetPredictionWithNonEmptyDefinition.

@Test
@SuppressWarnings("unchecked")
public void testGetPredictionWithNonEmptyDefinition() {
    LIMEExplainabilityRequest request = new LIMEExplainabilityRequest(EXECUTION_ID, SERVICE_URL, MODEL_IDENTIFIER, List.of(new NamedTypedValue("input1", new UnitValue("number", "number", new IntNode(20))), new NamedTypedValue("input2", new StructureValue("number", Map.of("input2b", new UnitValue("number", new IntNode(55))))), new NamedTypedValue("input3", new CollectionValue("number", List.of(new UnitValue("number", new IntNode(100)))))), List.of(new NamedTypedValue("output1", new UnitValue("number", "number", new IntNode(20))), new NamedTypedValue("output2", new StructureValue("number", Map.of("output2b", new UnitValue("number", new IntNode(55))))), new NamedTypedValue("output3", new CollectionValue("number", List.of(new UnitValue("number", new IntNode(100)))))));
    Prediction prediction = handler.getPrediction(request);
    // Inputs
    assertEquals(3, prediction.getInput().getFeatures().size());
    Optional<Feature> oInput1 = prediction.getInput().getFeatures().stream().filter(f -> f.getName().equals("input1")).findFirst();
    assertTrue(oInput1.isPresent());
    Feature input1 = oInput1.get();
    assertEquals(Type.NUMBER, input1.getType());
    assertEquals(20, input1.getValue().asNumber());
    Optional<Feature> oInput2 = prediction.getInput().getFeatures().stream().filter(f -> f.getName().equals("input2")).findFirst();
    assertTrue(oInput2.isPresent());
    Feature input2 = oInput2.get();
    assertEquals(Type.COMPOSITE, input2.getType());
    assertTrue(input2.getValue().getUnderlyingObject() instanceof List);
    List<Feature> input2Object = (List<Feature>) input2.getValue().getUnderlyingObject();
    assertEquals(1, input2Object.size());
    Optional<Feature> oInput2Child = input2Object.stream().filter(f -> f.getName().equals("input2b")).findFirst();
    assertTrue(oInput2Child.isPresent());
    Feature input2Child = oInput2Child.get();
    assertEquals(Type.NUMBER, input2Child.getType());
    assertEquals(55, input2Child.getValue().asNumber());
    Optional<Feature> oInput3 = prediction.getInput().getFeatures().stream().filter(f -> f.getName().equals("input3")).findFirst();
    assertTrue(oInput3.isPresent());
    Feature input3 = oInput3.get();
    assertEquals(Type.COMPOSITE, input3.getType());
    assertTrue(input3.getValue().getUnderlyingObject() instanceof List);
    List<Feature> input3Object = (List<Feature>) input3.getValue().getUnderlyingObject();
    assertEquals(1, input3Object.size());
    Feature input3Child = input3Object.get(0);
    assertEquals(Type.NUMBER, input3Child.getType());
    assertEquals(100, input3Child.getValue().asNumber());
    // Outputs
    assertEquals(3, prediction.getOutput().getOutputs().size());
    Optional<Output> oOutput1 = prediction.getOutput().getOutputs().stream().filter(o -> o.getName().equals("output1")).findFirst();
    assertTrue(oOutput1.isPresent());
    Output output1 = oOutput1.get();
    assertEquals(Type.NUMBER, output1.getType());
    assertEquals(20, output1.getValue().asNumber());
    Optional<Output> oOutput2 = prediction.getOutput().getOutputs().stream().filter(o -> o.getName().equals("output2")).findFirst();
    assertTrue(oOutput2.isPresent());
    Output output2 = oOutput2.get();
    assertEquals(Type.COMPOSITE, input2.getType());
    assertTrue(output2.getValue().getUnderlyingObject() instanceof List);
    List<Output> output2Object = (List<Output>) output2.getValue().getUnderlyingObject();
    assertEquals(1, output2Object.size());
    Optional<Output> oOutput2Child = output2Object.stream().filter(f -> f.getName().equals("output2b")).findFirst();
    assertTrue(oOutput2Child.isPresent());
    Output output2Child = oOutput2Child.get();
    assertEquals(Type.NUMBER, output2Child.getType());
    assertEquals(55, output2Child.getValue().asNumber());
    Optional<Output> oOutput3 = prediction.getOutput().getOutputs().stream().filter(o -> o.getName().equals("output3")).findFirst();
    assertTrue(oOutput3.isPresent());
    Output output3 = oOutput3.get();
    assertEquals(Type.COMPOSITE, output3.getType());
    assertTrue(output3.getValue().getUnderlyingObject() instanceof List);
    List<Output> output3Object = (List<Output>) output3.getValue().getUnderlyingObject();
    assertEquals(1, output3Object.size());
    Output output3Child = output3Object.get(0);
    assertEquals(Type.NUMBER, output3Child.getType());
    assertEquals(100, output3Child.getValue().asNumber());
}
Also used : Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) BeforeEach(org.junit.jupiter.api.BeforeEach) LIMEExplainabilityResult(org.kie.kogito.explainability.api.LIMEExplainabilityResult) IntNode(com.fasterxml.jackson.databind.node.IntNode) BaseExplainabilityRequest(org.kie.kogito.explainability.api.BaseExplainabilityRequest) Feature(org.kie.kogito.explainability.model.Feature) Prediction(org.kie.kogito.explainability.model.Prediction) ArgumentMatchers.eq(org.mockito.ArgumentMatchers.eq) Value(org.kie.kogito.explainability.model.Value) Saliency(org.kie.kogito.explainability.model.Saliency) StructureValue(org.kie.kogito.tracing.typedvalue.StructureValue) Assertions.assertFalse(org.junit.jupiter.api.Assertions.assertFalse) Map(java.util.Map) PredictionProviderFactory(org.kie.kogito.explainability.PredictionProviderFactory) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) BaseExplainabilityResult(org.kie.kogito.explainability.api.BaseExplainabilityResult) SimplePrediction(org.kie.kogito.explainability.model.SimplePrediction) NamedTypedValue(org.kie.kogito.explainability.api.NamedTypedValue) LimeExplainer(org.kie.kogito.explainability.local.lime.LimeExplainer) ExplainabilityStatus(org.kie.kogito.explainability.api.ExplainabilityStatus) FeatureImportance(org.kie.kogito.explainability.model.FeatureImportance) Type(org.kie.kogito.explainability.model.Type) PredictionProvider(org.kie.kogito.explainability.model.PredictionProvider) UnitValue(org.kie.kogito.tracing.typedvalue.UnitValue) Mockito.verify(org.mockito.Mockito.verify) Consumer(java.util.function.Consumer) Test(org.junit.jupiter.api.Test) List(java.util.List) LIMEExplainabilityRequest(org.kie.kogito.explainability.api.LIMEExplainabilityRequest) CollectionValue(org.kie.kogito.tracing.typedvalue.CollectionValue) Output(org.kie.kogito.explainability.model.Output) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) Optional(java.util.Optional) SaliencyModel(org.kie.kogito.explainability.api.SaliencyModel) Collections(java.util.Collections) ModelIdentifier(org.kie.kogito.explainability.api.ModelIdentifier) Mockito.mock(org.mockito.Mockito.mock) LIMEExplainabilityRequest(org.kie.kogito.explainability.api.LIMEExplainabilityRequest) Prediction(org.kie.kogito.explainability.model.Prediction) SimplePrediction(org.kie.kogito.explainability.model.SimplePrediction) UnitValue(org.kie.kogito.tracing.typedvalue.UnitValue) Feature(org.kie.kogito.explainability.model.Feature) NamedTypedValue(org.kie.kogito.explainability.api.NamedTypedValue) IntNode(com.fasterxml.jackson.databind.node.IntNode) CollectionValue(org.kie.kogito.tracing.typedvalue.CollectionValue) Output(org.kie.kogito.explainability.model.Output) StructureValue(org.kie.kogito.tracing.typedvalue.StructureValue) List(java.util.List) Test(org.junit.jupiter.api.Test)

Aggregations

Test (org.junit.jupiter.api.Test)7 CollectionValue (org.kie.kogito.tracing.typedvalue.CollectionValue)7 UnitValue (org.kie.kogito.tracing.typedvalue.UnitValue)7 CounterfactualSearchDomainUnitValue (org.kie.kogito.explainability.api.CounterfactualSearchDomainUnitValue)6 NamedTypedValue (org.kie.kogito.explainability.api.NamedTypedValue)6 StructureValue (org.kie.kogito.tracing.typedvalue.StructureValue)5 List (java.util.List)4 IntNode (com.fasterxml.jackson.databind.node.IntNode)3 TextNode (com.fasterxml.jackson.databind.node.TextNode)3 ArrayList (java.util.ArrayList)3 Collections.emptyList (java.util.Collections.emptyList)3 Feature (org.kie.kogito.explainability.model.Feature)3 TypedValue (org.kie.kogito.tracing.typedvalue.TypedValue)3 DoubleNode (com.fasterxml.jackson.databind.node.DoubleNode)2 CounterfactualExplainabilityRequest (org.kie.kogito.explainability.api.CounterfactualExplainabilityRequest)2 CounterfactualSearchDomainCollectionValue (org.kie.kogito.explainability.api.CounterfactualSearchDomainCollectionValue)2 Output (org.kie.kogito.explainability.model.Output)2 Collections (java.util.Collections)1 Map (java.util.Map)1 Optional (java.util.Optional)1