Search in sources :

Example 1 with RuleExecutorException

use of org.ihtsdo.drools.exception.RuleExecutorException in project snomed-drools by IHTSDO.

the class RuleExecutorTest method testInitFailure.

@Test
public void testInitFailure() {
    final RuleExecutor ruleExecutor1 = new RuleExecutor("non-existant-directory");
    try {
        ruleExecutor1.execute(RULE_SET_NAMES, Collections.singleton(new ConceptImpl("1")), conceptService, descriptionService, relationshipService, true, false);
        Assert.fail("Should have thrown exception.");
    } catch (RuleExecutorException e) {
    // Pass
    }
}
Also used : ConceptImpl(org.ihtsdo.drools.unittest.domain.ConceptImpl) RuleExecutorException(org.ihtsdo.drools.exception.RuleExecutorException) BadRequestRuleExecutorException(org.ihtsdo.drools.exception.BadRequestRuleExecutorException) RuleExecutor(org.ihtsdo.drools.RuleExecutor) Test(org.junit.Test)

Example 2 with RuleExecutorException

use of org.ihtsdo.drools.exception.RuleExecutorException in project snomed-drools by IHTSDO.

the class RuleExecutor method addAssertionGroup.

public void addAssertionGroup(String assertionGroupName, File ruleSetDirectory) throws RuleExecutorException {
    // Create the in-memory File System and add the resources files  to it
    KieFileSystem kieFileSystem = kieServices.newKieFileSystem();
    try {
        final RuleLoader ruleLoader = new RuleLoader(kieFileSystem);
        Files.walkFileTree(ruleSetDirectory.toPath(), ruleLoader);
        int rulesLoaded = ruleLoader.getRulesLoaded();
        if (rulesLoaded == 0) {
            logger.warn("No rules loaded. Rules directory: {}", ruleSetDirectory.getAbsolutePath());
        } else {
            logger.info("{} rules loaded.", ruleLoader.getRulesLoaded());
            totalRulesLoaded += rulesLoaded;
        }
        assertionGroupRuleCounts.put(assertionGroupName, totalRulesLoaded);
    } catch (IOException e) {
        throw new RuleExecutorException("Failed to load rule set " + assertionGroupName, e);
    }
    // Create the builder for the resources of the File System
    KieBuilder kieBuilder = kieServices.newKieBuilder(kieFileSystem);
    // Build the KieBases
    kieBuilder.buildAll();
    // Check for errors
    if (kieBuilder.getResults().hasMessages(Message.Level.ERROR)) {
        throw new RuleExecutorException(kieBuilder.getResults().toString());
    }
    // Get the Release ID (mvn style: groupId, artifactId,version)
    ReleaseId relId = kieBuilder.getKieModule().getReleaseId();
    // Create the Container, wrapping the KieModule with the given ReleaseId
    assertionGroupContainers.put(assertionGroupName, kieServices.newKieContainer(relId));
}
Also used : KieFileSystem(org.kie.api.builder.KieFileSystem) RuleExecutorException(org.ihtsdo.drools.exception.RuleExecutorException) BadRequestRuleExecutorException(org.ihtsdo.drools.exception.BadRequestRuleExecutorException) IOException(java.io.IOException) ReleaseId(org.kie.api.builder.ReleaseId) KieBuilder(org.kie.api.builder.KieBuilder)

Example 3 with RuleExecutorException

use of org.ihtsdo.drools.exception.RuleExecutorException in project snomed-drools by IHTSDO.

the class RuleExecutor method newTestResourceProvider.

/**
 * TestResourceProvider should be created once and used by other services to load test resources such as the case significant words list.
 * Calling this method again will load the resources again so can be useful if the resources change.
 * @param resourceManager The resource manager to use to load the test resource files.
 * @return A TestResourceProvider which uses the resourceManager.
 * @throws RuleExecutorException if there is a problem loading the test resources.
 */
public TestResourceProvider newTestResourceProvider(ResourceManager resourceManager) throws RuleExecutorException {
    try {
        TestResourceProvider testResourceProvider = new TestResourceProvider(resourceManager);
        testResourcesEmpty = !testResourceProvider.isAnyResourcesLoaded();
        return testResourceProvider;
    } catch (IOException e) {
        testResourcesEmpty = true;
        throw new RuleExecutorException("Failed to load test resources.", e);
    }
}
Also used : RuleExecutorException(org.ihtsdo.drools.exception.RuleExecutorException) BadRequestRuleExecutorException(org.ihtsdo.drools.exception.BadRequestRuleExecutorException) IOException(java.io.IOException) TestResourceProvider(org.ihtsdo.drools.service.TestResourceProvider)

Example 4 with RuleExecutorException

use of org.ihtsdo.drools.exception.RuleExecutorException in project snomed-drools by IHTSDO.

the class RuleExecutor method newTestResourceProvider.

public TestResourceProvider newTestResourceProvider(String awsKey, String awsSecretKey, String bucket, String path) throws RuleExecutorException {
    try {
        AmazonS3 amazonS3 = AmazonS3ClientBuilder.standard().withRegion("us-east-1").withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(awsKey, awsSecretKey))).build();
        ManualResourceConfiguration resourceConfiguration = new ManualResourceConfiguration(true, true, null, new ResourceConfiguration.Cloud(bucket, path));
        ResourceManager resourceManager = new ResourceManager(resourceConfiguration, new SimpleStorageResourceLoader(amazonS3));
        TestResourceProvider testResourceProvider = new TestResourceProvider(resourceManager);
        testResourcesEmpty = !testResourceProvider.isAnyResourcesLoaded();
        return testResourceProvider;
    } catch (IOException e) {
        testResourcesEmpty = true;
        throw new RuleExecutorException("Failed to load test resources.", e);
    }
}
Also used : AmazonS3(com.amazonaws.services.s3.AmazonS3) AWSStaticCredentialsProvider(com.amazonaws.auth.AWSStaticCredentialsProvider) RuleExecutorException(org.ihtsdo.drools.exception.RuleExecutorException) BadRequestRuleExecutorException(org.ihtsdo.drools.exception.BadRequestRuleExecutorException) SimpleStorageResourceLoader(org.springframework.cloud.aws.core.io.s3.SimpleStorageResourceLoader) ResourceManager(org.ihtsdo.otf.resourcemanager.ResourceManager) IOException(java.io.IOException) ManualResourceConfiguration(org.ihtsdo.otf.resourcemanager.ManualResourceConfiguration) TestResourceProvider(org.ihtsdo.drools.service.TestResourceProvider) BasicAWSCredentials(com.amazonaws.auth.BasicAWSCredentials) ResourceConfiguration(org.ihtsdo.otf.resourcemanager.ResourceConfiguration) ManualResourceConfiguration(org.ihtsdo.otf.resourcemanager.ManualResourceConfiguration)

Example 5 with RuleExecutorException

use of org.ihtsdo.drools.exception.RuleExecutorException in project snomed-drools by IHTSDO.

the class RuleExecutor method execute.

/**
 * Validate a concept using drools rules available to this executor.
 * A temporary identifier should be assigned to the concepts or any of it's descriptions and relationships
 * if the component is new and does not yet have an SCTID. The identifier of the component is used to identify invalid content.
 *
 * Passing services in with every invocation of this method allows the implementation to capture content context. For example
 * services relevant to the content branch being worked on.
 * @param ruleSetNames The rule sets to use during validation.
 * @param concepts The concepts to be validated.
 * @param conceptService An implementation of the ConceptService class for use in validation rules.
 * @param descriptionService An implementation of the DescriptionService class for use in validation rules.
 * @param relationshipService An implementation of the RelationshipService class for use in validation rules.
 * @param includePublishedComponents Include the published components of the given concept in results if found to be invalid.
 *                                   Published content will be used during validation regardless just not returned.
 * @param includeInferredRelationships Include the inferred relationships of the given concept during validation and
 *                                     in results if found to be invalid.
 * @return A list of content found to be invalid is returned.
 */
public List<InvalidContent> execute(Set<String> ruleSetNames, Collection<? extends Concept> concepts, ConceptService conceptService, DescriptionService descriptionService, RelationshipService relationshipService, boolean includePublishedComponents, boolean includeInferredRelationships) throws RuleExecutorException {
    for (Concept concept : concepts) {
        assertComponentIdsPresent(concept);
    }
    Date start = new Date();
    final List<List<InvalidContent>> sessionInvalidContent = new ArrayList<>();
    final List<InvalidContent> exceptionContents = new ArrayList<>();
    for (String ruleSetName : ruleSetNames) {
        final KieContainer kieContainer = assertionGroupContainers.get(ruleSetName);
        if (kieContainer == null) {
            throw new RuleExecutorException("Rule set not found for name '" + ruleSetName + "'");
        }
        int threads = concepts.size() == 1 ? 1 : 10;
        ExecutorService executorService = Executors.newFixedThreadPool(threads);
        List<StatelessKieSession> sessions = new ArrayList<>();
        for (int s = 0; s < threads; s++) {
            // List per thread to avoid concurrency issues.
            ArrayList<InvalidContent> invalidContent = new ArrayList<>();
            sessionInvalidContent.add(invalidContent);
            sessions.add(newStatelessKieSession(kieContainer, conceptService, descriptionService, relationshipService, invalidContent));
        }
        List<Concept> conceptList = new ArrayList<>(concepts);
        List<Callable<String>> tasks = new ArrayList<>();
        String total = String.format("%,d", concepts.size());
        int i = 0;
        while (i < concepts.size()) {
            Set<Component> components = new HashSet<>();
            Concept concept = conceptList.get(i++);
            addConcept(components, concept, includeInferredRelationships);
            int sessionIndex = tasks.size();
            tasks.add(() -> {
                try {
                    StatelessKieSession statelessKieSession = sessions.get(sessionIndex);
                    statelessKieSession.execute(components);
                    components.clear();
                    ((StatelessKnowledgeSessionImpl) statelessKieSession).newWorkingMemory();
                } catch (Exception e) {
                    exceptionContents.add(new InvalidContent(concept.getId(), concept, "An error occurred while running concept validation. Technical detail: " + e.getMessage(), Severity.ERROR));
                }
                return null;
            });
            if (tasks.size() == threads) {
                runTasks(executorService, tasks);
                tasks.clear();
            }
            if (i % 10_000 == 0) {
                logger.info("Validated {} of {}", String.format("%,d", i), total);
            }
        }
        if (!tasks.isEmpty()) {
            runTasks(executorService, tasks);
        }
        executorService.shutdown();
        logger.info("Validated {} of {}", String.format("%,d", i), total);
        logger.info("Rule execution took {} seconds", (new Date().getTime() - start.getTime()) / 1000);
    }
    List<InvalidContent> invalidContent = sessionInvalidContent.stream().flatMap(Collection::stream).filter(Objects::nonNull).collect(Collectors.toList());
    invalidContent.addAll(exceptionContents);
    invalidContent = removeDuplicates(invalidContent);
    if (!includePublishedComponents) {
        Set<InvalidContent> publishedInvalidContent = new HashSet<>();
        for (InvalidContent invalidContentItem : invalidContent) {
            logger.info("invalidContentItem : {}, {}, {}, {}, {}", invalidContentItem.getConceptId(), invalidContentItem.isIgnorePublishedCheck(), invalidContentItem.isPublished(), invalidContentItem.getSeverity(), invalidContentItem.getMessage());
            if (!invalidContentItem.isIgnorePublishedCheck() && invalidContentItem.isPublished()) {
                publishedInvalidContent.add(invalidContentItem);
            }
        }
        invalidContent.removeAll(publishedInvalidContent);
    }
    if (testResourcesEmpty) {
        invalidContent.add(0, InvalidContent.getGeneralWarning("Test resources were not available so assertions like case significance and US specific terms " + "checks will not have run."));
    }
    return invalidContent;
}
Also used : Callable(java.util.concurrent.Callable) StatelessKieSession(org.kie.api.runtime.StatelessKieSession) KieContainer(org.kie.api.runtime.KieContainer) RuleExecutorException(org.ihtsdo.drools.exception.RuleExecutorException) BadRequestRuleExecutorException(org.ihtsdo.drools.exception.BadRequestRuleExecutorException) InvalidContent(org.ihtsdo.drools.response.InvalidContent) RuleExecutorException(org.ihtsdo.drools.exception.RuleExecutorException) IOException(java.io.IOException) BadRequestRuleExecutorException(org.ihtsdo.drools.exception.BadRequestRuleExecutorException) StatelessKnowledgeSessionImpl(org.drools.core.impl.StatelessKnowledgeSessionImpl) ExecutorService(java.util.concurrent.ExecutorService)

Aggregations

RuleExecutorException (org.ihtsdo.drools.exception.RuleExecutorException)7 IOException (java.io.IOException)5 BadRequestRuleExecutorException (org.ihtsdo.drools.exception.BadRequestRuleExecutorException)5 TestResourceProvider (org.ihtsdo.drools.service.TestResourceProvider)2 KieBuilder (org.kie.api.builder.KieBuilder)2 KieFileSystem (org.kie.api.builder.KieFileSystem)2 ReleaseId (org.kie.api.builder.ReleaseId)2 AWSStaticCredentialsProvider (com.amazonaws.auth.AWSStaticCredentialsProvider)1 BasicAWSCredentials (com.amazonaws.auth.BasicAWSCredentials)1 AmazonS3 (com.amazonaws.services.s3.AmazonS3)1 File (java.io.File)1 Callable (java.util.concurrent.Callable)1 ExecutorService (java.util.concurrent.ExecutorService)1 StatelessKnowledgeSessionImpl (org.drools.core.impl.StatelessKnowledgeSessionImpl)1 RuleExecutor (org.ihtsdo.drools.RuleExecutor)1 InvalidContent (org.ihtsdo.drools.response.InvalidContent)1 ConceptImpl (org.ihtsdo.drools.unittest.domain.ConceptImpl)1 ManualResourceConfiguration (org.ihtsdo.otf.resourcemanager.ManualResourceConfiguration)1 ResourceConfiguration (org.ihtsdo.otf.resourcemanager.ResourceConfiguration)1 ResourceManager (org.ihtsdo.otf.resourcemanager.ResourceManager)1