Search in sources :

Example 1 with ModelElement

use of de.tum.in.www1.artemis.domain.modeling.ModelElement in project ArTEMiS by ls1intum.

the class AssessmentKnowledgeIntegrationTest method testSetModelAssessmentKnowledgeToModelElements.

/**
 * Tests that a ModelAssessmentKnowledge is correctly set to model elements
 * based on the ModelAssessmentKnowledge of the respective exercise
 *
 * @throws Exception might be thrown from Network Call to Artemis API
 */
@Test
@WithMockUser(username = "instructor1", roles = "INSTRUCTOR")
public void testSetModelAssessmentKnowledgeToModelElements() throws Exception {
    ModelingSubmission submission1 = ModelFactory.generateModelingSubmission(FileUtils.loadFileFromResources("test-data/model-submission/model.54727.json"), true);
    submission1.setId(1L);
    ModelingSubmission submission2 = ModelFactory.generateModelingSubmission(FileUtils.loadFileFromResources("test-data/model-submission/model.54727.cpy.json"), true);
    submission2.setId(2L);
    ModelingSubmission submission3 = ModelFactory.generateModelingSubmission(FileUtils.loadFileFromResources("test-data/model-submission/model.54727.cpy.json"), true);
    submission3.setId(3L);
    ModelingSubmission submission4 = ModelFactory.generateModelingSubmission(FileUtils.loadFileFromResources("test-data/model-submission/model.54727.cpy.json"), true);
    submission4.setId(4L);
    Course course = database.addEmptyCourse();
    ModelingExercise exercise1 = modelingExerciseUtilService.createModelingExercise(course.getId());
    ModelingExercise exercise2 = modelingExerciseUtilService.createModelingExercise(course.getId());
    modelAssessmentKnowledgeService = new ModelAssessmentKnowledgeService(modelAssessmentKnowledgeRepository, modelingExerciseRepository);
    exercise1.setKnowledge(modelAssessmentKnowledgeService.createNewKnowledge());
    exercise2.setKnowledge(modelAssessmentKnowledgeService.createNewKnowledge());
    ModelClusterFactory modelClusterFactory = new ModelClusterFactory();
    List<ModelCluster> modelClusters1 = modelClusterFactory.buildClusters(List.of(submission1, submission2), exercise1);
    List<ModelCluster> modelClusters2 = modelClusterFactory.buildClusters(List.of(submission3, submission4), exercise2);
    ModelCluster modelCluster = modelClusters1.get(0);
    for (ModelElement element : modelCluster.getModelElements()) {
        assertThat(element.getKnowledge().getId()).isEqualTo(exercise1.getKnowledge().getId());
    }
    modelCluster = modelClusters2.get(0);
    for (ModelElement element : modelCluster.getModelElements()) {
        assertThat(element.getKnowledge().getId()).isEqualTo(exercise2.getKnowledge().getId());
    }
    assertThat(exercise1.getKnowledge().getId()).isNotEqualTo(exercise2.getKnowledge().getId());
}
Also used : ModelClusterFactory(de.tum.in.www1.artemis.service.compass.controller.ModelClusterFactory) ModelElement(de.tum.in.www1.artemis.domain.modeling.ModelElement) ModelingSubmission(de.tum.in.www1.artemis.domain.modeling.ModelingSubmission) ModelingExercise(de.tum.in.www1.artemis.domain.modeling.ModelingExercise) ModelAssessmentKnowledgeService(de.tum.in.www1.artemis.service.ModelAssessmentKnowledgeService) ModelCluster(de.tum.in.www1.artemis.domain.modeling.ModelCluster) Course(de.tum.in.www1.artemis.domain.Course) WithMockUser(org.springframework.security.test.context.support.WithMockUser) Test(org.junit.jupiter.api.Test)

Example 2 with ModelElement

use of de.tum.in.www1.artemis.domain.modeling.ModelElement in project ArTEMiS by ls1intum.

the class CompassService method getSuggestionResult.

/**
 * Selects the feedback suggestion for each element in submission and creates a result from them
 * Returns null if no feedback can be selected or the submission has already a manual feedback
 *
 * @param modelingSubmission the submission to select feedbacks for
 * @param modelingExercise the modeling exercise to which the submission belongs
 * @return the semi-automatic result that has the feedbacks inside
 */
public Result getSuggestionResult(ModelingSubmission modelingSubmission, ModelingExercise modelingExercise) {
    Result result = getAutomaticResultForSubmission(modelingSubmission);
    if (result != null) {
        List<Feedback> feedbacksForSuggestion = new ArrayList<>();
        ModelClusterFactory clusterBuilder = new ModelClusterFactory();
        List<UMLElement> elements = clusterBuilder.getModelElements(modelingSubmission);
        List<ModelElement> modelElements = modelElementRepository.findByModelElementIdIn(elements.stream().map(UMLElement::getJSONElementID).collect(Collectors.toList()));
        List<Long> clusterIds = modelElements.stream().map(ModelElement::getCluster).map(ModelCluster::getId).collect(Collectors.toList());
        List<ModelCluster> modelClusters = modelClusterRepository.findAllByIdInWithEagerElements(clusterIds);
        List<String> references = modelClusters.stream().flatMap(modelCluster -> modelCluster.getModelElements().stream()).map(modelElement1 -> modelElement1.getModelElementType() + ":" + modelElement1.getModelElementId()).collect(Collectors.toList());
        List<Feedback> feedbacks = feedbackRepository.findByReferenceInAndResult_Submission_Participation_Exercise(references, modelingExercise);
        for (ModelElement modelElement : modelElements) {
            if (modelElement != null) {
                ModelCluster cluster = modelClusters.get(modelClusters.indexOf(modelElement.getCluster()));
                Set<ModelElement> similarElements = cluster.getModelElements();
                List<String> similarReferences = similarElements.stream().map(element -> element.getModelElementType() + ":" + element.getModelElementId()).toList();
                List<Feedback> similarFeedbacks = feedbacks.stream().filter(feedback -> similarReferences.contains(feedback.getReference())).collect(Collectors.toList());
                Feedback suggestedFeedback = FeedbackSelector.selectFeedback(modelElement, similarFeedbacks, result);
                if (suggestedFeedback != null) {
                    feedbacksForSuggestion.add(suggestedFeedback);
                }
            }
        }
        if (feedbacksForSuggestion.isEmpty()) {
            return null;
        }
        // Note, that a result is always initialized with an empty list -> no NPE here
        result.getFeedbacks().clear();
        result.getFeedbacks().addAll(feedbacksForSuggestion);
        result.setHasFeedback(false);
        result.setAssessmentType(AssessmentType.SEMI_AUTOMATIC);
    }
    return result;
}
Also used : java.util(java.util) Feedback(de.tum.in.www1.artemis.domain.Feedback) Logger(org.slf4j.Logger) FeedbackSelector(de.tum.in.www1.artemis.service.compass.controller.FeedbackSelector) de.tum.in.www1.artemis.repository(de.tum.in.www1.artemis.repository) TimeLogUtil(de.tum.in.www1.artemis.service.util.TimeLogUtil) LoggerFactory(org.slf4j.LoggerFactory) AssessmentType(de.tum.in.www1.artemis.domain.enumeration.AssessmentType) Collectors(java.util.stream.Collectors) UMLElement(de.tum.in.www1.artemis.service.compass.umlmodel.UMLElement) Result(de.tum.in.www1.artemis.domain.Result) ModelElement(de.tum.in.www1.artemis.domain.modeling.ModelElement) ModelClusterFactory(de.tum.in.www1.artemis.service.compass.controller.ModelClusterFactory) Service(org.springframework.stereotype.Service) ModelCluster(de.tum.in.www1.artemis.domain.modeling.ModelCluster) ModelingSubmission(de.tum.in.www1.artemis.domain.modeling.ModelingSubmission) StudentParticipation(de.tum.in.www1.artemis.domain.participation.StudentParticipation) ModelingExercise(de.tum.in.www1.artemis.domain.modeling.ModelingExercise) UMLElement(de.tum.in.www1.artemis.service.compass.umlmodel.UMLElement) ModelCluster(de.tum.in.www1.artemis.domain.modeling.ModelCluster) Result(de.tum.in.www1.artemis.domain.Result) ModelClusterFactory(de.tum.in.www1.artemis.service.compass.controller.ModelClusterFactory) ModelElement(de.tum.in.www1.artemis.domain.modeling.ModelElement) Feedback(de.tum.in.www1.artemis.domain.Feedback)

Example 3 with ModelElement

use of de.tum.in.www1.artemis.domain.modeling.ModelElement in project ArTEMiS by ls1intum.

the class FeedbackSelector method selectFeedback.

/**
 * Selects the feedback to suggest from the list of feedback then sets up the selected feedback for the given element and result
 *
 * @param modelElement the UML model element the Feedback will be suggested for
 * @param feedbackList the list of feedback to choose from
 * @param result the result the selected feedback will belong to
 * @return the feedback that is selected for suggestion
 */
public static Feedback selectFeedback(ModelElement modelElement, List<Feedback> feedbackList, Result result) {
    if (feedbackList == null || feedbackList.isEmpty()) {
        return null;
    }
    // counts the amount of feedback elements that have the same credits assigned, i.e. maps "credits -> amount" for every unique credit number
    Map<Double, Integer> creditCount = new HashMap<>();
    // collects the feedback texts of the feedback elements that have the same credits assigned, i.e. maps "credits -> set of feedback text" for every unique credit number
    Map<Double, Set<String>> creditFeedbackText = new HashMap<>();
    // collects associated grading instruction of the feedback that have the same credits assigned, i.e. maps "credits -> GradingInstruction" for every unique credit number
    Map<Double, GradingInstruction> creditGradingInstruction = new HashMap<>();
    for (Feedback existingFeedback : feedbackList) {
        double credits = existingFeedback.getCredits();
        creditCount.put(credits, creditCount.getOrDefault(credits, 0) + 1);
        if (existingFeedback.getText() != null) {
            Set<String> feedbackTextForCredits = creditFeedbackText.getOrDefault(credits, new HashSet<>());
            feedbackTextForCredits.add(existingFeedback.getText());
            creditFeedbackText.put(credits, feedbackTextForCredits);
        }
        if (existingFeedback.getGradingInstruction() != null) {
            creditGradingInstruction.put(credits, existingFeedback.getGradingInstruction());
        }
    }
    double maxCount = creditCount.values().stream().mapToInt(i -> i).max().orElse(0);
    double confidence = maxCount / feedbackList.size();
    double maxCountCredits = creditCount.entrySet().stream().filter(entry -> entry.getValue() == maxCount).map(Map.Entry::getKey).findFirst().orElse(0.0);
    Set<String> feedbackTextForMaxCountCredits = creditFeedbackText.getOrDefault(maxCountCredits, new HashSet<>());
    String text = feedbackTextForMaxCountCredits.stream().filter(Objects::nonNull).max(Comparator.comparingInt(String::length)).orElse("");
    GradingInstruction gradingInstruction = creditGradingInstruction.getOrDefault(maxCountCredits, new GradingInstruction());
    if (confidence < CompassConfiguration.ELEMENT_CONFIDENCE_THRESHOLD) {
        return null;
    }
    Feedback feedback = new Feedback();
    feedback.setCredits(roundCredits(maxCountCredits));
    feedback.setPositive(feedback.getCredits() >= 0);
    feedback.setReference(buildReferenceString(modelElement));
    feedback.setType(FeedbackType.AUTOMATIC);
    feedback.setResult(result);
    if (gradingInstruction.getId() != null) {
        feedback.setGradingInstruction(gradingInstruction);
    } else {
        feedback.setText(text);
    }
    return feedback;
}
Also used : BigDecimal(java.math.BigDecimal) Result(de.tum.in.www1.artemis.domain.Result) ModelElement(de.tum.in.www1.artemis.domain.modeling.ModelElement) java.util(java.util) Feedback(de.tum.in.www1.artemis.domain.Feedback) GradingInstruction(de.tum.in.www1.artemis.domain.GradingInstruction) FeedbackType(de.tum.in.www1.artemis.domain.enumeration.FeedbackType) RoundingMode(java.math.RoundingMode) CompassConfiguration(de.tum.in.www1.artemis.service.compass.utils.CompassConfiguration) GradingInstruction(de.tum.in.www1.artemis.domain.GradingInstruction) Feedback(de.tum.in.www1.artemis.domain.Feedback)

Example 4 with ModelElement

use of de.tum.in.www1.artemis.domain.modeling.ModelElement in project ArTEMiS by ls1intum.

the class ModelClusterFactoryTest method testBuildingClustersWithSimilarElements.

@Test
public void testBuildingClustersWithSimilarElements() throws Exception {
    ModelingSubmission submission1 = ModelFactory.generateModelingSubmission(FileUtils.loadFileFromResources("test-data/model-submission/model.54727.json"), true);
    submission1.setId(1L);
    ModelingSubmission submission2 = ModelFactory.generateModelingSubmission(FileUtils.loadFileFromResources("test-data/model-submission/model.54727.cpy.json"), true);
    submission2.setId(2L);
    ModelingSubmission submission3 = ModelFactory.generateModelingSubmission(FileUtils.loadFileFromResources("test-data/model-submission/model.54727.cpy.json"), true);
    submission3.setId(3L);
    ModelingSubmission submission4 = ModelFactory.generateModelingSubmission(FileUtils.loadFileFromResources("test-data/model-submission/model.54727.cpy.json"), true);
    submission4.setId(4L);
    ModelingSubmission submission5 = ModelFactory.generateModelingSubmission(FileUtils.loadFileFromResources("test-data/model-submission/model.one-element.json"), true);
    submission5.setId(5L);
    ModelingExercise exercise = new ModelingExercise();
    List<ModelCluster> modelClusters = modelClusterFactory.buildClusters(List.of(submission1, submission2, submission3, submission4, submission5), exercise);
    assertThat(modelClusters).as("model clusters created").hasSize(10);
    ModelCluster modelCluster = modelClusters.get(0);
    assertThat(modelCluster.getModelElements()).as("all elements are created").hasSize(4);
    for (ModelElement element : modelCluster.getModelElements()) {
        assertThat(element.getCluster()).as("created elements keeps the cluster").isEqualTo(modelCluster);
    }
}
Also used : ModelElement(de.tum.in.www1.artemis.domain.modeling.ModelElement) ModelingSubmission(de.tum.in.www1.artemis.domain.modeling.ModelingSubmission) ModelingExercise(de.tum.in.www1.artemis.domain.modeling.ModelingExercise) ModelCluster(de.tum.in.www1.artemis.domain.modeling.ModelCluster) Test(org.junit.jupiter.api.Test)

Example 5 with ModelElement

use of de.tum.in.www1.artemis.domain.modeling.ModelElement in project Artemis by ls1intum.

the class CompassService method getSuggestionResult.

/**
 * Selects the feedback suggestion for each element in submission and creates a result from them
 * Returns null if no feedback can be selected or the submission has already a manual feedback
 *
 * @param modelingSubmission the submission to select feedbacks for
 * @param modelingExercise the modeling exercise to which the submission belongs
 * @return the semi-automatic result that has the feedbacks inside
 */
public Result getSuggestionResult(ModelingSubmission modelingSubmission, ModelingExercise modelingExercise) {
    Result result = getAutomaticResultForSubmission(modelingSubmission);
    if (result != null) {
        List<Feedback> feedbacksForSuggestion = new ArrayList<>();
        ModelClusterFactory clusterBuilder = new ModelClusterFactory();
        List<UMLElement> elements = clusterBuilder.getModelElements(modelingSubmission);
        List<ModelElement> modelElements = modelElementRepository.findByModelElementIdIn(elements.stream().map(UMLElement::getJSONElementID).collect(Collectors.toList()));
        List<Long> clusterIds = modelElements.stream().map(ModelElement::getCluster).map(ModelCluster::getId).collect(Collectors.toList());
        List<ModelCluster> modelClusters = modelClusterRepository.findAllByIdInWithEagerElements(clusterIds);
        List<String> references = modelClusters.stream().flatMap(modelCluster -> modelCluster.getModelElements().stream()).map(modelElement1 -> modelElement1.getModelElementType() + ":" + modelElement1.getModelElementId()).collect(Collectors.toList());
        List<Feedback> feedbacks = feedbackRepository.findByReferenceInAndResult_Submission_Participation_Exercise(references, modelingExercise);
        for (ModelElement modelElement : modelElements) {
            if (modelElement != null) {
                ModelCluster cluster = modelClusters.get(modelClusters.indexOf(modelElement.getCluster()));
                Set<ModelElement> similarElements = cluster.getModelElements();
                List<String> similarReferences = similarElements.stream().map(element -> element.getModelElementType() + ":" + element.getModelElementId()).toList();
                List<Feedback> similarFeedbacks = feedbacks.stream().filter(feedback -> similarReferences.contains(feedback.getReference())).collect(Collectors.toList());
                Feedback suggestedFeedback = FeedbackSelector.selectFeedback(modelElement, similarFeedbacks, result);
                if (suggestedFeedback != null) {
                    feedbacksForSuggestion.add(suggestedFeedback);
                }
            }
        }
        if (feedbacksForSuggestion.isEmpty()) {
            return null;
        }
        // Note, that a result is always initialized with an empty list -> no NPE here
        result.getFeedbacks().clear();
        result.getFeedbacks().addAll(feedbacksForSuggestion);
        result.setHasFeedback(false);
        result.setAssessmentType(AssessmentType.SEMI_AUTOMATIC);
    }
    return result;
}
Also used : java.util(java.util) Feedback(de.tum.in.www1.artemis.domain.Feedback) Logger(org.slf4j.Logger) FeedbackSelector(de.tum.in.www1.artemis.service.compass.controller.FeedbackSelector) de.tum.in.www1.artemis.repository(de.tum.in.www1.artemis.repository) TimeLogUtil(de.tum.in.www1.artemis.service.util.TimeLogUtil) LoggerFactory(org.slf4j.LoggerFactory) AssessmentType(de.tum.in.www1.artemis.domain.enumeration.AssessmentType) Collectors(java.util.stream.Collectors) UMLElement(de.tum.in.www1.artemis.service.compass.umlmodel.UMLElement) Result(de.tum.in.www1.artemis.domain.Result) ModelElement(de.tum.in.www1.artemis.domain.modeling.ModelElement) ModelClusterFactory(de.tum.in.www1.artemis.service.compass.controller.ModelClusterFactory) Service(org.springframework.stereotype.Service) ModelCluster(de.tum.in.www1.artemis.domain.modeling.ModelCluster) ModelingSubmission(de.tum.in.www1.artemis.domain.modeling.ModelingSubmission) StudentParticipation(de.tum.in.www1.artemis.domain.participation.StudentParticipation) ModelingExercise(de.tum.in.www1.artemis.domain.modeling.ModelingExercise) UMLElement(de.tum.in.www1.artemis.service.compass.umlmodel.UMLElement) ModelCluster(de.tum.in.www1.artemis.domain.modeling.ModelCluster) Result(de.tum.in.www1.artemis.domain.Result) ModelClusterFactory(de.tum.in.www1.artemis.service.compass.controller.ModelClusterFactory) ModelElement(de.tum.in.www1.artemis.domain.modeling.ModelElement) Feedback(de.tum.in.www1.artemis.domain.Feedback)

Aggregations

ModelElement (de.tum.in.www1.artemis.domain.modeling.ModelElement)12 Feedback (de.tum.in.www1.artemis.domain.Feedback)8 Result (de.tum.in.www1.artemis.domain.Result)8 Test (org.junit.jupiter.api.Test)8 ModelCluster (de.tum.in.www1.artemis.domain.modeling.ModelCluster)6 ModelingExercise (de.tum.in.www1.artemis.domain.modeling.ModelingExercise)6 ModelingSubmission (de.tum.in.www1.artemis.domain.modeling.ModelingSubmission)6 ModelClusterFactory (de.tum.in.www1.artemis.service.compass.controller.ModelClusterFactory)4 java.util (java.util)4 Course (de.tum.in.www1.artemis.domain.Course)2 GradingInstruction (de.tum.in.www1.artemis.domain.GradingInstruction)2 AssessmentType (de.tum.in.www1.artemis.domain.enumeration.AssessmentType)2 FeedbackType (de.tum.in.www1.artemis.domain.enumeration.FeedbackType)2 SimilarElementCount (de.tum.in.www1.artemis.domain.modeling.SimilarElementCount)2 StudentParticipation (de.tum.in.www1.artemis.domain.participation.StudentParticipation)2 de.tum.in.www1.artemis.repository (de.tum.in.www1.artemis.repository)2 ModelAssessmentKnowledgeService (de.tum.in.www1.artemis.service.ModelAssessmentKnowledgeService)2 FeedbackSelector (de.tum.in.www1.artemis.service.compass.controller.FeedbackSelector)2 UMLElement (de.tum.in.www1.artemis.service.compass.umlmodel.UMLElement)2 CompassConfiguration (de.tum.in.www1.artemis.service.compass.utils.CompassConfiguration)2