Search in sources :

Example 36 with Participation

use of de.tum.in.www1.artemis.domain.participation.Participation in project ArTEMiS by ls1intum.

the class RepositoryService method getFilesWithInformationAboutChange.

/**
 * Gets the files of the repository and checks whether they were changed during a student participation.
 * Compares the files from the students' repository with the files of the template repository.
 *
 * @param repository the students' repository with possibly new files and changed files
 * @param templateRepository the template repository with default files on which the student started working on
 * @return a map of files with the information if they were changed/are new.
 */
public Map<String, Boolean> getFilesWithInformationAboutChange(Repository repository, Repository templateRepository) {
    Map<String, Boolean> filesWithInformationAboutChange = new HashMap<>();
    var repoFiles = gitService.listFilesAndFolders(repository).entrySet().stream().filter(entry -> entry.getValue() == FileType.FILE).map(Map.Entry::getKey).toList();
    Map<String, File> templateRepoFiles = gitService.listFilesAndFolders(templateRepository).entrySet().stream().filter(entry -> entry.getValue() == FileType.FILE).collect(Collectors.toMap(entry -> entry.getKey().toString(), Map.Entry::getKey));
    repoFiles.forEach(file -> {
        String fileName = file.toString();
        if (templateRepoFiles.get(fileName) == null) {
            filesWithInformationAboutChange.put(fileName, true);
        } else {
            File templateFile = templateRepoFiles.get(fileName);
            try {
                if (FileUtils.contentEquals(file, templateFile)) {
                    filesWithInformationAboutChange.put(fileName, false);
                } else {
                    filesWithInformationAboutChange.put(fileName, true);
                }
            } catch (IOException e) {
                log.error("Comparing files '{}' with '{}' failed: {}", fileName, templateFile, e.getMessage());
            }
        }
    });
    return filesWithInformationAboutChange;
}
Also used : Logger(org.slf4j.Logger) GitAPIException(org.eclipse.jgit.api.errors.GitAPIException) LoggerFactory(org.slf4j.LoggerFactory) GitService(de.tum.in.www1.artemis.service.connectors.GitService) HashMap(java.util.HashMap) FileUtils(org.apache.commons.io.FileUtils) Collectors(java.util.stream.Collectors) StandardCharsets(java.nio.charset.StandardCharsets) java.nio.file(java.nio.file) Principal(java.security.Principal) de.tum.in.www1.artemis.domain(de.tum.in.www1.artemis.domain) java.io(java.io) Service(org.springframework.stereotype.Service) UserRepository(de.tum.in.www1.artemis.repository.UserRepository) Map(java.util.Map) FileMove(de.tum.in.www1.artemis.web.rest.dto.FileMove) Optional(java.util.Optional) File(de.tum.in.www1.artemis.domain.File) HashMap(java.util.HashMap) HashMap(java.util.HashMap) Map(java.util.Map) File(de.tum.in.www1.artemis.domain.File)

Example 37 with Participation

use of de.tum.in.www1.artemis.domain.participation.Participation in project ArTEMiS by ls1intum.

the class ScoreService method updateOrCreateParticipantScore.

/**
 * Either updates an existing participant score or creates a new participant score if a new result comes in
 * The annotation "@Transactional" is ok because it means that this method does not support run in an outer transactional context, instead the outer transaction is paused
 *
 * @param createdOrUpdatedResult newly created or updated result
 */
// ok (see JavaDoc)
@Transactional(propagation = Propagation.NOT_SUPPORTED)
public void updateOrCreateParticipantScore(Result createdOrUpdatedResult) {
    if (createdOrUpdatedResult.getScore() == null || createdOrUpdatedResult.getCompletionDate() == null) {
        return;
    }
    // There is a deadlock problem with programming exercises here if we use the participation from the result (reason unknown at the moment)
    // therefore we get the participation from the database
    Optional<StudentParticipation> studentParticipationOptional = getStudentParticipationForResult(createdOrUpdatedResult);
    if (studentParticipationOptional.isEmpty()) {
        return;
    }
    StudentParticipation studentParticipation = studentParticipationOptional.get();
    // we ignore test runs of exams
    if (studentParticipation.isTestRun()) {
        return;
    }
    Exercise exercise = studentParticipation.getExercise();
    ParticipantScore existingParticipationScoreForExerciseAndParticipant = getExistingParticipationScore(studentParticipation, exercise);
    // there already exists a participant score -> we need to update it
    if (existingParticipationScoreForExerciseAndParticipant != null) {
        updateExistingParticipantScore(existingParticipationScoreForExerciseAndParticipant, createdOrUpdatedResult, exercise);
    } else {
        // there does not already exist a participant score -> we need to create it
        createNewParticipantScore(createdOrUpdatedResult, studentParticipation, exercise);
    }
}
Also used : ParticipantScore(de.tum.in.www1.artemis.domain.scores.ParticipantScore) StudentParticipation(de.tum.in.www1.artemis.domain.participation.StudentParticipation) Transactional(org.springframework.transaction.annotation.Transactional)

Example 38 with Participation

use of de.tum.in.www1.artemis.domain.participation.Participation in project ArTEMiS by ls1intum.

the class SubmissionExportService method createZipFileFromParticipations.

/**
 * Creates a zip file from a list of participations for an exercise.
 *
 * The outputDir is used to store the zip file and temporary files used for zipping so make
 * sure to delete it if it's no longer used.
 *
 * @param exercise the exercise in question
 * @param participations a list of participations to include
 * @param enableFilterAfterDueDate true, if all submissions that have been submitted after the due date should not be included in the file
 * @param lateSubmissionFilter an optional date filter for submissions
 * @param outputDir directory to store the temporary files in
 * @param exportErrors a list of errors for submissions that couldn't be exported and are not included in the file
 * @param reportData   a list of all exercises and their statistics
 * @return the zipped file
 * @throws IOException if an error occurred while zipping
 */
private Optional<File> createZipFileFromParticipations(Exercise exercise, List<StudentParticipation> participations, boolean enableFilterAfterDueDate, @Nullable ZonedDateTime lateSubmissionFilter, Path outputDir, List<String> exportErrors, List<ArchivalReportEntry> reportData) throws IOException {
    Course course = exercise.getCourseViaExerciseGroupOrCourseMember();
    // Create unique name for directory
    String zipGroupName = course.getShortName() + "-" + exercise.getTitle() + "-" + exercise.getId();
    String cleanZipGroupName = fileService.removeIllegalCharacters(zipGroupName);
    String zipFileName = cleanZipGroupName + "-" + ZonedDateTime.now().format(DateTimeFormatter.ofPattern("yyyyMMdd-Hmss")) + ".zip";
    // Create directory
    Path submissionsFolderPath = Path.of(outputDir.toString(), "zippedSubmissions", zipGroupName);
    Path zipFilePath = Path.of(outputDir.toString(), "zippedSubmissions", zipFileName);
    File submissionFolder = submissionsFolderPath.toFile();
    if (!submissionFolder.exists() && !submissionFolder.mkdirs()) {
        log.error("Couldn't create dir: {}", submissionFolder);
        exportErrors.add("Cannot create directory: " + submissionFolder.toPath());
        return Optional.empty();
    }
    // Create counter for log entry
    MutableInt skippedEntries = new MutableInt();
    // Save all Submissions
    List<Path> submissionFilePaths = participations.stream().map(participation -> {
        Submission latestSubmission = latestSubmission(participation, enableFilterAfterDueDate, lateSubmissionFilter);
        if (latestSubmission == null) {
            skippedEntries.increment();
            return Optional.<Path>empty();
        }
        // create file path
        String submissionFileName = exercise.getTitle() + "-" + participation.getParticipantIdentifier() + "-" + latestSubmission.getId() + this.getFileEndingForSubmission(latestSubmission);
        Path submissionFilePath = Path.of(submissionsFolderPath.toString(), submissionFileName);
        // store file
        try {
            this.saveSubmissionToFile(exercise, latestSubmission, submissionFilePath.toFile());
            return Optional.of(submissionFilePath);
        } catch (Exception ex) {
            String message = "Could not create file " + submissionFilePath + "  for exporting: " + ex.getMessage();
            log.error(message, ex);
            exportErrors.add(message);
            return Optional.<Path>empty();
        }
    }).flatMap(Optional::stream).collect(Collectors.toList());
    // Add report entry
    reportData.add(new ArchivalReportEntry(exercise, fileService.removeIllegalCharacters(exercise.getTitle()), participations.size(), submissionFilePaths.size(), skippedEntries.intValue()));
    if (submissionFilePaths.isEmpty()) {
        return Optional.empty();
    }
    // zip stores submissions
    try {
        zipFileService.createZipFile(zipFilePath, submissionFilePaths, submissionsFolderPath);
    } finally {
        log.debug("Delete all temporary files");
        fileService.deleteFiles(submissionFilePaths);
    }
    return Optional.of(zipFilePath.toFile());
}
Also used : Path(java.nio.file.Path) ArchivalReportEntry(de.tum.in.www1.artemis.service.archival.ArchivalReportEntry) Submission(de.tum.in.www1.artemis.domain.Submission) MutableInt(org.apache.commons.lang.mutable.MutableInt) Course(de.tum.in.www1.artemis.domain.Course) File(java.io.File) BadRequestAlertException(de.tum.in.www1.artemis.web.rest.errors.BadRequestAlertException) IOException(java.io.IOException)

Example 39 with Participation

use of de.tum.in.www1.artemis.domain.participation.Participation in project ArTEMiS by ls1intum.

the class SubmissionExportService method exportStudentSubmissions.

/**
 * Exports student submissions to a zip file for an exercise.
 *
 * The outputDir is used to store the zip file and temporary files used for zipping so make
 * sure to delete it if it's no longer used.
 *
 * @param exerciseId the id of the exercise to be exported
 * @param submissionExportOptions the options for the export
 * @param outputDir directory to store the temporary files in
 * @param exportErrors a list of errors for submissions that couldn't be exported and are not included in the file
 * @param reportData   a list of all exercises and their statistics
 * @return a reference to the zipped file
 * @throws IOException if an error occurred while zipping
 */
public Optional<File> exportStudentSubmissions(Long exerciseId, SubmissionExportOptionsDTO submissionExportOptions, Path outputDir, List<String> exportErrors, List<ArchivalReportEntry> reportData) throws IOException {
    Optional<Exercise> exerciseOpt = exerciseRepository.findWithEagerStudentParticipationsStudentAndSubmissionsById(exerciseId);
    if (exerciseOpt.isEmpty()) {
        return Optional.empty();
    }
    Exercise exercise = exerciseOpt.get();
    // Select the participations that should be exported
    List<StudentParticipation> exportedStudentParticipations;
    if (submissionExportOptions.isExportAllParticipants()) {
        exportedStudentParticipations = new ArrayList<>(exercise.getStudentParticipations());
    } else {
        List<String> participantIds = Arrays.stream(submissionExportOptions.getParticipantIdentifierList().split(",")).map(String::trim).toList();
        exportedStudentParticipations = exercise.getStudentParticipations().stream().filter(participation -> participantIds.contains(participation.getParticipantIdentifier())).collect(Collectors.toList());
    }
    boolean enableFilterAfterDueDate = false;
    ZonedDateTime filterLateSubmissionsDate = null;
    if (submissionExportOptions.isFilterLateSubmissions()) {
        if (submissionExportOptions.getFilterLateSubmissionsDate() == null) {
            enableFilterAfterDueDate = true;
        } else {
            filterLateSubmissionsDate = submissionExportOptions.getFilterLateSubmissionsDate();
        }
    }
    // Sort the student participations by id
    exportedStudentParticipations.sort(Comparator.comparing(DomainObject::getId));
    return this.createZipFileFromParticipations(exercise, exportedStudentParticipations, enableFilterAfterDueDate, filterLateSubmissionsDate, outputDir, exportErrors, reportData);
}
Also used : Exercise(de.tum.in.www1.artemis.domain.Exercise) ZonedDateTime(java.time.ZonedDateTime) StudentParticipation(de.tum.in.www1.artemis.domain.participation.StudentParticipation)

Example 40 with Participation

use of de.tum.in.www1.artemis.domain.participation.Participation in project ArTEMiS by ls1intum.

the class FileUploadSubmissionService method save.

/**
 * Saves the given submission. Is used for creating and updating file upload submissions. Rolls back if inserting fails - occurs for concurrent createFileUploadSubmission() calls.
 *
 * @param fileUploadSubmission the submission that should be saved
 * @param file                 the file that will be saved on the server
 * @param participation        the participation the submission belongs to
 * @param exercise             the exercise the submission belongs to
 * @return the fileUploadSubmission entity that was saved to the database
 * @throws IOException if file can't be saved
 * @throws EmptyFileException if file is empty
 */
public FileUploadSubmission save(FileUploadSubmission fileUploadSubmission, MultipartFile file, StudentParticipation participation, FileUploadExercise exercise) throws IOException, EmptyFileException {
    final Optional<ZonedDateTime> dueDate = exerciseDateService.getDueDate(participation);
    if (dueDate.isPresent() && exerciseDateService.isAfterDueDate(participation) && participation.getInitializationDate().isBefore(dueDate.get())) {
        throw new ResponseStatusException(HttpStatus.FORBIDDEN);
    }
    if (file.isEmpty()) {
        throw new EmptyFileException(file.getOriginalFilename());
    }
    final var oldFilePath = fileUploadSubmission.getFilePath();
    final var multipartFileHash = DigestUtils.md5Hex(file.getInputStream());
    // We need to set id for newly created submissions
    if (fileUploadSubmission.getId() == null) {
        fileUploadSubmission = fileUploadSubmissionRepository.save(fileUploadSubmission);
    }
    final var newLocalFilePath = saveFileForSubmission(file, fileUploadSubmission, exercise);
    final var newFilePath = fileService.publicPathForActualPath(newLocalFilePath, fileUploadSubmission.getId());
    // We need to ensure that we can access the store file and the stored file is the same as was passed to us in the request
    final var storedFileHash = DigestUtils.md5Hex(Files.newInputStream(Path.of(newLocalFilePath)));
    if (!multipartFileHash.equals(storedFileHash)) {
        throw new IOException("The file " + file.getName() + "could not be stored");
    }
    // Note: we can only delete the file, if the file name was changed (i.e. the new file name is different), otherwise this will cause issues
    if (oldFilePath != null) {
        // check if we already had a file associated with this submission
        if (!oldFilePath.equals(newFilePath)) {
            // different name
            // IMPORTANT: only delete the file when it has changed the name
            fileUploadSubmission.onDelete();
        } else {
            // same name
            // IMPORTANT: invalidate the cache so that the new file with the same name will be downloaded (and not a potentially cached one)
            fileService.resetOnPath(newLocalFilePath);
        }
    }
    // NOTE: from now on we always set submitted to true to prevent problems here! Except for late submissions of course exercises to prevent issues in auto-save
    if (exercise.isExamExercise() || exerciseDateService.isBeforeDueDate(participation)) {
        fileUploadSubmission.setSubmitted(true);
    }
    fileUploadSubmission.setSubmissionDate(ZonedDateTime.now());
    fileUploadSubmission.setType(SubmissionType.MANUAL);
    fileUploadSubmission.setParticipation(participation);
    // remove result from submission (in the unlikely case it is passed here), so that students cannot inject a result
    fileUploadSubmission.setResults(new ArrayList<>());
    // Note: we save before the new file path is set to potentially remove the old file on the file system
    fileUploadSubmission = fileUploadSubmissionRepository.save(fileUploadSubmission);
    fileUploadSubmission.setFilePath(newFilePath);
    // Note: we save again so that the new file is stored on the file system
    fileUploadSubmission = fileUploadSubmissionRepository.save(fileUploadSubmission);
    participation.addSubmission(fileUploadSubmission);
    participation.setInitializationState(InitializationState.FINISHED);
    StudentParticipation savedParticipation = studentParticipationRepository.save(participation);
    if (fileUploadSubmission.getId() == null) {
        Optional<Submission> optionalSubmission = savedParticipation.findLatestSubmission();
        if (optionalSubmission.isPresent()) {
            fileUploadSubmission = (FileUploadSubmission) optionalSubmission.get();
        }
    }
    return fileUploadSubmission;
}
Also used : Submission(de.tum.in.www1.artemis.domain.Submission) FileUploadSubmission(de.tum.in.www1.artemis.domain.FileUploadSubmission) ZonedDateTime(java.time.ZonedDateTime) EmptyFileException(de.tum.in.www1.artemis.exception.EmptyFileException) IOException(java.io.IOException) ResponseStatusException(org.springframework.web.server.ResponseStatusException) StudentParticipation(de.tum.in.www1.artemis.domain.participation.StudentParticipation)

Aggregations

StudentParticipation (de.tum.in.www1.artemis.domain.participation.StudentParticipation)181 WithMockUser (org.springframework.security.test.context.support.WithMockUser)138 Test (org.junit.jupiter.api.Test)124 ProgrammingExerciseStudentParticipation (de.tum.in.www1.artemis.domain.participation.ProgrammingExerciseStudentParticipation)70 Participation (de.tum.in.www1.artemis.domain.participation.Participation)54 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)50 ZonedDateTime (java.time.ZonedDateTime)47 ModelingSubmission (de.tum.in.www1.artemis.domain.modeling.ModelingSubmission)44 PreAuthorize (org.springframework.security.access.prepost.PreAuthorize)44 ModelingExercise (de.tum.in.www1.artemis.domain.modeling.ModelingExercise)42 EntityNotFoundException (de.tum.in.www1.artemis.web.rest.errors.EntityNotFoundException)42 AbstractSpringIntegrationBambooBitbucketJiraTest (de.tum.in.www1.artemis.AbstractSpringIntegrationBambooBitbucketJiraTest)38 QuizExercise (de.tum.in.www1.artemis.domain.quiz.QuizExercise)38 TextPlagiarismResult (de.tum.in.www1.artemis.domain.plagiarism.text.TextPlagiarismResult)34 Collectors (java.util.stream.Collectors)34 HttpHeaders (org.springframework.http.HttpHeaders)34 de.tum.in.www1.artemis.domain (de.tum.in.www1.artemis.domain)33 Result (de.tum.in.www1.artemis.domain.Result)33 Exam (de.tum.in.www1.artemis.domain.exam.Exam)32 ModelingPlagiarismResult (de.tum.in.www1.artemis.domain.plagiarism.modeling.ModelingPlagiarismResult)32