Search in sources :

Example 6 with JobExecutionProgress

use of org.folio.rest.jaxrs.model.JobExecutionProgress in project mod-source-record-manager by folio-org.

the class JobExecutionProgressServiceImplTest method shouldReturnFailedFutureWhenJobExecutionDoesNotExist.

@Test
public void shouldReturnFailedFutureWhenJobExecutionDoesNotExist(TestContext context) {
    Async async = context.async();
    int totalRecords = 62;
    String jobExecId = UUID.randomUUID().toString();
    Future<JobExecutionProgress> future = jobExecutionProgressService.initializeJobExecutionProgress(jobExecId, totalRecords, TENANT_ID);
    future.onComplete(ar -> {
        context.assertTrue(ar.failed());
        async.complete();
    });
}
Also used : JobExecutionProgress(org.folio.rest.jaxrs.model.JobExecutionProgress) Async(io.vertx.ext.unit.Async) Test(org.junit.Test) AbstractRestTest(org.folio.rest.impl.AbstractRestTest)

Example 7 with JobExecutionProgress

use of org.folio.rest.jaxrs.model.JobExecutionProgress in project mod-source-record-manager by folio-org.

the class JobExecutionProgressDaoImpl method updateByJobExecutionId.

@Override
public Future<JobExecutionProgress> updateByJobExecutionId(String jobExecutionId, UnaryOperator<JobExecutionProgress> progressMutator, String tenantId) {
    String rollbackMessage = String.format(ROLLBACK_MESSAGE, jobExecutionId);
    Promise<JobExecutionProgress> promise = Promise.promise();
    Promise<SQLConnection> tx = Promise.promise();
    PostgresClient pgClient = pgClientFactory.createInstance(tenantId);
    getSelectResult(tx, pgClient, jobExecutionId, tenantId).map(progressResults -> {
        Optional<JobExecutionProgress> optionalJobExecutionProgress = mapResultSetToOptionalJobExecutionProgress(progressResults);
        if (optionalJobExecutionProgress.isEmpty()) {
            throw new NotFoundException(rollbackMessage);
        }
        return progressMutator.apply(optionalJobExecutionProgress.get());
    }).compose(mutatedProgress -> updateProgressByJobExecutionId(tx.future(), mutatedProgress, tenantId)).onComplete(updateAr -> {
        if (updateAr.succeeded()) {
            pgClient.endTx(tx.future(), endTx -> promise.complete(updateAr.result()));
        } else {
            LOGGER.error(rollbackMessage, updateAr.cause());
            pgClient.rollbackTx(tx.future(), r -> promise.fail(updateAr.cause()));
        }
    });
    return promise.future();
}
Also used : ValidationHelper(org.folio.rest.tools.utils.ValidationHelper) Promise(io.vertx.core.Promise) PostgresClient.convertToPsqlStandard(org.folio.rest.persist.PostgresClient.convertToPsqlStandard) SQLConnection(org.folio.rest.persist.SQLConnection) Autowired(org.springframework.beans.factory.annotation.Autowired) UnaryOperator(java.util.function.UnaryOperator) Future(io.vertx.core.Future) NotFoundException(javax.ws.rs.NotFoundException) PostgresClient(org.folio.rest.persist.PostgresClient) Tuple(io.vertx.sqlclient.Tuple) PostgresClientFactory(org.folio.dao.util.PostgresClientFactory) RowIterator(io.vertx.sqlclient.RowIterator) Logger(org.apache.logging.log4j.Logger) JobExecutionProgress(org.folio.rest.jaxrs.model.JobExecutionProgress) Row(io.vertx.sqlclient.Row) Optional(java.util.Optional) RowSet(io.vertx.sqlclient.RowSet) AsyncResult(io.vertx.core.AsyncResult) Repository(org.springframework.stereotype.Repository) LogManager(org.apache.logging.log4j.LogManager) JobExecutionProgress(org.folio.rest.jaxrs.model.JobExecutionProgress) SQLConnection(org.folio.rest.persist.SQLConnection) NotFoundException(javax.ws.rs.NotFoundException) PostgresClient(org.folio.rest.persist.PostgresClient)

Example 8 with JobExecutionProgress

use of org.folio.rest.jaxrs.model.JobExecutionProgress in project mod-source-record-manager by folio-org.

the class RecordProcessedEventHandlingServiceImplTest method shouldIncrementCurrentlyFailedAndUpdateProgressOnHandleEvent.

@Test
public void shouldIncrementCurrentlyFailedAndUpdateProgressOnHandleEvent(TestContext context) {
    // given
    Async async = context.async();
    HashMap<String, String> payloadContext = new HashMap<>();
    DataImportEventPayload dataImportEventPayload = new DataImportEventPayload().withEventType(DataImportEventTypes.DI_ERROR.value()).withContext(payloadContext);
    Future<Boolean> future = jobExecutionService.initializeJobExecutions(initJobExecutionsRqDto, params).compose(initJobExecutionsRsDto -> jobExecutionService.setJobProfileToJobExecution(initJobExecutionsRsDto.getParentJobExecutionId(), jobProfileInfo, params)).compose(jobExecution -> {
        dataImportEventPayload.setJobExecutionId(jobExecution.getId());
        return chunkProcessingService.processChunk(rawRecordsDto, jobExecution.getId(), params);
    });
    // when
    Future<JobExecutionProgress> jobFuture = future.compose(ar -> recordProcessedEventHandlingService.handle(Json.encode(dataImportEventPayload), params)).compose(ar -> jobExecutionProgressService.getByJobExecutionId(dataImportEventPayload.getJobExecutionId(), TENANT_ID));
    // then
    jobFuture.onComplete(ar -> {
        context.assertTrue(ar.succeeded());
        JobExecutionProgress updatedProgress = ar.result();
        context.assertEquals(1, updatedProgress.getCurrentlyFailed());
        context.assertEquals(0, updatedProgress.getCurrentlySucceeded());
        context.assertEquals(rawRecordsDto.getRecordsMetadata().getTotal(), updatedProgress.getTotal());
        Async async2 = context.async();
        jobFuture.compose(jobAr -> jobExecutionService.getJobExecutionById(dataImportEventPayload.getJobExecutionId(), TENANT_ID)).onComplete(jobAr -> {
            context.assertTrue(jobAr.succeeded());
            context.assertTrue(jobAr.result().isPresent());
            JobExecution jobExecution = jobAr.result().get();
            context.assertEquals(PARSING_IN_PROGRESS, jobExecution.getStatus());
            async2.complete();
        });
        async.complete();
    });
}
Also used : TestContext(io.vertx.ext.unit.TestContext) MappingParametersProvider(org.folio.services.mappers.processor.MappingParametersProvider) JobMonitoringDaoImpl(org.folio.dao.JobMonitoringDaoImpl) JournalServiceImpl(org.folio.services.journal.JournalServiceImpl) MockitoAnnotations(org.mockito.MockitoAnnotations) MarcRecordAnalyzer(org.folio.dataimport.util.marc.MarcRecordAnalyzer) JobProfileInfo(org.folio.rest.jaxrs.model.JobProfileInfo) JobExecution(org.folio.rest.jaxrs.model.JobExecution) Spy(org.mockito.Spy) JsonObject(io.vertx.core.json.JsonObject) MappingRuleDaoImpl(org.folio.dao.MappingRuleDaoImpl) WireMock.post(com.github.tomakehurst.wiremock.client.WireMock.post) ERROR(org.folio.rest.jaxrs.model.JobExecution.Status.ERROR) InitialRecord(org.folio.rest.jaxrs.model.InitialRecord) MappingRulesSnapshotDaoImpl(org.folio.dao.MappingRulesSnapshotDaoImpl) DataImportEventPayload(org.folio.DataImportEventPayload) UUID(java.util.UUID) Future(io.vertx.core.Future) OkapiConnectionParams(org.folio.dataimport.util.OkapiConnectionParams) MarcImportEventsHandler(org.folio.verticle.consumers.util.MarcImportEventsHandler) Optional(java.util.Optional) AbstractRestTest(org.folio.rest.impl.AbstractRestTest) JobExecutionSourceChunkDaoImpl(org.folio.dao.JobExecutionSourceChunkDaoImpl) RunTestOnContext(io.vertx.ext.unit.junit.RunTestOnContext) Mockito.mock(org.mockito.Mockito.mock) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) Async(io.vertx.ext.unit.Async) Json(io.vertx.core.json.Json) OKAPI_TENANT_HEADER(org.folio.rest.util.OkapiConnectionParams.OKAPI_TENANT_HEADER) RegexPattern(com.github.tomakehurst.wiremock.matching.RegexPattern) RunWith(org.junit.runner.RunWith) PARSING_IN_PROGRESS(org.folio.rest.jaxrs.model.JobExecution.Status.PARSING_IN_PROGRESS) HashMap(java.util.HashMap) RecordsMetadata(org.folio.rest.jaxrs.model.RecordsMetadata) OKAPI_URL_HEADER(org.folio.dataimport.util.RestUtil.OKAPI_URL_HEADER) HrIdFieldServiceImpl(org.folio.services.afterprocessing.HrIdFieldServiceImpl) JobExecutionProgressServiceImpl(org.folio.services.progress.JobExecutionProgressServiceImpl) WireMock.ok(com.github.tomakehurst.wiremock.client.WireMock.ok) WireMock(com.github.tomakehurst.wiremock.client.WireMock) PostgresClientFactory(org.folio.dao.util.PostgresClientFactory) DataType(org.folio.rest.jaxrs.model.JobProfileInfo.DataType) TestUtil(org.folio.TestUtil) InitJobExecutionsRqDto(org.folio.rest.jaxrs.model.InitJobExecutionsRqDto) MappingParameters(org.folio.processing.mapping.defaultmapper.processor.parameters.MappingParameters) Before(org.junit.Before) WireMock.get(com.github.tomakehurst.wiremock.client.WireMock.get) InjectMocks(org.mockito.InjectMocks) UrlPathPattern(com.github.tomakehurst.wiremock.matching.UrlPathPattern) JobExecutionProgressDaoImpl(org.folio.dao.JobExecutionProgressDaoImpl) RUNNING_COMPLETE(org.folio.rest.jaxrs.model.JobExecution.UiStatus.RUNNING_COMPLETE) Vertx(io.vertx.core.Vertx) ReflectionTestUtils(org.springframework.test.util.ReflectionTestUtils) IOException(java.io.IOException) Test(org.junit.Test) COMMITTED(org.folio.rest.jaxrs.model.JobExecution.Status.COMMITTED) Mockito.when(org.mockito.Mockito.when) VertxUnitRunner(io.vertx.ext.unit.junit.VertxUnitRunner) MappingParamsSnapshotDaoImpl(org.folio.dao.MappingParamsSnapshotDaoImpl) WireMock.verify(com.github.tomakehurst.wiremock.client.WireMock.verify) WireMock.putRequestedFor(com.github.tomakehurst.wiremock.client.WireMock.putRequestedFor) RawRecordsDto(org.folio.rest.jaxrs.model.RawRecordsDto) JsonArray(io.vertx.core.json.JsonArray) JobExecutionProgress(org.folio.rest.jaxrs.model.JobExecutionProgress) Rule(org.junit.Rule) WireMock.created(com.github.tomakehurst.wiremock.client.WireMock.created) File(org.folio.rest.jaxrs.model.File) JobExecutionDaoImpl(org.folio.dao.JobExecutionDaoImpl) DataImportEventTypes(org.folio.rest.jaxrs.model.DataImportEventTypes) OKAPI_TOKEN_HEADER(org.folio.rest.util.OkapiConnectionParams.OKAPI_TOKEN_HEADER) Collections(java.util.Collections) JournalRecordDaoImpl(org.folio.dao.JournalRecordDaoImpl) KafkaConfig(org.folio.kafka.KafkaConfig) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) JobExecution(org.folio.rest.jaxrs.model.JobExecution) JobExecutionProgress(org.folio.rest.jaxrs.model.JobExecutionProgress) HashMap(java.util.HashMap) Async(io.vertx.ext.unit.Async) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) DataImportEventPayload(org.folio.DataImportEventPayload) AbstractRestTest(org.folio.rest.impl.AbstractRestTest) Test(org.junit.Test)

Example 9 with JobExecutionProgress

use of org.folio.rest.jaxrs.model.JobExecutionProgress in project mod-source-record-manager by folio-org.

the class JobExecutionProgressServiceImplTest method shouldInitProgress.

@Test
public void shouldInitProgress(TestContext context) {
    Async async = context.async();
    int expectedTotalRecords = 62;
    Future<JobExecutionProgress> future = jobExecutionService.initializeJobExecutions(initJobExecutionsRqDto, params).compose(initJobExecutionsRsDto -> jobExecutionProgressService.initializeJobExecutionProgress(initJobExecutionsRsDto.getParentJobExecutionId(), expectedTotalRecords, TENANT_ID)).compose(progress -> jobExecutionProgressService.getByJobExecutionId(progress.getJobExecutionId(), TENANT_ID));
    future.onComplete(ar -> {
        context.assertTrue(ar.succeeded());
        JobExecutionProgress progress = ar.result();
        context.assertEquals(expectedTotalRecords, progress.getTotal());
        jobMonitoringService.getByJobExecutionId(progress.getJobExecutionId(), params.getTenantId()).onSuccess(optionalJobMonitoring -> {
            context.assertTrue(optionalJobMonitoring.isPresent());
            JobMonitoring jobMonitoring = optionalJobMonitoring.get();
            context.assertNotNull(jobMonitoring.getId());
            context.assertEquals(progress.getJobExecutionId(), jobMonitoring.getJobExecutionId());
            context.assertNotNull(jobMonitoring.getLastEventTimestamp());
            context.assertFalse(jobMonitoring.getNotificationSent());
            async.complete();
        });
    });
}
Also used : TestContext(io.vertx.ext.unit.TestContext) Async(io.vertx.ext.unit.Async) Arrays(java.util.Arrays) OKAPI_TENANT_HEADER(org.folio.rest.util.OkapiConnectionParams.OKAPI_TENANT_HEADER) JobMonitoringDaoImpl(org.folio.dao.JobMonitoringDaoImpl) RunWith(org.junit.runner.RunWith) HashMap(java.util.HashMap) JobExecutionProgressServiceImpl(org.folio.services.progress.JobExecutionProgressServiceImpl) OKAPI_URL_HEADER(org.folio.dataimport.util.RestUtil.OKAPI_URL_HEADER) PostgresClientFactory(org.folio.dao.util.PostgresClientFactory) CompositeFuture(io.vertx.core.CompositeFuture) MockitoAnnotations(org.mockito.MockitoAnnotations) JobExecutionProgressService(org.folio.services.progress.JobExecutionProgressService) Spy(org.mockito.Spy) InitJobExecutionsRqDto(org.folio.rest.jaxrs.model.InitJobExecutionsRqDto) Before(org.junit.Before) InjectMocks(org.mockito.InjectMocks) GenericCompositeFuture(org.folio.okapi.common.GenericCompositeFuture) JobExecutionProgressDaoImpl(org.folio.dao.JobExecutionProgressDaoImpl) JobMonitoring(org.folio.rest.jaxrs.model.JobMonitoring) Vertx(io.vertx.core.Vertx) Test(org.junit.Test) VertxUnitRunner(io.vertx.ext.unit.junit.VertxUnitRunner) UUID(java.util.UUID) Future(io.vertx.core.Future) OkapiConnectionParams(org.folio.dataimport.util.OkapiConnectionParams) JobExecutionProgress(org.folio.rest.jaxrs.model.JobExecutionProgress) Rule(org.junit.Rule) File(org.folio.rest.jaxrs.model.File) JobExecutionDaoImpl(org.folio.dao.JobExecutionDaoImpl) AbstractRestTest(org.folio.rest.impl.AbstractRestTest) OKAPI_TOKEN_HEADER(org.folio.rest.util.OkapiConnectionParams.OKAPI_TOKEN_HEADER) RunTestOnContext(io.vertx.ext.unit.junit.RunTestOnContext) Collections(java.util.Collections) JobExecutionProgress(org.folio.rest.jaxrs.model.JobExecutionProgress) Async(io.vertx.ext.unit.Async) JobMonitoring(org.folio.rest.jaxrs.model.JobMonitoring) Test(org.junit.Test) AbstractRestTest(org.folio.rest.impl.AbstractRestTest)

Example 10 with JobExecutionProgress

use of org.folio.rest.jaxrs.model.JobExecutionProgress in project mod-source-record-manager by folio-org.

the class JobExecutionProgressServiceImplTest method shouldReturnFailedFutureOnUpdateWhenProgressDoesNotExist.

@Test
public void shouldReturnFailedFutureOnUpdateWhenProgressDoesNotExist(TestContext context) {
    Async async = context.async();
    int succeededRecords = 7;
    String jobExecutionId = UUID.randomUUID().toString();
    Future<JobExecutionProgress> future = jobExecutionProgressService.updateJobExecutionProgress(jobExecutionId, progressToUpdate -> progressToUpdate.withCurrentlySucceeded(succeededRecords), TENANT_ID);
    future.onComplete(ar -> {
        context.assertTrue(ar.failed());
        async.complete();
    });
}
Also used : JobExecutionProgress(org.folio.rest.jaxrs.model.JobExecutionProgress) Async(io.vertx.ext.unit.Async) Test(org.junit.Test) AbstractRestTest(org.folio.rest.impl.AbstractRestTest)

Aggregations

JobExecutionProgress (org.folio.rest.jaxrs.model.JobExecutionProgress)12 Async (io.vertx.ext.unit.Async)9 AbstractRestTest (org.folio.rest.impl.AbstractRestTest)9 Test (org.junit.Test)9 Future (io.vertx.core.Future)8 PostgresClientFactory (org.folio.dao.util.PostgresClientFactory)8 Vertx (io.vertx.core.Vertx)6 TestContext (io.vertx.ext.unit.TestContext)6 RunTestOnContext (io.vertx.ext.unit.junit.RunTestOnContext)6 VertxUnitRunner (io.vertx.ext.unit.junit.VertxUnitRunner)6 HashMap (java.util.HashMap)6 UUID (java.util.UUID)6 OkapiConnectionParams (org.folio.dataimport.util.OkapiConnectionParams)6 OKAPI_URL_HEADER (org.folio.dataimport.util.RestUtil.OKAPI_URL_HEADER)6 File (org.folio.rest.jaxrs.model.File)6 InitJobExecutionsRqDto (org.folio.rest.jaxrs.model.InitJobExecutionsRqDto)6 OKAPI_TENANT_HEADER (org.folio.rest.util.OkapiConnectionParams.OKAPI_TENANT_HEADER)6 OKAPI_TOKEN_HEADER (org.folio.rest.util.OkapiConnectionParams.OKAPI_TOKEN_HEADER)6 Before (org.junit.Before)6 Rule (org.junit.Rule)6