Search in sources :

Example 6 with ResilientJobConfig

use of com.synopsys.integration.wait.ResilientJobConfig in project hub-alert by blackducksoftware.

the class ScalingPerformanceTest method createAndTestJobs.

private void createAndTestJobs(AlertRequestUtility alertRequestUtility, BlackDuckProviderService blackDuckProviderService, ConfigurationManager configurationManager, LocalDateTime startingTime, List<String> jobIds, int numberOfJobsToCreate, String blackDuckProviderID) throws Exception {
    // create slack jobs
    createSlackJobs(configurationManager, startingTime, jobIds, numberOfJobsToCreate, 10, blackDuckProviderID, blackDuckProviderService.getBlackDuckProviderKey(), blackDuckProviderService.getBlackDuckProjectName());
    LocalDateTime startingNotificationSearchDateTime = LocalDateTime.now();
    // trigger BD notification
    blackDuckProviderService.triggerBlackDuckNotification();
    logTimeElapsedWithMessage("Triggering the Black Duck notification took %s", startingNotificationSearchDateTime, LocalDateTime.now());
    LocalDateTime startingNotificationWaitForTenJobs = LocalDateTime.now();
    // check that all jobs have processed the notification successfully, log how long it took
    ResilientJobConfig resilientJobConfig = new ResilientJobConfig(intLogger, 900, startingNotificationSearchDateTime.atZone(ZoneId.systemDefault()).toInstant().toEpochMilli(), 30);
    NotificationWaitJobTask notificationWaitJobTask = new NotificationWaitJobTask(intLogger, dateTimeFormatter, gson, alertRequestUtility, startingNotificationSearchDateTime, jobIds);
    notificationWaitJobTask.setFailOnJobFailure(false);
    boolean isComplete = WaitJob.waitFor(resilientJobConfig, notificationWaitJobTask, "scaling notification wait");
    logTimeElapsedWithMessage("Waiting for " + numberOfJobsToCreate + " jobs to process the notification took %s", startingNotificationWaitForTenJobs, LocalDateTime.now());
    intLogger.info("Finished waiting for the notification to be processed: " + isComplete);
    assertTrue(isComplete);
}
Also used : LocalDateTime(java.time.LocalDateTime) NotificationWaitJobTask(com.synopsys.integration.alert.performance.utility.NotificationWaitJobTask) ResilientJobConfig(com.synopsys.integration.wait.ResilientJobConfig)

Example 7 with ResilientJobConfig

use of com.synopsys.integration.wait.ResilientJobConfig in project hub-alert by blackducksoftware.

the class ComponentUnknownVersionNotificationSerializationTest method testNotificationSerialization.

@Test
// performance test
@Ignore
@Disabled
void testNotificationSerialization() throws IntegrationException, InterruptedException {
    LocalDateTime searchStartTime = LocalDateTime.now().minusMinutes(1);
    AlertRequestUtility alertRequestUtility = IntegrationPerformanceTestRunner.createAlertRequestUtility(webApplicationContext);
    BlackDuckProviderService blackDuckProviderService = new BlackDuckProviderService(alertRequestUtility, gson);
    configureJob(alertRequestUtility, blackDuckProviderService);
    ExternalId externalId = new ExternalId(Forge.MAVEN);
    externalId.setGroup("commons-fileupload");
    externalId.setName("commons-fileupload");
    Predicate<ProjectVersionComponentVersionView> componentFilter = (component) -> component.getComponentName().equals("Apache Commons FileUpload");
    blackDuckProviderService.triggerBlackDuckNotification(() -> externalId, componentFilter);
    try {
        ResilientJobConfig resilientJobConfig = new ResilientJobConfig(intLogger, 300, searchStartTime.atZone(ZoneId.systemDefault()).toInstant().toEpochMilli(), 20);
        NotificationReceivedWaitJobTask notificationWaitJobTask = new NotificationReceivedWaitJobTask(notificationAccessor, searchStartTime, "Apache Commons FileUpload", null, NotificationType.COMPONENT_UNKNOWN_VERSION);
        boolean isComplete = WaitJob.waitFor(resilientJobConfig, notificationWaitJobTask, "notification serialization test notification wait");
        if (isComplete) {
            String notificationContent = notificationWaitJobTask.getNotificationContent().orElseThrow(() -> new IllegalStateException("Expected notification is missing."));
            BlackDuckResponseResolver resolver = blackDuckProviderService.getBlackDuckServicesFactory().getBlackDuckResponseResolver();
            ComponentUnknownVersionNotificationView notificationView = resolver.resolve(notificationContent, ComponentUnknownVersionNotificationView.class);
            assertNotNull(notificationView.getContent());
            assertTrue(StringUtils.isNotBlank(notificationView.getContent().getComponentName()));
            BlackDuckApiClient apiClient = blackDuckProviderService.getBlackDuckServicesFactory().getBlackDuckApiClient();
            Optional<HttpUrl> componentUrl = HttpUrl.createSafely(notificationView.getContent().getBomComponent());
            if (componentUrl.isPresent()) {
                apiClient.delete(componentUrl.get());
            }
        }
    } catch (InterruptedException ex) {
    // if a timeout happens that's ok we are trying to ensure deserialization is correct.
    }
}
Also used : LocalDateTime(java.time.LocalDateTime) IntegrationException(com.synopsys.integration.exception.IntegrationException) DescriptorMocker(com.synopsys.integration.alert.util.DescriptorMocker) LoggerFactory(org.slf4j.LoggerFactory) Autowired(org.springframework.beans.factory.annotation.Autowired) FrequencyType(com.synopsys.integration.alert.common.enumeration.FrequencyType) BlackDuckResponseResolver(com.synopsys.integration.blackduck.http.transform.subclass.BlackDuckResponseResolver) ResilientJobConfig(com.synopsys.integration.wait.ResilientJobConfig) Disabled(org.junit.jupiter.api.Disabled) StringUtils(org.apache.commons.lang3.StringUtils) TestTags(com.synopsys.integration.alert.test.common.TestTags) HttpUrl(com.synopsys.integration.rest.HttpUrl) BeforeAll(org.junit.jupiter.api.BeforeAll) Gson(com.google.gson.Gson) Map(java.util.Map) Tag(org.junit.jupiter.api.Tag) TestPropertyKey(com.synopsys.integration.alert.test.common.TestPropertyKey) ExternalId(com.synopsys.integration.bdio.model.externalid.ExternalId) WebAppConfiguration(org.springframework.test.context.web.WebAppConfiguration) ProjectVersionComponentVersionView(com.synopsys.integration.blackduck.api.generated.view.ProjectVersionComponentVersionView) Slf4jIntLogger(com.synopsys.integration.log.Slf4jIntLogger) Predicate(java.util.function.Predicate) WebApplicationContext(org.springframework.web.context.WebApplicationContext) TestPropertySource(org.springframework.test.context.TestPropertySource) IntegrationPerformanceTestRunner(com.synopsys.integration.alert.performance.utility.IntegrationPerformanceTestRunner) ApplicationConfiguration(com.synopsys.integration.alert.configuration.ApplicationConfiguration) NotificationType(com.synopsys.integration.blackduck.api.manual.enumeration.NotificationType) ZoneId(java.time.ZoneId) WaitJob(com.synopsys.integration.wait.WaitJob) Test(org.junit.jupiter.api.Test) SlackDescriptor(com.synopsys.integration.alert.channel.slack.descriptor.SlackDescriptor) List(java.util.List) SpringBootTest(org.springframework.boot.test.context.SpringBootTest) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) Optional(java.util.Optional) DatabaseDataSource(com.synopsys.integration.alert.database.DatabaseDataSource) ChannelKeys(com.synopsys.integration.alert.descriptor.api.model.ChannelKeys) TestProperties(com.synopsys.integration.alert.test.common.TestProperties) Assertions.assertNotNull(org.junit.jupiter.api.Assertions.assertNotNull) Forge(com.synopsys.integration.bdio.model.Forge) LocalDateTime(java.time.LocalDateTime) HashMap(java.util.HashMap) IntLogger(com.synopsys.integration.log.IntLogger) AlertRequestUtility(com.synopsys.integration.alert.performance.utility.AlertRequestUtility) BlackDuckProviderService(com.synopsys.integration.alert.performance.utility.BlackDuckProviderService) BlackDuckApiClient(com.synopsys.integration.blackduck.service.BlackDuckApiClient) ConfigurationManager(com.synopsys.integration.alert.performance.utility.ConfigurationManager) ChannelDescriptor(com.synopsys.integration.alert.common.descriptor.ChannelDescriptor) NotificationAccessor(com.synopsys.integration.alert.common.persistence.accessor.NotificationAccessor) Application(com.synopsys.integration.alert.Application) ComponentUnknownVersionNotificationView(com.synopsys.integration.blackduck.api.manual.view.ComponentUnknownVersionNotificationView) Ignore(org.junit.Ignore) ContextConfiguration(org.springframework.test.context.ContextConfiguration) FieldValueModel(com.synopsys.integration.alert.common.rest.model.FieldValueModel) BlackDuckResponseResolver(com.synopsys.integration.blackduck.http.transform.subclass.BlackDuckResponseResolver) AlertRequestUtility(com.synopsys.integration.alert.performance.utility.AlertRequestUtility) BlackDuckProviderService(com.synopsys.integration.alert.performance.utility.BlackDuckProviderService) ExternalId(com.synopsys.integration.bdio.model.externalid.ExternalId) BlackDuckApiClient(com.synopsys.integration.blackduck.service.BlackDuckApiClient) HttpUrl(com.synopsys.integration.rest.HttpUrl) ComponentUnknownVersionNotificationView(com.synopsys.integration.blackduck.api.manual.view.ComponentUnknownVersionNotificationView) ResilientJobConfig(com.synopsys.integration.wait.ResilientJobConfig) ProjectVersionComponentVersionView(com.synopsys.integration.blackduck.api.generated.view.ProjectVersionComponentVersionView) Ignore(org.junit.Ignore) Test(org.junit.jupiter.api.Test) SpringBootTest(org.springframework.boot.test.context.SpringBootTest) Disabled(org.junit.jupiter.api.Disabled)

Example 8 with ResilientJobConfig

use of com.synopsys.integration.wait.ResilientJobConfig in project blackduck-alert by blackducksoftware.

the class IntegrationPerformanceTestRunner method runTest.

public void runTest(Runnable createGlobalConfigFunction, Map<String, FieldValueModel> channelFields, String jobName) throws IntegrationException, InterruptedException {
    intLogger.info(String.format("Starting time %s", dateTimeFormatter.format(LocalDateTime.now())));
    String blackDuckProviderID = createBlackDuckConfiguration();
    createGlobalConfigFunction.run();
    LocalDateTime jobStartingTime = LocalDateTime.now();
    String jobId = configurationManager.createJob(channelFields, jobName, blackDuckProviderID, blackDuckProviderService.getBlackDuckProjectName());
    String jobMessage = String.format("Creating the Job %s jobs took", jobName);
    logTimeElapsedWithMessage(jobMessage + " %s", jobStartingTime, LocalDateTime.now());
    LocalDateTime startingSearchDateTime = LocalDateTime.now();
    // trigger BD notification
    blackDuckProviderService.triggerBlackDuckNotification();
    intLogger.info("Triggered the Black Duck notification.");
    ResilientJobConfig resilientJobConfig = new ResilientJobConfig(intLogger, 600, startingSearchDateTime.atZone(ZoneId.systemDefault()).toInstant().toEpochMilli(), 20);
    NotificationWaitJobTask notificationWaitJobTask = new NotificationWaitJobTask(intLogger, dateTimeFormatter, gson, alertRequestUtility, startingSearchDateTime, jobId);
    boolean isComplete = WaitJob.waitFor(resilientJobConfig, notificationWaitJobTask, "int performance test runner notification wait");
    intLogger.info("Finished waiting for the notification to be processed: " + isComplete);
    assertTrue(isComplete);
}
Also used : LocalDateTime(java.time.LocalDateTime) ResilientJobConfig(com.synopsys.integration.wait.ResilientJobConfig)

Example 9 with ResilientJobConfig

use of com.synopsys.integration.wait.ResilientJobConfig in project blackduck-alert by blackducksoftware.

the class IntegrationPerformanceTestRunnerV2 method waitForJobToFinish.

private void waitForJobToFinish(LocalDateTime startingNotificationTime, WaitJobCondition waitJobCondition) throws IntegrationException, InterruptedException {
    ResilientJobConfig resilientJobConfig = new ResilientJobConfig(intLogger, waitTimeoutInSeconds, startingNotificationTime.atZone(ZoneId.systemDefault()).toInstant().toEpochMilli(), 2);
    boolean isComplete = WaitJob.waitFor(resilientJobConfig, waitJobCondition, "int performance test runner notification wait");
    intLogger.info("Finished waiting for the notification to be processed: " + isComplete);
    assertTrue(isComplete);
}
Also used : ResilientJobConfig(com.synopsys.integration.wait.ResilientJobConfig)

Example 10 with ResilientJobConfig

use of com.synopsys.integration.wait.ResilientJobConfig in project blackduck-alert by blackducksoftware.

the class NotificationRemovalTest method testDeletion.

@Test
// performance test
@Ignore
@Disabled
void testDeletion() throws IntegrationException, InterruptedException {
    providerConfig = createBlackDuckConfiguration();
    OffsetDateTime testStartTime = OffsetDateTime.now();
    OffsetDateTime notificationCreatedAtTime = OffsetDateTime.now();
    // create 1000 processed notifications not for removal
    createABatchOfNotifications(providerConfig, testStartTime, true);
    // create 9000 for removal with varying dates and processed flags
    for (int index = 0; index < 9; index++) {
        boolean processed = index % 2 == 0 ? true : false;
        // update the createdAt time to be 1 month older
        notificationCreatedAtTime = notificationCreatedAtTime.minusMonths(1);
        createABatchOfNotifications(providerConfig, notificationCreatedAtTime, processed);
    }
    OffsetDateTime oldestNotificationCreationTime = notificationCreatedAtTime;
    purgeTask = new PurgeTask(schedulingDescriptorKey, taskScheduler, notificationAccessor, systemMessageAccessor, taskManager, configurationModelConfigurationAccessor);
    LocalDateTime startTime = LocalDateTime.now();
    purgeTask.runTask();
    WaitJobCondition waitJobCondition = () -> {
        List<AlertNotificationModel> notificationsInDatabase = getAllNotificationsInDatabase(oldestNotificationCreationTime, testStartTime);
        return notificationsInDatabase.size() == BATCH_SIZE && notificationsInDatabase.stream().allMatch(AlertNotificationModel::getProcessed);
    };
    ResilientJobConfig resilientJobConfig = new ResilientJobConfig(LOGGER, 600, startTime.atZone(ZoneId.systemDefault()).toInstant().toEpochMilli(), 1);
    boolean isComplete = WaitJob.waitFor(resilientJobConfig, waitJobCondition, "int performance test runner notification wait");
    logTimeElapsedWithMessage("Purge of notifications duration: %s", startTime, LocalDateTime.now());
    List<AlertNotificationModel> remainingNotifications = getAllNotificationsInDatabase(oldestNotificationCreationTime, testStartTime);
    assertTrue(isComplete);
    assertEquals(BATCH_SIZE, remainingNotifications.size());
}
Also used : LocalDateTime(java.time.LocalDateTime) AlertNotificationModel(com.synopsys.integration.alert.common.rest.model.AlertNotificationModel) OffsetDateTime(java.time.OffsetDateTime) List(java.util.List) ArrayList(java.util.ArrayList) LinkedList(java.util.LinkedList) WaitJobCondition(com.synopsys.integration.wait.WaitJobCondition) PurgeTask(com.synopsys.integration.alert.component.scheduling.workflow.PurgeTask) ResilientJobConfig(com.synopsys.integration.wait.ResilientJobConfig) Ignore(org.junit.Ignore) Test(org.junit.jupiter.api.Test) AlertIntegrationTest(com.synopsys.integration.alert.util.AlertIntegrationTest) Disabled(org.junit.jupiter.api.Disabled)

Aggregations

ResilientJobConfig (com.synopsys.integration.wait.ResilientJobConfig)18 LocalDateTime (java.time.LocalDateTime)10 List (java.util.List)5 BlackDuckApiClient (com.synopsys.integration.blackduck.service.BlackDuckApiClient)4 HttpUrl (com.synopsys.integration.rest.HttpUrl)4 Test (org.junit.jupiter.api.Test)4 Ignore (org.junit.Ignore)3 Disabled (org.junit.jupiter.api.Disabled)3 Gson (com.google.gson.Gson)2 Application (com.synopsys.integration.alert.Application)2 SlackDescriptor (com.synopsys.integration.alert.channel.slack.descriptor.SlackDescriptor)2 ChannelDescriptor (com.synopsys.integration.alert.common.descriptor.ChannelDescriptor)2 FrequencyType (com.synopsys.integration.alert.common.enumeration.FrequencyType)2 NotificationAccessor (com.synopsys.integration.alert.common.persistence.accessor.NotificationAccessor)2 AlertNotificationModel (com.synopsys.integration.alert.common.rest.model.AlertNotificationModel)2 FieldValueModel (com.synopsys.integration.alert.common.rest.model.FieldValueModel)2 PurgeTask (com.synopsys.integration.alert.component.scheduling.workflow.PurgeTask)2 ApplicationConfiguration (com.synopsys.integration.alert.configuration.ApplicationConfiguration)2 DatabaseDataSource (com.synopsys.integration.alert.database.DatabaseDataSource)2 ChannelKeys (com.synopsys.integration.alert.descriptor.api.model.ChannelKeys)2