Search in sources :

Example 11 with AutoScalingPolicy

use of com.netflix.titus.api.appscale.model.AutoScalingPolicy in project titus-control-plane by Netflix.

the class DefaultAppScaleManagerTest method submitTwoJobs.

private List<String> submitTwoJobs(DefaultAppScaleManager appScaleManager, String jobIdOne, String jobIdTwo, InMemoryPolicyStore policyStore) throws Exception {
    // call - createAutoScalingPolicy
    AutoScalingPolicy autoScalingPolicyOne = AutoScalingPolicyTests.buildStepScalingPolicy(jobIdOne);
    appScaleManager.createAutoScalingPolicy(autoScalingPolicyOne).toBlocking().single();
    // call - createAutoScalingPolicy
    AutoScalingPolicy autoScalingPolicyTwo = AutoScalingPolicyTests.buildStepScalingPolicy(jobIdTwo);
    appScaleManager.createAutoScalingPolicy(autoScalingPolicyTwo).toBlocking().single();
    // call - processPendingPolicies
    List<AutoScalingPolicy> savedPolicies = policyStore.retrievePolicies(false).toList().toBlocking().first();
    AutoScalingPolicyTests.waitForCondition(() -> savedPolicies.size() == 2);
    Assertions.assertThat(savedPolicies.size()).isEqualTo(2);
    return savedPolicies.stream().map(policy -> policy.getRefId()).collect(Collectors.toList());
}
Also used : AutoScalePolicyException(com.netflix.titus.api.appscale.service.AutoScalePolicyException) Completable(rx.Completable) JobModel(com.netflix.titus.api.jobmanager.model.job.JobModel) LoggerFactory(org.slf4j.LoggerFactory) AutoScalableTarget(com.netflix.titus.api.appscale.model.AutoScalableTarget) ArrayList(java.util.ArrayList) Observable(rx.Observable) Answer(org.mockito.stubbing.Answer) InvocationOnMock(org.mockito.invocation.InvocationOnMock) JobGroupInfo(com.netflix.titus.api.jobmanager.model.job.JobGroupInfo) AutoScalingPolicy(com.netflix.titus.api.appscale.model.AutoScalingPolicy) Arrays.asList(java.util.Arrays.asList) Map(java.util.Map) JobState(com.netflix.titus.api.jobmanager.model.job.JobState) Schedulers(rx.schedulers.Schedulers) Assertions(org.assertj.core.api.Assertions) CallMetadata(com.netflix.titus.api.model.callmetadata.CallMetadata) JobDescriptor(com.netflix.titus.api.jobmanager.model.job.JobDescriptor) Job(com.netflix.titus.api.jobmanager.model.job.Job) Logger(org.slf4j.Logger) JobUpdateEvent(com.netflix.titus.api.jobmanager.model.job.event.JobUpdateEvent) ServiceJobExt(com.netflix.titus.api.jobmanager.model.job.ext.ServiceJobExt) Awaitility.await(com.jayway.awaitility.Awaitility.await) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Test(org.junit.Test) UUID(java.util.UUID) Mockito.when(org.mockito.Mockito.when) JobManagerEvent(com.netflix.titus.api.jobmanager.model.job.event.JobManagerEvent) Collectors(java.util.stream.Collectors) PolicyType(com.netflix.titus.api.appscale.model.PolicyType) InMemoryPolicyStore(com.netflix.titus.runtime.store.v3.memory.InMemoryPolicyStore) Capacity(com.netflix.titus.api.jobmanager.model.job.Capacity) TimeUnit(java.util.concurrent.TimeUnit) CountDownLatch(java.util.concurrent.CountDownLatch) List(java.util.List) V3JobOperations(com.netflix.titus.api.jobmanager.service.V3JobOperations) DefaultRegistry(com.netflix.spectator.api.DefaultRegistry) Optional(java.util.Optional) TitusRuntime(com.netflix.titus.common.runtime.TitusRuntime) Mockito.mock(org.mockito.Mockito.mock) AutoScalingPolicy(com.netflix.titus.api.appscale.model.AutoScalingPolicy)

Example 12 with AutoScalingPolicy

use of com.netflix.titus.api.appscale.model.AutoScalingPolicy in project titus-control-plane by Netflix.

the class DefaultAppScaleManagerTest method checkBulkPolicyCreations.

@Test
public void checkBulkPolicyCreations() {
    AutoScalingPolicyTests.MockAlarmClient mockAlarmClient = new AutoScalingPolicyTests.MockAlarmClient();
    AutoScalingPolicyTests.MockAppAutoScalingClient mockAppAutoScalingClient = new AutoScalingPolicyTests.MockAppAutoScalingClient();
    InMemoryPolicyStore policyStore = new InMemoryPolicyStore();
    final int totalJobs = 300;
    List<String> jobIds = new ArrayList<>(totalJobs);
    for (int i = 0; i < totalJobs; i++) {
        String jobId = UUID.randomUUID().toString();
        jobIds.add(jobId);
    }
    V3JobOperations v3JobOperations = mockV3OperationsForJobs(jobIds);
    DefaultAppScaleManager appScaleManager = new DefaultAppScaleManager(policyStore, mockAlarmClient, mockAppAutoScalingClient, v3JobOperations, new DefaultRegistry(), AutoScalingPolicyTests.mockAppScaleManagerConfiguration(), Schedulers.computation(), mock(TitusRuntime.class));
    final CountDownLatch latch = new CountDownLatch(totalJobs);
    for (int i = 0; i < totalJobs; i++) {
        final AutoScalingPolicy autoScalingPolicy = AutoScalingPolicyTests.buildStepScalingPolicy(jobIds.get(i));
        appScaleManager.createAutoScalingPolicy(autoScalingPolicy).subscribe(pid -> {
            log.info("Policy created {}", pid);
            latch.countDown();
        });
    }
    await().timeout(30, TimeUnit.SECONDS).until(() -> mockAppAutoScalingClient.getNumPolicies() == totalJobs);
}
Also used : V3JobOperations(com.netflix.titus.api.jobmanager.service.V3JobOperations) ArrayList(java.util.ArrayList) InMemoryPolicyStore(com.netflix.titus.runtime.store.v3.memory.InMemoryPolicyStore) TitusRuntime(com.netflix.titus.common.runtime.TitusRuntime) CountDownLatch(java.util.concurrent.CountDownLatch) AutoScalingPolicy(com.netflix.titus.api.appscale.model.AutoScalingPolicy) DefaultRegistry(com.netflix.spectator.api.DefaultRegistry) Test(org.junit.Test)

Example 13 with AutoScalingPolicy

use of com.netflix.titus.api.appscale.model.AutoScalingPolicy in project titus-control-plane by Netflix.

the class DefaultAppScaleManagerTest method checkCreatePolicyFlow.

private void checkCreatePolicyFlow(PolicyType policyType) throws Exception {
    // create instance of DefaultAppScaleManager
    AutoScalingPolicyTests.MockAlarmClient mockAlarmClient = new AutoScalingPolicyTests.MockAlarmClient();
    AutoScalingPolicyTests.MockAppAutoScalingClient mockAppAutoScalingClient = new AutoScalingPolicyTests.MockAppAutoScalingClient();
    InMemoryPolicyStore policyStore = new InMemoryPolicyStore();
    String jobIdOne = UUID.randomUUID().toString();
    String jobIdTwo = UUID.randomUUID().toString();
    V3JobOperations v3JobOperations = mockV3Operations(jobIdOne, jobIdTwo);
    DefaultAppScaleManager appScaleManager = new DefaultAppScaleManager(policyStore, mockAlarmClient, mockAppAutoScalingClient, v3JobOperations, new DefaultRegistry(), AutoScalingPolicyTests.mockAppScaleManagerConfiguration(), Schedulers.immediate(), mock(TitusRuntime.class));
    AutoScalingPolicy autoScalingPolicyOne;
    AutoScalingPolicy autoScalingPolicyTwo;
    if (policyType == PolicyType.StepScaling) {
        autoScalingPolicyOne = AutoScalingPolicyTests.buildStepScalingPolicy(jobIdOne);
        autoScalingPolicyTwo = AutoScalingPolicyTests.buildStepScalingPolicy(jobIdTwo);
    } else {
        autoScalingPolicyOne = AutoScalingPolicyTests.buildTargetTrackingPolicy(jobIdOne);
        autoScalingPolicyTwo = AutoScalingPolicyTests.buildTargetTrackingPolicy(jobIdTwo);
    }
    // call - createAutoScalingPolicy
    appScaleManager.createAutoScalingPolicy(autoScalingPolicyOne).toBlocking().single();
    String policyRefIdTwo = appScaleManager.createAutoScalingPolicy(autoScalingPolicyTwo).toBlocking().single();
    AutoScalingPolicyTests.waitForCondition(() -> {
        List<AutoScalingPolicy> policies = policyStore.retrievePolicies(false).toList().toBlocking().first();
        return policies.size() == 2 && mockAppAutoScalingClient.getNumPolicies() == 2 && mockAppAutoScalingClient.getNumScalableTargets() == 2;
    });
    // verify counts in CloudAlarmClient, AppAutoScaleClient and AppScalePolicyStore
    List<AutoScalingPolicy> policiesStored = policyStore.retrievePolicies(false).toList().toBlocking().first();
    Assertions.assertThat(policiesStored.size()).isEqualTo(2);
    Assertions.assertThat(mockAppAutoScalingClient.getNumPolicies()).isEqualTo(2);
    Assertions.assertThat(mockAppAutoScalingClient.getNumScalableTargets()).isEqualTo(2);
    if (policyType == PolicyType.StepScaling) {
        Assertions.assertThat(mockAlarmClient.getNumOfAlarmsCreated()).isEqualTo(2);
    }
    appScaleManager.removeAutoScalingPolicy(policyRefIdTwo).await();
    AutoScalingPolicyTests.waitForCondition(() -> {
        List<AutoScalingPolicy> policies = policyStore.retrievePolicies(false).toList().toBlocking().first();
        return policies.size() == 1 && mockAppAutoScalingClient.getNumPolicies() == 1 && mockAppAutoScalingClient.getNumScalableTargets() == 1;
    });
    // verify counts in CloudAlarmClient, AppAutoScaleClient and AppScalePolicyStore
    policiesStored = policyStore.retrievePolicies(false).toList().toBlocking().first();
    Assertions.assertThat(policiesStored.size()).isEqualTo(1);
    Assertions.assertThat(mockAppAutoScalingClient.getNumPolicies()).isEqualTo(1);
    Assertions.assertThat(mockAppAutoScalingClient.getNumScalableTargets()).isEqualTo(1);
    if (policyType == PolicyType.StepScaling) {
        Assertions.assertThat(mockAlarmClient.getNumOfAlarmsCreated()).isEqualTo(1);
    }
}
Also used : V3JobOperations(com.netflix.titus.api.jobmanager.service.V3JobOperations) InMemoryPolicyStore(com.netflix.titus.runtime.store.v3.memory.InMemoryPolicyStore) TitusRuntime(com.netflix.titus.common.runtime.TitusRuntime) AutoScalingPolicy(com.netflix.titus.api.appscale.model.AutoScalingPolicy) DefaultRegistry(com.netflix.spectator.api.DefaultRegistry)

Example 14 with AutoScalingPolicy

use of com.netflix.titus.api.appscale.model.AutoScalingPolicy in project titus-control-plane by Netflix.

the class InMemoryPolicyStore method updateStatusMessage.

@Override
public Completable updateStatusMessage(String policyRefId, String statusMessage) {
    return Completable.fromCallable(() -> {
        AutoScalingPolicy autoScalingPolicy = policyMap.get(policyRefId);
        AutoScalingPolicy policySaved = AutoScalingPolicy.newBuilder().withAutoScalingPolicy(autoScalingPolicy).withStatusMessage(statusMessage).build();
        return policyMap.put(policyRefId, policySaved);
    });
}
Also used : AutoScalingPolicy(com.netflix.titus.api.appscale.model.AutoScalingPolicy)

Example 15 with AutoScalingPolicy

use of com.netflix.titus.api.appscale.model.AutoScalingPolicy in project titus-control-plane by Netflix.

the class InMemoryPolicyStore method updatePolicyId.

@Override
public Completable updatePolicyId(String policyRefId, String policyId) {
    return Completable.fromCallable(() -> {
        AutoScalingPolicy autoScalingPolicy = policyMap.get(policyRefId);
        AutoScalingPolicy policySaved = AutoScalingPolicy.newBuilder().withAutoScalingPolicy(autoScalingPolicy).withPolicyId(policyId).build();
        return policyMap.put(policyRefId, policySaved);
    });
}
Also used : AutoScalingPolicy(com.netflix.titus.api.appscale.model.AutoScalingPolicy)

Aggregations

AutoScalingPolicy (com.netflix.titus.api.appscale.model.AutoScalingPolicy)21 PolicyConfiguration (com.netflix.titus.api.appscale.model.PolicyConfiguration)8 StepScalingPolicyConfiguration (com.netflix.titus.api.appscale.model.StepScalingPolicyConfiguration)8 Test (org.junit.Test)7 DefaultRegistry (com.netflix.spectator.api.DefaultRegistry)6 AlarmConfiguration (com.netflix.titus.api.appscale.model.AlarmConfiguration)6 StepAdjustment (com.netflix.titus.api.appscale.model.StepAdjustment)4 BoundStatement (com.datastax.driver.core.BoundStatement)3 ResultSet (com.datastax.driver.core.ResultSet)3 Row (com.datastax.driver.core.Row)3 Session (com.datastax.driver.core.Session)3 PolicyStatus (com.netflix.titus.api.appscale.model.PolicyStatus)3 PolicyType (com.netflix.titus.api.appscale.model.PolicyType)3 V3JobOperations (com.netflix.titus.api.jobmanager.service.V3JobOperations)3 TitusRuntime (com.netflix.titus.common.runtime.TitusRuntime)3 InMemoryPolicyStore (com.netflix.titus.runtime.store.v3.memory.InMemoryPolicyStore)3 IntegrationNotParallelizableTest (com.netflix.titus.testkit.junit.category.IntegrationNotParallelizableTest)3 List (java.util.List)3 Optional (java.util.Optional)3 UUID (java.util.UUID)3