Search in sources :

Example 11 with RetentionStrategy

use of org.graylog2.plugin.indexer.retention.RetentionStrategy in project graylog2-server by Graylog2.

the class V20161116172100_DefaultIndexSetMigration method getRetentionStrategyConfig.

private RetentionStrategyConfig getRetentionStrategyConfig(IndexManagementConfig indexManagementConfig) {
    final String strategyName = indexManagementConfig.retentionStrategy();
    final Provider<RetentionStrategy> provider = retentionStrategies.get(strategyName);
    checkState(provider != null, "Couldn't retrieve retention strategy provider for <" + strategyName + ">");
    final RetentionStrategy retentionStrategy = provider.get();
    @SuppressWarnings("unchecked") final Class<RetentionStrategyConfig> configClass = (Class<RetentionStrategyConfig>) retentionStrategy.configurationClass();
    final RetentionStrategyConfig retentionStrategyConfig = clusterConfigService.get(configClass);
    checkState(retentionStrategyConfig != null, "Couldn't retrieve retention strategy config for <" + strategyName + ">");
    return retentionStrategyConfig;
}
Also used : RetentionStrategyConfig(org.graylog2.plugin.indexer.retention.RetentionStrategyConfig) RetentionStrategy(org.graylog2.plugin.indexer.retention.RetentionStrategy)

Example 12 with RetentionStrategy

use of org.graylog2.plugin.indexer.retention.RetentionStrategy in project graylog2-server by Graylog2.

the class AbstractIndexCountBasedRetentionStrategyTest method shouldIgnoreWriteAliasWhenDeterminingRetainedIndices.

@Test
public void shouldIgnoreWriteAliasWhenDeterminingRetainedIndices() {
    final String indexWithWriteIndexAlias = "index1";
    final String writeIndexAlias = "WriteIndexAlias";
    when(indexSet.getWriteIndexAlias()).thenReturn(writeIndexAlias);
    indexMap.put(indexWithWriteIndexAlias, Collections.singleton(writeIndexAlias));
    retentionStrategy.retain(indexSet);
    final ArgumentCaptor<List> retainedIndexName = ArgumentCaptor.forClass(List.class);
    verify(retentionStrategy, times(1)).retain(retainedIndexName.capture(), eq(indexSet));
    assertThat(retainedIndexName.getValue()).containsExactly("index2");
    verify(activityWriter, times(2)).write(any(Activity.class));
}
Also used : Activity(org.graylog2.shared.system.activities.Activity) List(java.util.List) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Test(org.junit.Test)

Example 13 with RetentionStrategy

use of org.graylog2.plugin.indexer.retention.RetentionStrategy in project graylog2-server by Graylog2.

the class V20161124104700_AddRetentionRotationAndDefaultFlagToIndexSetMigrationTest method upgradeWhenOneAlreadyHasStrategiesSet.

@Test
public void upgradeWhenOneAlreadyHasStrategiesSet() throws Exception {
    final String rotationStrategyClass = MessageCountRotationStrategy.class.getCanonicalName();
    final String retentionStrategyClass = DeletionRetentionStrategy.class.getCanonicalName();
    final RotationStrategyConfig rotationStrategy = MessageCountRotationStrategyConfig.createDefault();
    final RetentionStrategyConfig retentionStrategy = DeletionRetentionStrategyConfig.createDefault();
    final IndexSetConfig config1 = IndexSetConfig.builder().id("id-1").title("title-1").indexPrefix("prefix-1").shards(1).replicas(0).rotationStrategy(rotationStrategy).retentionStrategyClass(retentionStrategyClass).retentionStrategy(retentionStrategy).creationDate(ZonedDateTime.of(2016, 10, 12, 0, 0, 0, 0, ZoneOffset.UTC)).indexAnalyzer("standard").indexTemplateName("template-1").indexOptimizationMaxNumSegments(1).indexOptimizationDisabled(false).build();
    final IndexSetConfig config2 = IndexSetConfig.builder().id("id-2").title("title-2").indexPrefix("prefix-2").shards(1).replicas(0).rotationStrategyClass(rotationStrategyClass).rotationStrategy(rotationStrategy).retentionStrategyClass(retentionStrategyClass).retentionStrategy(retentionStrategy).creationDate(ZonedDateTime.of(2016, 10, 13, 0, 0, 0, 0, ZoneOffset.UTC)).indexAnalyzer("standard").indexTemplateName("template-2").indexOptimizationMaxNumSegments(1).indexOptimizationDisabled(false).build();
    when(clusterConfigService.get(IndexManagementConfig.class)).thenReturn(IndexManagementConfig.create(rotationStrategyClass, retentionStrategyClass));
    when(indexSetService.findAll()).thenReturn(Lists.newArrayList(config1, config2));
    migration.upgrade();
    verify(indexSetService).save(config1.toBuilder().rotationStrategyClass(rotationStrategyClass).build());
    verify(indexSetService, never()).save(config2);
    verify(clusterConfigService).write(V20161124104700_AddRetentionRotationAndDefaultFlagToIndexSetMigration.MigrationCompleted.create(Collections.singleton("id-1"), Collections.singleton("id-2"), "id-1"));
}
Also used : DeletionRetentionStrategyConfig(org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig) RetentionStrategyConfig(org.graylog2.plugin.indexer.retention.RetentionStrategyConfig) IndexSetConfig(org.graylog2.indexer.indexset.IndexSetConfig) MessageCountRotationStrategyConfig(org.graylog2.indexer.rotation.strategies.MessageCountRotationStrategyConfig) RotationStrategyConfig(org.graylog2.plugin.indexer.rotation.RotationStrategyConfig) Test(org.junit.Test)

Example 14 with RetentionStrategy

use of org.graylog2.plugin.indexer.retention.RetentionStrategy in project graylog2-server by Graylog2.

the class V20161124104700_AddRetentionRotationAndDefaultFlagToIndexSetMigrationTest method upgradeDoeNotRunWhenAlreadyComplete.

@Test
public void upgradeDoeNotRunWhenAlreadyComplete() throws Exception {
    final String rotationStrategyClass = MessageCountRotationStrategy.class.getCanonicalName();
    final String retentionStrategyClass = DeletionRetentionStrategy.class.getCanonicalName();
    final RotationStrategyConfig rotationStrategy = MessageCountRotationStrategyConfig.createDefault();
    final RetentionStrategyConfig retentionStrategy = DeletionRetentionStrategyConfig.createDefault();
    final IndexSetConfig config1 = IndexSetConfig.builder().id("id-1").title("title-1").indexPrefix("prefix-1").shards(1).replicas(0).rotationStrategy(rotationStrategy).retentionStrategy(retentionStrategy).creationDate(ZonedDateTime.of(2016, 10, 12, 0, 0, 0, 0, ZoneOffset.UTC)).indexAnalyzer("standard").indexTemplateName("template-1").indexOptimizationMaxNumSegments(1).indexOptimizationDisabled(false).build();
    final IndexSetConfig config2 = IndexSetConfig.builder().id("id-2").title("title-2").indexPrefix("prefix-2").shards(1).replicas(0).rotationStrategy(rotationStrategy).retentionStrategy(retentionStrategy).creationDate(ZonedDateTime.of(2016, 10, 13, 0, 0, 0, 0, ZoneOffset.UTC)).indexAnalyzer("standard").indexTemplateName("template-2").indexOptimizationMaxNumSegments(1).indexOptimizationDisabled(false).build();
    when(clusterConfigService.get(IndexManagementConfig.class)).thenReturn(IndexManagementConfig.create(rotationStrategyClass, retentionStrategyClass));
    when(indexSetService.findAll()).thenReturn(Lists.newArrayList(config1, config2));
    when(clusterConfigService.get(V20161124104700_AddRetentionRotationAndDefaultFlagToIndexSetMigration.MigrationCompleted.class)).thenReturn(V20161124104700_AddRetentionRotationAndDefaultFlagToIndexSetMigration.MigrationCompleted.create(Collections.emptySet(), Collections.emptySet(), "id-1"));
    migration.upgrade();
    verify(indexSetService, never()).save(any(IndexSetConfig.class));
    verify(clusterConfigService, never()).write(V20161124104700_AddRetentionRotationAndDefaultFlagToIndexSetMigration.MigrationCompleted.class);
}
Also used : DeletionRetentionStrategyConfig(org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig) RetentionStrategyConfig(org.graylog2.plugin.indexer.retention.RetentionStrategyConfig) IndexSetConfig(org.graylog2.indexer.indexset.IndexSetConfig) MessageCountRotationStrategyConfig(org.graylog2.indexer.rotation.strategies.MessageCountRotationStrategyConfig) RotationStrategyConfig(org.graylog2.plugin.indexer.rotation.RotationStrategyConfig) Test(org.junit.Test)

Example 15 with RetentionStrategy

use of org.graylog2.plugin.indexer.retention.RetentionStrategy in project graylog2-server by Graylog2.

the class V20161124104700_AddRetentionRotationAndDefaultFlagToIndexSetMigrationTest method upgrade.

@Test
public void upgrade() throws Exception {
    final String rotationStrategyClass = MessageCountRotationStrategy.class.getCanonicalName();
    final String retentionStrategyClass = DeletionRetentionStrategy.class.getCanonicalName();
    final RotationStrategyConfig rotationStrategy = MessageCountRotationStrategyConfig.createDefault();
    final RetentionStrategyConfig retentionStrategy = DeletionRetentionStrategyConfig.createDefault();
    final IndexSetConfig config1 = IndexSetConfig.builder().id("id-1").title("title-1").indexPrefix("prefix-1").shards(1).replicas(0).rotationStrategy(rotationStrategy).retentionStrategy(retentionStrategy).creationDate(ZonedDateTime.of(2016, 10, 12, 0, 0, 0, 0, ZoneOffset.UTC)).indexAnalyzer("standard").indexTemplateName("template-1").indexOptimizationMaxNumSegments(1).indexOptimizationDisabled(false).build();
    final IndexSetConfig config2 = IndexSetConfig.builder().id("id-2").title("title-2").indexPrefix("prefix-2").shards(1).replicas(0).rotationStrategy(rotationStrategy).retentionStrategy(retentionStrategy).creationDate(ZonedDateTime.of(2016, 10, 10, 0, 0, 0, 0, ZoneOffset.UTC)).indexAnalyzer("standard").indexTemplateName("template-2").indexOptimizationMaxNumSegments(1).indexOptimizationDisabled(false).build();
    when(clusterConfigService.get(IndexManagementConfig.class)).thenReturn(IndexManagementConfig.create(rotationStrategyClass, retentionStrategyClass));
    when(indexSetService.findAll()).thenReturn(Lists.newArrayList(config1, config2));
    migration.upgrade();
    verify(indexSetService).save(config1.toBuilder().rotationStrategyClass(rotationStrategyClass).retentionStrategyClass(retentionStrategyClass).build());
    verify(indexSetService).save(config2.toBuilder().rotationStrategyClass(rotationStrategyClass).retentionStrategyClass(retentionStrategyClass).build());
    verify(clusterConfigService).write(V20161124104700_AddRetentionRotationAndDefaultFlagToIndexSetMigration.MigrationCompleted.create(ImmutableSet.of("id-1", "id-2"), Collections.emptySet(), "id-2"));
}
Also used : DeletionRetentionStrategyConfig(org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig) RetentionStrategyConfig(org.graylog2.plugin.indexer.retention.RetentionStrategyConfig) IndexSetConfig(org.graylog2.indexer.indexset.IndexSetConfig) MessageCountRotationStrategyConfig(org.graylog2.indexer.rotation.strategies.MessageCountRotationStrategyConfig) RotationStrategyConfig(org.graylog2.plugin.indexer.rotation.RotationStrategyConfig) Test(org.junit.Test)

Aggregations

IndexSetConfig (org.graylog2.indexer.indexset.IndexSetConfig)11 RetentionStrategyConfig (org.graylog2.plugin.indexer.retention.RetentionStrategyConfig)9 Test (org.junit.Test)9 DeletionRetentionStrategyConfig (org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig)6 MessageCountRotationStrategyConfig (org.graylog2.indexer.rotation.strategies.MessageCountRotationStrategyConfig)6 RotationStrategyConfig (org.graylog2.plugin.indexer.rotation.RotationStrategyConfig)6 RetentionStrategy (org.graylog2.plugin.indexer.retention.RetentionStrategy)4 IndexManagementConfig (org.graylog2.indexer.management.IndexManagementConfig)3 InternalServerErrorException (javax.ws.rs.InternalServerErrorException)2 IndexSet (org.graylog2.indexer.IndexSet)2 DefaultIndexSetConfig (org.graylog2.indexer.indexset.DefaultIndexSetConfig)2 DeletionRetentionStrategy (org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy)2 Timed (com.codahale.metrics.annotation.Timed)1 JsonMappingException (com.fasterxml.jackson.databind.JsonMappingException)1 SchemaFactoryWrapper (com.fasterxml.jackson.module.jsonSchema.factories.SchemaFactoryWrapper)1 DuplicateKeyException (com.mongodb.DuplicateKeyException)1 ApiOperation (io.swagger.annotations.ApiOperation)1 ZonedDateTime (java.time.ZonedDateTime)1 List (java.util.List)1 GET (javax.ws.rs.GET)1