Search in sources :

Example 16 with Indices

use of org.graylog2.indexer.indices.Indices in project graylog2-server by Graylog2.

the class SizeBasedRotationStrategyTest method testRotate.

@Test
public void testRotate() throws Exception {
    final CommonStats commonStats = new CommonStats();
    commonStats.store = new StoreStats(1000, 0);
    final IndexStatistics stats = IndexStatistics.create("name", commonStats, commonStats, Collections.<ShardRouting>emptyList());
    when(indices.getIndexStats("name")).thenReturn(stats);
    when(indexSet.getNewestIndex()).thenReturn("name");
    when(indexSet.getConfig()).thenReturn(indexSetConfig);
    when(indexSetConfig.rotationStrategy()).thenReturn(SizeBasedRotationStrategyConfig.create(100L));
    final SizeBasedRotationStrategy strategy = new SizeBasedRotationStrategy(indices, nodeId, auditEventSender);
    strategy.rotate(indexSet);
    verify(indexSet, times(1)).cycle();
    reset(indexSet);
}
Also used : IndexStatistics(org.graylog2.indexer.indices.IndexStatistics) CommonStats(org.elasticsearch.action.admin.indices.stats.CommonStats) StoreStats(org.elasticsearch.index.store.StoreStats) Test(org.junit.Test)

Example 17 with Indices

use of org.graylog2.indexer.indices.Indices in project graylog2-server by Graylog2.

the class IndexRangesResource method rebuildIndexSet.

@POST
@Timed
@Path("/index_set/{indexSetId}/rebuild")
@RequiresPermissions(RestPermissions.INDEXRANGES_REBUILD)
@ApiOperation(value = "Rebuild/sync index range information for the given index set.", notes = "This triggers a systemjob that scans every index in the given index set and stores meta information " + "about what indices contain messages in what timeranges. It atomically overwrites " + "already existing meta information.")
@ApiResponses(value = { @ApiResponse(code = 202, message = "Rebuild/sync systemjob triggered.") })
@Produces(MediaType.APPLICATION_JSON)
@AuditEvent(type = AuditEventTypes.ES_INDEX_RANGE_UPDATE_JOB)
public Response rebuildIndexSet(@ApiParam(name = "indexSetId") @PathParam("indexSetId") @NotBlank final String indexSetId) {
    final IndexSet indexSet = indexSetRegistry.get(indexSetId).orElseThrow(() -> new javax.ws.rs.NotFoundException("Index set <" + indexSetId + "> not found!"));
    submitIndexRangesJob(Collections.singleton(indexSet));
    return Response.accepted().build();
}
Also used : IndexSet(org.graylog2.indexer.IndexSet) Path(javax.ws.rs.Path) RequiresPermissions(org.apache.shiro.authz.annotation.RequiresPermissions) POST(javax.ws.rs.POST) Produces(javax.ws.rs.Produces) Timed(com.codahale.metrics.annotation.Timed) ApiOperation(io.swagger.annotations.ApiOperation) AuditEvent(org.graylog2.audit.jersey.AuditEvent) ApiResponses(io.swagger.annotations.ApiResponses)

Example 18 with Indices

use of org.graylog2.indexer.indices.Indices in project graylog2-server by Graylog2.

the class IndicesResource method indexSetClosed.

@GET
@Timed
@Path("/{indexSetId}/closed")
@ApiOperation(value = "Get a list of closed indices that can be reopened.")
@Produces(MediaType.APPLICATION_JSON)
public ClosedIndices indexSetClosed(@ApiParam(name = "indexSetId") @PathParam("indexSetId") String indexSetId) {
    final IndexSet indexSet = getIndexSet(indexSetRegistry, indexSetId);
    final Set<String> closedIndices = indices.getClosedIndices(indexSet).stream().filter((indexName) -> isPermitted(RestPermissions.INDICES_READ, indexName) && indexSetRegistry.isManagedIndex(indexName)).collect(Collectors.toSet());
    return ClosedIndices.create(closedIndices, closedIndices.size());
}
Also used : IndicesReadRequest(org.graylog2.rest.models.system.indexer.requests.IndicesReadRequest) AllIndices(org.graylog2.rest.models.system.indexer.responses.AllIndices) Produces(javax.ws.rs.Produces) LoggerFactory(org.slf4j.LoggerFactory) Path(javax.ws.rs.Path) ApiParam(io.swagger.annotations.ApiParam) IndexStats(org.graylog2.rest.models.system.indexer.responses.IndexStats) ClosedIndices(org.graylog2.rest.models.system.indexer.responses.ClosedIndices) Valid(javax.validation.Valid) ApiOperation(io.swagger.annotations.ApiOperation) MediaType(javax.ws.rs.core.MediaType) Indices(org.graylog2.indexer.indices.Indices) Locale(java.util.Locale) Map(java.util.Map) BadRequestException(javax.ws.rs.BadRequestException) IndexSet(org.graylog2.indexer.IndexSet) DELETE(javax.ws.rs.DELETE) NoAuditEvent(org.graylog2.audit.jersey.NoAuditEvent) Set(java.util.Set) IndexStatistics(org.graylog2.indexer.indices.IndexStatistics) NotNull(javax.validation.constraints.NotNull) Collectors(java.util.stream.Collectors) NotFoundException(javax.ws.rs.NotFoundException) Timed(com.codahale.metrics.annotation.Timed) AuditEventTypes(org.graylog2.audit.AuditEventTypes) CommonStats(org.elasticsearch.action.admin.indices.stats.CommonStats) RequiresAuthentication(org.apache.shiro.authz.annotation.RequiresAuthentication) PathParam(javax.ws.rs.PathParam) GET(javax.ws.rs.GET) HashMap(java.util.HashMap) ApiResponses(io.swagger.annotations.ApiResponses) Function(java.util.function.Function) Inject(javax.inject.Inject) ShardRouting(org.graylog2.rest.models.system.indexer.responses.ShardRouting) RequiresPermissions(org.apache.shiro.authz.annotation.RequiresPermissions) ImmutableList(com.google.common.collect.ImmutableList) AuditEvent(org.graylog2.audit.jersey.AuditEvent) Api(io.swagger.annotations.Api) IndexSetRegistry(org.graylog2.indexer.IndexSetRegistry) Logger(org.slf4j.Logger) POST(javax.ws.rs.POST) Cluster(org.graylog2.indexer.cluster.Cluster) ForbiddenException(javax.ws.rs.ForbiddenException) IndexInfo(org.graylog2.rest.models.system.indexer.responses.IndexInfo) RestResource(org.graylog2.shared.rest.resources.RestResource) ApiResponse(io.swagger.annotations.ApiResponse) OpenIndicesInfo(org.graylog2.rest.models.system.indexer.responses.OpenIndicesInfo) RestPermissions(org.graylog2.shared.security.RestPermissions) TooManyAliasesException(org.graylog2.indexer.indices.TooManyAliasesException) IndexSet(org.graylog2.indexer.IndexSet) Path(javax.ws.rs.Path) Produces(javax.ws.rs.Produces) Timed(com.codahale.metrics.annotation.Timed) GET(javax.ws.rs.GET) ApiOperation(io.swagger.annotations.ApiOperation)

Example 19 with Indices

use of org.graylog2.indexer.indices.Indices in project graylog2-server by Graylog2.

the class IndicesResource method indexSetReopened.

@GET
@Timed
@Path("/{indexSetId}/reopened")
@ApiOperation(value = "Get a list of reopened indices, which will not be cleaned by retention cleaning")
@Produces(MediaType.APPLICATION_JSON)
public ClosedIndices indexSetReopened(@ApiParam(name = "indexSetId") @PathParam("indexSetId") String indexSetId) {
    final IndexSet indexSet = getIndexSet(indexSetRegistry, indexSetId);
    final Set<String> reopenedIndices = indices.getReopenedIndices(indexSet).stream().filter((indexName) -> isPermitted(RestPermissions.INDICES_READ, indexName) && indexSetRegistry.isManagedIndex(indexName)).collect(Collectors.toSet());
    return ClosedIndices.create(reopenedIndices, reopenedIndices.size());
}
Also used : IndicesReadRequest(org.graylog2.rest.models.system.indexer.requests.IndicesReadRequest) AllIndices(org.graylog2.rest.models.system.indexer.responses.AllIndices) Produces(javax.ws.rs.Produces) LoggerFactory(org.slf4j.LoggerFactory) Path(javax.ws.rs.Path) ApiParam(io.swagger.annotations.ApiParam) IndexStats(org.graylog2.rest.models.system.indexer.responses.IndexStats) ClosedIndices(org.graylog2.rest.models.system.indexer.responses.ClosedIndices) Valid(javax.validation.Valid) ApiOperation(io.swagger.annotations.ApiOperation) MediaType(javax.ws.rs.core.MediaType) Indices(org.graylog2.indexer.indices.Indices) Locale(java.util.Locale) Map(java.util.Map) BadRequestException(javax.ws.rs.BadRequestException) IndexSet(org.graylog2.indexer.IndexSet) DELETE(javax.ws.rs.DELETE) NoAuditEvent(org.graylog2.audit.jersey.NoAuditEvent) Set(java.util.Set) IndexStatistics(org.graylog2.indexer.indices.IndexStatistics) NotNull(javax.validation.constraints.NotNull) Collectors(java.util.stream.Collectors) NotFoundException(javax.ws.rs.NotFoundException) Timed(com.codahale.metrics.annotation.Timed) AuditEventTypes(org.graylog2.audit.AuditEventTypes) CommonStats(org.elasticsearch.action.admin.indices.stats.CommonStats) RequiresAuthentication(org.apache.shiro.authz.annotation.RequiresAuthentication) PathParam(javax.ws.rs.PathParam) GET(javax.ws.rs.GET) HashMap(java.util.HashMap) ApiResponses(io.swagger.annotations.ApiResponses) Function(java.util.function.Function) Inject(javax.inject.Inject) ShardRouting(org.graylog2.rest.models.system.indexer.responses.ShardRouting) RequiresPermissions(org.apache.shiro.authz.annotation.RequiresPermissions) ImmutableList(com.google.common.collect.ImmutableList) AuditEvent(org.graylog2.audit.jersey.AuditEvent) Api(io.swagger.annotations.Api) IndexSetRegistry(org.graylog2.indexer.IndexSetRegistry) Logger(org.slf4j.Logger) POST(javax.ws.rs.POST) Cluster(org.graylog2.indexer.cluster.Cluster) ForbiddenException(javax.ws.rs.ForbiddenException) IndexInfo(org.graylog2.rest.models.system.indexer.responses.IndexInfo) RestResource(org.graylog2.shared.rest.resources.RestResource) ApiResponse(io.swagger.annotations.ApiResponse) OpenIndicesInfo(org.graylog2.rest.models.system.indexer.responses.OpenIndicesInfo) RestPermissions(org.graylog2.shared.security.RestPermissions) TooManyAliasesException(org.graylog2.indexer.indices.TooManyAliasesException) IndexSet(org.graylog2.indexer.IndexSet) Path(javax.ws.rs.Path) Produces(javax.ws.rs.Produces) Timed(com.codahale.metrics.annotation.Timed) GET(javax.ws.rs.GET) ApiOperation(io.swagger.annotations.ApiOperation)

Example 20 with Indices

use of org.graylog2.indexer.indices.Indices in project graylog2-server by Graylog2.

the class Searches method search.

public SearchResult search(SearchesConfig config) {
    Set<IndexRange> indices = determineAffectedIndicesWithRanges(config.range(), config.filter());
    Set<String> indexNames = Sets.newHashSet();
    for (IndexRange index : indices) {
        indexNames.add(index.indexName());
    }
    SearchRequest request = searchRequest(config, indexNames).request();
    SearchResponse r = c.search(request).actionGet();
    recordEsMetrics(r, config.range());
    return new SearchResult(r.getHits(), indices, config.query(), request.source(), r.getTook());
}
Also used : IndexRange(org.graylog2.indexer.ranges.IndexRange) SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchResult(org.graylog2.indexer.results.SearchResult) SearchResponse(org.elasticsearch.action.search.SearchResponse)

Aggregations

IndexSet (org.graylog2.indexer.IndexSet)14 Test (org.junit.Test)12 IndexRange (org.graylog2.indexer.ranges.IndexRange)10 Set (java.util.Set)9 ZonedDateTime (java.time.ZonedDateTime)6 Map (java.util.Map)6 DateTime (org.joda.time.DateTime)6 Timed (com.codahale.metrics.annotation.Timed)5 ApiOperation (io.swagger.annotations.ApiOperation)5 ApiResponses (io.swagger.annotations.ApiResponses)5 IOException (java.io.IOException)5 HashMap (java.util.HashMap)5 Collectors (java.util.stream.Collectors)5 Inject (javax.inject.Inject)5 POST (javax.ws.rs.POST)5 Path (javax.ws.rs.Path)5 Produces (javax.ws.rs.Produces)5 AuditEvent (org.graylog2.audit.jersey.AuditEvent)5 MongoIndexRange (org.graylog2.indexer.ranges.MongoIndexRange)5 TimeRange (org.graylog2.plugin.indexer.searches.timeranges.TimeRange)5