Search in sources :

Example 76 with IndexNotFoundException

use of org.elasticsearch.index.IndexNotFoundException in project metron by apache.

the class ElasticsearchMetaAlertDaoTest method testUpdateShouldThrowExceptionOnMissingSensorIndex.

@Test
public void testUpdateShouldThrowExceptionOnMissingSensorIndex() throws Exception {
    ElasticsearchDao elasticsearchDao = mock(ElasticsearchDao.class);
    ElasticsearchMetaAlertRetrieveLatestDao elasticsearchMetaAlertRetrieveLatestDao = mock(ElasticsearchMetaAlertRetrieveLatestDao.class);
    MetaAlertConfig metaAlertConfig = mock(MetaAlertConfig.class);
    ElasticsearchMetaAlertUpdateDao emauDao = spy(new ElasticsearchMetaAlertUpdateDao(elasticsearchDao, elasticsearchMetaAlertRetrieveLatestDao, metaAlertConfig, 1));
    doThrow(new IndexNotFoundException("bro")).when(emauDao).getMetaAlertsForAlert("alert_one");
    Document update = new Document(new HashMap<>(), "alert_one", "", 0L);
    assertThrows(IndexNotFoundException.class, () -> emauDao.update(update, Optional.empty()));
}
Also used : MetaAlertConfig(org.apache.metron.indexing.dao.metaalert.MetaAlertConfig) IndexNotFoundException(org.elasticsearch.index.IndexNotFoundException) Document(org.apache.metron.indexing.dao.update.Document) Test(org.junit.jupiter.api.Test)

Example 77 with IndexNotFoundException

use of org.elasticsearch.index.IndexNotFoundException in project crate by crate.

the class InternalBlobTableInfoFactory method resolveIndexMetadata.

private IndexMetadata resolveIndexMetadata(String tableName, ClusterState state) {
    String indexName = BlobIndex.fullIndexName(tableName);
    Index index;
    try {
        index = indexNameExpressionResolver.concreteIndices(state, IndicesOptions.strictExpandOpen(), indexName)[0];
    } catch (IndexNotFoundException ex) {
        throw new RelationUnknown(indexName, ex);
    }
    return state.metadata().index(index);
}
Also used : RelationUnknown(io.crate.exceptions.RelationUnknown) IndexNotFoundException(org.elasticsearch.index.IndexNotFoundException) BlobIndex(io.crate.blob.v2.BlobIndex) Index(org.elasticsearch.index.Index)

Example 78 with IndexNotFoundException

use of org.elasticsearch.index.IndexNotFoundException in project crate by crate.

the class RoutingProvider method forIndices.

public Routing forIndices(ClusterState state, String[] concreteIndices, Map<String, Set<String>> routingValuesByIndex, boolean ignoreMissingShards, ShardSelection shardSelection) {
    Set<IndexShardRoutingTable> shards;
    try {
        shards = computeTargetedShards(state, concreteIndices, routingValuesByIndex);
    } catch (IndexNotFoundException e) {
        return new Routing(Collections.emptyMap());
    }
    Map<String, Map<String, IntIndexedContainer>> locations = new TreeMap<>();
    for (IndexShardRoutingTable shard : shards) {
        final ShardIterator shardIt;
        switch(shardSelection) {
            case ANY:
                if (awarenessAttributes.isEmpty()) {
                    shardIt = shard.activeInitializingShardsIt(seed);
                } else {
                    shardIt = shard.preferAttributesActiveInitializingShardsIt(awarenessAttributes, state.getNodes(), seed);
                }
                break;
            case PRIMARIES:
                shardIt = shard.primaryActiveInitializingShardIt();
                break;
            default:
                throw new AssertionError("Invalid ShardSelection: " + shardSelection);
        }
        fillLocationsFromShardIterator(ignoreMissingShards, locations, shardIt);
    }
    return new Routing(locations);
}
Also used : IndexShardRoutingTable(org.elasticsearch.cluster.routing.IndexShardRoutingTable) ShardIterator(org.elasticsearch.cluster.routing.ShardIterator) IndexNotFoundException(org.elasticsearch.index.IndexNotFoundException) ShardRouting(org.elasticsearch.cluster.routing.ShardRouting) TreeMap(java.util.TreeMap) ImmutableOpenMap(org.elasticsearch.common.collect.ImmutableOpenMap) Map(java.util.Map) TreeMap(java.util.TreeMap)

Example 79 with IndexNotFoundException

use of org.elasticsearch.index.IndexNotFoundException in project crate by crate.

the class InsertFromValues method execute.

private CompletableFuture<ShardResponse.CompressedResult> execute(NodeLimits nodeLimits, ClusterState state, Collection<ShardUpsertRequest> shardUpsertRequests, TransportShardUpsertAction shardUpsertAction, ScheduledExecutorService scheduler) {
    ShardResponse.CompressedResult compressedResult = new ShardResponse.CompressedResult();
    if (shardUpsertRequests.isEmpty()) {
        return CompletableFuture.completedFuture(compressedResult);
    }
    CompletableFuture<ShardResponse.CompressedResult> result = new CompletableFuture<>();
    AtomicInteger numRequests = new AtomicInteger(shardUpsertRequests.size());
    AtomicReference<Throwable> lastFailure = new AtomicReference<>(null);
    Consumer<ShardUpsertRequest> countdown = request -> {
        if (numRequests.decrementAndGet() == 0) {
            Throwable throwable = lastFailure.get();
            if (throwable == null) {
                result.complete(compressedResult);
            } else {
                throwable = SQLExceptions.unwrap(throwable, t -> t instanceof RuntimeException);
                // we want to report duplicate key exceptions
                if (!SQLExceptions.isDocumentAlreadyExistsException(throwable) && (partitionWasDeleted(throwable, request.index()) || partitionClosed(throwable, request.index()) || mixedArgumentTypesFailure(throwable))) {
                    result.complete(compressedResult);
                } else {
                    result.completeExceptionally(throwable);
                }
            }
        }
    };
    for (ShardUpsertRequest request : shardUpsertRequests) {
        String nodeId;
        try {
            nodeId = state.routingTable().shardRoutingTable(request.shardId()).primaryShard().currentNodeId();
        } catch (IndexNotFoundException e) {
            lastFailure.set(e);
            if (!IndexParts.isPartitioned(request.index())) {
                synchronized (compressedResult) {
                    compressedResult.markAsFailed(request.items());
                }
            }
            countdown.accept(request);
            continue;
        }
        final ConcurrencyLimit nodeLimit = nodeLimits.get(nodeId);
        final long startTime = nodeLimit.startSample();
        ActionListener<ShardResponse> listener = new ActionListener<>() {

            @Override
            public void onResponse(ShardResponse shardResponse) {
                Throwable throwable = shardResponse.failure();
                if (throwable == null) {
                    nodeLimit.onSample(startTime, false);
                    synchronized (compressedResult) {
                        compressedResult.update(shardResponse);
                    }
                } else {
                    nodeLimit.onSample(startTime, true);
                    lastFailure.set(throwable);
                }
                countdown.accept(request);
            }

            @Override
            public void onFailure(Exception e) {
                nodeLimit.onSample(startTime, true);
                Throwable t = SQLExceptions.unwrap(e);
                if (!partitionWasDeleted(t, request.index())) {
                    synchronized (compressedResult) {
                        compressedResult.markAsFailed(request.items());
                    }
                }
                lastFailure.set(t);
                countdown.accept(request);
            }
        };
        shardUpsertAction.execute(request, new RetryListener<>(scheduler, l -> shardUpsertAction.execute(request, l), listener, BackoffPolicy.limitedDynamic(nodeLimit)));
    }
    return result;
}
Also used : GeneratedColumns(io.crate.execution.dml.upsert.GeneratedColumns) IndexParts(io.crate.metadata.IndexParts) INDEX_CLOSED_BLOCK(org.elasticsearch.cluster.metadata.IndexMetadata.INDEX_CLOSED_BLOCK) Arrays(java.util.Arrays) TransportShardUpsertAction(io.crate.execution.dml.upsert.TransportShardUpsertAction) ShardIterator(org.elasticsearch.cluster.routing.ShardIterator) ShardedRequests(io.crate.execution.engine.indexing.ShardedRequests) TableFunctionRelation(io.crate.analyze.relations.TableFunctionRelation) NodeLimits(io.crate.execution.jobs.NodeLimits) TransportCreatePartitionsAction(org.elasticsearch.action.admin.indices.create.TransportCreatePartitionsAction) RetryListener(io.crate.execution.support.RetryListener) DependencyCarrier(io.crate.planner.DependencyCarrier) ClusterState(org.elasticsearch.cluster.ClusterState) RowN(io.crate.data.RowN) SymbolEvaluator(io.crate.analyze.SymbolEvaluator) TableStats(io.crate.statistics.TableStats) ClusterBlock(org.elasticsearch.cluster.block.ClusterBlock) ColumnIndexWriterProjection(io.crate.execution.dsl.projection.ColumnIndexWriterProjection) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) IntArrayList(com.carrotsearch.hppc.IntArrayList) IndexNotFoundException(org.elasticsearch.index.IndexNotFoundException) Map(java.util.Map) TypeGuessEstimateRowSize(io.crate.breaker.TypeGuessEstimateRowSize) ConcurrencyLimit(io.crate.concurrent.limits.ConcurrencyLimit) SelectSymbol(io.crate.expression.symbol.SelectSymbol) GroupRowsByShard(io.crate.execution.engine.indexing.GroupRowsByShard) DocTableInfo(io.crate.metadata.doc.DocTableInfo) Collection(java.util.Collection) InMemoryBatchIterator(io.crate.data.InMemoryBatchIterator) Set(java.util.Set) UUID(java.util.UUID) InputRow(io.crate.expression.InputRow) ShardRequest(io.crate.execution.dml.ShardRequest) ExecutionPlan(io.crate.planner.ExecutionPlan) List(java.util.List) OrderBy(io.crate.analyze.OrderBy) Row(io.crate.data.Row) Symbol(io.crate.expression.symbol.Symbol) RowShardResolver(io.crate.execution.engine.collect.RowShardResolver) Assignments(io.crate.expression.symbol.Assignments) Row1(io.crate.data.Row1) ShardRouting(org.elasticsearch.cluster.routing.ShardRouting) Input(io.crate.data.Input) SENTINEL(io.crate.data.SentinelRow.SENTINEL) ClusterService(org.elasticsearch.cluster.service.ClusterService) CollectExpression(io.crate.execution.engine.collect.CollectExpression) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) Operation(io.crate.metadata.table.Operation) AtomicReference(java.util.concurrent.atomic.AtomicReference) Function(java.util.function.Function) Supplier(java.util.function.Supplier) InsertSourceFromCells(io.crate.execution.dml.upsert.InsertSourceFromCells) ArrayList(java.util.ArrayList) BackoffPolicy(org.elasticsearch.action.bulk.BackoffPolicy) Metadata(org.elasticsearch.cluster.metadata.Metadata) ClusterBlockException(org.elasticsearch.cluster.block.ClusterBlockException) ShardLocation(io.crate.execution.engine.indexing.ShardLocation) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) StreamSupport(java.util.stream.StreamSupport) ColumnValidationException(io.crate.exceptions.ColumnValidationException) Nullable(javax.annotation.Nullable) FutureActionListener(io.crate.action.FutureActionListener) ProjectionBuilder(io.crate.execution.dsl.projection.builder.ProjectionBuilder) BULK_REQUEST_TIMEOUT_SETTING(io.crate.execution.engine.indexing.ShardingUpsertExecutor.BULK_REQUEST_TIMEOUT_SETTING) Iterator(java.util.Iterator) Reference(io.crate.metadata.Reference) DataType(io.crate.types.DataType) AcknowledgedResponse(org.elasticsearch.action.support.master.AcknowledgedResponse) RamAccounting(io.crate.breaker.RamAccounting) Consumer(java.util.function.Consumer) RowConsumer(io.crate.data.RowConsumer) ShardResponse(io.crate.execution.dml.ShardResponse) ShardUpsertRequest(io.crate.execution.dml.upsert.ShardUpsertRequest) CollectionBucket(io.crate.data.CollectionBucket) TableFunctionImplementation(io.crate.metadata.tablefunctions.TableFunctionImplementation) IndexNameResolver(io.crate.execution.engine.indexing.IndexNameResolver) NotSerializableExceptionWrapper(org.elasticsearch.common.io.stream.NotSerializableExceptionWrapper) AbstractTableRelation(io.crate.analyze.relations.AbstractTableRelation) PlannerContext(io.crate.planner.PlannerContext) InputColumns(io.crate.execution.dsl.projection.builder.InputColumns) SQLExceptions(io.crate.exceptions.SQLExceptions) InputFactory(io.crate.expression.InputFactory) CreatePartitionsRequest(org.elasticsearch.action.admin.indices.create.CreatePartitionsRequest) ActionListener(org.elasticsearch.action.ActionListener) ConcurrencyLimit(io.crate.concurrent.limits.ConcurrencyLimit) ShardUpsertRequest(io.crate.execution.dml.upsert.ShardUpsertRequest) AtomicReference(java.util.concurrent.atomic.AtomicReference) IndexNotFoundException(org.elasticsearch.index.IndexNotFoundException) ClusterBlockException(org.elasticsearch.cluster.block.ClusterBlockException) ColumnValidationException(io.crate.exceptions.ColumnValidationException) ShardResponse(io.crate.execution.dml.ShardResponse) CompletableFuture(java.util.concurrent.CompletableFuture) FutureActionListener(io.crate.action.FutureActionListener) ActionListener(org.elasticsearch.action.ActionListener) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) IndexNotFoundException(org.elasticsearch.index.IndexNotFoundException)

Example 80 with IndexNotFoundException

use of org.elasticsearch.index.IndexNotFoundException in project crate by crate.

the class InsertFromValues method resolveAndGroupShardRequests.

private static <TReq extends ShardRequest<TReq, TItem>, TItem extends ShardRequest.Item> Map<ShardLocation, TReq> resolveAndGroupShardRequests(ShardedRequests<TReq, TItem> shardedRequests, ClusterService clusterService) {
    var itemsByMissingIndex = shardedRequests.itemsByMissingIndex().entrySet().iterator();
    while (itemsByMissingIndex.hasNext()) {
        var entry = itemsByMissingIndex.next();
        var index = entry.getKey();
        var requestItems = entry.getValue();
        var requestItemsIterator = requestItems.iterator();
        while (requestItemsIterator.hasNext()) {
            var itemAndRoutingAndSourceInfo = requestItemsIterator.next();
            ShardLocation shardLocation;
            try {
                shardLocation = getShardLocation(index, itemAndRoutingAndSourceInfo.item().id(), itemAndRoutingAndSourceInfo.routing(), clusterService);
            } catch (IndexNotFoundException e) {
                if (IndexParts.isPartitioned(index)) {
                    requestItemsIterator.remove();
                    continue;
                } else {
                    throw e;
                }
            }
            shardedRequests.add(itemAndRoutingAndSourceInfo.item(), 0, shardLocation, null);
            requestItemsIterator.remove();
        }
        if (requestItems.isEmpty()) {
            itemsByMissingIndex.remove();
        }
    }
    return shardedRequests.itemsByShard();
}
Also used : ShardLocation(io.crate.execution.engine.indexing.ShardLocation) IndexNotFoundException(org.elasticsearch.index.IndexNotFoundException)

Aggregations

IndexNotFoundException (org.elasticsearch.index.IndexNotFoundException)92 ClusterState (org.elasticsearch.cluster.ClusterState)22 ShardNotFoundException (org.elasticsearch.index.shard.ShardNotFoundException)21 ShardId (org.elasticsearch.index.shard.ShardId)19 ShardRouting (org.elasticsearch.cluster.routing.ShardRouting)16 Index (org.elasticsearch.index.Index)16 Map (java.util.Map)15 ArrayList (java.util.ArrayList)14 IOException (java.io.IOException)13 IndexMetadata (org.elasticsearch.cluster.metadata.IndexMetadata)12 List (java.util.List)11 IndicesOptions (org.elasticsearch.action.support.IndicesOptions)11 ClusterName (org.elasticsearch.cluster.ClusterName)9 DiscoveryNode (org.elasticsearch.cluster.node.DiscoveryNode)9 Settings (org.elasticsearch.common.settings.Settings)9 Matchers.containsString (org.hamcrest.Matchers.containsString)9 HashMap (java.util.HashMap)8 Nullable (javax.annotation.Nullable)8 RoutingNode (org.elasticsearch.cluster.routing.RoutingNode)8 RoutingNodes (org.elasticsearch.cluster.routing.RoutingNodes)8