Search in sources :

Example 86 with ElasticsearchException

use of org.elasticsearch.ElasticsearchException in project elasticsearch-indexing-proxy by codelibs.

the class RequestSender method process.

private void process(final long filePosition) {
    if (logger.isDebugEnabled()) {
        logger.debug("RequestSender(" + index + ") processes " + filePosition);
    }
    path = dataPath.resolve(String.format(dataFileFormat, filePosition) + IndexingProxyPlugin.DATA_EXTENTION);
    if (FileAccessUtils.existsFile(path)) {
        logger.info("[Sender][{}] Indexing: {}", index, path.toAbsolutePath());
        requestPosition = 0;
        try {
            processRequests(AccessController.doPrivileged((PrivilegedAction<IndexingProxyStreamInput>) () -> {
                try {
                    return new IndexingProxyStreamInput(Files.newInputStream(path), namedWriteableRegistry);
                } catch (final IOException e) {
                    throw new ElasticsearchException("Failed to read " + path.toAbsolutePath(), e);
                }
            }));
        // continue
        } catch (final Exception e) {
            retryWithError("Failed to access " + path.toAbsolutePath(), e);
        // retry
        }
    } else {
        if (logger.isDebugEnabled()) {
            logger.debug("{} does not exist.", path.toAbsolutePath());
        }
        long next = getNextValue(filePosition);
        for (long i = 0; i < senderLookupFiles; i++) {
            if (FileAccessUtils.existsFile(dataPath.resolve(String.format(dataFileFormat, next) + IndexingProxyPlugin.DATA_EXTENTION))) {
                logger.warn("[Sender][" + index + "] file_id " + filePosition + " is skipped. Moving to file_id " + next);
                processNext(next);
                return;
            // continue
            }
            next = getNextValue(next);
        }
        threadPool.schedule(senderInterval, Names.GENERIC, this);
    // retry
    }
}
Also used : PrivilegedAction(java.security.PrivilegedAction) IndexingProxyStreamInput(org.codelibs.elasticsearch.idxproxy.stream.IndexingProxyStreamInput) IOException(java.io.IOException) ElasticsearchException(org.elasticsearch.ElasticsearchException) ElasticsearchException(org.elasticsearch.ElasticsearchException) IOException(java.io.IOException)

Example 87 with ElasticsearchException

use of org.elasticsearch.ElasticsearchException in project bw-calendar-engine by Bedework.

the class BwIndexEsImpl method newIndex.

@Override
public String newIndex(final String name) throws CalFacadeException {
    try {
        final String newName = name + newIndexSuffix();
        targetIndex = newName;
        final IndicesAdminClient idx = getAdminIdx();
        final CreateIndexRequestBuilder cirb = idx.prepareCreate(newName);
        final File f = new File(idxpars.getIndexerConfig());
        final byte[] sbBytes = Streams.copyToByteArray(f);
        cirb.setSource(sbBytes);
        final CreateIndexRequest cir = cirb.request();
        final ActionFuture<CreateIndexResponse> af = idx.create(cir);
        /*resp = */
        af.actionGet();
        index(new UpdateInfo());
        info("Index created: change token set to " + currentChangeToken());
        return newName;
    } catch (final ElasticsearchException ese) {
        // Failed somehow
        error(ese);
        return null;
    } catch (final CalFacadeException cfe) {
        throw cfe;
    } catch (final Throwable t) {
        error(t);
        throw new CalFacadeException(t);
    }
}
Also used : CreateIndexRequestBuilder(org.elasticsearch.action.admin.indices.create.CreateIndexRequestBuilder) IndicesAdminClient(org.elasticsearch.client.IndicesAdminClient) ElasticsearchException(org.elasticsearch.ElasticsearchException) CreateIndexRequest(org.elasticsearch.action.admin.indices.create.CreateIndexRequest) CreateIndexResponse(org.elasticsearch.action.admin.indices.create.CreateIndexResponse) File(java.io.File) UpdateInfo(org.bedework.util.elasticsearch.DocBuilderBase.UpdateInfo) CalFacadeException(org.bedework.calfacade.exc.CalFacadeException)

Example 88 with ElasticsearchException

use of org.elasticsearch.ElasticsearchException in project bw-calendar-engine by Bedework.

the class BwIndexEsImpl method setAlias.

@Override
public int setAlias(final String index, final String alias) throws CalFacadeException {
    // IndicesAliasesResponse resp = null;
    try {
        /* Other is the alias name - index is the index we were just indexing into
       */
        final IndicesAdminClient idx = getAdminIdx();
        final GetAliasesRequestBuilder igarb = idx.prepareGetAliases(alias);
        final ActionFuture<GetAliasesResponse> getAliasesAf = idx.getAliases(igarb.request());
        final GetAliasesResponse garesp = getAliasesAf.actionGet();
        final ImmutableOpenMap<String, List<AliasMetaData>> aliasesmeta = garesp.getAliases();
        final IndicesAliasesRequestBuilder iarb = idx.prepareAliases();
        final Iterator<String> it = aliasesmeta.keysIt();
        while (it.hasNext()) {
            final String indexName = it.next();
            for (final AliasMetaData amd : aliasesmeta.get(indexName)) {
                if (amd.getAlias().equals(alias)) {
                    iarb.removeAlias(indexName, alias);
                }
            }
        }
        iarb.addAlias(index, alias);
        final ActionFuture<IndicesAliasesResponse> af = idx.aliases(iarb.request());
        /*resp = */
        af.actionGet();
        return 0;
    } catch (final ElasticsearchException ese) {
        // Failed somehow
        error(ese);
        return -1;
    } catch (final CalFacadeException cfe) {
        throw cfe;
    } catch (final Throwable t) {
        throw new CalFacadeException(t);
    }
}
Also used : GetAliasesResponse(org.elasticsearch.action.admin.indices.alias.get.GetAliasesResponse) GetAliasesRequestBuilder(org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequestBuilder) IndicesAdminClient(org.elasticsearch.client.IndicesAdminClient) ElasticsearchException(org.elasticsearch.ElasticsearchException) CalFacadeException(org.bedework.calfacade.exc.CalFacadeException) AliasMetaData(org.elasticsearch.cluster.metadata.AliasMetaData) ArrayList(java.util.ArrayList) List(java.util.List) IndicesAliasesResponse(org.elasticsearch.action.admin.indices.alias.IndicesAliasesResponse) IndicesAliasesRequestBuilder(org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequestBuilder)

Example 89 with ElasticsearchException

use of org.elasticsearch.ElasticsearchException in project bw-calendar-engine by Bedework.

the class BwIndexEsImpl method currentChangeToken.

@Override
public String currentChangeToken() throws CalFacadeException {
    UpdateInfo ui;
    try {
        final GetRequestBuilder grb = getClient().prepareGet(targetIndex, docTypeUpdateTracker, updateTrackerId).setFields("count", "_timestamp");
        final GetResponse gr = grb.execute().actionGet();
        if (!gr.isExists()) {
            return null;
        }
        final EntityBuilder er = getEntityBuilder(gr.getFields());
        ui = er.makeUpdateInfo();
    } catch (final ElasticsearchException ese) {
        warn("Exception getting UpdateInfo: " + ese.getLocalizedMessage());
        ui = new UpdateInfo();
    }
    synchronized (updateInfo) {
        UpdateInfo tui = updateInfo.get(targetIndex);
        if ((tui != null) && (tui.getCount() >= (0111111111 - 1000))) {
            // Reset before we overflow
            tui = null;
        }
        if ((tui == null) || (!tui.getCount().equals(ui.getCount()))) {
            updateInfo.put(targetIndex, ui);
        } else {
            ui = tui;
        }
    }
    return ui.getChangeToken();
}
Also used : ElasticsearchException(org.elasticsearch.ElasticsearchException) GetResponse(org.elasticsearch.action.get.GetResponse) UpdateInfo(org.bedework.util.elasticsearch.DocBuilderBase.UpdateInfo) GetRequestBuilder(org.elasticsearch.action.get.GetRequestBuilder)

Example 90 with ElasticsearchException

use of org.elasticsearch.ElasticsearchException in project bw-calendar-engine by Bedework.

the class BwIndexEsImpl method markTransaction.

@Override
public void markTransaction() throws CalFacadeException {
    final UpdateInfo ui = updateInfo.get(targetIndex);
    if ((ui != null) && !ui.isUpdate()) {
        return;
    }
    try {
        final UpdateRequestBuilder urb = getClient().prepareUpdate(targetIndex, docTypeUpdateTracker, updateTrackerId).setRetryOnConflict(20).setRefresh(true);
        urb.setScript("ctx._source.count += 1", ScriptService.ScriptType.INLINE);
        final UpdateResponse ur = urb.execute().actionGet();
    } catch (final ElasticsearchException ese) {
        warn("Exception updating UpdateInfo: " + ese.getLocalizedMessage());
        index(new UpdateInfo());
    }
}
Also used : UpdateResponse(org.elasticsearch.action.update.UpdateResponse) UpdateRequestBuilder(org.elasticsearch.action.update.UpdateRequestBuilder) ElasticsearchException(org.elasticsearch.ElasticsearchException) UpdateInfo(org.bedework.util.elasticsearch.DocBuilderBase.UpdateInfo)

Aggregations

ElasticsearchException (org.elasticsearch.ElasticsearchException)309 IOException (java.io.IOException)127 Settings (org.elasticsearch.common.settings.Settings)32 XContentBuilder (org.elasticsearch.common.xcontent.XContentBuilder)30 HashMap (java.util.HashMap)29 ClusterState (org.elasticsearch.cluster.ClusterState)29 ArrayList (java.util.ArrayList)28 Matchers.containsString (org.hamcrest.Matchers.containsString)25 List (java.util.List)20 Map (java.util.Map)20 AtomicReference (java.util.concurrent.atomic.AtomicReference)20 ParameterizedMessage (org.apache.logging.log4j.message.ParameterizedMessage)18 XContentParser (org.elasticsearch.common.xcontent.XContentParser)17 Path (java.nio.file.Path)16 Test (org.junit.Test)16 ActionListener (org.elasticsearch.action.ActionListener)15 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)14 Version (org.elasticsearch.Version)14 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)13 ResourceNotFoundException (org.elasticsearch.ResourceNotFoundException)13