Search in sources :

Example 6 with GetAliasesResponse

use of org.elasticsearch.client.GetAliasesResponse in project herd by FINRAOS.

the class IndexFunctionsDaoTest method testDeleteIndexDocumentsFunctionThrowsElasticsearchRestClientException.

@Test(expected = ElasticsearchRestClientException.class)
public void testDeleteIndexDocumentsFunctionThrowsElasticsearchRestClientException() throws Exception {
    // Build mocks
    GetAliasesResponse getAliasesResponse = mock(GetAliasesResponse.class);
    IndicesClient indicesClient = mock(IndicesClient.class);
    RestHighLevelClient restHighLevelClient = mock(RestHighLevelClient.class);
    AliasMetadata AliasMetadata = mock(AliasMetadata.class);
    // Create objects needed for the test.
    Set<AliasMetadata> AliasMetadataSet = new HashSet<>();
    AliasMetadataSet.add(AliasMetadata);
    Map<String, Set<AliasMetadata>> aliases = new HashMap<>();
    aliases.put(SEARCH_INDEX_ALIAS_BDEF, AliasMetadataSet);
    List<Long> businessObjectDefinitionIds = new ArrayList<>();
    businessObjectDefinitionIds.add(SEARCH_INDEX_BUSINESS_OBJECT_DEFINITION_DOCUMENT_ID);
    // Mock the calls to external methods
    when(elasticsearchRestHighLevelClientFactory.getRestHighLevelClient()).thenReturn(restHighLevelClient);
    when(restHighLevelClient.indices()).thenReturn(indicesClient);
    when(indicesClient.getAlias(any(GetAliasesRequest.class), eq(RequestOptions.DEFAULT))).thenReturn(getAliasesResponse);
    when(getAliasesResponse.getAliases()).thenReturn(aliases);
    when(restHighLevelClient.bulk(any(BulkRequest.class), eq(RequestOptions.DEFAULT))).thenThrow(new IOException());
    // Call the method under test
    indexFunctionsDao.deleteIndexDocuments(SEARCH_INDEX_NAME, businessObjectDefinitionIds);
}
Also used : AliasMetadata(org.elasticsearch.cluster.metadata.AliasMetadata) Set(java.util.Set) HashSet(java.util.HashSet) GetAliasesResponse(org.elasticsearch.client.GetAliasesResponse) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) IndicesClient(org.elasticsearch.client.IndicesClient) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) IOException(java.io.IOException) BulkRequest(org.elasticsearch.action.bulk.BulkRequest) GetAliasesRequest(org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 7 with GetAliasesResponse

use of org.elasticsearch.client.GetAliasesResponse in project search by craftercms.

the class ElasticsearchAdminServiceImpl method doDeleteIndexes.

/**
 * Performs the index delete using the given Elasticsearch client
 */
protected void doDeleteIndexes(RestHighLevelClient client, String aliasName) {
    try {
        GetAliasesResponse indices = client.indices().getAlias(new GetAliasesRequest(aliasName), RequestOptions.DEFAULT);
        Set<String> actualIndices = indices.getAliases().keySet();
        logger.info("Deleting indices {}", actualIndices);
        client.indices().delete(new DeleteIndexRequest(actualIndices.toArray(new String[] {})), RequestOptions.DEFAULT);
    } catch (IOException e) {
        throw new ElasticsearchException(aliasName, "Error deleting index " + aliasName, e);
    }
}
Also used : GetAliasesResponse(org.elasticsearch.client.GetAliasesResponse) DeleteIndexRequest(org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest) GetAliasesRequest(org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest) IOException(java.io.IOException) ElasticsearchException(org.craftercms.search.elasticsearch.exception.ElasticsearchException)

Example 8 with GetAliasesResponse

use of org.elasticsearch.client.GetAliasesResponse in project ccd-definition-store-api by hmcts.

the class HighLevelCCDElasticClient method upsertMapping.

@Override
public boolean upsertMapping(String aliasName, String caseTypeMapping) throws IOException {
    log.info("upsert mapping of most recent index for alias {}", aliasName);
    GetAliasesResponse aliasesResponse = getAlias(aliasName);
    String currentIndex = getCurrentAliasIndex(aliasName, aliasesResponse);
    log.info("upsert mapping of index {}", currentIndex);
    PutMappingRequest request = new PutMappingRequest(currentIndex);
    request.type(config.getCasesIndexType());
    request.source(caseTypeMapping, XContentType.JSON);
    AcknowledgedResponse acknowledgedResponse = elasticClient.indices().putMapping(request, RequestOptions.DEFAULT);
    log.info("mapping upserted: {}", acknowledgedResponse.isAcknowledged());
    return acknowledgedResponse.isAcknowledged();
}
Also used : GetAliasesResponse(org.elasticsearch.client.GetAliasesResponse) PutMappingRequest(org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest) AcknowledgedResponse(org.elasticsearch.action.support.master.AcknowledgedResponse)

Example 9 with GetAliasesResponse

use of org.elasticsearch.client.GetAliasesResponse in project open-commerce-search by CommerceExperts.

the class ElasticsearchIndexClient method getAliases.

/**
 * Get actual index names with potential aliases.
 *
 * @param indexNames
 * @return
 */
public Map<String, Set<AliasMetadata>> getAliases(String indexName) {
    try {
        GetAliasesRequest getAliasesRequest = new GetAliasesRequest().indices(indexName);
        GetAliasesResponse response = highLevelClient.indices().getAlias(getAliasesRequest, RequestOptions.DEFAULT);
        return response.getAliases();
    } catch (IOException e) {
        log.info("Could not get alias for index {}", indexName);
    }
    return Collections.emptyMap();
}
Also used : GetAliasesResponse(org.elasticsearch.client.GetAliasesResponse) GetAliasesRequest(org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest) IOException(java.io.IOException)

Example 10 with GetAliasesResponse

use of org.elasticsearch.client.GetAliasesResponse in project hopsworks by logicalclocks.

the class PyPiLibraryElasticIndexer method execute.

@Timeout
@TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)
public void execute(Timer timer) throws ElasticException {
    long errorRescheduleTimeout = 600000;
    LOGGER.log(Level.INFO, "Running PyPi Indexer");
    try {
        ClusterHealthResponse clusterHealthResponse = elasticClientCtrl.clusterHealthGet();
        GetIndexTemplatesResponse templateResponse = elasticClientCtrl.templateGet(Settings.ELASTIC_PYPI_LIBRARIES_ALIAS);
        // If elasticsearch is down or template not in elastic reschedule the timer
        if (clusterHealthResponse.getStatus().equals(ClusterHealthStatus.RED)) {
            scheduleTimer(errorRescheduleTimeout);
            LOGGER.log(Level.INFO, "Elastic currently down, rescheduling indexing for pypi libraries");
            return;
        } else if (templateResponse.getIndexTemplates().isEmpty()) {
            scheduleTimer(errorRescheduleTimeout);
            LOGGER.log(Level.INFO, "Elastic template " + Settings.ELASTIC_PYPI_LIBRARIES_ALIAS + " currently missing, " + "rescheduling indexing for pypi libraries");
            return;
        }
    } catch (Exception e) {
        scheduleTimer(errorRescheduleTimeout);
        LOGGER.log(Level.SEVERE, "Exception occurred trying to index pypi libraries, rescheduling timer", e);
        return;
    }
    String newIndex = Settings.ELASTIC_PYPI_LIBRARIES_INDEX_PATTERN_PREFIX + System.currentTimeMillis();
    try {
        GetAliasesResponse pypiAlias = elasticClientCtrl.getAliases(Settings.ELASTIC_PYPI_LIBRARIES_ALIAS);
        if (!pypiAlias.getAliases().isEmpty()) {
            this.isIndexed = true;
        }
        String[] indicesToDelete = elasticClientCtrl.mngIndicesGetBySimplifiedRegex(Settings.ELASTIC_PYPI_LIBRARIES_INDEX_REGEX);
        Element body = Jsoup.connect(settings.getPyPiSimpleEndpoint()).maxBodySize(0).get().body();
        Elements elements = body.getElementsByTag("a");
        CreateIndexRequest createIndexRequest = new CreateIndexRequest(newIndex);
        elasticClientCtrl.mngIndexCreate(createIndexRequest);
        final int bulkSize = 100;
        int currentBulkSize = 0;
        int currentId = 0;
        BulkRequest bulkRequest = new BulkRequest();
        LOGGER.log(Level.INFO, "Starting to index libraries from pypi simple index");
        for (Element library : elements) {
            IndexRequest indexRequest = new IndexRequest().index(newIndex).id(String.valueOf(currentId)).source(jsonBuilder().startObject().field("library", library.text()).endObject());
            bulkRequest.add(indexRequest);
            currentBulkSize += 1;
            currentId += 1;
            if (currentBulkSize == bulkSize) {
                elasticClientCtrl.bulkUpdateDoc(bulkRequest);
                bulkRequest = new BulkRequest();
                currentBulkSize = 0;
            }
        }
        // Also send last batch
        if (bulkRequest.numberOfActions() > 0) {
            elasticClientCtrl.bulkUpdateDoc(bulkRequest);
        }
        if (pypiAlias.getAliases().isEmpty()) {
            elasticClientCtrl.createAlias(Settings.ELASTIC_PYPI_LIBRARIES_ALIAS, newIndex);
        } else {
            String currentSearchIndex = pypiAlias.getAliases().keySet().iterator().next();
            elasticClientCtrl.aliasSwitchIndex(Settings.ELASTIC_PYPI_LIBRARIES_ALIAS, currentSearchIndex, newIndex);
        }
        this.isIndexed = true;
        LOGGER.log(Level.INFO, "Finished indexing");
        for (String index : indicesToDelete) {
            DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest().indices(index);
            elasticClientCtrl.mngIndexDelete(deleteIndexRequest);
        }
    } catch (Exception ex) {
        LOGGER.log(Level.SEVERE, "Indexing pypi libraries failed", ex);
        scheduleTimer(errorRescheduleTimeout);
        if (elasticClientCtrl.mngIndexExists(newIndex)) {
            DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest().indices(newIndex);
            elasticClientCtrl.mngIndexDelete(deleteIndexRequest);
        }
        return;
    }
    String rawInterval = settings.getPyPiIndexerTimerInterval();
    Long intervalValue = settings.getConfTimeValue(rawInterval);
    TimeUnit intervalTimeunit = settings.getConfTimeTimeUnit(rawInterval);
    scheduleTimer(intervalTimeunit.toMillis(intervalValue));
}
Also used : ClusterHealthResponse(org.elasticsearch.action.admin.cluster.health.ClusterHealthResponse) GetAliasesResponse(org.elasticsearch.client.GetAliasesResponse) Element(org.jsoup.nodes.Element) DeleteIndexRequest(org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest) Elements(org.jsoup.select.Elements) IndexRequest(org.elasticsearch.action.index.IndexRequest) DeleteIndexRequest(org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest) CreateIndexRequest(org.elasticsearch.client.indices.CreateIndexRequest) ElasticException(io.hops.hopsworks.exceptions.ElasticException) GetIndexTemplatesResponse(org.elasticsearch.client.indices.GetIndexTemplatesResponse) BulkRequest(org.elasticsearch.action.bulk.BulkRequest) TimeUnit(java.util.concurrent.TimeUnit) CreateIndexRequest(org.elasticsearch.client.indices.CreateIndexRequest) TransactionAttribute(javax.ejb.TransactionAttribute) Timeout(javax.ejb.Timeout)

Aggregations

GetAliasesResponse (org.elasticsearch.client.GetAliasesResponse)18 GetAliasesRequest (org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest)15 Set (java.util.Set)13 HashMap (java.util.HashMap)10 BulkRequest (org.elasticsearch.action.bulk.BulkRequest)10 IndicesClient (org.elasticsearch.client.IndicesClient)10 RestHighLevelClient (org.elasticsearch.client.RestHighLevelClient)10 HashSet (java.util.HashSet)9 AliasMetadata (org.elasticsearch.cluster.metadata.AliasMetadata)9 Test (org.junit.Test)9 IOException (java.io.IOException)7 BulkResponse (org.elasticsearch.action.bulk.BulkResponse)5 ArrayList (java.util.ArrayList)4 DeleteIndexRequest (org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest)4 AcknowledgedResponse (org.elasticsearch.action.support.master.AcknowledgedResponse)3 ElasticException (io.hops.hopsworks.exceptions.ElasticException)2 IndicesAliasesRequest (org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest)2 GetAliasesRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest)2 GetAliasesResponse (org.graylog.shaded.elasticsearch7.org.elasticsearch.client.GetAliasesResponse)2 JsonNode (com.fasterxml.jackson.databind.JsonNode)1