Search in sources :

Example 1 with MetadataFeature

use of org.apache.sdap.mudrod.recommendation.structure.MetadataFeature in project incubator-sdap-mudrod by apache.

the class NormalizeFeatures method normalizeMetadataVariables.

public void normalizeMetadataVariables(ESDriver es) {
    es.createBulkProcessor();
    SearchResponse scrollResp = es.getClient().prepareSearch(indexName).setTypes(metadataType).setScroll(new TimeValue(60000)).setQuery(QueryBuilders.matchAllQuery()).setSize(100).execute().actionGet();
    while (true) {
        for (SearchHit hit : scrollResp.getHits().getHits()) {
            Map<String, Object> metadata = hit.getSource();
            Map<String, Object> updatedValues = new HashMap<>();
            // !!!important change to other normalizer class when using other metadata
            MetadataFeature normalizer = new PODAACMetadataFeature();
            normalizer.normalizeMetadataVariables(metadata, updatedValues);
            UpdateRequest ur = es.generateUpdateRequest(indexName, metadataType, hit.getId(), updatedValues);
            es.getBulkProcessor().add(ur);
        }
        scrollResp = es.getClient().prepareSearchScroll(scrollResp.getScrollId()).setScroll(new TimeValue(600000)).execute().actionGet();
        if (scrollResp.getHits().getHits().length == 0) {
            break;
        }
    }
    es.destroyBulkProcessor();
}
Also used : MetadataFeature(org.apache.sdap.mudrod.recommendation.structure.MetadataFeature) PODAACMetadataFeature(org.apache.sdap.mudrod.recommendation.structure.PODAACMetadataFeature) SearchHit(org.elasticsearch.search.SearchHit) HashMap(java.util.HashMap) UpdateRequest(org.elasticsearch.action.update.UpdateRequest) PODAACMetadataFeature(org.apache.sdap.mudrod.recommendation.structure.PODAACMetadataFeature) TimeValue(org.elasticsearch.common.unit.TimeValue) SearchResponse(org.elasticsearch.action.search.SearchResponse)

Aggregations

HashMap (java.util.HashMap)1 MetadataFeature (org.apache.sdap.mudrod.recommendation.structure.MetadataFeature)1 PODAACMetadataFeature (org.apache.sdap.mudrod.recommendation.structure.PODAACMetadataFeature)1 SearchResponse (org.elasticsearch.action.search.SearchResponse)1 UpdateRequest (org.elasticsearch.action.update.UpdateRequest)1 TimeValue (org.elasticsearch.common.unit.TimeValue)1 SearchHit (org.elasticsearch.search.SearchHit)1