Examples with IndexingHelper org.codelibs.fess.helper.IndexingHelper used on opensource projects

Example 1 with IndexingHelper

use of org.codelibs.fess.helper.IndexingHelper in project fess by codelibs.

the class FessCrawlerThread method getChildUrlSet.

protected Set<RequestData> getChildUrlSet(final FessEsClient fessEsClient, final String id) {
    final FessConfig fessConfig = ComponentUtil.getFessConfig();
    final IndexingHelper indexingHelper = ComponentUtil.getIndexingHelper();
    final List<Map<String, Object>> docList = indexingHelper.getChildDocumentList(fessEsClient, id, new String[] { fessConfig.getIndexFieldUrl() });
    if (docList.isEmpty()) {
        return null;
    }
    if (logger.isDebugEnabled()) {
        logger.debug("Found documents: " + docList);
    }
    final Set<RequestData> urlSet = new HashSet<>(docList.size());
    for (final Map<String, Object> doc : docList) {
        final String url = DocumentUtil.getValue(doc, fessConfig.getIndexFieldUrl(), String.class);
        if (StringUtil.isNotBlank(url)) {
            urlSet.add(RequestDataBuilder.newRequestData().get().url(url).build());
        }
    }
    return urlSet;
}

Example 2 with IndexingHelper

use of org.codelibs.fess.helper.IndexingHelper in project fess by codelibs.

the class FileListIndexUpdateCallbackImpl method deleteDocuments.

protected void deleteDocuments() {
    final FessEsClient fessEsClient = ComponentUtil.getFessEsClient();
    final IndexingHelper indexingHelper = ComponentUtil.getIndexingHelper();
    for (final String url : deleteUrlList) {
        indexingHelper.deleteDocumentByUrl(fessEsClient, url);
    }
    if (logger.isDebugEnabled()) {
        logger.debug("Deleted " + deleteUrlList);
    }
    deleteUrlList.clear();
}

Example 3 with IndexingHelper

use of org.codelibs.fess.helper.IndexingHelper in project fess by codelibs.

the class IndexUpdateCallbackImpl method commit.

@Override
public void commit() {
    synchronized (docList) {
        if (!docList.isEmpty()) {
            final IndexingHelper indexingHelper = ComponentUtil.getIndexingHelper();
            final FessEsClient fessEsClient = ComponentUtil.getFessEsClient();
            indexingHelper.sendDocuments(fessEsClient, docList);
        }
    }
}

Example 4 with IndexingHelper

use of org.codelibs.fess.helper.IndexingHelper in project fess by codelibs.

the class FileListIndexUpdateCallbackImpl method deleteDocument.

protected boolean deleteDocument(final Map<String, String> paramMap, final Map<String, Object> dataMap) {
    if (logger.isDebugEnabled()) {
        logger.debug("Deleting {}", dataMap);
    }
    final FessConfig fessConfig = ComponentUtil.getFessConfig();
    // required check
    if (!dataMap.containsKey(fessConfig.getIndexFieldUrl()) || dataMap.get(fessConfig.getIndexFieldUrl()) == null) {
        logger.warn("Could not delete a doc. Invalid data: {}", dataMap);
        return false;
    }
    synchronized (indexUpdateCallback) {
        final long maxAccessCount = getMaxAccessCount(paramMap, dataMap);
        final String url = dataMap.get(fessConfig.getIndexFieldUrl()).toString();
        if (maxAccessCount != 1L) {
            final SearchEngineClient searchEngineClient = ComponentUtil.getSearchEngineClient();
            final IndexingHelper indexingHelper = ComponentUtil.getIndexingHelper();
            final long count = indexingHelper.deleteDocumentByQuery(searchEngineClient, QueryBuilders.prefixQuery(fessConfig.getIndexFieldUrl(), url));
            if (logger.isDebugEnabled()) {
                logger.debug("Deleted {} docs for {}*", count, url);
            }
        } else {
            deleteUrlList.add(url);
            if (deleteUrlList.size() >= maxDeleteDocumentCacheSize) {
                deleteDocuments();
            }
        }
    }
    return true;
}

Example 5 with IndexingHelper

use of org.codelibs.fess.helper.IndexingHelper in project fess by codelibs.

the class IndexUpdateCallbackImpl method commit.

@Override
public void commit() {
    synchronized (docList) {
        if (!docList.isEmpty()) {
            final IndexingHelper indexingHelper = ComponentUtil.getIndexingHelper();
            final SearchEngineClient searchEngineClient = ComponentUtil.getSearchEngineClient();
            indexingHelper.sendDocuments(searchEngineClient, docList);
        }
    }
}