Search in sources :

Example 6 with FessSystemException

use of org.codelibs.fess.exception.FessSystemException in project fess by codelibs.

the class ViewHelper method getCachedFacetResponse.

public FacetResponse getCachedFacetResponse(final String query) {
    final OptionalThing<FessUserBean> userBean = ComponentUtil.getComponent(FessLoginAssist.class).getSavedUserBean();
    final String permissionKey = userBean.map(user -> StreamUtil.stream(user.getPermissions()).get(stream -> stream.sorted().distinct().collect(Collectors.joining("\n")))).orElse(StringUtil.EMPTY);
    try {
        return facetCache.get(query + "\n" + permissionKey, () -> {
            final SearchHelper searchHelper = ComponentUtil.getSearchHelper();
            final SearchForm params = new SearchForm() {

                @Override
                public int getPageSize() {
                    return 0;
                }

                @Override
                public int getStartPosition() {
                    return 0;
                }
            };
            params.q = query;
            final SearchRenderData data = new SearchRenderData();
            searchHelper.search(params, data, userBean);
            if (logger.isDebugEnabled()) {
                logger.debug("loaded facet data: {}", data);
            }
            return data.getFacetResponse();
        });
    } catch (final ExecutionException e) {
        throw new FessSystemException("Cannot load facet from cache.", e);
    }
}
Also used : Constants(org.codelibs.fess.Constants) BufferedInputStream(java.io.BufferedInputStream) URLDecoder(java.net.URLDecoder) Date(java.util.Date) OptionalThing(org.dbflute.optional.OptionalThing) StringUtils(org.apache.commons.lang3.StringUtils) ActionRuntime(org.lastaflute.web.ruts.process.ActionRuntime) LaResponseUtil(org.lastaflute.web.util.LaResponseUtil) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig) StreamResponse(org.lastaflute.web.response.StreamResponse) StreamUtil.split(org.codelibs.core.stream.StreamUtil.split) Matcher(java.util.regex.Matcher) StreamUtil(org.codelibs.core.stream.StreamUtil) Locale(java.util.Locale) Map(java.util.Map) LaFunctions(org.lastaflute.taglib.function.LaFunctions) SearchRenderData(org.codelibs.fess.entity.SearchRenderData) HttpSession(javax.servlet.http.HttpSession) Context(com.github.jknack.handlebars.Context) FacetQueryView(org.codelibs.fess.entity.FacetQueryView) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Set(java.util.Set) CoreLibConstants(org.codelibs.core.CoreLibConstants) Collectors(java.util.stream.Collectors) ActionResponse(org.lastaflute.web.response.ActionResponse) List(java.util.List) Logger(org.apache.logging.log4j.Logger) ComponentUtil(org.codelibs.fess.util.ComponentUtil) PostConstruct(javax.annotation.PostConstruct) LaServletContextUtil(org.lastaflute.web.util.LaServletContextUtil) FileTemplateLoader(com.github.jknack.handlebars.io.FileTemplateLoader) Pattern(java.util.regex.Pattern) CacheBuilder(com.google.common.cache.CacheBuilder) UnsupportedEncodingException(java.io.UnsupportedEncodingException) ResponseData(org.codelibs.fess.crawler.entity.ResponseData) DocumentUtil(org.codelibs.fess.util.DocumentUtil) FessSystemException(org.codelibs.fess.exception.FessSystemException) HighlightField(org.opensearch.search.fetch.subphase.highlight.HighlightField) HashMap(java.util.HashMap) ClientAbortException(org.apache.catalina.connector.ClientAbortException) Function(java.util.function.Function) CrawlingConfig(org.codelibs.fess.es.config.exentity.CrawlingConfig) LaRequestUtil(org.lastaflute.web.util.LaRequestUtil) ArrayList(java.util.ArrayList) CrawlerClient(org.codelibs.fess.crawler.client.CrawlerClient) UserAgentType(org.codelibs.fess.helper.UserAgentHelper.UserAgentType) HashSet(java.util.HashSet) FessUserBean(org.codelibs.fess.mylasta.action.FessUserBean) ResourceUtil(org.codelibs.fess.util.ResourceUtil) HttpServletRequest(javax.servlet.http.HttpServletRequest) Handlebars(com.github.jknack.handlebars.Handlebars) CharUtil(org.codelibs.fess.crawler.util.CharUtil) DynamicProperties(org.codelibs.core.misc.DynamicProperties) StringUtil(org.codelibs.core.lang.StringUtil) IOException(java.io.IOException) StringEscapeUtils(org.apache.commons.text.StringEscapeUtils) FacetResponse(org.codelibs.fess.util.FacetResponse) SessionTrackingMode(javax.servlet.SessionTrackingMode) File(java.io.File) CloseableUtil(org.codelibs.core.io.CloseableUtil) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) Consumer(java.util.function.Consumer) SearchForm(org.codelibs.fess.app.web.base.SearchForm) URLEncoder(java.net.URLEncoder) FessLoginAssist(org.codelibs.fess.app.web.base.login.FessLoginAssist) ServletContext(javax.servlet.ServletContext) HighlightInfo(org.codelibs.fess.entity.HighlightInfo) SimpleDateFormat(com.ibm.icu.text.SimpleDateFormat) Cache(com.google.common.cache.Cache) LogManager(org.apache.logging.log4j.LogManager) RequestDataBuilder(org.codelibs.fess.crawler.builder.RequestDataBuilder) CrawlerClientFactory(org.codelibs.fess.crawler.client.CrawlerClientFactory) Text(org.opensearch.common.text.Text) InputStream(java.io.InputStream) Template(com.github.jknack.handlebars.Template) FessUserBean(org.codelibs.fess.mylasta.action.FessUserBean) SearchRenderData(org.codelibs.fess.entity.SearchRenderData) ExecutionException(java.util.concurrent.ExecutionException) SearchForm(org.codelibs.fess.app.web.base.SearchForm) FessLoginAssist(org.codelibs.fess.app.web.base.login.FessLoginAssist) FessSystemException(org.codelibs.fess.exception.FessSystemException)

Example 7 with FessSystemException

use of org.codelibs.fess.exception.FessSystemException in project fess by codelibs.

the class ViewHelper method writeContent.

protected StreamResponse writeContent(final String configId, final String url, final CrawlerClient client) {
    final StreamResponse response = new StreamResponse(StringUtil.EMPTY);
    final ResponseData responseData = client.execute(RequestDataBuilder.newRequestData().get().url(url).build());
    if (responseData.getHttpStatusCode() == 404) {
        response.httpStatus(responseData.getHttpStatusCode());
        CloseableUtil.closeQuietly(responseData);
        return response;
    }
    writeFileName(response, responseData);
    writeContentType(response, responseData);
    writeNoCache(response, responseData);
    response.stream(out -> {
        try (final InputStream is = new BufferedInputStream(responseData.getResponseBody())) {
            out.write(is);
        } catch (final IOException e) {
            if (!(e.getCause() instanceof ClientAbortException)) {
                throw new FessSystemException("Failed to write a content. configId: " + configId + ", url: " + url, e);
            }
        } finally {
            CloseableUtil.closeQuietly(responseData);
        }
        if (logger.isDebugEnabled()) {
            logger.debug("Finished to write {}", url);
        }
    });
    return response;
}
Also used : BufferedInputStream(java.io.BufferedInputStream) BufferedInputStream(java.io.BufferedInputStream) InputStream(java.io.InputStream) StreamResponse(org.lastaflute.web.response.StreamResponse) ResponseData(org.codelibs.fess.crawler.entity.ResponseData) ClientAbortException(org.apache.catalina.connector.ClientAbortException) IOException(java.io.IOException) FessSystemException(org.codelibs.fess.exception.FessSystemException)

Example 8 with FessSystemException

use of org.codelibs.fess.exception.FessSystemException in project fess by codelibs.

the class CrawlingInfoHelper method store.

public synchronized void store(final String sessionId, final boolean create) {
    CrawlingInfo crawlingInfo = create ? null : getCrawlingInfoService().getLast(sessionId);
    if (crawlingInfo == null) {
        crawlingInfo = new CrawlingInfo(sessionId);
        try {
            getCrawlingInfoService().store(crawlingInfo);
        } catch (final Exception e) {
            throw new FessSystemException("No crawling session.", e);
        }
    }
    if (infoMap != null) {
        final List<CrawlingInfoParam> crawlingInfoParamList = new ArrayList<>();
        for (final Map.Entry<String, String> entry : infoMap.entrySet()) {
            final CrawlingInfoParam crawlingInfoParam = new CrawlingInfoParam();
            crawlingInfoParam.setCrawlingInfoId(crawlingInfo.getId());
            crawlingInfoParam.setKey(entry.getKey());
            crawlingInfoParam.setValue(entry.getValue());
            crawlingInfoParamList.add(crawlingInfoParam);
        }
        getCrawlingInfoService().storeInfo(crawlingInfoParamList);
    }
    infoMap = null;
}
Also used : CrawlingInfo(org.codelibs.fess.es.config.exentity.CrawlingInfo) ArrayList(java.util.ArrayList) HashMap(java.util.HashMap) LinkedHashMap(java.util.LinkedHashMap) Map(java.util.Map) FessSystemException(org.codelibs.fess.exception.FessSystemException) UnsupportedEncodingException(java.io.UnsupportedEncodingException) CrawlingInfoParam(org.codelibs.fess.es.config.exentity.CrawlingInfoParam) FessSystemException(org.codelibs.fess.exception.FessSystemException)

Example 9 with FessSystemException

use of org.codelibs.fess.exception.FessSystemException in project fess by codelibs.

the class ProcessHelper method startProcess.

public synchronized JobProcess startProcess(final String sessionId, final List<String> cmdList, final Consumer<ProcessBuilder> pbCall) {
    final ProcessBuilder pb = new ProcessBuilder(cmdList);
    pbCall.accept(pb);
    destroyProcess(sessionId);
    JobProcess jobProcess;
    try {
        jobProcess = new JobProcess(pb.start());
        destroyProcess(sessionId, runningProcessMap.putIfAbsent(sessionId, jobProcess));
        return jobProcess;
    } catch (final IOException e) {
        throw new FessSystemException("Crawler Process terminated.", e);
    }
}
Also used : IOException(java.io.IOException) JobProcess(org.codelibs.fess.util.JobProcess) FessSystemException(org.codelibs.fess.exception.FessSystemException)

Example 10 with FessSystemException

use of org.codelibs.fess.exception.FessSystemException in project fess by codelibs.

the class ViewHelper method asContentResponse.

public StreamResponse asContentResponse(final Map<String, Object> doc) {
    if (logger.isDebugEnabled()) {
        logger.debug("writing the content of: {}", doc);
    }
    final FessConfig fessConfig = ComponentUtil.getFessConfig();
    final CrawlingConfigHelper crawlingConfigHelper = ComponentUtil.getCrawlingConfigHelper();
    final String configId = DocumentUtil.getValue(doc, fessConfig.getIndexFieldConfigId(), String.class);
    if (configId == null) {
        throw new FessSystemException("configId is null.");
    }
    if (configId.length() < 2) {
        throw new FessSystemException("Invalid configId: " + configId);
    }
    final CrawlingConfig config = crawlingConfigHelper.getCrawlingConfig(configId);
    if (config == null) {
        throw new FessSystemException("No crawlingConfig: " + configId);
    }
    final String url = DocumentUtil.getValue(doc, fessConfig.getIndexFieldUrl(), String.class);
    final CrawlerClientFactory crawlerClientFactory = config.initializeClientFactory(() -> ComponentUtil.getComponent(CrawlerClientFactory.class));
    final CrawlerClient client = crawlerClientFactory.getClient(url);
    if (client == null) {
        throw new FessSystemException("No CrawlerClient: " + configId + ", url: " + url);
    }
    return writeContent(configId, url, client);
}
Also used : CrawlingConfig(org.codelibs.fess.es.config.exentity.CrawlingConfig) CrawlerClientFactory(org.codelibs.fess.crawler.client.CrawlerClientFactory) CrawlerClient(org.codelibs.fess.crawler.client.CrawlerClient) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig) FessSystemException(org.codelibs.fess.exception.FessSystemException)

Aggregations

FessSystemException (org.codelibs.fess.exception.FessSystemException)23 UnsupportedEncodingException (java.io.UnsupportedEncodingException)7 IOException (java.io.IOException)6 File (java.io.File)5 ArrayList (java.util.ArrayList)5 Execute (org.lastaflute.web.Execute)5 BufferedReader (java.io.BufferedReader)4 InputStream (java.io.InputStream)4 InputStreamReader (java.io.InputStreamReader)4 Reader (java.io.Reader)4 Map (java.util.Map)4 PostConstruct (javax.annotation.PostConstruct)4 HashMap (java.util.HashMap)3 Secured (org.codelibs.fess.annotation.Secured)3 BufferedInputStream (java.io.BufferedInputStream)2 Path (java.nio.file.Path)2 Date (java.util.Date)2 List (java.util.List)2 ExecutionException (java.util.concurrent.ExecutionException)2 Consumer (java.util.function.Consumer)2