Search in sources :

Example 1 with StringUtil

use of org.codelibs.core.lang.StringUtil in project fess by codelibs.

the class QueryStringBuilder method appendConditions.

protected void appendConditions(final StringBuilder queryBuf, final Map<String, String[]> conditions) {
    if (conditions == null) {
        return;
    }
    final FessConfig fessConfig = ComponentUtil.getFessConfig();
    final int maxQueryLength = fessConfig.getQueryMaxLengthAsInteger();
    stream(conditions.get(SearchRequestParams.AS_OCCURRENCE)).of(stream -> stream.filter(this::isOccurrence).findFirst().ifPresent(q -> queryBuf.insert(0, q + ":")));
    stream(conditions.get(SearchRequestParams.AS_Q)).of(stream -> stream.filter(q -> StringUtil.isNotBlank(q) && q.length() <= maxQueryLength).forEach(q -> queryBuf.append(' ').append(q)));
    stream(conditions.get(SearchRequestParams.AS_EPQ)).of(stream -> stream.filter(q -> StringUtil.isNotBlank(q) && q.length() <= maxQueryLength).forEach(q -> queryBuf.append(" \"").append(escape(q, "\"")).append('"')));
    stream(conditions.get(SearchRequestParams.AS_OQ)).of(stream -> stream.filter(q -> StringUtil.isNotBlank(q) && q.length() <= maxQueryLength).forEach(oq -> split(oq, " ").get(s -> s.filter(StringUtil::isNotBlank).reduce((q1, q2) -> escape(q1, "(", ")") + OR + escape(q2, "(", ")"))).ifPresent(q -> {
        appendQuery(queryBuf, q);
    })));
    stream(conditions.get(SearchRequestParams.AS_NQ)).of(stream -> stream.filter(q -> StringUtil.isNotBlank(q) && q.length() <= maxQueryLength).forEach(eq -> {
        final String nq = split(eq, " ").get(s -> s.filter(StringUtil::isNotBlank).map(q -> "NOT " + q).collect(Collectors.joining(" ")));
        queryBuf.append(' ').append(nq);
    }));
    stream(conditions.get(SearchRequestParams.AS_FILETYPE)).of(stream -> stream.filter(q -> StringUtil.isNotBlank(q) && q.length() <= maxQueryLength).forEach(q -> queryBuf.append(" filetype:\"").append(q.trim()).append('"')));
    stream(conditions.get(SearchRequestParams.AS_SITESEARCH)).of(stream -> stream.filter(q -> StringUtil.isNotBlank(q) && q.length() <= maxQueryLength).forEach(q -> queryBuf.append(" site:").append(q.trim())));
    stream(conditions.get(SearchRequestParams.AS_TIMESTAMP)).of(stream -> stream.filter(q -> StringUtil.isNotBlank(q) && q.length() <= maxQueryLength).forEach(q -> queryBuf.append(" timestamp:").append(q.trim())));
}
Also used : Constants(org.codelibs.fess.Constants) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig) StreamUtil.split(org.codelibs.core.stream.StreamUtil.split) StreamUtil.stream(org.codelibs.core.stream.StreamUtil.stream) SearchRequestParams(org.codelibs.fess.entity.SearchRequestParams) Map(java.util.Map) StringUtil(org.codelibs.core.lang.StringUtil) Collectors(java.util.stream.Collectors) RelatedQueryHelper(org.codelibs.fess.helper.RelatedQueryHelper) StringUtil(org.codelibs.core.lang.StringUtil) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig)

Example 2 with StringUtil

use of org.codelibs.core.lang.StringUtil in project fess by codelibs.

the class ApiAdminElevatewordAction method createEditBody.

protected EditBody createEditBody(final ElevateWord entity) {
    final EditBody body = new EditBody();
    copyBeanToBean(entity, body, copyOp -> {
        copyOp.excludeNull();
        copyOp.exclude(Constants.PERMISSIONS);
    });
    final PermissionHelper permissionHelper = ComponentUtil.getPermissionHelper();
    body.permissions = stream(entity.getPermissions()).get(stream -> stream.map(s -> permissionHelper.decode(s)).filter(StringUtil::isNotBlank).distinct().collect(Collectors.joining("\n")));
    return body;
}
Also used : Constants(org.codelibs.fess.Constants) FessSystemException(org.codelibs.fess.exception.FessSystemException) ElevateWordPager(org.codelibs.fess.app.pager.ElevateWordPager) PermissionHelper(org.codelibs.fess.helper.PermissionHelper) ApiResult(org.codelibs.fess.app.web.api.ApiResult) Status(org.codelibs.fess.app.web.api.ApiResult.Status) StreamResponse(org.lastaflute.web.response.StreamResponse) CrudMode(org.codelibs.fess.app.web.CrudMode) CommonPoolUtil(org.codelibs.core.concurrent.CommonPoolUtil) OutputStreamWriter(java.io.OutputStreamWriter) Path(java.nio.file.Path) StreamUtil.stream(org.codelibs.core.stream.StreamUtil.stream) ElevateWordService(org.codelibs.fess.app.service.ElevateWordService) Files(java.nio.file.Files) FessApiAdminAction(org.codelibs.fess.app.web.api.admin.FessApiAdminAction) BufferedWriter(java.io.BufferedWriter) Resource(javax.annotation.Resource) StringUtil(org.codelibs.core.lang.StringUtil) Reader(java.io.Reader) JsonResponse(org.lastaflute.web.response.JsonResponse) InputStreamReader(java.io.InputStreamReader) Collectors(java.util.stream.Collectors) UploadForm(org.codelibs.fess.app.web.admin.elevateword.UploadForm) ApiUpdateResponse(org.codelibs.fess.app.web.api.ApiResult.ApiUpdateResponse) List(java.util.List) ComponentUtil(org.codelibs.fess.util.ComponentUtil) AdminElevatewordAction.getElevateWord(org.codelibs.fess.app.web.admin.elevateword.AdminElevatewordAction.getElevateWord) Writer(java.io.Writer) Execute(org.lastaflute.web.Execute) ElevateWord(org.codelibs.fess.es.config.exentity.ElevateWord) BufferedReader(java.io.BufferedReader) SuggestHelper(org.codelibs.fess.helper.SuggestHelper) InputStream(java.io.InputStream) PermissionHelper(org.codelibs.fess.helper.PermissionHelper) StringUtil(org.codelibs.core.lang.StringUtil)

Example 3 with StringUtil

use of org.codelibs.core.lang.StringUtil in project fess by codelibs.

the class ApiAdminFileconfigAction method createEditBody.

protected EditBody createEditBody(final FileConfig entity) {
    final EditBody body = new EditBody();
    copyBeanToBean(entity, body, copyOp -> {
        copyOp.excludeNull();
        copyOp.exclude(Constants.PERMISSIONS, Constants.VIRTUAL_HOSTS);
    });
    final PermissionHelper permissionHelper = ComponentUtil.getPermissionHelper();
    body.permissions = stream(entity.getPermissions()).get(stream -> stream.map(s -> permissionHelper.decode(s)).filter(StringUtil::isNotBlank).distinct().collect(Collectors.joining("\n")));
    body.virtualHosts = stream(entity.getVirtualHosts()).get(stream -> stream.filter(StringUtil::isNotBlank).distinct().map(String::trim).collect(Collectors.joining("\n")));
    return body;
}
Also used : Constants(org.codelibs.fess.Constants) StreamUtil.stream(org.codelibs.core.stream.StreamUtil.stream) FessApiAdminAction(org.codelibs.fess.app.web.api.admin.FessApiAdminAction) AdminFileconfigAction.getFileConfig(org.codelibs.fess.app.web.admin.fileconfig.AdminFileconfigAction.getFileConfig) Resource(javax.annotation.Resource) StringUtil(org.codelibs.core.lang.StringUtil) ApiConfigResponse(org.codelibs.fess.app.web.api.ApiResult.ApiConfigResponse) JsonResponse(org.lastaflute.web.response.JsonResponse) PermissionHelper(org.codelibs.fess.helper.PermissionHelper) Collectors(java.util.stream.Collectors) ApiResult(org.codelibs.fess.app.web.api.ApiResult) ApiUpdateResponse(org.codelibs.fess.app.web.api.ApiResult.ApiUpdateResponse) FileConfigPager(org.codelibs.fess.app.pager.FileConfigPager) Status(org.codelibs.fess.app.web.api.ApiResult.Status) List(java.util.List) CrudMode(org.codelibs.fess.app.web.CrudMode) ComponentUtil(org.codelibs.fess.util.ComponentUtil) FileConfigService(org.codelibs.fess.app.service.FileConfigService) Execute(org.lastaflute.web.Execute) ApiResponse(org.codelibs.fess.app.web.api.ApiResult.ApiResponse) FileConfig(org.codelibs.fess.es.config.exentity.FileConfig) PermissionHelper(org.codelibs.fess.helper.PermissionHelper) StringUtil(org.codelibs.core.lang.StringUtil)

Example 4 with StringUtil

use of org.codelibs.core.lang.StringUtil in project fess by codelibs.

the class ApiAdminDataconfigAction method createEditBody.

protected EditBody createEditBody(final DataConfig entity) {
    final EditBody body = new EditBody();
    copyBeanToBean(entity, body, copyOp -> {
        copyOp.excludeNull();
        copyOp.exclude(Constants.PERMISSIONS, Constants.VIRTUAL_HOSTS);
    });
    final PermissionHelper permissionHelper = ComponentUtil.getPermissionHelper();
    body.permissions = stream(entity.getPermissions()).get(stream -> stream.map(s -> permissionHelper.decode(s)).filter(StringUtil::isNotBlank).distinct().collect(Collectors.joining("\n")));
    body.virtualHosts = stream(entity.getVirtualHosts()).get(stream -> stream.filter(StringUtil::isNotBlank).distinct().map(String::trim).collect(Collectors.joining("\n")));
    return body;
}
Also used : DataConfigService(org.codelibs.fess.app.service.DataConfigService) DataConfig(org.codelibs.fess.es.config.exentity.DataConfig) Constants(org.codelibs.fess.Constants) StreamUtil.stream(org.codelibs.core.stream.StreamUtil.stream) FessApiAdminAction(org.codelibs.fess.app.web.api.admin.FessApiAdminAction) Resource(javax.annotation.Resource) StringUtil(org.codelibs.core.lang.StringUtil) ApiConfigResponse(org.codelibs.fess.app.web.api.ApiResult.ApiConfigResponse) JsonResponse(org.lastaflute.web.response.JsonResponse) PermissionHelper(org.codelibs.fess.helper.PermissionHelper) Collectors(java.util.stream.Collectors) ApiResult(org.codelibs.fess.app.web.api.ApiResult) ApiUpdateResponse(org.codelibs.fess.app.web.api.ApiResult.ApiUpdateResponse) DataConfigPager(org.codelibs.fess.app.pager.DataConfigPager) Status(org.codelibs.fess.app.web.api.ApiResult.Status) List(java.util.List) CrudMode(org.codelibs.fess.app.web.CrudMode) AdminDataconfigAction.getDataConfig(org.codelibs.fess.app.web.admin.dataconfig.AdminDataconfigAction.getDataConfig) ComponentUtil(org.codelibs.fess.util.ComponentUtil) Execute(org.lastaflute.web.Execute) ApiResponse(org.codelibs.fess.app.web.api.ApiResult.ApiResponse) PermissionHelper(org.codelibs.fess.helper.PermissionHelper) StringUtil(org.codelibs.core.lang.StringUtil)

Example 5 with StringUtil

use of org.codelibs.core.lang.StringUtil in project fess by codelibs.

the class SuggestJob method executeSuggestCreator.

protected void executeSuggestCreator() {
    final List<String> cmdList = new ArrayList<>();
    final String cpSeparator = SystemUtils.IS_OS_WINDOWS ? ";" : ":";
    final ServletContext servletContext = ComponentUtil.getComponent(ServletContext.class);
    final ProcessHelper processHelper = ComponentUtil.getProcessHelper();
    final FessConfig fessConfig = ComponentUtil.getFessConfig();
    cmdList.add(fessConfig.getJavaCommandPath());
    // -cp
    cmdList.add("-cp");
    final StringBuilder buf = new StringBuilder(100);
    ResourceUtil.getOverrideConfPath().ifPresent(p -> {
        buf.append(p);
        buf.append(cpSeparator);
    });
    final String confPath = System.getProperty(Constants.FESS_CONF_PATH);
    if (StringUtil.isNotBlank(confPath)) {
        buf.append(confPath);
        buf.append(cpSeparator);
    }
    // WEB-INF/env/suggest/resources
    buf.append("WEB-INF");
    buf.append(File.separator);
    buf.append("env");
    buf.append(File.separator);
    buf.append(getExecuteType());
    buf.append(File.separator);
    buf.append("resources");
    buf.append(cpSeparator);
    // WEB-INF/classes
    buf.append("WEB-INF");
    buf.append(File.separator);
    buf.append("classes");
    // target/classes
    final String userDir = System.getProperty("user.dir");
    final File targetDir = new File(userDir, "target");
    final File targetClassesDir = new File(targetDir, "classes");
    if (targetClassesDir.isDirectory()) {
        buf.append(cpSeparator);
        buf.append(targetClassesDir.getAbsolutePath());
    }
    // WEB-INF/lib
    appendJarFile(cpSeparator, buf, new File(servletContext.getRealPath("/WEB-INF/lib")), "WEB-INF" + File.separator + "lib" + File.separator);
    // WEB-INF/env/suggest/lib
    appendJarFile(cpSeparator, buf, new File(servletContext.getRealPath("/WEB-INF/env/" + getExecuteType() + "/lib")), "WEB-INF" + File.separator + "env" + File.separator + getExecuteType() + File.separator + "lib" + File.separator);
    // WEB-INF/plugin
    appendJarFile(cpSeparator, buf, new File(servletContext.getRealPath("/WEB-INF/plugin")), "WEB-INF" + File.separator + "plugin" + File.separator);
    final File targetLibDir = new File(targetDir, "fess" + File.separator + "WEB-INF" + File.separator + "lib");
    if (targetLibDir.isDirectory()) {
        appendJarFile(cpSeparator, buf, targetLibDir, targetLibDir.getAbsolutePath() + File.separator);
    }
    cmdList.add(buf.toString());
    if (useLocalFesen) {
        final String httpAddress = System.getProperty(Constants.FESS_ES_HTTP_ADDRESS);
        if (StringUtil.isNotBlank(httpAddress)) {
            cmdList.add("-D" + Constants.FESS_ES_HTTP_ADDRESS + "=" + httpAddress);
        }
    }
    final String systemLastaEnv = System.getProperty("lasta.env");
    if (StringUtil.isNotBlank(systemLastaEnv)) {
        if ("web".equals(systemLastaEnv)) {
            cmdList.add("-Dlasta.env=" + getExecuteType());
        } else {
            cmdList.add("-Dlasta.env=" + systemLastaEnv);
        }
    } else if (StringUtil.isNotBlank(lastaEnv)) {
        cmdList.add("-Dlasta.env=" + lastaEnv);
    } else {
        cmdList.add("-Dlasta.env=" + getExecuteType());
    }
    addFessConfigProperties(cmdList);
    addFessSystemProperties(cmdList);
    addSystemProperty(cmdList, Constants.FESS_CONF_PATH, null, null);
    cmdList.add("-Dfess." + getExecuteType() + ".process=true");
    if (logFilePath == null) {
        final String value = System.getProperty("fess.log.path");
        logFilePath = value != null ? value : new File(targetDir, "logs").getAbsolutePath();
    }
    cmdList.add("-Dfess.log.path=" + logFilePath);
    addSystemProperty(cmdList, "fess.log.name", "fess-" + getExecuteType(), "-" + getExecuteType());
    if (logLevel == null) {
        addSystemProperty(cmdList, "fess.log.level", null, null);
    } else {
        cmdList.add("-Dfess.log.level=" + logLevel);
    }
    stream(fessConfig.getJvmSuggestOptionsAsArray()).of(stream -> stream.filter(StringUtil::isNotBlank).forEach(value -> cmdList.add(value)));
    File ownTmpDir = null;
    final String tmpDir = System.getProperty("java.io.tmpdir");
    if (fessConfig.isUseOwnTmpDir() && StringUtil.isNotBlank(tmpDir)) {
        ownTmpDir = new File(tmpDir, "fessTmpDir_" + sessionId);
        if (ownTmpDir.mkdirs()) {
            cmdList.add("-Djava.io.tmpdir=" + ownTmpDir.getAbsolutePath());
        } else {
            ownTmpDir = null;
        }
    }
    if (!jvmOptions.isEmpty()) {
        jvmOptions.stream().filter(StringUtil::isNotBlank).forEach(cmdList::add);
    }
    cmdList.add(SuggestCreator.class.getCanonicalName());
    cmdList.add("--sessionId");
    cmdList.add(sessionId);
    final File propFile = ComponentUtil.getSystemHelper().createTempFile(getExecuteType() + "_", ".properties");
    try {
        cmdList.add("-p");
        cmdList.add(propFile.getAbsolutePath());
        createSystemProperties(cmdList, propFile);
        final File baseDir = new File(servletContext.getRealPath("/WEB-INF")).getParentFile();
        if (logger.isInfoEnabled()) {
            logger.info("SuggestCreator: \nDirectory={}\nOptions={}", baseDir, cmdList);
        }
        final JobProcess jobProcess = processHelper.startProcess(sessionId, cmdList, pb -> {
            pb.directory(baseDir);
            pb.redirectErrorStream(true);
        });
        final InputStreamThread it = jobProcess.getInputStreamThread();
        it.start();
        final Process currentProcess = jobProcess.getProcess();
        currentProcess.waitFor();
        it.join(5000);
        final int exitValue = currentProcess.exitValue();
        if (logger.isInfoEnabled()) {
            logger.info("SuggestCreator: Exit Code={} - Process Output:\n{}", exitValue, it.getOutput());
        }
        if (exitValue != 0) {
            final StringBuilder out = new StringBuilder();
            if (processTimeout) {
                out.append("Process is terminated due to ").append(timeout).append(" second exceeded.\n");
            }
            out.append("Exit Code: ").append(exitValue).append("\nOutput:\n").append(it.getOutput());
            throw new JobProcessingException(out.toString());
        }
        ComponentUtil.getPopularWordHelper().clearCache();
    } catch (final JobProcessingException e) {
        throw e;
    } catch (final Exception e) {
        throw new JobProcessingException("SuggestCreator Process terminated.", e);
    } finally {
        try {
            processHelper.destroyProcess(sessionId);
        } finally {
            if (propFile != null && !propFile.delete()) {
                logger.warn("Failed to delete {}.", propFile.getAbsolutePath());
            }
            deleteTempDir(ownTmpDir);
        }
    }
}
Also used : Constants(org.codelibs.fess.Constants) StreamUtil.stream(org.codelibs.core.stream.StreamUtil.stream) ProcessHelper(org.codelibs.fess.helper.ProcessHelper) SystemUtils(org.apache.commons.lang3.SystemUtils) InputStreamThread(org.codelibs.fess.util.InputStreamThread) StringUtil(org.codelibs.core.lang.StringUtil) SuggestCreator(org.codelibs.fess.exec.SuggestCreator) File(java.io.File) ArrayList(java.util.ArrayList) ResourceUtil(org.codelibs.fess.util.ResourceUtil) List(java.util.List) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig) Logger(org.apache.logging.log4j.Logger) ComponentUtil(org.codelibs.fess.util.ComponentUtil) ServletContext(javax.servlet.ServletContext) RandomStringUtils(org.apache.commons.lang3.RandomStringUtils) JobProcessingException(org.codelibs.fess.exception.JobProcessingException) TimeoutTask(org.codelibs.core.timer.TimeoutTask) LogManager(org.apache.logging.log4j.LogManager) JobProcess(org.codelibs.fess.util.JobProcess) SuggestCreator(org.codelibs.fess.exec.SuggestCreator) ArrayList(java.util.ArrayList) JobProcess(org.codelibs.fess.util.JobProcess) ProcessHelper(org.codelibs.fess.helper.ProcessHelper) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig) JobProcess(org.codelibs.fess.util.JobProcess) JobProcessingException(org.codelibs.fess.exception.JobProcessingException) JobProcessingException(org.codelibs.fess.exception.JobProcessingException) InputStreamThread(org.codelibs.fess.util.InputStreamThread) ServletContext(javax.servlet.ServletContext) StringUtil(org.codelibs.core.lang.StringUtil) File(java.io.File)

Aggregations

StringUtil (org.codelibs.core.lang.StringUtil)46 ComponentUtil (org.codelibs.fess.util.ComponentUtil)41 Constants (org.codelibs.fess.Constants)40 StreamUtil.stream (org.codelibs.core.stream.StreamUtil.stream)38 Collectors (java.util.stream.Collectors)35 List (java.util.List)30 StreamUtil.split (org.codelibs.core.stream.StreamUtil.split)27 PermissionHelper (org.codelibs.fess.helper.PermissionHelper)26 ArrayList (java.util.ArrayList)25 Resource (javax.annotation.Resource)20 CrudMode (org.codelibs.fess.app.web.CrudMode)19 FessConfig (org.codelibs.fess.mylasta.direction.FessConfig)19 OptionalThing (org.dbflute.optional.OptionalThing)19 Execute (org.lastaflute.web.Execute)19 Map (java.util.Map)17 Stream (java.util.stream.Stream)17 SystemHelper (org.codelibs.fess.helper.SystemHelper)17 OptionalEntity (org.dbflute.optional.OptionalEntity)17 LogManager (org.apache.logging.log4j.LogManager)16 Logger (org.apache.logging.log4j.Logger)16