Search in sources :

Example 26 with StopWatch

use of core.framework.util.StopWatch in project core-ng-project by neowu.

the class MockElasticSearch method createClient.

@Override
protected Client createClient() {
    StopWatch watch = new StopWatch();
    try {
        Settings.Builder settings = Settings.builder();
        settings.put(Environment.PATH_HOME_SETTING.getKey(), dataPath).put(NetworkModule.HTTP_ENABLED.getKey(), false).put(NetworkService.GLOBAL_NETWORK_BINDHOST_SETTING.getKey(), "_local_").put(DiscoveryModule.DISCOVERY_TYPE_SETTING.getKey(), "single-node");
        MockNode node = new MockNode(settings.build());
        node.start();
        return node.client();
    } catch (NodeValidationException e) {
        throw new Error(e);
    } finally {
        logger.info("create local elasticsearch node, dataPath={}, elapsedTime={}", dataPath, watch.elapsedTime());
    }
}
Also used : NodeValidationException(org.elasticsearch.node.NodeValidationException) Settings(org.elasticsearch.common.settings.Settings) StopWatch(core.framework.util.StopWatch)

Example 27 with StopWatch

use of core.framework.util.StopWatch in project core-ng-project by neowu.

the class HTTPClientBuilder method build.

public HTTPClient build() {
    StopWatch watch = new StopWatch();
    try {
        HttpClientBuilder builder = HttpClients.custom();
        builder.setUserAgent(userAgent);
        builder.setKeepAliveStrategy((response, context) -> keepAliveTimeout.toMillis());
        builder.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE).setSSLContext(new SSLContextBuilder().loadTrustMaterial(TrustSelfSignedStrategy.INSTANCE).build());
        // builder use PoolingHttpClientConnectionManager by default, and connTimeToLive will be set by keepAlive value
        builder.setDefaultSocketConfig(SocketConfig.custom().setSoKeepAlive(true).build());
        builder.setDefaultRequestConfig(RequestConfig.custom().setSocketTimeout((int) timeout.toMillis()).setConnectionRequestTimeout((int) timeout.toMillis()).setConnectTimeout((int) timeout.toMillis()).build());
        builder.setMaxConnPerRoute(maxConnections).setMaxConnTotal(maxConnections);
        builder.disableAuthCaching();
        builder.disableConnectionState();
        // retry should be handled in framework level with better trace log
        builder.disableAutomaticRetries();
        if (!enableRedirect)
            builder.disableRedirectHandling();
        if (!enableCookie)
            builder.disableCookieManagement();
        CloseableHttpClient httpClient = builder.build();
        return new HTTPClientImpl(httpClient, userAgent, slowOperationThreshold);
    } catch (NoSuchAlgorithmException | KeyManagementException | KeyStoreException e) {
        throw new Error(e);
    } finally {
        logger.info("create http client, elapsedTime={}", watch.elapsedTime());
    }
}
Also used : HTTPClientImpl(core.framework.impl.http.HTTPClientImpl) CloseableHttpClient(org.apache.http.impl.client.CloseableHttpClient) HttpClientBuilder(org.apache.http.impl.client.HttpClientBuilder) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException) KeyStoreException(java.security.KeyStoreException) SSLContextBuilder(org.apache.http.ssl.SSLContextBuilder) KeyManagementException(java.security.KeyManagementException) StopWatch(core.framework.util.StopWatch)

Example 28 with StopWatch

use of core.framework.util.StopWatch in project core-ng-project by neowu.

the class HTTPServer method start.

public void start() {
    if (httpPort == null && httpsPort == null) {
        // by default start http
        httpPort = 8080;
    }
    StopWatch watch = new StopWatch();
    try {
        Undertow.Builder builder = Undertow.builder();
        if (httpPort != null)
            builder.addHttpListener(httpPort, "0.0.0.0");
        if (httpsPort != null)
            builder.addHttpsListener(httpsPort, "0.0.0.0", new SSLContextBuilder().build());
        // same logic as io.undertow.Undertow.Builder(), but use overridden availableProcessors value
        int ioThreads = Math.max(Threads.availableProcessors(), 2);
        int workerThreads = ioThreads * 8;
        builder.setHandler(handler()).setServerOption(UndertowOptions.DECODE_URL, false).setServerOption(UndertowOptions.ENABLE_HTTP2, true).setServerOption(UndertowOptions.ENABLE_RFC6265_COOKIE_VALIDATION, true).setServerOption(UndertowOptions.MAX_ENTITY_SIZE, // max post body is 10M
        10L * 1024 * 1024).setIoThreads(ioThreads).setWorkerThreads(workerThreads);
        server = builder.build();
        server.start();
    } finally {
        logger.info("http server started, httpPort={}, httpsPort={}, gzip={}, elapsedTime={}", httpPort, httpsPort, gzip, watch.elapsedTime());
    }
}
Also used : Undertow(io.undertow.Undertow) StopWatch(core.framework.util.StopWatch)

Example 29 with StopWatch

use of core.framework.util.StopWatch in project core-ng-project by neowu.

the class ElasticSearchImpl method indices.

@Override
public List<ElasticSearchIndex> indices() {
    StopWatch watch = new StopWatch();
    try {
        AdminClient adminClient = client().admin();
        ClusterStateResponse response = adminClient.cluster().state(new ClusterStateRequest().clear().metaData(true)).actionGet();
        ImmutableOpenMap<String, IndexMetaData> indices = response.getState().getMetaData().indices();
        List<ElasticSearchIndex> results = new ArrayList<>(indices.size());
        for (ObjectObjectCursor<String, IndexMetaData> cursor : indices) {
            IndexMetaData metaData = cursor.value;
            ElasticSearchIndex index = new ElasticSearchIndex();
            index.index = metaData.getIndex().getName();
            index.state = metaData.getState();
            results.add(index);
        }
        return results;
    } catch (ElasticsearchException e) {
        // due to elastic search uses async executor to run, we have to wrap the exception to retain the original place caused the exception
        throw new SearchException(e);
    } finally {
        logger.info("indices, elapsedTime={}", watch.elapsedTime());
    }
}
Also used : ClusterStateResponse(org.elasticsearch.action.admin.cluster.state.ClusterStateResponse) ClusterStateRequest(org.elasticsearch.action.admin.cluster.state.ClusterStateRequest) ArrayList(java.util.ArrayList) SearchException(core.framework.search.SearchException) ElasticsearchException(org.elasticsearch.ElasticsearchException) StopWatch(core.framework.util.StopWatch) IndexMetaData(org.elasticsearch.cluster.metadata.IndexMetaData) ElasticSearchIndex(core.framework.search.ElasticSearchIndex) AdminClient(org.elasticsearch.client.AdminClient)

Example 30 with StopWatch

use of core.framework.util.StopWatch in project core-ng-project by neowu.

the class ElasticSearchImpl method createClient.

protected Client createClient() {
    if (addresses.isEmpty())
        throw new Error("addresses must not be empty");
    StopWatch watch = new StopWatch();
    try {
        Settings.Builder settings = Settings.builder();
        settings.put(NetworkService.TCP_CONNECT_TIMEOUT.getKey(), new TimeValue(timeout.toMillis())).put(TransportClient.CLIENT_TRANSPORT_PING_TIMEOUT.getKey(), new TimeValue(timeout.toMillis())).put(TransportClient.CLIENT_TRANSPORT_PING_TIMEOUT.getKey(), new TimeValue(timeout.toMillis())).put(TransportClient.CLIENT_TRANSPORT_IGNORE_CLUSTER_NAME.getKey(), // refer to https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html
        "true");
        if (sniff) {
            settings.put(TransportClient.CLIENT_TRANSPORT_SNIFF.getKey(), true);
        }
        TransportClient client = new PreBuiltTransportClient(settings.build());
        addresses.forEach(client::addTransportAddress);
        return client;
    } finally {
        logger.info("create elasticsearch client, addresses={}, elapsedTime={}", addresses, watch.elapsedTime());
    }
}
Also used : TransportClient(org.elasticsearch.client.transport.TransportClient) PreBuiltTransportClient(org.elasticsearch.transport.client.PreBuiltTransportClient) PreBuiltTransportClient(org.elasticsearch.transport.client.PreBuiltTransportClient) Settings(org.elasticsearch.common.settings.Settings) TimeValue(org.elasticsearch.common.unit.TimeValue) StopWatch(core.framework.util.StopWatch)

Aggregations

StopWatch (core.framework.util.StopWatch)79 IOException (java.io.IOException)21 UncheckedIOException (java.io.UncheckedIOException)21 SearchException (core.framework.search.SearchException)10 ElasticsearchException (org.elasticsearch.ElasticsearchException)10 ArrayList (java.util.ArrayList)6 HTMLTemplate (core.framework.impl.template.HTMLTemplate)4 BytesParam (core.framework.impl.log.filter.BytesParam)3 Map (java.util.Map)3 BsonDocument (org.bson.BsonDocument)3 BulkWriteOptions (com.mongodb.client.model.BulkWriteOptions)2 UpdateOptions (com.mongodb.client.model.UpdateOptions)2 DeleteResult (com.mongodb.client.result.DeleteResult)2 TemplateContext (core.framework.impl.template.TemplateContext)2 Headers (org.apache.kafka.common.header.Headers)2 Bson (org.bson.conversions.Bson)2 BulkRequestBuilder (org.elasticsearch.action.bulk.BulkRequestBuilder)2 BulkResponse (org.elasticsearch.action.bulk.BulkResponse)2 Settings (org.elasticsearch.common.settings.Settings)2 MongoClient (com.mongodb.MongoClient)1