Search in sources :

Example 36 with GetRequest

use of org.elasticsearch.action.get.GetRequest in project pinpoint by naver.

the class ElasticsearchExecutorInterceptor method recordeESattributes.

// TODO max dsl limit need
private void recordeESattributes(SpanEventRecorder recorder, Object target, Object[] args, Object result, Throwable throwable) {
    if (recordESVersion) {
        if (target instanceof ClusterInfoAccessor) {
            // record elasticsearch version and cluster name.
            recorder.recordAttribute(ElasticsearchConstants.ARGS_VERSION_ANNOTATION_KEY, ((ClusterInfoAccessor) target)._$PINPOINT$_getClusterInfo());
        }
    }
    if (recordDsl) {
        if (args[0] instanceof SearchRequest) {
            SearchRequest request = (SearchRequest) args[0];
            recorder.recordAttribute(ElasticsearchConstants.ARGS_DSL_ANNOTATION_KEY, StringUtils.abbreviate(request.source().toString(), 256));
        } else if (args[0] instanceof GetRequest) {
            // GetRequest request = (GetRequest) args[0];
            recorder.recordAttribute(ElasticsearchConstants.ARGS_DSL_ANNOTATION_KEY, StringUtils.abbreviate(args[0].toString(), 256));
        } else if (args[0] instanceof IndexRequest) {
            // IndexRequest request = (IndexRequest) args[0];
            recorder.recordAttribute(ElasticsearchConstants.ARGS_DSL_ANNOTATION_KEY, StringUtils.abbreviate(args[0].toString(), 256));
        } else if (args[0] instanceof DeleteRequest) {
            // DeleteRequest request = (DeleteRequest) args[0];
            recorder.recordAttribute(ElasticsearchConstants.ARGS_DSL_ANNOTATION_KEY, StringUtils.abbreviate(args[0].toString(), 256));
        } else if (args[0] instanceof UpdateRequest) {
            // UpdateRequest request = (UpdateRequest) args[0];
            recorder.recordAttribute(ElasticsearchConstants.ARGS_DSL_ANNOTATION_KEY, StringUtils.abbreviate(args[0].toString(), 256));
        }
    }
}
Also used : SearchRequest(org.elasticsearch.action.search.SearchRequest) UpdateRequest(org.elasticsearch.action.update.UpdateRequest) ClusterInfoAccessor(com.navercorp.pinpoint.plugin.elasticsearch.accessor.ClusterInfoAccessor) GetRequest(org.elasticsearch.action.get.GetRequest) IndexRequest(org.elasticsearch.action.index.IndexRequest) DeleteRequest(org.elasticsearch.action.delete.DeleteRequest)

Example 37 with GetRequest

use of org.elasticsearch.action.get.GetRequest in project gora by apache.

the class ElasticsearchStore method exists.

@Override
public boolean exists(K key) throws GoraException {
    GetRequest getRequest = new GetRequest(elasticsearchMapping.getIndexName(), (String) key);
    getRequest.fetchSourceContext(new FetchSourceContext(false)).storedFields("_none_");
    try {
        return client.exists(getRequest, RequestOptions.DEFAULT);
    } catch (IOException ex) {
        throw new GoraException(ex);
    }
}
Also used : GoraException(org.apache.gora.util.GoraException) FetchSourceContext(org.elasticsearch.search.fetch.subphase.FetchSourceContext) GetRequest(org.elasticsearch.action.get.GetRequest) IOException(java.io.IOException)

Example 38 with GetRequest

use of org.elasticsearch.action.get.GetRequest in project graylog2-server by Graylog2.

the class MessagesAdapterES7 method get.

@Override
public ResultMessage get(String messageId, String index) throws DocumentNotFoundException {
    final GetRequest getRequest = new GetRequest(index, messageId);
    final GetResponse result = this.client.execute((c, requestOptions) -> c.get(getRequest, requestOptions));
    if (!result.isExists()) {
        throw new DocumentNotFoundException(index, messageId);
    }
    return ResultMessage.parseFromSource(messageId, index, result.getSource());
}
Also used : DocumentNotFoundException(org.graylog2.indexer.messages.DocumentNotFoundException) GetRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.get.GetRequest) GetResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.get.GetResponse)

Example 39 with GetRequest

use of org.elasticsearch.action.get.GetRequest in project vorto by eclipse.

the class ElasticSearchService method modelIndexExist.

@SuppressWarnings("unused")
private boolean modelIndexExist(ModelId modelId) {
    PreConditions.notNull(modelId, "modelId must not be null.");
    GetRequest getRequest = new GetRequest(VORTO_INDEX, DOC, modelId.getPrettyFormat());
    getRequest.fetchSourceContext(new FetchSourceContext(false));
    getRequest.storedFields("_none_");
    try {
        return client.exists(getRequest, RequestOptions.DEFAULT);
    } catch (IOException e) {
        throw new IndexingException(String.format("Error while querying if model '%s' exist.", modelId.getPrettyFormat()), e);
    }
}
Also used : FetchSourceContext(org.elasticsearch.search.fetch.subphase.FetchSourceContext) GetRequest(org.elasticsearch.action.get.GetRequest) IOException(java.io.IOException)

Example 40 with GetRequest

use of org.elasticsearch.action.get.GetRequest in project incubator-gobblin by apache.

the class ElasticsearchWriterIntegrationTest method testMalformedDocs.

/**
 * Sends two docs in a single batch with different field types
 * Triggers Elasticsearch server to send back an exception due to malformed docs
 * @throws IOException
 */
public void testMalformedDocs(WriterVariant writerVariant, RecordTypeGenerator recordVariant, MalformedDocPolicy malformedDocPolicy) throws IOException {
    String indexName = writerVariant.getName().toLowerCase();
    String indexType = (recordVariant.getName() + malformedDocPolicy.name()).toLowerCase();
    Config config = writerVariant.getConfigBuilder().setIdMappingEnabled(true).setIndexName(indexName).setIndexType(indexType).setHttpPort(_esTestServer.getHttpPort()).setTransportPort(_esTestServer.getTransportPort()).setTypeMapperClassName(recordVariant.getTypeMapperClassName()).setMalformedDocPolicy(malformedDocPolicy).build();
    TestClient testClient = writerVariant.getTestClient(config);
    testClient.recreateIndex(indexName);
    String id1 = TestUtils.generateRandomAlphaString(10);
    String id2 = TestUtils.generateRandomAlphaString(10);
    Object testRecord1 = recordVariant.getRecord(id1, PayloadType.LONG);
    Object testRecord2 = recordVariant.getRecord(id2, PayloadType.MAP);
    SequentialBasedBatchAccumulator<Object> batchAccumulator = new SequentialBasedBatchAccumulator<>(config);
    BatchAsyncDataWriter elasticsearchWriter = writerVariant.getBatchAsyncDataWriter(config);
    BufferedAsyncDataWriter bufferedAsyncDataWriter = new BufferedAsyncDataWriter(batchAccumulator, elasticsearchWriter);
    DataWriter writer = AsyncWriterManager.builder().failureAllowanceRatio(0.0).retriesEnabled(false).config(config).asyncDataWriter(bufferedAsyncDataWriter).build();
    try {
        writer.write(testRecord1);
        writer.write(testRecord2);
        writer.commit();
        writer.close();
        if (malformedDocPolicy == MalformedDocPolicy.FAIL) {
            Assert.fail("Should have thrown an exception if malformed doc policy was set to Fail");
        }
    } catch (Exception e) {
        switch(malformedDocPolicy) {
            case IGNORE:
            case WARN:
                {
                    Assert.fail("Should not have failed if malformed doc policy was set to ignore or warn", e);
                    break;
                }
            case FAIL:
                {
                    // pass through
                    break;
                }
            default:
                {
                    throw new RuntimeException("This test does not handle this policyType : " + malformedDocPolicy.toString());
                }
        }
    }
    // Irrespective of policy, first doc should be inserted and second doc should fail
    int docsIndexed = 0;
    try {
        {
            GetResponse response = testClient.get(new GetRequest(indexName, indexType, id1));
            Assert.assertEquals(response.getId(), id1, "Response id matches request");
            System.out.println(malformedDocPolicy + ":" + response.toString());
            if (response.isExists()) {
                docsIndexed++;
            }
        }
        {
            GetResponse response = testClient.get(new GetRequest(indexName, indexType, id2));
            Assert.assertEquals(response.getId(), id2, "Response id matches request");
            System.out.println(malformedDocPolicy + ":" + response.toString());
            if (response.isExists()) {
                docsIndexed++;
            }
        }
        // only one doc should be found
        Assert.assertEquals(docsIndexed, 1, "Only one document should be indexed");
    } catch (Exception e) {
        Assert.fail("Failed to get a response", e);
    } finally {
        testClient.close();
    }
}
Also used : Config(com.typesafe.config.Config) BufferedAsyncDataWriter(org.apache.gobblin.writer.BufferedAsyncDataWriter) SequentialBasedBatchAccumulator(org.apache.gobblin.writer.SequentialBasedBatchAccumulator) GetResponse(org.elasticsearch.action.get.GetResponse) IOException(java.io.IOException) GetRequest(org.elasticsearch.action.get.GetRequest) BatchAsyncDataWriter(org.apache.gobblin.writer.BatchAsyncDataWriter) DataWriter(org.apache.gobblin.writer.DataWriter) BufferedAsyncDataWriter(org.apache.gobblin.writer.BufferedAsyncDataWriter) BatchAsyncDataWriter(org.apache.gobblin.writer.BatchAsyncDataWriter)

Aggregations

GetRequest (org.elasticsearch.action.get.GetRequest)45 GetResponse (org.elasticsearch.action.get.GetResponse)29 Test (org.junit.Test)14 IOException (java.io.IOException)13 IndexRequest (org.elasticsearch.action.index.IndexRequest)9 HashMap (java.util.HashMap)7 TikaDocument (org.icij.extract.document.TikaDocument)7 FetchSourceContext (org.elasticsearch.search.fetch.subphase.FetchSourceContext)6 ArrayList (java.util.ArrayList)5 DocumentFactory (org.icij.extract.document.DocumentFactory)5 ByteArrayInputStream (java.io.ByteArrayInputStream)4 ParsingReader (org.apache.tika.parser.ParsingReader)4 ElasticsearchException (org.elasticsearch.ElasticsearchException)4 BulkItemResponse (org.elasticsearch.action.bulk.BulkItemResponse)4 DeleteRequest (org.elasticsearch.action.delete.DeleteRequest)4 SearchRequest (org.elasticsearch.action.search.SearchRequest)4 UpdateRequest (org.elasticsearch.action.update.UpdateRequest)4 PathIdentifier (org.icij.extract.document.PathIdentifier)4 BulkRequest (org.elasticsearch.action.bulk.BulkRequest)3 MultiGetRequest (org.elasticsearch.action.get.MultiGetRequest)3