Search in sources :

Example 41 with XContentType

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.XContentType in project elasticsearch by elastic.

the class AggregationProfileShardResultTests method testFromXContent.

public void testFromXContent() throws IOException {
    AggregationProfileShardResult profileResult = createTestItem(2);
    XContentType xContentType = randomFrom(XContentType.values());
    boolean humanReadable = randomBoolean();
    BytesReference originalBytes = toXContent(profileResult, xContentType, humanReadable);
    AggregationProfileShardResult parsed;
    try (XContentParser parser = createParser(xContentType.xContent(), originalBytes)) {
        XContentParserUtils.ensureExpectedToken(parser.nextToken(), XContentParser.Token.START_OBJECT, parser::getTokenLocation);
        XContentParserUtils.ensureFieldName(parser, parser.nextToken(), AggregationProfileShardResult.AGGREGATIONS);
        XContentParserUtils.ensureExpectedToken(parser.nextToken(), XContentParser.Token.START_ARRAY, parser::getTokenLocation);
        parsed = AggregationProfileShardResult.fromXContent(parser);
        assertEquals(XContentParser.Token.END_OBJECT, parser.nextToken());
        assertNull(parser.nextToken());
    }
    assertToXContentEquivalent(originalBytes, toXContent(parsed, xContentType, humanReadable), xContentType);
}
Also used : BytesReference(org.elasticsearch.common.bytes.BytesReference) XContentType(org.elasticsearch.common.xcontent.XContentType) XContentParser(org.elasticsearch.common.xcontent.XContentParser)

Example 42 with XContentType

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.XContentType in project elasticsearch by elastic.

the class CollectorResultTests method testFromXContent.

public void testFromXContent() throws IOException {
    CollectorResult collectorResult = createTestItem(1);
    XContentType xContentType = randomFrom(XContentType.values());
    boolean humanReadable = randomBoolean();
    BytesReference originalBytes = toXContent(collectorResult, xContentType, humanReadable);
    CollectorResult parsed;
    try (XContentParser parser = createParser(xContentType.xContent(), originalBytes)) {
        ensureExpectedToken(XContentParser.Token.START_OBJECT, parser.nextToken(), parser::getTokenLocation);
        parsed = CollectorResult.fromXContent(parser);
        assertNull(parser.nextToken());
    }
    assertToXContentEquivalent(originalBytes, toXContent(parsed, xContentType, humanReadable), xContentType);
}
Also used : BytesReference(org.elasticsearch.common.bytes.BytesReference) XContentType(org.elasticsearch.common.xcontent.XContentType) XContentParser(org.elasticsearch.common.xcontent.XContentParser)

Example 43 with XContentType

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.XContentType in project crate by crate.

the class MetadataStateFormat method read.

/**
 * Reads the state from a given file and compares the expected version against the actual version of
 * the state.
 */
public final T read(NamedXContentRegistry namedXContentRegistry, Path file) throws IOException {
    try (Directory dir = newDirectory(file.getParent())) {
        try (IndexInput indexInput = dir.openInput(file.getFileName().toString(), IOContext.DEFAULT)) {
            // We checksum the entire file before we even go and parse it. If it's corrupted we barf right here.
            CodecUtil.checksumEntireFile(indexInput);
            CodecUtil.checkHeader(indexInput, STATE_FILE_CODEC, MIN_COMPATIBLE_STATE_FILE_VERSION, STATE_FILE_VERSION);
            final XContentType xContentType = XContentType.values()[indexInput.readInt()];
            if (xContentType != FORMAT) {
                throw new IllegalStateException("expected state in " + file + " to be " + FORMAT + " format but was " + xContentType);
            }
            long filePointer = indexInput.getFilePointer();
            long contentSize = indexInput.length() - CodecUtil.footerLength() - filePointer;
            try (IndexInput slice = indexInput.slice("state_xcontent", filePointer, contentSize)) {
                try (InputStreamIndexInput in = new InputStreamIndexInput(slice, contentSize)) {
                    try (XContentParser parser = XContentFactory.xContent(FORMAT).createParser(namedXContentRegistry, LoggingDeprecationHandler.INSTANCE, in)) {
                        return fromXContent(parser);
                    }
                }
            }
        } catch (CorruptIndexException | IndexFormatTooOldException | IndexFormatTooNewException ex) {
            // we trick this into a dedicated exception with the original stacktrace
            throw new CorruptStateException(ex);
        }
    }
}
Also used : XContentType(org.elasticsearch.common.xcontent.XContentType) IndexFormatTooOldException(org.apache.lucene.index.IndexFormatTooOldException) InputStreamIndexInput(org.elasticsearch.common.lucene.store.InputStreamIndexInput) IndexInput(org.apache.lucene.store.IndexInput) InputStreamIndexInput(org.elasticsearch.common.lucene.store.InputStreamIndexInput) CorruptIndexException(org.apache.lucene.index.CorruptIndexException) IndexFormatTooNewException(org.apache.lucene.index.IndexFormatTooNewException) XContentParser(org.elasticsearch.common.xcontent.XContentParser) Directory(org.apache.lucene.store.Directory) SimpleFSDirectory(org.apache.lucene.store.SimpleFSDirectory)

Example 44 with XContentType

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.XContentType in project crate by crate.

the class ColumnPolicyIntegrationTest method testStrictPartitionedTableInsert.

@Test
public void testStrictPartitionedTableInsert() throws Exception {
    execute("create table numbers (" + "  num int, " + "  odd boolean," + "  prime boolean" + ") partitioned by (odd) with (column_policy='strict', number_of_replicas=0)");
    ensureYellow();
    GetIndexTemplatesResponse response = client().admin().indices().prepareGetTemplates(PartitionName.templateName(sqlExecutor.getCurrentSchema(), "numbers")).execute().actionGet();
    assertThat(response.getIndexTemplates().size(), is(1));
    IndexTemplateMetadata template = response.getIndexTemplates().get(0);
    CompressedXContent mappingStr = template.mappings().get(Constants.DEFAULT_MAPPING_TYPE);
    assertThat(mappingStr, is(notNullValue()));
    Tuple<XContentType, Map<String, Object>> typeAndMap = XContentHelper.convertToMap(mappingStr.compressedReference(), false, XContentType.JSON);
    @SuppressWarnings("unchecked") Map<String, Object> mapping = (Map<String, Object>) typeAndMap.v2().get(Constants.DEFAULT_MAPPING_TYPE);
    assertThat(decodeMappingValue(mapping.get("dynamic")), is(ColumnPolicy.STRICT));
    execute("insert into numbers (num, odd, prime) values (?, ?, ?)", new Object[] { 6, true, false });
    execute("refresh table numbers");
    Map<String, Object> sourceMap = getSourceMap(new PartitionName(new RelationName("doc", "numbers"), Arrays.asList("true")).asIndexName());
    assertThat(decodeMappingValue(sourceMap.get("dynamic")), is(ColumnPolicy.STRICT));
    assertThrowsMatches(() -> execute("insert into numbers (num, odd, prime, perfect) values (?, ?, ?, ?)", new Object[] { 28, true, false, true }), isSQLError(is("Column perfect unknown"), UNDEFINED_COLUMN, NOT_FOUND, 4043));
}
Also used : PartitionName(io.crate.metadata.PartitionName) XContentType(org.elasticsearch.common.xcontent.XContentType) GetIndexTemplatesResponse(org.elasticsearch.action.admin.indices.template.get.GetIndexTemplatesResponse) IndexTemplateMetadata(org.elasticsearch.cluster.metadata.IndexTemplateMetadata) CompressedXContent(org.elasticsearch.common.compress.CompressedXContent) RelationName(io.crate.metadata.RelationName) Matchers.containsString(org.hamcrest.Matchers.containsString) Map(java.util.Map) Test(org.junit.Test)

Example 45 with XContentType

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.XContentType in project crate by crate.

the class ESTestCaseTests method testShuffleXContentExcludeFields.

public void testShuffleXContentExcludeFields() throws IOException {
    XContentType xContentType = randomFrom(XContentType.values());
    try (XContentBuilder builder = XContentBuilder.builder(xContentType.xContent())) {
        builder.startObject();
        {
            builder.field("field1", "value1");
            builder.field("field2", "value2");
            {
                builder.startObject("object1");
                {
                    builder.field("inner1", "value1");
                    builder.field("inner2", "value2");
                    builder.field("inner3", "value3");
                }
                builder.endObject();
            }
            {
                builder.startObject("object2");
                {
                    builder.field("inner4", "value4");
                    builder.field("inner5", "value5");
                    builder.field("inner6", "value6");
                }
                builder.endObject();
            }
        }
        builder.endObject();
        BytesReference bytes = BytesReference.bytes(builder);
        final LinkedHashMap<String, Object> initialMap;
        try (XContentParser parser = createParser(xContentType.xContent(), bytes)) {
            initialMap = (LinkedHashMap<String, Object>) parser.mapOrdered();
        }
        List<String> expectedInnerKeys1 = Arrays.asList("inner1", "inner2", "inner3");
        Set<List<String>> distinctTopLevelKeys = new HashSet<>();
        Set<List<String>> distinctInnerKeys2 = new HashSet<>();
        for (int i = 0; i < 10; i++) {
            try (XContentParser parser = createParser(xContentType.xContent(), bytes)) {
                try (XContentBuilder shuffledBuilder = shuffleXContent(parser, randomBoolean(), "object1")) {
                    try (XContentParser shuffledParser = createParser(shuffledBuilder)) {
                        Map<String, Object> shuffledMap = shuffledParser.mapOrdered();
                        assertEquals("both maps should contain the same mappings", initialMap, shuffledMap);
                        List<String> shuffledKeys = new ArrayList<>(shuffledMap.keySet());
                        distinctTopLevelKeys.add(shuffledKeys);
                        @SuppressWarnings("unchecked") Map<String, Object> innerMap1 = (Map<String, Object>) shuffledMap.get("object1");
                        List<String> actualInnerKeys1 = new ArrayList<>(innerMap1.keySet());
                        assertEquals("object1 should have been left untouched", expectedInnerKeys1, actualInnerKeys1);
                        @SuppressWarnings("unchecked") Map<String, Object> innerMap2 = (Map<String, Object>) shuffledMap.get("object2");
                        List<String> actualInnerKeys2 = new ArrayList<>(innerMap2.keySet());
                        distinctInnerKeys2.add(actualInnerKeys2);
                    }
                }
            }
        }
        // out of 10 shuffling runs we expect to have at least more than 1 distinct output for both top level keys and inner object2
        assertThat(distinctTopLevelKeys.size(), greaterThan(1));
        assertThat(distinctInnerKeys2.size(), greaterThan(1));
    }
}
Also used : BytesReference(org.elasticsearch.common.bytes.BytesReference) ArrayList(java.util.ArrayList) XContentType(org.elasticsearch.common.xcontent.XContentType) ArrayList(java.util.ArrayList) List(java.util.List) LinkedHashMap(java.util.LinkedHashMap) Map(java.util.Map) XContentBuilder(org.elasticsearch.common.xcontent.XContentBuilder) XContentParser(org.elasticsearch.common.xcontent.XContentParser) HashSet(java.util.HashSet)

Aggregations

XContentType (org.elasticsearch.common.xcontent.XContentType)86 BytesReference (org.elasticsearch.common.bytes.BytesReference)50 XContentParser (org.elasticsearch.common.xcontent.XContentParser)45 Map (java.util.Map)20 XContentBuilder (org.elasticsearch.common.xcontent.XContentBuilder)20 HashMap (java.util.HashMap)14 IndexRequest (org.elasticsearch.action.index.IndexRequest)11 DocWriteRequest (org.elasticsearch.action.DocWriteRequest)10 IOException (java.io.IOException)9 DeleteRequest (org.elasticsearch.action.delete.DeleteRequest)8 ArrayList (java.util.ArrayList)6 Collections.emptyMap (java.util.Collections.emptyMap)6 List (java.util.List)6 ElasticsearchException (org.elasticsearch.ElasticsearchException)6 BulkRequest (org.elasticsearch.action.bulk.BulkRequest)6 UpdateRequest (org.elasticsearch.action.update.UpdateRequest)6 Collections.singletonMap (java.util.Collections.singletonMap)5 HttpEntity (org.apache.http.HttpEntity)5 ByteArrayEntity (org.apache.http.entity.ByteArrayEntity)5 GetRequest (org.elasticsearch.action.get.GetRequest)5