Search in sources :

Example 16 with NamedWriteableAwareStreamInput

use of org.opensearch.common.io.stream.NamedWriteableAwareStreamInput in project OpenSearch by opensearch-project.

the class GeoTileGridTests method testSerializationPreBounds.

public void testSerializationPreBounds() throws Exception {
    Version noBoundsSupportVersion = VersionUtils.randomVersionBetween(random(), LegacyESVersion.V_7_0_0, LegacyESVersion.V_7_5_0);
    GeoTileGridAggregationBuilder builder = createTestAggregatorBuilder();
    try (BytesStreamOutput output = new BytesStreamOutput()) {
        output.setVersion(LegacyESVersion.V_7_6_0);
        builder.writeTo(output);
        try (StreamInput in = new NamedWriteableAwareStreamInput(output.bytes().streamInput(), new NamedWriteableRegistry(Collections.emptyList()))) {
            in.setVersion(noBoundsSupportVersion);
            GeoTileGridAggregationBuilder readBuilder = new GeoTileGridAggregationBuilder(in);
            assertThat(readBuilder.geoBoundingBox(), equalTo(new GeoBoundingBox(new GeoPoint(Double.NaN, Double.NaN), new GeoPoint(Double.NaN, Double.NaN))));
        }
    }
}
Also used : NamedWriteableRegistry(org.opensearch.common.io.stream.NamedWriteableRegistry) GeoPoint(org.opensearch.common.geo.GeoPoint) GeoTileGridAggregationBuilder(org.opensearch.search.aggregations.bucket.geogrid.GeoTileGridAggregationBuilder) Version(org.opensearch.Version) LegacyESVersion(org.opensearch.LegacyESVersion) StreamInput(org.opensearch.common.io.stream.StreamInput) NamedWriteableAwareStreamInput(org.opensearch.common.io.stream.NamedWriteableAwareStreamInput) NamedWriteableAwareStreamInput(org.opensearch.common.io.stream.NamedWriteableAwareStreamInput) GeoBoundingBox(org.opensearch.common.geo.GeoBoundingBox) BytesStreamOutput(org.opensearch.common.io.stream.BytesStreamOutput)

Example 17 with NamedWriteableAwareStreamInput

use of org.opensearch.common.io.stream.NamedWriteableAwareStreamInput in project OpenSearch by opensearch-project.

the class BaseAggregationTestCase method testSerialization.

/**
 * Test serialization and deserialization of the test AggregatorFactory.
 */
public void testSerialization() throws IOException {
    AB testAgg = createTestAggregatorBuilder();
    try (BytesStreamOutput output = new BytesStreamOutput()) {
        output.writeNamedWriteable(testAgg);
        try (StreamInput in = new NamedWriteableAwareStreamInput(output.bytes().streamInput(), namedWriteableRegistry())) {
            AggregationBuilder deserialized = in.readNamedWriteable(AggregationBuilder.class);
            assertEquals(testAgg, deserialized);
            assertEquals(testAgg.hashCode(), deserialized.hashCode());
            assertNotSame(testAgg, deserialized);
        }
    }
}
Also used : ValuesSourceAggregationBuilder(org.opensearch.search.aggregations.support.ValuesSourceAggregationBuilder) StreamInput(org.opensearch.common.io.stream.StreamInput) NamedWriteableAwareStreamInput(org.opensearch.common.io.stream.NamedWriteableAwareStreamInput) NamedWriteableAwareStreamInput(org.opensearch.common.io.stream.NamedWriteableAwareStreamInput) BytesStreamOutput(org.opensearch.common.io.stream.BytesStreamOutput)

Example 18 with NamedWriteableAwareStreamInput

use of org.opensearch.common.io.stream.NamedWriteableAwareStreamInput in project OpenSearch by opensearch-project.

the class OpenSearchTestCase method copyInstance.

protected static <T> T copyInstance(T original, NamedWriteableRegistry namedWriteableRegistry, Writeable.Writer<T> writer, Writeable.Reader<T> reader, Version version) throws IOException {
    try (BytesStreamOutput output = new BytesStreamOutput()) {
        output.setVersion(version);
        writer.write(output, original);
        try (StreamInput in = new NamedWriteableAwareStreamInput(output.bytes().streamInput(), namedWriteableRegistry)) {
            in.setVersion(version);
            return reader.read(in);
        }
    }
}
Also used : NamedWriteableAwareStreamInput(org.opensearch.common.io.stream.NamedWriteableAwareStreamInput) StreamInput(org.opensearch.common.io.stream.StreamInput) NamedWriteableAwareStreamInput(org.opensearch.common.io.stream.NamedWriteableAwareStreamInput) BytesStreamOutput(org.opensearch.common.io.stream.BytesStreamOutput)

Example 19 with NamedWriteableAwareStreamInput

use of org.opensearch.common.io.stream.NamedWriteableAwareStreamInput in project OpenSearch by opensearch-project.

the class PublicationTransportHandler method handleIncomingPublishRequest.

private PublishWithJoinResponse handleIncomingPublishRequest(BytesTransportRequest request) throws IOException {
    final Compressor compressor = CompressorFactory.compressor(request.bytes());
    StreamInput in = request.bytes().streamInput();
    try {
        if (compressor != null) {
            in = new InputStreamStreamInput(compressor.threadLocalInputStream(in));
        }
        in = new NamedWriteableAwareStreamInput(in, namedWriteableRegistry);
        in.setVersion(request.version());
        // If true we received full cluster state - otherwise diffs
        if (in.readBoolean()) {
            final ClusterState incomingState;
            // Close early to release resources used by the de-compression as early as possible
            try (StreamInput input = in) {
                incomingState = ClusterState.readFrom(input, transportService.getLocalNode());
            } catch (Exception e) {
                logger.warn("unexpected error while deserializing an incoming cluster state", e);
                throw e;
            }
            fullClusterStateReceivedCount.incrementAndGet();
            logger.debug("received full cluster state version [{}] with size [{}]", incomingState.version(), request.bytes().length());
            final PublishWithJoinResponse response = acceptState(incomingState);
            lastSeenClusterState.set(incomingState);
            return response;
        } else {
            final ClusterState lastSeen = lastSeenClusterState.get();
            if (lastSeen == null) {
                logger.debug("received diff for but don't have any local cluster state - requesting full state");
                incompatibleClusterStateDiffReceivedCount.incrementAndGet();
                throw new IncompatibleClusterStateVersionException("have no local cluster state");
            } else {
                ClusterState incomingState;
                try {
                    final Diff<ClusterState> diff;
                    // Close stream early to release resources used by the de-compression as early as possible
                    try (StreamInput input = in) {
                        diff = ClusterState.readDiffFrom(input, lastSeen.nodes().getLocalNode());
                    }
                    // might throw IncompatibleClusterStateVersionException
                    incomingState = diff.apply(lastSeen);
                } catch (IncompatibleClusterStateVersionException e) {
                    incompatibleClusterStateDiffReceivedCount.incrementAndGet();
                    throw e;
                } catch (Exception e) {
                    logger.warn("unexpected error while deserializing an incoming cluster state", e);
                    throw e;
                }
                compatibleClusterStateDiffReceivedCount.incrementAndGet();
                logger.debug("received diff cluster state version [{}] with uuid [{}], diff size [{}]", incomingState.version(), incomingState.stateUUID(), request.bytes().length());
                final PublishWithJoinResponse response = acceptState(incomingState);
                lastSeenClusterState.compareAndSet(lastSeen, incomingState);
                return response;
            }
        }
    } finally {
        IOUtils.close(in);
    }
}
Also used : ClusterState(org.opensearch.cluster.ClusterState) InputStreamStreamInput(org.opensearch.common.io.stream.InputStreamStreamInput) NamedWriteableAwareStreamInput(org.opensearch.common.io.stream.NamedWriteableAwareStreamInput) StreamInput(org.opensearch.common.io.stream.StreamInput) Compressor(org.opensearch.common.compress.Compressor) NamedWriteableAwareStreamInput(org.opensearch.common.io.stream.NamedWriteableAwareStreamInput) IncompatibleClusterStateVersionException(org.opensearch.cluster.IncompatibleClusterStateVersionException) OpenSearchException(org.opensearch.OpenSearchException) IncompatibleClusterStateVersionException(org.opensearch.cluster.IncompatibleClusterStateVersionException) IOException(java.io.IOException) TransportException(org.opensearch.transport.TransportException) InputStreamStreamInput(org.opensearch.common.io.stream.InputStreamStreamInput)

Example 20 with NamedWriteableAwareStreamInput

use of org.opensearch.common.io.stream.NamedWriteableAwareStreamInput in project OpenSearch by opensearch-project.

the class PercolatorFieldMapperTests method assertQueryBuilder.

private void assertQueryBuilder(BytesRef actual, QueryBuilder expected) throws IOException {
    try (InputStream in = new ByteArrayInputStream(actual.bytes, actual.offset, actual.length)) {
        try (StreamInput input = new NamedWriteableAwareStreamInput(new InputStreamStreamInput(in), writableRegistry())) {
            // Query builder's content is stored via BinaryFieldMapper, which has a custom encoding
            // to encode multiple binary values into a single binary doc values field.
            // This is the reason we need to first need to read the number of values and
            // then the length of the field value in bytes.
            input.readVInt();
            input.readVInt();
            QueryBuilder queryBuilder = input.readNamedWriteable(QueryBuilder.class);
            assertThat(queryBuilder, equalTo(expected));
        }
    }
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) NamedWriteableAwareStreamInput(org.opensearch.common.io.stream.NamedWriteableAwareStreamInput) InputStreamStreamInput(org.opensearch.common.io.stream.InputStreamStreamInput) StreamInput(org.opensearch.common.io.stream.StreamInput) NamedWriteableAwareStreamInput(org.opensearch.common.io.stream.NamedWriteableAwareStreamInput) BoostingQueryBuilder(org.opensearch.index.query.BoostingQueryBuilder) BoolQueryBuilder(org.opensearch.index.query.BoolQueryBuilder) HasChildQueryBuilder(org.opensearch.join.query.HasChildQueryBuilder) ConstantScoreQueryBuilder(org.opensearch.index.query.ConstantScoreQueryBuilder) FunctionScoreQueryBuilder(org.opensearch.index.query.functionscore.FunctionScoreQueryBuilder) QueryBuilder(org.opensearch.index.query.QueryBuilder) ScriptQueryBuilder(org.opensearch.index.query.ScriptQueryBuilder) DisMaxQueryBuilder(org.opensearch.index.query.DisMaxQueryBuilder) RangeQueryBuilder(org.opensearch.index.query.RangeQueryBuilder) HasParentQueryBuilder(org.opensearch.join.query.HasParentQueryBuilder) MatchAllQueryBuilder(org.opensearch.index.query.MatchAllQueryBuilder) InputStreamStreamInput(org.opensearch.common.io.stream.InputStreamStreamInput)

Aggregations

NamedWriteableAwareStreamInput (org.opensearch.common.io.stream.NamedWriteableAwareStreamInput)43 StreamInput (org.opensearch.common.io.stream.StreamInput)37 BytesStreamOutput (org.opensearch.common.io.stream.BytesStreamOutput)32 NamedWriteableRegistry (org.opensearch.common.io.stream.NamedWriteableRegistry)21 Version (org.opensearch.Version)11 InputStreamStreamInput (org.opensearch.common.io.stream.InputStreamStreamInput)7 ByteArrayInputStream (java.io.ByteArrayInputStream)5 IOException (java.io.IOException)5 XContentBuilder (org.opensearch.common.xcontent.XContentBuilder)5 QueryBuilder (org.opensearch.index.query.QueryBuilder)5 SearchModule (org.opensearch.search.SearchModule)5 InputStream (java.io.InputStream)4 BytesRef (org.apache.lucene.util.BytesRef)4 ClusterState (org.opensearch.cluster.ClusterState)4 BytesReference (org.opensearch.common.bytes.BytesReference)4 AliasFilter (org.opensearch.search.internal.AliasFilter)4 ArrayList (java.util.ArrayList)3 LegacyESVersion (org.opensearch.LegacyESVersion)3 ShardId (org.opensearch.index.shard.ShardId)3 HashMap (java.util.HashMap)2