Examples with Document com.yahoo.vespa.http.client.core.Document used on opensource projects

Example 16 with Document

use of com.yahoo.vespa.http.client.core.Document in project vespa by vespa-engine.

the class OperationProcessorTest method testBlockingOfOperationsToSameDocIdMany.

@Test
public void testBlockingOfOperationsToSameDocIdMany() {
    SessionParams sessionParams = new SessionParams.Builder().addCluster(new Cluster.Builder().addEndpoint(Endpoint.create("host")).build()).setConnectionParams(new ConnectionParams.Builder().setEnableV3Protocol(true).build()).build();
    OperationProcessor operationProcessor = new OperationProcessor(new IncompleteResultsThrottler(1000, 1000, null, null), (docId, documentResult) -> queue.add(documentResult), sessionParams, null);
    Queue<Document> documentQueue = new ArrayDeque<>();
    for (int x = 0; x < 100; x++) {
        Document document = new Document("doc:a:b", String.valueOf(x), null);
        operationProcessor.sendDocument(document);
        documentQueue.add(document);
    }
    for (int x = 0; x < 100; x++) {
        assertThat(queue.size(), is(x));
        // Only one operations should be in flight.
        assertThat(operationProcessor.getIncompleteResultQueueSize(), is(1));
        Document document = documentQueue.poll();
        operationProcessor.resultReceived(new EndpointResult(document.getOperationId(), new Result.Detail(Endpoint.create("host"))), 0);
        assertThat(queue.size(), is(x + 1));
        if (x < 99) {
            assertThat(operationProcessor.getIncompleteResultQueueSize(), is(1));
        } else {
            assertThat(operationProcessor.getIncompleteResultQueueSize(), is(0));
        }
    }
}

Example 17 with Document

use of com.yahoo.vespa.http.client.core.Document in project vespa by vespa-engine.

the class ApacheGatewayConnection method getDataWithStartAndEndOfFeed.

private ByteBuffer[] getDataWithStartAndEndOfFeed(List<Document> docs, int version) {
    List<ByteBuffer> data = new ArrayList<ByteBuffer>();
    if (version == 2 || version == 3) {
        for (Document doc : docs) {
            int operationSize = doc.size() + startOfFeed.length + endOfFeed.length;
            StringBuilder envelope = new StringBuilder();
            Encoder.encode(doc.getOperationId(), envelope);
            envelope.append(' ');
            envelope.append(Integer.toHexString(operationSize));
            envelope.append('\n');
            data.add(StandardCharsets.US_ASCII.encode(envelope.toString()));
            data.add(ByteBuffer.wrap(startOfFeed));
            data.add(doc.getData());
            data.add(ByteBuffer.wrap(endOfFeed));
        }
    } else {
        throw new IllegalArgumentException("Protocol version " + version + " unsupported by client.");
    }
    return data.toArray(new ByteBuffer[data.size()]);
}

Example 18 with Document

use of com.yahoo.vespa.http.client.core.Document in project vespa by vespa-engine.

the class MultiClusterSessionOutputStream method close.

@Override
public void close() throws IOException {
    Document document = new Document(documentId.toString(), toByteArray(), context);
    operationProcessor.sendDocument(document);
    super.close();
}