Search in sources :

Example 21 with Response

use of build.bazel.remote.execution.v2.BatchReadBlobsResponse.Response in project bazel-buildfarm by bazelbuild.

the class ContentAddressableStorageService method instanceFindMissingBlobs.

void instanceFindMissingBlobs(Instance instance, FindMissingBlobsRequest request, StreamObserver<FindMissingBlobsResponse> responseObserver) {
    Stopwatch stopwatch = Stopwatch.createStarted();
    FindMissingBlobsResponse.Builder builder = FindMissingBlobsResponse.newBuilder();
    ListenableFuture<FindMissingBlobsResponse.Builder> responseFuture = transform(instance.findMissingBlobs(request.getBlobDigestsList(), TracingMetadataUtils.fromCurrentContext()), builder::addAllMissingBlobDigests, directExecutor());
    addCallback(responseFuture, new FutureCallback<FindMissingBlobsResponse.Builder>() {

        @Override
        public void onSuccess(FindMissingBlobsResponse.Builder builder) {
            try {
                FindMissingBlobsResponse response = builder.build();
                responseObserver.onNext(response);
                responseObserver.onCompleted();
                long elapsedMicros = stopwatch.elapsed(MICROSECONDS);
                missingBlobs.observe(request.getBlobDigestsList().size());
                logger.log(Level.FINE, "FindMissingBlobs(" + instance.getName() + ") for " + request.getBlobDigestsList().size() + " blobs in " + elapsedMicros / 1000.0);
            } catch (Throwable t) {
                onFailure(t);
            }
        }

        @SuppressWarnings("NullableProblems")
        @Override
        public void onFailure(Throwable t) {
            Status status = Status.fromThrowable(t);
            if (status.getCode() != Code.CANCELLED) {
                logger.log(Level.SEVERE, format("findMissingBlobs(%s): %d", request.getInstanceName(), request.getBlobDigestsCount()), t);
                responseObserver.onError(t);
            }
        }
    }, directExecutor());
}
Also used : Status(io.grpc.Status) FindMissingBlobsResponse(build.bazel.remote.execution.v2.FindMissingBlobsResponse) Stopwatch(com.google.common.base.Stopwatch)

Example 22 with Response

use of build.bazel.remote.execution.v2.BatchReadBlobsResponse.Response in project bazel-buildfarm by bazelbuild.

the class StubInstance method putAllBlobs.

@Override
public Iterable<Digest> putAllBlobs(Iterable<ByteString> blobs, RequestMetadata requestMetadata) {
    long totalSize = 0;
    ImmutableList.Builder<Request> requests = ImmutableList.builder();
    for (ByteString blob : blobs) {
        checkState(totalSize + blob.size() <= maxBatchUpdateBlobsSize);
        requests.add(Request.newBuilder().setDigest(digestUtil.compute(blob)).setData(blob).build());
        totalSize += blob.size();
    }
    BatchUpdateBlobsRequest batchRequest = BatchUpdateBlobsRequest.newBuilder().setInstanceName(getName()).addAllRequests(requests.build()).build();
    BatchUpdateBlobsResponse batchResponse = deadlined(casBlockingStub).withInterceptors(attachMetadataInterceptor(requestMetadata)).batchUpdateBlobs(batchRequest);
    PutAllBlobsException exception = null;
    for (BatchUpdateBlobsResponse.Response response : batchResponse.getResponsesList()) {
        com.google.rpc.Status status = response.getStatus();
        if (Code.forNumber(status.getCode()) != Code.OK) {
            if (exception == null) {
                exception = new PutAllBlobsException();
            }
            exception.addFailedResponse(response);
        }
    }
    if (exception != null) {
        throw exception;
    }
    return Iterables.transform(batchResponse.getResponsesList(), BatchUpdateBlobsResponse.Response::getDigest);
}
Also used : OperationsGrpc(com.google.longrunning.OperationsGrpc) ByteStreamGrpc(com.google.bytestream.ByteStreamGrpc) OperationQueueGrpc(build.buildfarm.v1test.OperationQueueGrpc) ExecutionGrpc(build.bazel.remote.execution.v2.ExecutionGrpc) ActionCacheGrpc(build.bazel.remote.execution.v2.ActionCacheGrpc) WorkerProfileGrpc(build.buildfarm.v1test.WorkerProfileGrpc) AdminGrpc(build.buildfarm.v1test.AdminGrpc) CapabilitiesGrpc(build.bazel.remote.execution.v2.CapabilitiesGrpc) ContentAddressableStorageGrpc(build.bazel.remote.execution.v2.ContentAddressableStorageGrpc) ShutDownWorkerGrpc(build.buildfarm.v1test.ShutDownWorkerGrpc) BatchUpdateBlobsResponse(build.bazel.remote.execution.v2.BatchUpdateBlobsResponse) ImmutableList(com.google.common.collect.ImmutableList) ByteString(com.google.protobuf.ByteString) BackplaneStatusRequest(build.buildfarm.v1test.BackplaneStatusRequest) ReadRequest(com.google.bytestream.ByteStreamProto.ReadRequest) WaitExecutionRequest(build.bazel.remote.execution.v2.WaitExecutionRequest) ListOperationsRequest(com.google.longrunning.ListOperationsRequest) TakeOperationRequest(build.buildfarm.v1test.TakeOperationRequest) FindMissingBlobsRequest(build.bazel.remote.execution.v2.FindMissingBlobsRequest) WorkerProfileRequest(build.buildfarm.v1test.WorkerProfileRequest) ReindexAllCasRequest(build.buildfarm.v1test.ReindexAllCasRequest) WorkerListRequest(build.buildfarm.v1test.WorkerListRequest) Request(build.bazel.remote.execution.v2.BatchUpdateBlobsRequest.Request) BatchUpdateBlobsRequest(build.bazel.remote.execution.v2.BatchUpdateBlobsRequest) PrepareWorkerForGracefulShutDownRequest(build.buildfarm.v1test.PrepareWorkerForGracefulShutDownRequest) ReindexCasRequest(build.buildfarm.v1test.ReindexCasRequest) GetActionResultRequest(build.bazel.remote.execution.v2.GetActionResultRequest) GetClientStartTimeRequest(build.buildfarm.v1test.GetClientStartTimeRequest) GetTreeRequest(build.bazel.remote.execution.v2.GetTreeRequest) PollOperationRequest(build.buildfarm.v1test.PollOperationRequest) CancelOperationRequest(com.google.longrunning.CancelOperationRequest) GetCapabilitiesRequest(build.bazel.remote.execution.v2.GetCapabilitiesRequest) UpdateActionResultRequest(build.bazel.remote.execution.v2.UpdateActionResultRequest) GetOperationRequest(com.google.longrunning.GetOperationRequest) DeleteOperationRequest(com.google.longrunning.DeleteOperationRequest) BatchReadBlobsRequest(build.bazel.remote.execution.v2.BatchReadBlobsRequest) ShutDownWorkerGracefullyRequest(build.buildfarm.v1test.ShutDownWorkerGracefullyRequest) BatchReadBlobsResponse(build.bazel.remote.execution.v2.BatchReadBlobsResponse) ListOperationsResponse(com.google.longrunning.ListOperationsResponse) ReadResponse(com.google.bytestream.ByteStreamProto.ReadResponse) FindMissingBlobsResponse(build.bazel.remote.execution.v2.FindMissingBlobsResponse) GetTreeResponse(build.bazel.remote.execution.v2.GetTreeResponse) Response(build.bazel.remote.execution.v2.BatchReadBlobsResponse.Response) BatchUpdateBlobsResponse(build.bazel.remote.execution.v2.BatchUpdateBlobsResponse) BatchUpdateBlobsRequest(build.bazel.remote.execution.v2.BatchUpdateBlobsRequest)

Example 23 with Response

use of build.bazel.remote.execution.v2.BatchReadBlobsResponse.Response in project bazel-buildfarm by bazelbuild.

the class AbstractServerInstance method expireOperation.

protected void expireOperation(Operation operation) throws InterruptedException {
    ActionResult actionResult = ActionResult.newBuilder().setExitCode(-1).setStderrRaw(ByteString.copyFromUtf8("[BUILDFARM]: Action timed out with no response from worker")).build();
    ExecuteResponse executeResponse = ExecuteResponse.newBuilder().setResult(actionResult).setStatus(com.google.rpc.Status.newBuilder().setCode(Code.DEADLINE_EXCEEDED.getNumber())).build();
    ExecuteOperationMetadata metadata = expectExecuteOperationMetadata(operation);
    if (metadata == null) {
        throw new IllegalStateException("Operation " + operation.getName() + " did not contain valid metadata");
    }
    metadata = metadata.toBuilder().setStage(ExecutionStage.Value.COMPLETED).build();
    putOperation(operation.toBuilder().setDone(true).setMetadata(Any.pack(metadata)).setResponse(Any.pack(executeResponse)).build());
}
Also used : ActionResult(build.bazel.remote.execution.v2.ActionResult) ExecuteOperationMetadata(build.bazel.remote.execution.v2.ExecuteOperationMetadata) ExecuteResponse(build.bazel.remote.execution.v2.ExecuteResponse)

Example 24 with Response

use of build.bazel.remote.execution.v2.BatchReadBlobsResponse.Response in project tools_remote by bazelbuild.

the class ActionGroupingTest method makeExecute.

private RpcCallDetails makeExecute(ActionResult result) {
    ExecuteResponse response = ExecuteResponse.newBuilder().setResult(result).build();
    Operation operation = Operation.newBuilder().setResponse(Any.pack(response)).setDone(true).build();
    ExecuteDetails execute = ExecuteDetails.newBuilder().addResponses(operation).build();
    return RpcCallDetails.newBuilder().setExecute(execute).build();
}
Also used : ExecuteDetails(com.google.devtools.build.lib.remote.logging.RemoteExecutionLog.ExecuteDetails) ExecuteResponse(build.bazel.remote.execution.v2.ExecuteResponse) Operation(com.google.longrunning.Operation)

Example 25 with Response

use of build.bazel.remote.execution.v2.BatchReadBlobsResponse.Response in project tools_remote by bazelbuild.

the class LogParserUtils method extractExecuteResponse.

private static List<ExecuteResponse> extractExecuteResponse(List<Operation> operations) throws IOException {
    ArrayList<ExecuteResponse> result = new ArrayList<>();
    for (Operation o : operations) {
        StringBuilder error = new StringBuilder();
        ExecuteResponse response = getExecuteResponse(o, ExecuteResponse.class, error);
        if (response != null && (response.hasResult() || (response.hasStatus()) && response.getStatus().getCode() != Code.OK.value())) {
            result.add(response);
        }
    }
    return result;
}
Also used : ArrayList(java.util.ArrayList) ExecuteResponse(build.bazel.remote.execution.v2.ExecuteResponse) Operation(com.google.longrunning.Operation)

Aggregations

Digest (build.bazel.remote.execution.v2.Digest)10 ByteString (com.google.protobuf.ByteString)10 ExecuteResponse (build.bazel.remote.execution.v2.ExecuteResponse)9 Test (org.junit.Test)9 FindMissingBlobsResponse (build.bazel.remote.execution.v2.FindMissingBlobsResponse)7 Operation (com.google.longrunning.Operation)7 ContentAddressableStorageGrpc (build.bazel.remote.execution.v2.ContentAddressableStorageGrpc)6 FindMissingBlobsRequest (build.bazel.remote.execution.v2.FindMissingBlobsRequest)5 ImmutableList (com.google.common.collect.ImmutableList)5 ActionResult (build.bazel.remote.execution.v2.ActionResult)4 BatchReadBlobsResponse (build.bazel.remote.execution.v2.BatchReadBlobsResponse)4 BatchUpdateBlobsRequest (build.bazel.remote.execution.v2.BatchUpdateBlobsRequest)4 BatchUpdateBlobsResponse (build.bazel.remote.execution.v2.BatchUpdateBlobsResponse)4 GetTreeResponse (build.bazel.remote.execution.v2.GetTreeResponse)4 DigestUtil (build.buildfarm.common.DigestUtil)4 Stopwatch (com.google.common.base.Stopwatch)4 Request (build.bazel.remote.execution.v2.BatchUpdateBlobsRequest.Request)3 ExecuteOperationMetadata (build.bazel.remote.execution.v2.ExecuteOperationMetadata)3 ExecutionGrpc (build.bazel.remote.execution.v2.ExecutionGrpc)3 RequestMetadata (build.bazel.remote.execution.v2.RequestMetadata)3