Search in sources :

Example 6 with CacheRequest

use of okhttp3.internal.cache.CacheRequest in project okhttp by square.

the class JavaApiConverter method createOkResponseForCacheGet.

/**
   * Creates an OkHttp {@link Response} using the supplied {@link Request} and {@link CacheResponse}
   * to supply the data.
   */
static Response createOkResponseForCacheGet(Request request, CacheResponse javaResponse) throws IOException {
    // Build a cache request for the response to use.
    Headers responseHeaders = createHeaders(javaResponse.getHeaders());
    Headers varyHeaders;
    if (HttpHeaders.hasVaryAll(responseHeaders)) {
        // "*" means that this will be treated as uncacheable anyway.
        varyHeaders = new Headers.Builder().build();
    } else {
        varyHeaders = HttpHeaders.varyHeaders(request.headers(), responseHeaders);
    }
    Request cacheRequest = new Request.Builder().url(request.url()).method(request.method(), null).headers(varyHeaders).build();
    Response.Builder okResponseBuilder = new Response.Builder();
    // Request: Use the cacheRequest we built.
    okResponseBuilder.request(cacheRequest);
    // Status line: Java has this as one of the headers.
    StatusLine statusLine = StatusLine.parse(extractStatusLine(javaResponse));
    okResponseBuilder.protocol(statusLine.protocol);
    okResponseBuilder.code(statusLine.code);
    okResponseBuilder.message(statusLine.message);
    // Response headers
    Headers okHeaders = extractOkHeaders(javaResponse, okResponseBuilder);
    okResponseBuilder.headers(okHeaders);
    // Response body
    ResponseBody okBody = createOkBody(okHeaders, javaResponse);
    okResponseBuilder.body(okBody);
    // Handle SSL handshake information as needed.
    if (javaResponse instanceof SecureCacheResponse) {
        SecureCacheResponse javaSecureCacheResponse = (SecureCacheResponse) javaResponse;
        // Handshake doesn't support null lists.
        List<Certificate> peerCertificates;
        try {
            peerCertificates = javaSecureCacheResponse.getServerCertificateChain();
        } catch (SSLPeerUnverifiedException e) {
            peerCertificates = Collections.emptyList();
        }
        List<Certificate> localCertificates = javaSecureCacheResponse.getLocalCertificateChain();
        if (localCertificates == null) {
            localCertificates = Collections.emptyList();
        }
        String cipherSuiteString = javaSecureCacheResponse.getCipherSuite();
        CipherSuite cipherSuite = CipherSuite.forJavaName(cipherSuiteString);
        Handshake handshake = Handshake.get(null, cipherSuite, peerCertificates, localCertificates);
        okResponseBuilder.handshake(handshake);
    }
    return okResponseBuilder.build();
}
Also used : SecureCacheResponse(java.net.SecureCacheResponse) HttpHeaders(okhttp3.internal.http.HttpHeaders) Headers(okhttp3.Headers) JavaNetHeaders(okhttp3.internal.JavaNetHeaders) CipherSuite(okhttp3.CipherSuite) SSLPeerUnverifiedException(javax.net.ssl.SSLPeerUnverifiedException) CacheRequest(okhttp3.internal.cache.CacheRequest) Request(okhttp3.Request) ResponseBody(okhttp3.ResponseBody) CacheResponse(java.net.CacheResponse) Response(okhttp3.Response) SecureCacheResponse(java.net.SecureCacheResponse) StatusLine(okhttp3.internal.http.StatusLine) Certificate(java.security.cert.Certificate) Handshake(okhttp3.Handshake)

Example 7 with CacheRequest

use of okhttp3.internal.cache.CacheRequest in project okhttp by square.

the class URLEncodingTest method backdoorUrlToUri.

private URI backdoorUrlToUri(URL url) throws Exception {
    final AtomicReference<URI> uriReference = new AtomicReference<>();
    OkHttpClient.Builder builder = new OkHttpClient.Builder();
    Internal.instance.setCache(builder, new InternalCache() {

        @Override
        public Response get(Request request) throws IOException {
            uriReference.set(request.url().uri());
            throw new UnsupportedOperationException();
        }

        @Override
        public CacheRequest put(Response response) throws IOException {
            return null;
        }

        @Override
        public void remove(Request request) throws IOException {
        }

        @Override
        public void update(Response cached, Response network) {
        }

        @Override
        public void trackConditionalCacheHit() {
        }

        @Override
        public void trackResponse(CacheStrategy cacheStrategy) {
        }
    });
    try {
        HttpURLConnection connection = new OkUrlFactory(builder.build()).open(url);
        connection.getResponseCode();
    } catch (Exception expected) {
        if (expected.getCause() instanceof URISyntaxException) {
            expected.printStackTrace();
        }
    }
    return uriReference.get();
}
Also used : OkHttpClient(okhttp3.OkHttpClient) Request(okhttp3.Request) CacheRequest(okhttp3.internal.cache.CacheRequest) InternalCache(okhttp3.internal.cache.InternalCache) AtomicReference(java.util.concurrent.atomic.AtomicReference) IOException(java.io.IOException) URISyntaxException(java.net.URISyntaxException) URI(java.net.URI) URISyntaxException(java.net.URISyntaxException) IOException(java.io.IOException) Response(okhttp3.Response) OkUrlFactory(okhttp3.OkUrlFactory) HttpURLConnection(java.net.HttpURLConnection) CacheRequest(okhttp3.internal.cache.CacheRequest) CacheStrategy(okhttp3.internal.cache.CacheStrategy)

Example 8 with CacheRequest

use of okhttp3.internal.cache.CacheRequest in project okhttp by square.

the class Cache method put.

CacheRequest put(Response response) {
    String requestMethod = response.request().method();
    if (HttpMethod.invalidatesCache(response.request().method())) {
        try {
            remove(response.request());
        } catch (IOException ignored) {
        // The cache cannot be written.
        }
        return null;
    }
    if (!requestMethod.equals("GET")) {
        // so is high and the benefit is low.
        return null;
    }
    if (HttpHeaders.hasVaryAll(response)) {
        return null;
    }
    Entry entry = new Entry(response);
    DiskLruCache.Editor editor = null;
    try {
        editor = cache.edit(key(response.request().url()));
        if (editor == null) {
            return null;
        }
        entry.writeTo(editor);
        return new CacheRequestImpl(editor);
    } catch (IOException e) {
        abortQuietly(editor);
        return null;
    }
}
Also used : DiskLruCache(okhttp3.internal.cache.DiskLruCache) ByteString(okio.ByteString) IOException(java.io.IOException)

Example 9 with CacheRequest

use of okhttp3.internal.cache.CacheRequest in project buck by facebook.

the class ThriftArtifactCache method fetchImpl.

@Override
public CacheResult fetchImpl(RuleKey ruleKey, LazyPath output, HttpArtifactCacheEvent.Finished.Builder eventBuilder) throws IOException {
    BuckCacheFetchRequest fetchRequest = new BuckCacheFetchRequest();
    com.facebook.buck.artifact_cache.thrift.RuleKey thriftRuleKey = new com.facebook.buck.artifact_cache.thrift.RuleKey();
    thriftRuleKey.setHashString(ruleKey.getHashCode().toString());
    fetchRequest.setRuleKey(thriftRuleKey);
    fetchRequest.setRepository(repository);
    fetchRequest.setScheduleType(scheduleType);
    fetchRequest.setDistributedBuildModeEnabled(distributedBuildModeEnabled);
    BuckCacheRequest cacheRequest = new BuckCacheRequest();
    cacheRequest.setType(BuckCacheRequestType.FETCH);
    cacheRequest.setFetchRequest(fetchRequest);
    LOG.verbose("Will fetch key %s", thriftRuleKey);
    final ThriftArtifactCacheProtocol.Request request = ThriftArtifactCacheProtocol.createRequest(PROTOCOL, cacheRequest);
    Request.Builder builder = toOkHttpRequest(request);
    try (HttpResponse httpResponse = fetchClient.makeRequest(hybridThriftEndpoint, builder)) {
        if (httpResponse.statusCode() != 200) {
            String message = String.format("Failed to fetch cache artifact with HTTP status code [%d:%s] " + " to url [%s] for rule key [%s].", httpResponse.statusCode(), httpResponse.statusMessage(), httpResponse.requestUrl(), ruleKey.toString());
            LOG.error(message);
            return CacheResult.error(name, message);
        }
        try (ThriftArtifactCacheProtocol.Response response = ThriftArtifactCacheProtocol.parseResponse(PROTOCOL, httpResponse.getBody())) {
            eventBuilder.getFetchBuilder().setResponseSizeBytes(httpResponse.contentLength());
            BuckCacheResponse cacheResponse = response.getThriftData();
            if (!cacheResponse.isWasSuccessful()) {
                LOG.warn("Request was unsuccessful: %s", cacheResponse.getErrorMessage());
                return CacheResult.error(name, cacheResponse.getErrorMessage());
            }
            BuckCacheFetchResponse fetchResponse = cacheResponse.getFetchResponse();
            if (LOG.isDebugEnabled()) {
                LOG.debug("Debug info for cache fetch request: request=[%s] response=[%s]", ThriftUtil.thriftToDebugJson(cacheRequest), ThriftUtil.thriftToDebugJson(cacheResponse));
            }
            if (!fetchResponse.isArtifactExists()) {
                LOG.verbose("Artifact did not exist.");
                return CacheResult.miss();
            }
            LOG.verbose("Got artifact.  Attempting to read payload.");
            Path tmp = createTempFileForDownload();
            ThriftArtifactCacheProtocol.Response.ReadPayloadInfo readResult;
            try (OutputStream tmpFile = projectFilesystem.newFileOutputStream(tmp)) {
                readResult = response.readPayload(tmpFile);
                LOG.verbose("Successfully read payload: %d bytes.", readResult.getBytesRead());
            }
            ArtifactMetadata metadata = fetchResponse.getMetadata();
            if (LOG.isVerboseEnabled()) {
                LOG.verbose(String.format("Fetched artifact with rule key [%s] contains the following metadata: [%s]", ruleKey, ThriftUtil.thriftToDebugJson(metadata)));
            }
            eventBuilder.setTarget(Optional.ofNullable(metadata.getBuildTarget())).getFetchBuilder().setAssociatedRuleKeys(toImmutableSet(metadata.getRuleKeys())).setArtifactSizeBytes(readResult.getBytesRead());
            if (!metadata.isSetArtifactPayloadMd5()) {
                String msg = "Fetched artifact is missing the MD5 hash.";
                LOG.warn(msg);
            } else {
                eventBuilder.getFetchBuilder().setArtifactContentHash(metadata.getArtifactPayloadMd5());
                if (!readResult.getMd5Hash().equals(fetchResponse.getMetadata().getArtifactPayloadMd5())) {
                    String msg = String.format("The artifact fetched from cache is corrupted. ExpectedMD5=[%s] ActualMD5=[%s]", fetchResponse.getMetadata().getArtifactPayloadMd5(), readResult.getMd5Hash());
                    LOG.error(msg);
                    return CacheResult.error(name, msg);
                }
            }
            // This makes sure we don't have 'half downloaded files' in the dir cache.
            projectFilesystem.move(tmp, output.get(), StandardCopyOption.REPLACE_EXISTING);
            return CacheResult.hit(name, ImmutableMap.copyOf(fetchResponse.getMetadata().getMetadata()), readResult.getBytesRead());
        }
    }
}
Also used : Path(java.nio.file.Path) LazyPath(com.facebook.buck.io.LazyPath) BuckCacheFetchResponse(com.facebook.buck.artifact_cache.thrift.BuckCacheFetchResponse) RuleKey(com.facebook.buck.rules.RuleKey) OutputStream(java.io.OutputStream) BuckCacheStoreRequest(com.facebook.buck.artifact_cache.thrift.BuckCacheStoreRequest) Request(okhttp3.Request) BuckCacheRequest(com.facebook.buck.artifact_cache.thrift.BuckCacheRequest) BuckCacheFetchRequest(com.facebook.buck.artifact_cache.thrift.BuckCacheFetchRequest) HttpResponse(com.facebook.buck.slb.HttpResponse) BuckCacheRequest(com.facebook.buck.artifact_cache.thrift.BuckCacheRequest) BuckCacheFetchRequest(com.facebook.buck.artifact_cache.thrift.BuckCacheFetchRequest) BuckCacheResponse(com.facebook.buck.artifact_cache.thrift.BuckCacheResponse) HttpResponse(com.facebook.buck.slb.HttpResponse) BuckCacheFetchResponse(com.facebook.buck.artifact_cache.thrift.BuckCacheFetchResponse) BuckCacheResponse(com.facebook.buck.artifact_cache.thrift.BuckCacheResponse) ArtifactMetadata(com.facebook.buck.artifact_cache.thrift.ArtifactMetadata)

Example 10 with CacheRequest

use of okhttp3.internal.cache.CacheRequest in project buck by facebook.

the class ThriftArtifactCache method storeImpl.

@Override
protected void storeImpl(final ArtifactInfo info, final Path file, final HttpArtifactCacheEvent.Finished.Builder eventBuilder) throws IOException {
    final ByteSource artifact = new ByteSource() {

        @Override
        public InputStream openStream() throws IOException {
            return projectFilesystem.newFileInputStream(file);
        }
    };
    BuckCacheStoreRequest storeRequest = new BuckCacheStoreRequest();
    ArtifactMetadata artifactMetadata = infoToMetadata(info, artifact, repository, scheduleType, distributedBuildModeEnabled);
    storeRequest.setMetadata(artifactMetadata);
    PayloadInfo payloadInfo = new PayloadInfo();
    long artifactSizeBytes = artifact.size();
    payloadInfo.setSizeBytes(artifactSizeBytes);
    BuckCacheRequest cacheRequest = new BuckCacheRequest();
    cacheRequest.addToPayloads(payloadInfo);
    cacheRequest.setType(BuckCacheRequestType.STORE);
    cacheRequest.setStoreRequest(storeRequest);
    if (LOG.isVerboseEnabled()) {
        LOG.verbose(String.format("Storing artifact with metadata: [%s].", ThriftUtil.thriftToDebugJson(artifactMetadata)));
    }
    final ThriftArtifactCacheProtocol.Request request = ThriftArtifactCacheProtocol.createRequest(PROTOCOL, cacheRequest, artifact);
    Request.Builder builder = toOkHttpRequest(request);
    eventBuilder.getStoreBuilder().setRequestSizeBytes(request.getRequestLengthBytes());
    try (HttpResponse httpResponse = storeClient.makeRequest(hybridThriftEndpoint, builder)) {
        if (httpResponse.statusCode() != 200) {
            throw new IOException(String.format("Failed to store cache artifact with HTTP status code [%d:%s] " + " to url [%s] for build target [%s] that has size [%d] bytes.", httpResponse.statusCode(), httpResponse.statusMessage(), httpResponse.requestUrl(), info.getBuildTarget().orElse(null), artifactSizeBytes));
        }
        try (ThriftArtifactCacheProtocol.Response response = ThriftArtifactCacheProtocol.parseResponse(PROTOCOL, httpResponse.getBody())) {
            BuckCacheResponse cacheResponse = response.getThriftData();
            if (!cacheResponse.isWasSuccessful()) {
                reportFailure("Failed to store artifact with thriftErrorMessage=[%s] " + "url=[%s] artifactSizeBytes=[%d]", response.getThriftData().getErrorMessage(), httpResponse.requestUrl(), artifactSizeBytes);
            }
            eventBuilder.getStoreBuilder().setArtifactContentHash(storeRequest.getMetadata().artifactPayloadMd5);
            eventBuilder.getStoreBuilder().setWasStoreSuccessful(cacheResponse.isWasSuccessful());
            if (LOG.isDebugEnabled()) {
                LOG.debug("Debug info for cache store request: artifactMetadata=[%s] response=[%s]", ThriftUtil.thriftToDebugJson(artifactMetadata), ThriftUtil.thriftToDebugJson(cacheResponse));
            }
        }
    }
}
Also used : BuckCacheStoreRequest(com.facebook.buck.artifact_cache.thrift.BuckCacheStoreRequest) Request(okhttp3.Request) BuckCacheRequest(com.facebook.buck.artifact_cache.thrift.BuckCacheRequest) BuckCacheFetchRequest(com.facebook.buck.artifact_cache.thrift.BuckCacheFetchRequest) HttpResponse(com.facebook.buck.slb.HttpResponse) BuckCacheRequest(com.facebook.buck.artifact_cache.thrift.BuckCacheRequest) IOException(java.io.IOException) ByteSource(com.google.common.io.ByteSource) BuckCacheStoreRequest(com.facebook.buck.artifact_cache.thrift.BuckCacheStoreRequest) PayloadInfo(com.facebook.buck.artifact_cache.thrift.PayloadInfo) ArtifactMetadata(com.facebook.buck.artifact_cache.thrift.ArtifactMetadata) BuckCacheResponse(com.facebook.buck.artifact_cache.thrift.BuckCacheResponse)

Aggregations

URI (java.net.URI)9 IOException (java.io.IOException)7 HttpURLConnection (java.net.HttpURLConnection)7 Request (okhttp3.Request)7 Test (org.junit.Test)7 URLConnection (java.net.URLConnection)5 HttpsURLConnection (javax.net.ssl.HttpsURLConnection)5 AbstractResponseCache (okhttp3.AbstractResponseCache)5 Response (okhttp3.Response)5 MockResponse (okhttp3.mockwebserver.MockResponse)5 URL (java.net.URL)4 OkUrlFactory (okhttp3.OkUrlFactory)4 ByteArrayInputStream (java.io.ByteArrayInputStream)3 CacheResponse (java.net.CacheResponse)3 ResponseCache (java.net.ResponseCache)3 SecureCacheResponse (java.net.SecureCacheResponse)3 Headers (okhttp3.Headers)3 CacheRequest (okhttp3.internal.cache.CacheRequest)3 ArtifactMetadata (com.facebook.buck.artifact_cache.thrift.ArtifactMetadata)2 BuckCacheFetchRequest (com.facebook.buck.artifact_cache.thrift.BuckCacheFetchRequest)2