Search in sources :

Example 6 with DatasetResponseWithStatusCodeAndHeaders

use of won.protocol.rest.DatasetResponseWithStatusCodeAndHeaders in project webofneeds by researchstudio-sat.

the class CachingLinkedDataSource method fetchAndCacheIfAppropriate.

private DatasetResponseWithStatusCodeAndHeaders fetchAndCacheIfAppropriate(final URI resource, final URI requesterWebID, final LinkedDataCacheEntry linkedDataCacheEntry, final HttpHeaders headers) {
    DatasetResponseWithStatusCodeAndHeaders responseData = fetchWithEtagValidation(resource, requesterWebID, linkedDataCacheEntry, headers);
    Date expires = parseCacheControlMaxAgeValue(resource, responseData);
    if (expires == null) {
        expires = parseExpiresHeader(resource, responseData);
        if (expires != null && expires.getTime() == 0) {
            // Don't cache.
            return responseData;
        }
    }
    EnumSet<CacheControlFlag> cacheControlFlags = parseCacheControlHeaderFlags(resource, responseData);
    if (cacheControlFlags.contains(CacheControlFlag.NO_STORE) || cacheControlFlags.contains(CacheControlFlag.NO_CACHE)) {
        // we are not allowed to cache the result
        // make sure it's not in the cache from a previous request
        cache.remove(makeCacheKey(resource, requesterWebID));
        logger.debug("Fetched {}. Will not be cached due to Cache-Control headers sent by server", resource);
        return responseData;
    }
    Date responseDate = parseDateHeader(resource, responseData);
    if (responseDate != null && expires != null) {
        // old way of saying don't cache: Date header >= Expires header
        if (responseDate.equals(expires) || responseDate.after(expires)) {
            // we are not allowed to cache the result
            // make sure it's not in the cache from a previous request
            logger.debug("Fetched {}. Will not be cached due to Expires/Date header combination sent by server", resource);
            cache.remove(makeCacheKey(resource, requesterWebID));
            return responseData;
        }
    }
    // if we don't get a new etag, see if we have a 304 code - then we can use th
    // old etag
    String etag = responseData.getResponseHeaders().getFirst(HttpHeaders.ETAG);
    if (etag == null && responseData.getStatusCode() == HttpStatus.NOT_MODIFIED.value() && linkedDataCacheEntry != null) {
        etag = linkedDataCacheEntry.getEtag();
    }
    // cache the result
    LinkedDataCacheEntry entry = new LinkedDataCacheEntry(etag, expires, writeDatasetToByteArray(responseData.getDataset()), cacheControlFlags, responseData.getResponseHeaders(), responseData.getStatusCode());
    this.cache.put(new Element(makeCacheKey(resource, requesterWebID), entry));
    logger.debug("Fetched and cached {} ", resource);
    if (logger.isDebugEnabled()) {
        logger.debug("cache size: {} elements, in-memory size: {} bytes", cache.getSize(), cache.calculateInMemorySize());
    }
    return responseData;
}
Also used : DatasetResponseWithStatusCodeAndHeaders(won.protocol.rest.DatasetResponseWithStatusCodeAndHeaders) Element(net.sf.ehcache.Element) Date(java.util.Date)

Aggregations

DatasetResponseWithStatusCodeAndHeaders (won.protocol.rest.DatasetResponseWithStatusCodeAndHeaders)6 HttpHeaders (org.springframework.http.HttpHeaders)3 Date (java.util.Date)2 Element (net.sf.ehcache.Element)2 DistributedPubSubMediator (akka.cluster.pubsub.DistributedPubSubMediator)1 ParseException (java.text.ParseException)1 CountDownLatch (java.util.concurrent.CountDownLatch)1 CacheException (net.sf.ehcache.CacheException)1 Dataset (org.apache.jena.query.Dataset)1 Lock (org.apache.jena.shared.Lock)1 RestClientException (org.springframework.web.client.RestClientException)1 NeedEvent (won.matcher.service.common.event.NeedEvent)1 CrawlWrapperException (won.matcher.service.crawler.exception.CrawlWrapperException)1 CrawlUriMessage (won.matcher.service.crawler.msg.CrawlUriMessage)1 ResourceCrawlUriMessage (won.matcher.service.crawler.msg.ResourceCrawlUriMessage)1 IncorrectPropertyCountException (won.protocol.exception.IncorrectPropertyCountException)1 NeedState (won.protocol.model.NeedState)1 NeedModelWrapper (won.protocol.util.NeedModelWrapper)1