Page Speed Optimization Libraries
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
Classes | Public Member Functions | Static Public Attributes | List of all members
net_instaweb::CacheUrlAsyncFetcher Class Reference

#include "cache_url_async_fetcher.h"

Inheritance diagram for net_instaweb::CacheUrlAsyncFetcher:


class  AsyncOpHooks

Public Member Functions

 CacheUrlAsyncFetcher (const Hasher *lock_hasher, NamedLockManager *lock_manager, HTTPCache *cache, const GoogleString &fragment, AsyncOpHooks *async_op_hooks, UrlAsyncFetcher *fetcher)
 None of these are owned by CacheUrlAsyncFetcher.
virtual bool SupportsHttps () const
virtual void Fetch (const GoogleString &url, MessageHandler *message_handler, AsyncFetch *base_fetch)
HTTPCachehttp_cache () const
UrlAsyncFetcherfetcher () const
void set_backend_first_byte_latency_histogram (Histogram *x)
Histogrambackend_first_byte_latency_histogram () const
void set_fallback_responses_served (Variable *x)
Variablefallback_responses_served () const
void set_fallback_responses_served_while_revalidate (Variable *x)
Variablefallback_responses_served_while_revalidate () const
void set_num_conditional_refreshes (Variable *x)
Variablenum_conditional_refreshes () const
void set_num_proactively_freshen_user_facing_request (Variable *x)
Variablenum_proactively_freshen_user_facing_request () const
void set_respect_vary (bool x)
bool respect_vary () const
void set_ignore_recent_fetch_failed (bool x)
bool ignore_recent_fetch_failed () const
void set_serve_stale_if_fetch_error (bool x)
bool serve_stale_if_fetch_error () const
void set_serve_stale_while_revalidate_threshold_sec (int64 x)
int64 serve_stale_while_revalidate_threshold_sec () const
void set_default_cache_html (bool x)
bool default_cache_html () const
void set_proactively_freshen_user_facing_request (bool x)
bool proactively_freshen_user_facing_request () const
void set_own_fetcher (bool x)
void set_response_sequence (Sequence *x)
- Public Member Functions inherited from net_instaweb::UrlAsyncFetcher
virtual int64 timeout_ms ()
virtual void ShutDown ()
void set_fetch_with_gzip (bool x)
bool fetch_with_gzip () const
AsyncFetchEnableInflation (AsyncFetch *fetch) const

Static Public Attributes

static const int kNotInCacheStatus
- Static Public Attributes inherited from net_instaweb::UrlAsyncFetcher
static const int64 kUnspecifiedTimeout

Additional Inherited Members

- Protected Member Functions inherited from net_instaweb::UrlAsyncFetcher
 UrlAsyncFetcher ()

Detailed Description

Composes an asynchronous URL fetcher with an http cache, to generate an asynchronous caching URL fetcher.

This fetcher will asynchronously check the cache. If the url is found in cache and is still valid, the fetch's callback will be called right away. This includes any cached failures or that URL is uncacheable, unless set_ignore_recent_fetch_failed(true) is called. Otherwise (if fetcher != NULL) an async fetch will be performed in the fetcher, the result of which will be written into the cache. In case the fetch fails and there is a stale response in the cache, we serve the stale response.

If fetcher == NULL, this will only perform a cache lookup and then call the callback immediately.

In case of cache hit and resource is about to expire (80% of TTL or 5 mins which ever is minimum), it will trigger background fetch to freshen the value in cache. Background fetch only be triggered only if async_op_hooks_ != NULL, otherwise, fetcher object accessed by BackgroundFreshenFetch may be deleted by the time origin fetch finishes.

TODO(sligocki): In order to use this for fetching resources for rewriting we'd need to integrate resource locking in this class. Do we want that?

Member Function Documentation

virtual void net_instaweb::CacheUrlAsyncFetcher::Fetch ( const GoogleString url,
MessageHandler message_handler,
AsyncFetch fetch 

Asynchronously fetch a URL, set the response headers and stream the contents to fetch and call fetch->Done() when the fetch finishes.

There is an unchecked contract that response_headers are set before the response_writer or callback are used. Caution, several implementations do not satisfy this contract (but should).


TODO(sligocki): GoogleString -> GoogleUrl or at least StringPiece.

TODO(sligocki): Include the URL in the fetch, like the request headers.

Implements net_instaweb::UrlAsyncFetcher.

void net_instaweb::CacheUrlAsyncFetcher::set_response_sequence ( Sequence x)

By default, the CacheUrlAsyncFetcher will call its fetcher callbacks on whatever thread the cache or the fetcher happen to be on (e.g. the memcached thread). Setting the response_sequence ensures that cached responses call their callbacks by queueing on that sequence rather than executing them directly.

TODO(jmarantz): this currently only makes sense to call when there is no fetcher, as the implementation does not queue up fetcher-callbacks; only cache callbacks. But if we want to fully implement bandwidth optimization at the expense of latency, we are going to have to add sequencing of fetch callbacks in addition to cache callbacks. This is not hard to do, but we don't need it yet so it's not done.
virtual bool net_instaweb::CacheUrlAsyncFetcher::SupportsHttps ( ) const

Determine if the fetcher supports fetching using HTTPS. By default we assume a fetcher can.

Reimplemented from net_instaweb::UrlAsyncFetcher.

Member Data Documentation

const int net_instaweb::CacheUrlAsyncFetcher::kNotInCacheStatus

HTTP status code used to indicate that we failed the Fetch because result was not found in cache. (Only happens if fetcher_ == NULL).

The documentation for this class was generated from the following file: