Page Speed Optimization Libraries
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
Public Member Functions | Static Public Member Functions | Protected Types | Protected Member Functions | Static Protected Member Functions | Protected Attributes | List of all members
net_instaweb::SystemRewriteDriverFactory Class Referenceabstract

A server context with features specific to a psol port on a unix system. More...

#include "system_rewrite_driver_factory.h"

Inheritance diagram for net_instaweb::SystemRewriteDriverFactory:
net_instaweb::RewriteDriverFactory net_instaweb::ApacheRewriteDriverFactory

Public Member Functions

 SystemRewriteDriverFactory (const ProcessContext &process_context, SystemThreadSystem *thread_system, AbstractSharedMem *shared_mem_runtime, StringPiece hostname, int port)
void Init ()
AbstractSharedMemshared_mem_runtime () const
SharedMemStatisticsAllocateAndInitSharedMemStatistics (bool local, const StringPiece &name, const SystemRewriteOptions &options)
 Creates and ::Initializes a shared memory statistics object.
virtual void NonStaticInitStats (Statistics *statistics)=0
NonceGeneratorDefaultNonceGenerator ()
 Creates a HashedNonceGenerator initialized with data from /dev/random.
bool is_root_process () const
virtual void RootInit ()
virtual void ChildInit ()
virtual void ParentOrChildInit ()
void PostConfig (const std::vector< SystemServerContext * > &server_contexts, GoogleString *error_message, int *error_index, Statistics **global_statistics)
void SharedCircularBufferInit (bool is_root)
ParseAndSetOption1 (StringPiece option, StringPiece arg, bool process_scope, GoogleString *msg, MessageHandler *handler)
ParseAndSetOption2 (StringPiece option, StringPiece arg1, StringPiece arg2, bool process_scope, GoogleString *msg, MessageHandler *handler)
virtual HasherNewHasher ()
virtual TimerDefaultTimer ()
virtual ServerContextNewServerContext ()
virtual bool enable_property_cache () const
 Hook so implementations may disable the property cache.
GoogleString hostname_identifier ()
virtual void ShutDown ()
virtual void StopCacheActivity ()
SystemCachescaches ()
virtual void set_message_buffer_size (int x)
UrlAsyncFetcherGetFetcher (SystemRewriteOptions *config)
void set_track_original_content_length (bool x)
bool track_original_content_length () const
void list_outstanding_urls_on_error (bool x)
virtual int max_queue_size ()
virtual int queued_per_host ()
virtual int requests_per_host ()
 Normally 4, or #threads if that's more.
void set_static_asset_prefix (StringPiece s)
const GoogleStringstatic_asset_prefix ()
int num_rewrite_threads () const
void set_num_rewrite_threads (int x)
int num_expensive_rewrite_threads () const
void set_num_expensive_rewrite_threads (int x)
bool use_per_vhost_statistics () const
void set_use_per_vhost_statistics (bool x)
bool install_crash_handler () const
void set_install_crash_handler (bool x)
virtual bool UseBeaconResultsInFilters () const
virtual bool IsServerThreaded ()
virtual int LookupThreadLimit ()
virtual void StartController (const SystemRewriteOptions &options)
virtual void NameProcess (const char *name)
 Set the name of this process, for debugging visibility.
virtual void PrepareForkedProcess (const char *name)
virtual void PrepareControllerProcess ()
- Public Member Functions inherited from net_instaweb::RewriteDriverFactory
 RewriteDriverFactory (const ProcessContext &process_context, ThreadSystem *thread_system)
 Takes ownership of thread_system.
void InitializeDefaultOptions ()
void set_html_parse_message_handler (MessageHandler *message_handler)
void set_message_handler (MessageHandler *message_handler)
void set_file_system (FileSystem *file_system)
void set_hasher (Hasher *hasher)
void set_nonce_generator (NonceGenerator *nonce_generator)
void set_url_namer (UrlNamer *url_namer)
void set_signature (SHA1Signature *signature)
void set_timer (Timer *timer)
void set_usage_data_reporter (UsageDataReporter *reporter)
void set_slurp_directory (const StringPiece &directory)
void set_slurp_read_only (bool read_only)
void set_slurp_print_urls (bool read_only)
void set_force_caching (bool u)
void set_base_url_async_fetcher (UrlAsyncFetcher *url_fetcher)
bool set_filename_prefix (StringPiece p)
bool slurping_enabled () const
 Determines whether Slurping is enabled.
MessageHandlerhtml_parse_message_handler ()
MessageHandlermessage_handler ()
FileSystemfile_system ()
NonceGeneratornonce_generator ()
Hasherhasher ()
UrlNamerurl_namer ()
UserAgentMatcheruser_agent_matcher ()
StaticAssetManagerstatic_asset_manager ()
SHA1Signaturesignature ()
RewriteOptionsdefault_options ()
virtual RewriteOptionsManagerNewRewriteOptionsManager ()
Timertimer ()
NamedLockManagerlock_manager ()
QueuedWorkerPoolWorkerPool (WorkerPoolCategory pool)
Schedulerscheduler ()
UsageDataReporterusage_data_reporter ()
js_tokenizer_patterns () const
const std::vector< const
UserAgentNormalizer * > & 
user_agent_normalizers ()
virtual UrlAsyncFetcherComputeUrlAsyncFetcher ()
ServerContextCreateServerContext ()
void InitServerContext (ServerContext *server_context)
virtual void AddPlatformSpecificRewritePasses (RewriteDriver *driver)
virtual void AddPlatformSpecificDecodingPasses (RewriteDriver *driver)
virtual void ApplyPlatformSpecificConfiguration (RewriteDriver *driver)
ThreadSystemthread_system ()
const StringSet & created_directories () const
bool async_rewrites ()
RewriteStatsrewrite_stats ()
virtual bool HasWaveforms () const
 Returns true if the server supports waveforms in its status page.
Statisticsstatistics ()
void SetStatistics (Statistics *stats)
 Does not take ownership of Statistics.
void AddCreatedDirectory (const GoogleString &dir)
 Registers the directory as having been created by us.
virtual RewriteOptionsNewRewriteOptions ()
virtual RewriteOptionsNewRewriteOptionsForQuery ()
const GoogleStringversion_string () const
 get/set the version placed into the X-[Mod-]Page(s|-S)peed header.
void set_version_string (const StringPiece &version_string)
void defer_cleanup (Function *f)
template<class T >
void TakeOwnership (T *obj)
virtual bool IsDebugClient (const GoogleString &ip) const
 Base method that returns true if the given ip is a debug ip.
virtual ExperimentMatcherNewExperimentMatcher ()

Static Public Member Functions

static void InitApr ()
- Static Public Member Functions inherited from net_instaweb::RewriteDriverFactory
static void InitializeDefaultOptions (RewriteOptions *options)
static void InitStats (Statistics *statistics)
static void Initialize ()
 Initializes static variables. Initialize/Terminate calls must be paired.
static void Terminate ()

Protected Types

typedef std::set
< SystemServerContext * > 

Protected Member Functions

virtual void InitStaticAssetManager (StaticAssetManager *static_asset_manager)
 Initializes the StaticAssetManager.
virtual void SetupCaches (ServerContext *server_context)
virtual QueuedWorkerPoolCreateWorkerPool (WorkerPoolCategory pool, StringPiece name)
virtual void SetupMessageHandlers ()
virtual void ShutDownMessageHandlers ()
virtual void SetCircularBuffer (SharedCircularBuffer *buffer)
virtual void ShutDownFetchers ()
virtual UrlAsyncFetcherAllocateFetcher (SystemRewriteOptions *config)
virtual FileSystemDefaultFileSystem ()
virtual NamedLockManagerDefaultLockManager ()
virtual void AutoDetectThreadCounts ()
bool thread_counts_finalized ()
< CentralController
GetCentralController (NamedLockManager *lock_manager) override
 Delegate from RewriteDriverFactory to construct CentralController.
- Protected Member Functions inherited from net_instaweb::RewriteDriverFactory
bool FetchersComputed () const
StringPiece filename_prefix ()
bool TerminateServerContext (ServerContext *server_context)
virtual MessageHandlerDefaultHtmlParseMessageHandler ()=0
virtual MessageHandlerDefaultMessageHandler ()=0
virtual SHA1SignatureDefaultSignature ()
virtual ServerContextNewDecodingServerContext ()=0
virtual CriticalImagesFinderDefaultCriticalImagesFinder (ServerContext *server_context)
virtual CriticalSelectorFinderDefaultCriticalSelectorFinder (ServerContext *server_context)
virtual UrlNamerDefaultUrlNamer ()
virtual UserAgentMatcherDefaultUserAgentMatcher ()
virtual UsageDataReporterDefaultUsageDataReporter ()
virtual void AddPlatformSpecificUserAgentNormalizers (std::vector< const UserAgentNormalizer * > *out)
virtual int LowPriorityLoadSheddingThreshold () const
virtual SchedulerCreateScheduler ()
virtual void FetcherSetupHooks ()
 Called before creating the url fetchers.
virtual StringPiece LockFilePrefix ()
void InitStubDecodingServerContext (ServerContext *context)
void RebuildDecodingDriverForTests (ServerContext *server_context)
 For use in tests.
void reset_default_options (RewriteOptions *new_defaults)

Static Protected Member Functions

static void InitStats (Statistics *statistics)

Protected Attributes

SystemServerContextSet uninitialized_server_contexts_

Additional Inherited Members

- Public Types inherited from net_instaweb::RewriteDriverFactory
enum  WorkerPoolCategory { kHtmlWorkers, kRewriteWorkers, kLowPriorityRewriteWorkers, kNumWorkerPools }

Detailed Description

A server context with features specific to a psol port on a unix system.

Member Typedef Documentation

Once ServerContexts are initialized via RewriteDriverFactory::InitServerContext, they will be managed by the RewriteDriverFactory. But in the root process the ServerContexts will never be initialized. We track these here so that SystemRewriteDriverFactory::ChildInit can iterate over all the server contexts that need to be ChildInit'd, and so that we can free them in the Root process that does not run ChildInit.

Constructor & Destructor Documentation

net_instaweb::SystemRewriteDriverFactory::SystemRewriteDriverFactory ( const ProcessContext process_context,
SystemThreadSystem thread_system,
AbstractSharedMem shared_mem_runtime,
StringPiece  hostname,
int  port 

Takes ownership of thread_system.

On Posix systems implementers should leave shared_mem_runtime NULL, otherwise they should implement AbstractSharedMem for their platform and pass in an instance here. The factory takes ownership of the shared memory runtime if one is passed in. Implementers who don't want to support shared memory at all should set PAGESPEED_SUPPORT_POSIX_SHARED_MEM to false and pass in NULL, and the factory will use a NullSharedMem.

After construction, you must call Init() to finish the initialization.

Member Function Documentation

virtual UrlAsyncFetcher* net_instaweb::SystemRewriteDriverFactory::AllocateFetcher ( SystemRewriteOptions config)

Allocates a serf fetcher. Implementations may override this method to supply other kinds of fetchers. For example, ngx_pagespeed may return either a serf fetcher or an nginx-native fetcher depending on options.

virtual void net_instaweb::SystemRewriteDriverFactory::AutoDetectThreadCounts ( )

Updates num_rewrite_threads_ and num_expensive_rewrite_threads_ with sensible values if they are not explicitly set.

virtual QueuedWorkerPool* net_instaweb::SystemRewriteDriverFactory::CreateWorkerPool ( WorkerPoolCategory  pool,
StringPiece  name 

Subclasses can override this to create an appropriately-sized thread pool for their environment. The default implementation will always make one with a single thread.

Reimplemented from net_instaweb::RewriteDriverFactory.

virtual NamedLockManager* net_instaweb::SystemRewriteDriverFactory::DefaultLockManager ( )

They may also supply a custom lock manager. The default implementation will use the file system.

Reimplemented from net_instaweb::RewriteDriverFactory.

UrlAsyncFetcher* net_instaweb::SystemRewriteDriverFactory::GetFetcher ( SystemRewriteOptions config)

Finds a fetcher for the settings in this config, sharing with existing fetchers if possible, otherwise making a new one (and its required thread).

static void net_instaweb::SystemRewriteDriverFactory::InitApr ( )

If the server using this isn't using APR natively, call this to initialize the APR library.

static void net_instaweb::SystemRewriteDriverFactory::InitStats ( Statistics statistics)

Initializes all the statistics objects created transitively by SystemRewriteDriverFactory. Only subclasses should call this.

bool net_instaweb::SystemRewriteDriverFactory::is_root_process ( ) const

For shared memory resources the general setup we follow is to have the first running process (aka the root) create the necessary segments and fill in their shared data structures, while processes created to actually handle requests attach to already existing shared data structures.

During normal server startup[1], RootInit() must be called from the root process and ChildInit() in every child process.

Keep in mind, however, that when fork() is involved a process may effectively see both calls, in which case the 'ChildInit' call would come second and override the previous root status. Both calls are also invoked in the debug single-process mode (In Apache, "httpd -X".).

Note that these are not static methods – they are invoked on every SystemRewriteDriverFactory instance, which exist for the global configuration as well as all the virtual hosts.

Implementations should override RootInit and ChildInit for their setup. See ApacheRewriteDriverFactory for an example.

[1] Besides normal startup, Apache also uses a temporary process to syntax check the config file. That basically looks like a complete normal startup and shutdown to the code.

virtual bool net_instaweb::SystemRewriteDriverFactory::IsServerThreaded ( )

Check whether the server is threaded. For example, Nginx uses an event loop and can keep with the default of false, while Apache with a threaded multiprocessing module (MPM) overrides this method to return true.

< Most new servers are non-threaded nowadays.

Reimplemented in net_instaweb::ApacheRewriteDriverFactory.

void net_instaweb::SystemRewriteDriverFactory::list_outstanding_urls_on_error ( bool  x)

When Serf gets a system error during polling, to avoid spamming the log we just print the number of outstanding fetch URLs. To debug this it's useful to print the complete set of URLs, in which case this should be turned on.

virtual int net_instaweb::SystemRewriteDriverFactory::LookupThreadLimit ( )

Threaded implementing servers should return the maximum number of threads that might be used for handling user requests.

Reimplemented in net_instaweb::ApacheRewriteDriverFactory.

virtual int net_instaweb::SystemRewriteDriverFactory::max_queue_size ( )

When RateLimitBackgroundFetches is enabled the fetcher needs to apply some limits. An implementation may need to tune these based on conditions only observable at startup, in which case they can override these.

virtual ServerContext* net_instaweb::SystemRewriteDriverFactory::NewServerContext ( )

Creates a new ServerContext* object. ServerContext itself must be overridden per Factory as it has at least one pure virtual method.

Implements net_instaweb::RewriteDriverFactory.

virtual void net_instaweb::SystemRewriteDriverFactory::NonStaticInitStats ( Statistics statistics)
pure virtual

Hook for subclasses to init their stats and call SystemRewriteDriverFactory::InitStats().

Implemented in net_instaweb::ApacheRewriteDriverFactory.

virtual void net_instaweb::SystemRewriteDriverFactory::ParentOrChildInit ( )

This helper method contains init procedures invoked by both RootInit() and ChildInit()

Reimplemented in net_instaweb::ApacheRewriteDriverFactory.

virtual RewriteOptions::OptionSettingResult net_instaweb::SystemRewriteDriverFactory::ParseAndSetOption1 ( StringPiece  option,
StringPiece  arg,
bool  process_scope,
GoogleString msg,
MessageHandler handler 

Most options are parsed by and applied to the RewriteOptions via ParseAndSetOptionFromNameN, but process-scope options need to be set on the rewrite driver factory.

ParseAndSetOptionN will only apply changes to the rewrite driver factory if process_scope is true, but it should be called regardless in order to give more helpful error messages ("wrong scope" vs "no such option"). If an option is used out of scope an appropriate message is put in msg and either kOptionValueInvalid or kOptionOk is returned: invalid if the parser should abort with an error, ok if parsing should continue past the error.

void net_instaweb::SystemRewriteDriverFactory::PostConfig ( const std::vector< SystemServerContext * > &  server_contexts,
GoogleString error_message,
int *  error_index,
Statistics **  global_statistics 

After the whole configuration has been read, we need to do additional configuration that requires a global view.

  • server_contexts should be all server contexts on the system
  • error_message and error_index are set if there's an error, and error_index is the index in server_contexts of the one with the issue.
  • global_statistics is lazily initialized to a shared memory statistics owned by this factory if any of the server contexts require it.
virtual void net_instaweb::SystemRewriteDriverFactory::PrepareControllerProcess ( )

Once we've created the controller process, we need to initialize it like we would one of our normal parent or child processes. The controller manager will call this once it has a process it needs prepared.

virtual void net_instaweb::SystemRewriteDriverFactory::PrepareForkedProcess ( const char *  name)

Hook for handling any process-specific initialization the host webserver might need when we manually fork off a process. Children should call the superclass method when overriding (so it can set the process name). See NgxRewriteDriverFactory::PrepareForkedProcess.

void net_instaweb::SystemRewriteDriverFactory::set_track_original_content_length ( bool  x)

Tracks the size of resources fetched from origin and populates the X-Original-Content-Length header for resources derived from them.

virtual void net_instaweb::SystemRewriteDriverFactory::SetupCaches ( ServerContext server_context)

Called from InitServerContext, but virtualized separately as it is platform-specific. This method must call on the server context: set_http_cache, set_metadata_cache, set_filesystem_metadata_cache, and MakePropertyCaches.

Implements net_instaweb::RewriteDriverFactory.

Reimplemented in net_instaweb::ApacheRewriteDriverFactory.

virtual void net_instaweb::SystemRewriteDriverFactory::SetupMessageHandlers ( )
TODO(jefftk): create SystemMessageHandler and get rid of these hooks.

Reimplemented in net_instaweb::ApacheRewriteDriverFactory.

void net_instaweb::SystemRewriteDriverFactory::SharedCircularBufferInit ( bool  is_root)

Initialize SharedCircularBuffer and pass it to SystemMessageHandler and SystemHtmlParseMessageHandler. is_root is true if this is invoked from root (ie. parent) process.

virtual void net_instaweb::SystemRewriteDriverFactory::ShutDown ( )

Release all the resources. It also calls the base class ShutDown to release the base class resources.

Reimplemented from net_instaweb::RewriteDriverFactory.

virtual void net_instaweb::SystemRewriteDriverFactory::ShutDownFetchers ( )

Can be overridden by subclasses to shutdown any fetchers we don't know about.

virtual void net_instaweb::SystemRewriteDriverFactory::StartController ( const SystemRewriteOptions options)

By default this uses the ControllerManager to fork off some processes to handle the Controller. If you're on a system where fork doesn't make sense or running the Controller in its own process doesn't make sense, this is a hook where you can start the controller in whatever way makes sense for your platform.

virtual bool net_instaweb::SystemRewriteDriverFactory::UseBeaconResultsInFilters ( ) const

mod_pagespeed uses a beacon handler to collect data for critical images, css, etc., so filters should be configured accordingly.

Implements net_instaweb::RewriteDriverFactory.

The documentation for this class was generated from the following file: