Page History

...

Here is the explanation of each of all the fields:

• Wiki Markup
  Fetcher threads try to get a fetch item (url) from a queue of all the fetch items (this queue is actually a queue of queues. For details see \[0\]). If a thread doesn't get a fetch-item, it spinwaits for 500ms before polling the queue again. The 'spinWaiting' count tells us how many threads are in their "spinwaiting" state at a given instance.

• The 'active' count tells us how many threads are currently performing the activities related to the fetch of a fetch-item. This involves sending requests to the server, getting the bytes from the server, parsing, storing etc.
• 'pages' is a count for total pages fetched till a given point.
• 'errors' is a count for total errors seen.
• Next comes pages/s. First number comes from this:

  No Format
  ((((float)pages)*10)/elapsed)/10.0

  second one comes from this:

  No Format
  (actualPages*10)/10.0

  . "actualPages" holds the count of pages processed in the last 5 secs (when the calculation is done). First number can be seen as the overall speed for that execution. The second number can be regarded as the instanteous speed as it just uses the #pages in last 5 secs when this calculation is done.
• Next comes the kb/s values which are computed from: (((float)bytes)*8)/1024)/elapsed and

  No Format
  ((float)actualBytes)*8)/1024

  . This is similar to that of pages/sec.
• 'URLs' indicates how many urls are pending and 'queues' indicate the number of queues present. Queues are formed on the basis on hostname or ip depending on the configuration set.
  

...