Download Metrics
Contents
Download Metrics
Download Metrics in EPrints
Capturing Usage
Eprints uses the Apache web server and mod_perl (an accelerator for Perl CGI scripts). mod_perl provides hooks to the logging and control features of the Apache server. One of these hooks is to the access logging function. A log handler is registered with Apache by using the PerlLogHandler pragma e.g.
PerlLogHandler EPrints::Apache::LogHandler
This handler is called after a response has been generated by the web server (i.e. in response to requests coming in from web clients).
In the initial implementation we are only interested in requests to eprint objects. This ignores requests to static content (e.g. the home page), searches and user-specific content (e.g. depositing processes). This is because our current requirements are to capture usage to determine download metrics for objects contained in a repository, rather than analysing how a repository is used.
For the remainder of this section we will use the following example request (as you would otherwise see in an Apache access log file):
127.0.0.1 - - [25/May/2007:13:07:24 +0100] "GET /12614/01/Semantic_Web_Revisted.pdf HTTP/1.1" 200 130951 "http://www.w3.org/2001/sw/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
The pertinent parts of this entry are:
| field | example |
|---|---|
| requesting host | 127.0.0.1 |
| date/time of request (by server time) | 25/May/2007:13:07:24 +0100 |
| page requested | /12614/01/Semantic_Web_Revisted.pdf |
| HTTP response code (200 is 'ok') | 200 |
| referring web page | http://www.w3.org/2001/sw/ |
| user's web browser (or web crawler id) | Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) |
In the Eprints log handler any non-200 HTTP response code requests are ignored i.e. any errors or redirects. Requests are then split up into three sets, non-eprint requests, eprint abstract requests and full-text requests. This is based on the page requested field.
| /12614/ | /01/ | Semantic_Web_Revisted.pdf |
|---|---|---|
| eprint identifier | document identifier | file requested |
Request without an eprint identifier part are ignored. Requests with no document identifier part are defined as abstract requests. Requests with both eprint and document identifier parts are defined as full-text requests.
A simple filter is implemented in the capture code to handle requests to multi-file documents (e.g. HTML pages with inlined images). Any full-text requests where the referring web page (see above) is also a request to the same full-text are ignored. Assuming the referring web page is correctly supplied by the web client this will count requests to any part of a document, but ignore multiple requests resulting from complex media.
Storing Usage
Requests are stored in the Eprints database in the access_log table, see [Access_Log] for a description of the fields. This is almost a mirror of the normal Apache log file, but by storing the data in a table enables much simpler exposure and re-use of the datab.
Log data can potentially get very large, which may require a different technical approach in future (or periodic triaging of the access log data).
Exposing Usage
Eprints 3 includes a disabled script that can expose access logs through an OAI-PMH interface. This interface is located at /cgi/oai_accesslogs. The interface contains an EPrints::abort at the top that prevents all access - this is because usage logs are potentially sensitive data, and a repository will probably want to implement some kind of access restriction before providing access to their logs.
The OAI interface uses Eprints export plugins to convert access log objects (as read from the Eprints database) to XML suitable to be embedded in the OAI output. From an Eprints architecture perspective the mechanism to expose access log records is nearly identical to the exposure of eprint records (they are both datasets in Eprints).
| eprint | accesslog | |
|---|---|---|
| identifier | oai:<archive name>:<item id> | oai:<archive name>:<item id> |
| export plugins | (predefined list in cfg.d/oai.pl) | can_accept => "list/access" |
| from/until | last_mod field | datestamp field |
| OAI sets | arbitrary fields defined in cfg.d/oai.pl | not-supported |
A result set of records is generated from the OAI request. Each record is converted to XML, depending on the export plugin requested (via a mapping of metadataPrefix to plugin identifier), and output as an XML document.
Currently Eprints supports exporting access log records in OpenURL ContextObject format. To be OAI-compliant an interface should also support Dublin Core (but access logs don't make a lot of sense in DC).
