IRStats Technical Documentation - Revision history

Th.lauke@arcor.de: category and redirection updated

2019-08-08T16:43:48Z

category and redirection updated

Pm705 at 18:59, 12 November 2015

2015-11-12T18:59:28Z

Pm705 at 18:58, 12 November 2015

2015-11-12T18:58:26Z

Pm705 at 18:57, 12 November 2015

2015-11-12T18:57:07Z

Pm705 at 16:00, 8 February 2010

2010-02-08T16:00:52Z

Gobfrey at 19:41, 30 May 2007

2007-05-30T19:41:22Z

Gobfrey at 19:17, 30 May 2007

2007-05-30T19:17:08Z

Gobfrey at 18:55, 30 May 2007

2007-05-30T18:55:11Z

Gobfrey: /* Directory Structure */

2007-05-30T18:28:58Z

‎Directory Structure

Gobfrey at 18:25, 30 May 2007

2007-05-30T18:25:13Z

@@ Line 1: / Line 1: @@
-[[Category:IRStats]]
+[[Category:Obsolete]]
-<div style="border: 2px solid red; background-color: yellow;padding:10px">This is IRStats 1 documentation. IRStats 1 is now out of support. You may have been looking for [[IRStats 2]]</div>
+<div style="border: 2px solid red; background-color: yellow;padding:10px">This is IRStats 1 documentation. IRStats 1 is now out of support. You may have been looking for [[IRStats2]]</div>
 = Directory Structure =

@@ Line 1: / Line 1: @@
 [[Category:IRStats]]
-<div style="border: 2px solid red; background-color: yellow;padding:10px">This is IRStats1 documentation. IRStats1 is now out of support. You may have been looking for [[IRStats2]]</div>
+<div style="border: 2px solid red; background-color: yellow;padding:10px">This is IRStats 1 documentation. IRStats 1 is now out of support. You may have been looking for [[IRStats 2]]</div>
 = Directory Structure =

@@ Line 1: / Line 1: @@
 [[Category:IRStats]]
-<div style="border: 2px solid red; background-color: yellow;padding:10px">This is IRStats1 documentation. IRStats1 is now out of support. You may have been looking for IRStats2</div>
+<div style="border: 2px solid red; background-color: yellow;padding:10px">This is IRStats1 documentation. IRStats1 is now out of support. You may have been looking for [[IRStats2]]</div>
 = Directory Structure =

← Older revision		Revision as of 18:57, 12 November 2015
Line 1:		Line 1:
	[[Category:IRStats]]		[[Category:IRStats]]
		+	<div style="border: 2px solid red; background-color: yellow;padding:10px">This is IRStats1 documentation. IRStats1 is now out of support. You may have been looking for IRStats2</div>
	= Directory Structure =		= Directory Structure =

← Older revision		Revision as of 16:00, 8 February 2010
Line 1:		Line 1:
		+	[[Category:IRStats]]
	= Directory Structure =		= Directory Structure =

@@ Line 125: / Line 125: @@
 == Date ==
-A date object was implemented because there were some specific things that needed to do with dates.
+A date object was implemented because there were some specific things that needed to be done with dates.
 ===Functions===
@@ Line 140: / Line 140: @@
 *render(format_string) - returns a date string.  Format can be:
 **'short' - Calls render_abbreviated - returns a date like this: 05-Jul-77
 **'numerical' (default) - Calls render_numerical - returns a date like this: 19770705
 *clone - returns an new, identical date object.
 == Cache ==
 The interface to the cache.
 === Functions ===
@@ Line 170: / Line 165: @@
 == UserInterface::Controls ==
-This is used to generate the drop boxes in the stats cgi script.  If I had more time I'd document it fully, but my daughter's going to be born in less than 12 hours.
+This is used to generate the drop boxes in the stats cgi script.
+===Functions===
-== Page (depricated) ==
+new(params_obj, database_interface_object) - returns the object.
-Harkens back to the day when a page object contained views.
+start_date_control() - returns the html for the three drop-boxes for selecting the year, month and day of the start date.
 == View ==
@@ Line 187: / Line 185: @@
 *populate - The engine that powers IRStats.
-=== View::FullTextCountHTML ===
+=== View::DownloadCountHTML ===
-The FullTextCountHTML is an extremely simple view.  It retrieves one row from the database and does no processing, making it ideal for a quick walkthrough:
+The DownloadCountHTML is an extremely simple view.  It retrieves one row from the database and does no processing, making it ideal for a quick walkthrough:
 ==== Housekeeping ====
 At the top of the file, we need:
-  package IRStats::View::FullTextCountHTML;
+  package IRStats::View::DownloadCountHTML;
   use strict;
   use warnings;
@@ Line 205: / Line 203: @@
 ==== Configuration Constants ====
-We are interested in retreiving the fulltxt column, and a count as we will be aggregating.  The sql_params are set, so that we can filter on fulltext downloads, and we need to group as we are counting.  We also create our visualisation here.
+We aren't actually interested in any columns, just in the count, but we put that in the columns array anyway.
   sub initialise
+  {
          my ($self) = @_;
-        $self->{'sql_columns'} = [ 'fulltxt', 'COUNT(fulltxt)' ];
+         $self->{'sql_params'} = {columns => [ 'COUNT' ]};
          $self->{'visualisation'} = IRStats::Visualisation::HTML->new();
+  }
@@ Line 225: / Line 223: @@
 ==== populate ====
-Every populate function should start by checking the cache.
+Almost every populate function should start by checking the cache.
   sub populate
@@ Line 240: / Line 238: @@
      my $query = $self->{'database'}->get_stats(
              $self->{'params'},
              $self->{'sql_params'}
              );
@@ Line 266: / Line 263: @@
      my $query = $self->{'database'}->get_stats(
              $self->{'params'},
              $self->{'sql_params'}
      );
@@ Line 315: / Line 311: @@
 An HTML table that is rendered in several columns.
 ==== Configuration Constants ====
-$default_number_of_rows - an int representing the maximum number of rows the table should have.
+$default_number_of_rows - an int representing the maximum number of rows the table should have.  This is to prevent sending huge tables to browsers which may not be able to handle it.
 ==== Overridden Functions ====

@@ Line 85: / Line 85: @@
 Note that the leading IRStats:: has been left out for brevity.
 == Params ==
@@ Line 92: / Line 100: @@
 === Functions ===
-*new(CGI_object) - returns new object
+*new(Configuration, [ CGI_object | params_hash ]) - returns new object
 *mask(params_hash) - used when you want to temporarily overwrite parameter(s).  Overwrites values with contents of params_hash.  Overwritten values get pushed onto a stack.
 *unmask - Sets parameters back to how they were before the last mask.
 *get(param_name) - returns the value of a single parameter.
 *create_id - Uses MD5 to create a unique ID from the id_params (see Constants above).  This is called whenever get('id') is called.
@@ Line 101: / Line 108: @@
 == DatabaseInterface ==
 This object does what it says on the tin.  Any access to the database is done though it.
 === Functions ===
-*new() - returns object.
+*new(Configuration) - returns object.
-*retreive_set_names() - returns a list of eprint sets.  Currently 'group' and 'author' are implemented.  This is used to verify cgi input.
+*retreive_set_names() - returns a list of eprint sets.  This can be used to verify cgi input.
 *get_membership(eprint_id, set_name) - For a given eprint ID, which of a named set does it belong to.  For example, we can find out which authors eprint 12614 has by get_membership(12614, 'author').
 *get_citation(id, set, length) - returns a citation.  Every set member (eprint, author, group) has two citations.  short and full.  We only return a short citation if length == 'short'.  So, to get the short citation of a group 3: get_citation(3,'group','short').
-*get_code($id,$set) - UNWRITTEN - Set member have codes.  This how they are identified by the user.  For example author_lac is the member of the author set whose code is lac.  To get the code for group 3: get_code(3,'group').
+*get_stats(params_object, query_params_hash) - returns a dbi object containing the stats we are interested.  i.e. the params_object's date range and eprints sets, and only the columns in query params hash.  The query params hash can contain the following key/value pairs
+**columns => column_name_array - Which columns are we interested in?
-When retreiving statistics, IRStats filters by inner joining the irstats_true_accesses_table to other tables contining eprint IDs.  Sometimes it has to create these tables.
+**order => column_name - A hash containing a column name and directions (ASC or DESC)
 **limit => int - How many results to return
 **group_by => column_name - if we need to group by a column.
-get_stats works by examining the 'eprints' parameter and calling one of the following functions:
+**where => where_hash_array - if additional logic needs to be applied, this array contains hashes containing a column name, an operator and a value.  These are ANDed together.
-**get_list_stats
+*check_tables() - If any IRStats tables are missing, this function will create them.
-**get_top_stats
+*insert_main_table_row(column_array) - inserts the values in the array into the main table (taking into account any tables that contain only IDs).
-**get_set_stats
+*do_sql(sql_query_string) - takes a string and performs a query, returning the dbi object containing the results.  This is the only point where sql is sent to the database.
 == Date ==
-I implemented a date object because there were some specific things I needed to do with dates.
+A date object was implemented because there were some specific things that needed to do with dates.
 ===Functions===
@@ Line 150: / Line 134: @@
 *increment(period) - decrements by calling mod_date.
 *part(part_name, style - Returns the day, month or year.  For month, if style=='text', returns a three letter string, otherwise returns an integer.  For year, if style=='short', returns the last two digits, otherwise returns all four.
 *less_than(date_object) - compares itself to another date object.  Returns 1 if it's less than it, otherwise returns 0.
 *greater_than(date_object) - compares itself to another date object.  Returns 1 if it's greater than it, otherwise returns 0.

@@ Line 1: / Line 1: @@
 = Directory Structure =
@@ Line 25: / Line 24: @@
 Conceptually, where any images would be kept (e.g. national flags).  At the moment, only the img/graphs directory is used.  This is where generated graphs are stored.
 == /opt/irstats/perl_lib ==
@@ Line 35: / Line 87: @@
 == Params ==
-This object holds the parameters that are used to generate the statistics.  The most imortant of these are a date range and an eprint set.
+This object holds the parameters that are used to generate the statistics.  This is passed around the system.
 === Configuration Constants ===
 *$defaults - Any default parameters you wish to set.

@@ Line 1: / Line 1: @@
 = Directory Structure =
 == /opt/irstats/bin ==
@@ Line 16: / Line 13: @@
 == /opt/irstats/cache ==
-Contains cache files.  Feel free to delete these whenever you like.
+Contains cache files.  These should probably be deleted whenever the database is updated.
 == /opt/irstats/cgi ==
-Contains two scripts, 'get_view' and 'stats'.
+Contains two scripts, 'get_view and 'stats'.
 *get_view returns the output of a IRStats::View (see below), which is currently a chunk of html or csv, but could be almost anything.

@@ Line 1: / Line 1: @@
 = Directory Structure =
-== /opt/epstats ==
+== /opt/irstats ==
 Contains data files for GeoIP.  If I had had root access, I would have put them in the correct place.  They are linked to from the correct place.  These need regular updating, something which hasn't been implemented.
-== /opt/epstats/bin ==
+== /opt/irstats/bin ==
 Contains the scripts needed to update the table.
 *daily_update.sh - Runs all the scripts in the right order.
 *extract_metadata_from_archive.pl - Extracts eprint, author and group metadata from the repository by iterating over every eprint.
-*update_table.pl - Filters and processes new entries in the accesslog to update the epstats_true_acesses_table.  Uses 'SearchParser.pm' and 'repeatscache'.
+*update_table.pl - Filters and processes new entries in the accesslog to update the irstats_true_acesses_table.  Uses 'SearchParser.pm' and 'repeatscache'.
-* convert_ip_to_host.pl - Attempts to convert ip addresses of the new entries in epstats_true_acesses_table to hostnames.  Uses 'host_updated' to keep track of where it got to last time.
+* convert_ip_to_host.pl - Attempts to convert ip addresses of the new entries in irstats_true_acesses_table to hostnames.  Uses 'host_updated' to keep track of where it got to last time.
 Note that most of these scripts probably need to be tidied up.  They were written in a hurry and were never polished.
-== /opt/epstats/cache ==
+== /opt/irstats/cache ==
 Contains cache files.  Feel free to delete these whenever you like.
-== /opt/epstats/cgi ==
+== /opt/irstats/cgi ==
 Contains two scripts, 'get_view' and 'stats'.
-*get_view returns the output of a EPstats::View (see below), which is currently a chunk of html or csv, but could be almost anything.
+*get_view returns the output of a IRStats::View (see below), which is currently a chunk of html or csv, but could be almost anything.
 *stats is a handy cgi form that passes arguements to get_view
-== /opt/epstats/img ==
+== /opt/irstats/img ==
 Conceptually, where any images would be kept (e.g. national flags).  At the moment, only the img/graphs directory is used.  This is where generated graphs are stored.
-== /opt/epstats/perl_lib ==
+== /opt/irstats/perl_lib ==
-Contains all the epstats classes.
+Contains all the irstats classes.
-= EPStats Classes =
+= IRStats Classes =
-Note that the leading EPStats:: has been left out for brevity.
+Note that the leading IRStats:: has been left out for brevity.
 == Params ==
@@ Line 71: / Line 70: @@
 *get_code($id,$set) - UNWRITTEN - Set member have codes.  This how they are identified by the user.  For example author_lac is the member of the author set whose code is lac.  To get the code for group 3: get_code(3,'group').
-When retreiving statistics, EPStats filters by inner joining the epstats_true_accesses_table to other tables contining eprint IDs.  Sometimes it has to create these tables.
+When retreiving statistics, IRStats filters by inner joining the irstats_true_accesses_table to other tables contining eprint IDs.  Sometimes it has to create these tables.
 *create_top_table(param_object) - This creates a table containing the eprint IDs of the top X by fulltext download between two dates.
@@ Line 131: / Line 130: @@
 *new(start_date_obj, end_date_obj) - doesn't do anything, just returns the object.
-The following functions all return an array of hashes.  Each hash has the keys 'start_date' and 'end_date', and the values are both EPStats::Date objects.
+The following functions all return an array of hashes.  Each hash has the keys 'start_date' and 'end_date', and the values are both IRStats::Date objects.
 *calandar_months - Returns full months (each element starts on the 1st, and ends on the last day).
@@ Line 154: / Line 153: @@
 *new - passes arguments to superclass, then calls 'initialise'.
 *initialise - the Configuration Constants are set here.
-*populate - The engine that powers EPStats.
+*populate - The engine that powers IRStats.
 === View::FullTextCountHTML ===
@@ Line 161: / Line 160: @@
 ==== Housekeeping ====
 At the top of the file, we need:
-  package EPStats::View::FullTextCountHTML;
+  package IRStats::View::FullTextCountHTML;
   use strict;
   use warnings;
 Now, which modules will we use.  I've included perchardir, the graph making package, even though we're not using it.
-  use EPStats::DatabaseInterface;
+  use IRStats::DatabaseInterface;
-  use EPStats::Cache;
+  use IRStats::Cache;
-  use EPStats::Visualisation::HTML;
+  use IRStats::Visualisation::HTML;
-  use EPStats::View;
+  use IRStats::View;
   use perlchartdir;
 And link to superclass.
-  our @ISA = qw/ EPStats::View /;
+  our @ISA = qw/ IRStats::View /;
 ==== Configuration Constants ====
@@ Line 180: / Line 179: @@
          $self->{'sql_columns'} = [ 'fulltxt', 'COUNT(fulltxt)' ];
          $self->{'sql_params'} = {where => "fulltxt = 'F'", group_by => 'fulltxt'};
-         $self->{'visualisation'} = EPStats::Visualisation::HTML->new();
+         $self->{'visualisation'} = IRStats::Visualisation::HTML->new();
+  }
@@ Line 199: / Line 198: @@
+  {
      my ($self) = @_;
-     my $cache = EPStats::Cache->new($self->{'params'}->get('id'));
+     my $cache = IRStats::Cache->new($self->{'params'}->get('id'));
      if ($cache->exists)
+     {
@@ Line 214: / Line 213: @@
 Now we process them.  In this case, we don't even need a loop as we know there's only going to be one row.  We'll stick the result straight into some html, and save it.  Don't forget that if there isn't any data, you still have to output something.
      my @row = $query->fetchrow_array();
-     my $html = '<span class="epstats_view_fulltextcounthtml">' . ($row[1] ? $row[1] : '0') . "</span>";
+     my $html = '<span class="irstats_view_fulltextcounthtml">' . ($row[1] ? $row[1] : '0') . "</span>";
 A little housekeeping:
      $query->finish();
@@ Line 229: / Line 228: @@
 If we wanted to break our daterange into periods, we'd need to do something like this:
-  my $periods = EPStats::Periods->new($self->{'params'}->{'start_date'},$self->{'params'}->{'end_date'});
+  my $periods = IRStats::Periods->new($self->{'params'}->{'start_date'},$self->{'params'}->{'end_date'});
   foreach my $period ( @{$periods->calandar_months()} )
+  {