Autocompletion

EPrints 3 Reference: Directory Structure - Metadata Fields - Repository Configuration - XML Config Files - XML Export Format - EPrints data structure - Core API - Data Objects

For a how-to, please see Autocompletion and Authority Files (Romeo Autocomplete) or Adding an Auto-Completer to a non-workflow page

Autocompletion in EPrints 3 consists of several stages.

• A field in the workflow is configured to say what autocompletion URL to use, plus any additional parameters to pass to the script. This URL must be on the same server (e.g. foo.eprints.org) but does not have to be part of the EPrints system.
• The autocomplete script takes the text typed so far (and maybe the additional parameters) and returns a chunk of XML describing possible autocomplete options. This XML consists of a number of rows (how many is up to the script).
• Each row contains some HTML to show the person viewing plus a magic <ul> block which is hidden from display, but is used by the autocomplete JavaScript to autocomplete the page.

Autocomplete Scripts

EPrints autocomplete scripts live in /opt/eprints3/cgi/users/lookup/ you can add your own here, or maybe elsewhere if, for example, you needed to use PHP.

There are several kinds of autocomplete scripts:

• those that just use the existing data in your repository (these are dead easy as they work out of the box).
• ones which use a file which you place in your repository's cfg/autocomplete/ directory.
• more clever ones.

You may be able to find new autocomplete scripts and authority files on http://files.eprints.org/

Scripts are in (rough) order of complexity to use:

journal_by_name

Can only be used on the "publication" field. Looks up the publication in the existing publications in the repository and autocompletes the publication. If ISSN and/or publisher exist in the same input component as the journal field they will also be completed if data is available.

journal_by_issn

As above, but attached to the ISSN field.

event_by_name

Similar to journal_by_name. Is attached to the event_title field and autocompletes from existing repository data. If they are in the same (multi) input component it will also try and autocomplete event_location, event_dates and event_type.

name

Attached to a multiple compound name/id field (e.g. creators) looks up the name in the existing list in the repository. Can match on any id or given or family name. Populates all parts of the current row it can.

title_duplicates

This is a slightly odd script as it doesn't actually provide any autocomplete data. What it does is search the list of existing titles to see if there is a match. It only searches if there are 5 or more characters entered so far.

If it finds any matches it lists them with a warning that they might be a problem, but does not assist autocompletion. If many matches are made then a short title only is shown, if the list is only 4 or fewer then a full citation is shown.

This is set to "on" by default in the hope that it will reduce duplicate submissions.

simple_file

File needs an additional parameter to be passed to it. This is configured in the workflow. This parameter is the name of a file in the cfg/autocomplete directory. This file contains a list of values which are searched (case insensitively) and matches returned. A second parameter of "mode=prefix" can be set to only match values which start with the text being typed, rather than contain it. Add a second and subsequent parameter by using a semi-colon ( ; ) as delimiter (tested in 3.3.10)

simple_sql

Similar to simple_file but gets its values from a database table.

The table must be in the eprints database used by this repository and start with "ac_". The script needs a param. passed from workflow to indicate the name of the table WITHOUT the ac_ prefix. E.g. if the table was "ac_badgers" the parameter would be "table=badgers". The only field used is "value" which works like the lines in the text file. If you want this to be blindingly fast you can make sure "value" is indexed, and set mode=prefix. With those set autocompleting from a dictionary of half a million words worked cheerfully.

url_name_value

This works like simple_sql except for the fact it uses three columns. url, name and value. It searches and autocompletes using value, but the human-readable description is supplied by "name" and if url is set then a (more info) link is shown. The link opens a new window to avoid mid-form trauma.

file

This is for more complex autocompletion authority files. It works like simple_file except that the file format is more complicated. For a how-to, please see Autocompletion and Authority Files (Romeo Autocomplete)

The file consists of lines which contain:

• a value to search, (e.g. "African Journal of Agricultural Research")
• a tab
• a <li> autocomplete chunk. (with no line breaks) e.g.

 <li style='border-right: solid 50px #30FF30' >

external source

This takes all the ideas above, and extends them to make an API call to an external data source. This has the advantage that you are always referring to the authoritative source, but the disadvantage that you are reliant on both the network being up and the external source being available.

It breaks down into two parts:

• the autocompleter call in the web page
• the script being called

For an example, here is one way to query the RoMEO data directly:

First, set the autocompleter in the eprints workflow:

     <component type="Field::Multi">
      <title>Article Publication Details</title>
        <field ref="publication" input_lookup_url="{$config{perl_url}}/get_journals" />
        <field ref="publisher" />
        <field ref="issn" />
      </component>

Next have the script:

use strict;
use HTTP::Request;
use LWP::UserAgent;
use XML::Twig;

use Data::Dumper;
use EPrints;

my $journal_data = {};

sub urldecode{
  my ($url) = @_;
  $url =~ s/%([0-9a-f][0-9a-f])/pack("C",hex($1))/egi;
  $url =~ s/\x2B/ /; # swap '+' for ' '
  return $url;
}

# XML::Twig's routine for dealing with a journal entry
sub process_journal {
  my ( $twig, $journal ) = @_;

  # get the components
  my $title = urldecode( $journal->first_child('jtitle')->text );

  my $zetoc = urldecode( $journal->first_child('zetocpub')->text ) 
                  if $journal->first_child('zetocpub');
  my $romeo = urldecode( $journal->first_child('romeopub')->text )
                  if $journal->first_child('romeopub');
  my $issn  = urldecode( $journal->first_child('issn')->text )
                  if $journal->first_child('issn');

  my $publisher = $romeo;
  $publisher = $zetoc if (not $publisher && $zetoc);
  
  # build a lub of html based on the components
  my $html .= "<li>$title";
  $html .= "<br />published by $publisher" if $publisher;
  
  $html .= "<ul>";
  if ($title) {
      $html .= "<li id='for:value:component:_publication'>$title</li>";
  }
  if ($publisher) {
      $html .= "<li id='for:value:component:_publisher'>$publisher</li>";
  }
  if ($issn) {
    $html .= "<li id='for:value:component:_issn'>$issn</li>";
  }
  $html .= "</ul></li>\n";
  warn "\n\n$html\n\n";
  # save the html
  $journal_data->{$title} = $html;

  1; 
} ## end process_journal

# get a list of journals that match the query
sub get_journals {
  my $journal = shift;
  my @html = ();

  if ($journal) {
    return ("<ul><li>keep typing....</li></ul>") if (length($journal) < 3);

    $journal =~ s/\s+/\+/;
    my $query = "http://www.sherpa.ac.uk/romeoapi.php?qtype=starts&jtitle=$journal";

    my $request = HTTP::Request->new( GET => "$query" );

    my $ua = LWP::UserAgent->new();
    my $response = $ua->request($request);
    my $content = $response->content();

    my $twig = XML::Twig->new(
                       'keep_encoding' => 1,
                       'TwigRoots' => { 'journals' => 1 },
                       'TwigHandlers' => { 'journal' => \&process_journal, }
                      );
    $twig->parse($content);
    if (scalar keys %{$journal_data}) {
      push @html, "<ul class='journals'>\n";
      foreach my $title (sort keys %{$journal_data}) {
        push @html, "$journal_data->{$title}\n";
      } ## end of  foreach my $title (sort keys %{$journal_data})
      push @html, "</ul>\n";
    } ## end of if (scalar keys %{$journal_data}) ...
  } else {
    push @html, "<!-- No journal name supplied -->\n";
  }

  return (join "\n", @html)

} ## end get_journals

my $session = EPrints::Session->new();

# we need the send an initial content-type
print <<END;
<?xml version="1.0" encoding="UTF-8" ?>

END

# then we send the fragment of html for the autocompleter
print get_journals( lc $session->param( "q" ) );

$session->terminate;