EPrints Documentation - User contributions [en-gb]

Adding multilang fields

2016-07-07T10:29:37Z

Mamalos@eng.auth.gr: /* Warning for already populated repositories */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''! The users of this plugin are requested to use EPrints API to copy the values from the old fields to the new ones. A script that copies the '''title''' value of each eprint to '''ml_title''' field's '''en''' and '''el''' languages can be found here:[https://github.com/mamalos/eprints/blob/master/help_scripts/set_ml_title.pl]. Please, use this script at your own risk and don't forget to change its content to reflect the languages of your repository. The same script can be used for copying the '''abstract''' field's values to '''ml_abstract''' '''en''' and '''el''' by replacing title with abstract in this script.

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek languages ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. So, for the English language we can add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases can be added in file '''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new '''multilang''' should be added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
EPrints' search functionality is only aware of the '''title''' field, so its search mechanisms -which directly search EPrints' database- has no knowledge about '''ml_title'''. To address this, we should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We have chosen not to change EPrints' default search configuration files, in order not to affect future EPrints' upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

MultiLang Fields Bazaar Package

2016-07-07T10:28:42Z

Mamalos@eng.auth.gr:

[[Category:EPrints 3 Plugins]]

The MultiLang plugins introduces multiple language support in EPrints fields. The specific plugin replaces the '''title''' and '''abstract''' fields' types with their multilingual versions that calculate their values based on two newly added multilingual fields ('''ml_title''' and '''ml_abstract''' respectively). The wiki-page that explains how one can replace basic EPrints fields with multilang-fields -which basically explains how this plugin was created- is [[Adding multilang fields]].

== Installation Prerequisites ==
None.

== Installation ==

Install through the EPrints Bazaar.

== Warning for already populated repositories ==
Once this plugin is installed, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''! The users of this plugin are requested to use EPrints API to copy the values from the old fields to the new ones. A script that copies the '''title''' value of each eprint to '''ml_title''' field's '''en''' and '''el''' languages can be found here:[https://github.com/mamalos/eprints/blob/master/help_scripts/set_ml_title.pl]. Please, use this script at your own risk and don't forget to change its content to reflect the languages of your repository. The same script can be used for copying the '''abstract''' field's values to '''ml_abstract''' '''en''' and '''el''' by replacing title with abstract in this script.

== How to use the plugin ==
Once the plugin has been installed, the user needs to edit the workflow to contain the multilingual fields' versions instead of the default ones. This procedure is explained in the next section.

== Editing the workflow ==
In order for the plugin to be usable, the default field versions need to be commented out and the new multilingual ones need to be added. So, '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' has to be edited as follows:

<source lang="html4strict">


<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>

As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports the new '''ml_'''fields. Once the workflow has changed, the repository needs to be reloaded for the changes to take effect:

<source lang="bash">
./epadmin reload reponame
</source>

== Editing the phrase files ==
The plugin updates the EPrints system by providing an English and a Greek phrase file for the new fields it creates. The new fields are named '''ml_title''' and '''ml_abstract'''. So, if this plugin is used for repositories in other languages than these, the site administrator needs to provide the appropriate phrase files for their languages manually. What they can do is copy the phrases from the English language phrase file located in '''~eprints/archives/<reponame>/cfg/lang/en/phrases/local.xml''' and copy them to a new file located in '''~eprints/archives/<reponame>/cfg/lang/<your_lang>/phrases/'''. The file-name should have an '''xml''' extension (like '''local.xml''' in our example).

The phrases that need to be added are the following (excerpt from the Greek phrases file):
<source lang="html4strict">

<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>



<epp:phrase id="eprint_fieldname_ml_abstract">Περίληψη</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_abstract">Μία περίληψη των περιεχομένων. Εάν το τεκμήριο διαθέτει περίληψη, αυτή πρέπει να τοποθετηθεί εδώ. Υποστηρίζεται μόνον απλό κείμενο.</epp:phrase>

</source>

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

== How the plugin works ==
Basically, what the plugin does is that it replaces EPrints default '''title''' and '''abstract''' field types with a type that supports storing and retrieving information on multilingual fields. The procedure that we followed in order to achieve this functionality was the following:

* We introduced a new field's type, '''virtualwithvalue''', that would allow our fields to override EPrints fields' default behaviour.
* We added multilingual versions of the fields that are replaced (namely '''ml_title''' and '''ml_abstract''') to store multilingual content.
* We introduced our new fields' names, types and functionality to EPrints via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''
* We added the appropriate phrases in each language's phrase file.
* We added a custom lookup cgi script that uses the new '''ml_title''' field for the autocompletion feature.
* We changed EPrints basic and advanced search scripts to search our new multilingual fields instead of the original ones.

A screenshot showing how the two fields look when inserting a new document follows:
[[File:ruomo_multilang.jpg]]

MultiLang Fields Bazaar Package

2016-07-07T10:28:09Z

Mamalos@eng.auth.gr: /* Warning for already populated repositories */

[[Category:EPrints 3 Plugins]]

The MultiLang plugins introduces multiple language support in EPrints fields. The specific plugin replaces the '''title''' and '''abstract''' fields' types with their multilingual versions that calculate their values based on two newly added multilingual fields ('''ml_title''' and '''ml_abstract''' respectively). The wiki-page that explains how one can replace basic EPrints fields with multilang-fields -which basically explains how this plugin was created- is [[Adding multilang fields]].

== Installation Prerequisites ==
None.

== Installation ==

Install through the EPrints Bazaar.

== Warning for already populated repositories ==
Once this plugin is installed, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''! The users of this plugin are requested to use EPrints API to copy the values from the old fields to the new ones.

== How to use the plugin ==
Once the plugin has been installed, the user needs to edit the workflow to contain the multilingual fields' versions instead of the default ones. This procedure is explained in the next section. A script that copies the '''title''' value of each eprint to '''ml_title''' field's '''en''' and '''el''' languages can be found here:[https://github.com/mamalos/eprints/blob/master/help_scripts/set_ml_title.pl]. Please, use this script at your own risk and don't forget to change its content to reflect the languages of your repository. The same script can be used for copying the '''abstract''' field's values to '''ml_abstract''' '''en''' and '''el''' by replacing title with abstract in this script.

== Editing the workflow ==
In order for the plugin to be usable, the default field versions need to be commented out and the new multilingual ones need to be added. So, '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' has to be edited as follows:

<source lang="html4strict">


<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>

As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports the new '''ml_'''fields. Once the workflow has changed, the repository needs to be reloaded for the changes to take effect:

<source lang="bash">
./epadmin reload reponame
</source>

== Editing the phrase files ==
The plugin updates the EPrints system by providing an English and a Greek phrase file for the new fields it creates. The new fields are named '''ml_title''' and '''ml_abstract'''. So, if this plugin is used for repositories in other languages than these, the site administrator needs to provide the appropriate phrase files for their languages manually. What they can do is copy the phrases from the English language phrase file located in '''~eprints/archives/<reponame>/cfg/lang/en/phrases/local.xml''' and copy them to a new file located in '''~eprints/archives/<reponame>/cfg/lang/<your_lang>/phrases/'''. The file-name should have an '''xml''' extension (like '''local.xml''' in our example).

The phrases that need to be added are the following (excerpt from the Greek phrases file):
<source lang="html4strict">

<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>



<epp:phrase id="eprint_fieldname_ml_abstract">Περίληψη</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_abstract">Μία περίληψη των περιεχομένων. Εάν το τεκμήριο διαθέτει περίληψη, αυτή πρέπει να τοποθετηθεί εδώ. Υποστηρίζεται μόνον απλό κείμενο.</epp:phrase>

</source>

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

== How the plugin works ==
Basically, what the plugin does is that it replaces EPrints default '''title''' and '''abstract''' field types with a type that supports storing and retrieving information on multilingual fields. The procedure that we followed in order to achieve this functionality was the following:

* We introduced a new field's type, '''virtualwithvalue''', that would allow our fields to override EPrints fields' default behaviour.
* We added multilingual versions of the fields that are replaced (namely '''ml_title''' and '''ml_abstract''') to store multilingual content.
* We introduced our new fields' names, types and functionality to EPrints via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''
* We added the appropriate phrases in each language's phrase file.
* We added a custom lookup cgi script that uses the new '''ml_title''' field for the autocompletion feature.
* We changed EPrints basic and advanced search scripts to search our new multilingual fields instead of the original ones.

A screenshot showing how the two fields look when inserting a new document follows:
[[File:ruomo_multilang.jpg]]

MultiLang Fields Bazaar Package

2016-07-07T10:25:22Z

Mamalos@eng.auth.gr: /* How to use the plugin */

[[Category:EPrints 3 Plugins]]

The MultiLang plugins introduces multiple language support in EPrints fields. The specific plugin replaces the '''title''' and '''abstract''' fields' types with their multilingual versions that calculate their values based on two newly added multilingual fields ('''ml_title''' and '''ml_abstract''' respectively). The wiki-page that explains how one can replace basic EPrints fields with multilang-fields -which basically explains how this plugin was created- is [[Adding multilang fields]].

== Installation Prerequisites ==
None.

== Installation ==

Install through the EPrints Bazaar.

== Warning for already populated repositories ==
Once this plugin is installed, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''! The users are requested to use EPrints API to copy the values from the old fields to the new ones.

== How to use the plugin ==
Once the plugin has been installed, the user needs to edit the workflow to contain the multilingual fields' versions instead of the default ones. This procedure is explained in the next section. A script that copies the '''title''' value of each eprint to '''ml_title''' field's '''en''' and '''el''' languages can be found here:[https://github.com/mamalos/eprints/blob/master/help_scripts/set_ml_title.pl]. Please, use this script at your own risk and don't forget to change its content to reflect the languages of your repository. The same script can be used for copying the '''abstract''' field's values to '''ml_abstract''' '''en''' and '''el''' by replacing title with abstract in this script.

== Editing the workflow ==
In order for the plugin to be usable, the default field versions need to be commented out and the new multilingual ones need to be added. So, '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' has to be edited as follows:

<source lang="html4strict">


<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>

As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports the new '''ml_'''fields. Once the workflow has changed, the repository needs to be reloaded for the changes to take effect:

<source lang="bash">
./epadmin reload reponame
</source>

== Editing the phrase files ==
The plugin updates the EPrints system by providing an English and a Greek phrase file for the new fields it creates. The new fields are named '''ml_title''' and '''ml_abstract'''. So, if this plugin is used for repositories in other languages than these, the site administrator needs to provide the appropriate phrase files for their languages manually. What they can do is copy the phrases from the English language phrase file located in '''~eprints/archives/<reponame>/cfg/lang/en/phrases/local.xml''' and copy them to a new file located in '''~eprints/archives/<reponame>/cfg/lang/<your_lang>/phrases/'''. The file-name should have an '''xml''' extension (like '''local.xml''' in our example).

The phrases that need to be added are the following (excerpt from the Greek phrases file):
<source lang="html4strict">

<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>



<epp:phrase id="eprint_fieldname_ml_abstract">Περίληψη</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_abstract">Μία περίληψη των περιεχομένων. Εάν το τεκμήριο διαθέτει περίληψη, αυτή πρέπει να τοποθετηθεί εδώ. Υποστηρίζεται μόνον απλό κείμενο.</epp:phrase>

</source>

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

== How the plugin works ==
Basically, what the plugin does is that it replaces EPrints default '''title''' and '''abstract''' field types with a type that supports storing and retrieving information on multilingual fields. The procedure that we followed in order to achieve this functionality was the following:

* We introduced a new field's type, '''virtualwithvalue''', that would allow our fields to override EPrints fields' default behaviour.
* We added multilingual versions of the fields that are replaced (namely '''ml_title''' and '''ml_abstract''') to store multilingual content.
* We introduced our new fields' names, types and functionality to EPrints via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''
* We added the appropriate phrases in each language's phrase file.
* We added a custom lookup cgi script that uses the new '''ml_title''' field for the autocompletion feature.
* We changed EPrints basic and advanced search scripts to search our new multilingual fields instead of the original ones.

A screenshot showing how the two fields look when inserting a new document follows:
[[File:ruomo_multilang.jpg]]

MultiLang Fields Bazaar Package

2016-06-24T09:58:16Z

Mamalos@eng.auth.gr: /* Editing the phrase files */

[[Category:EPrints 3 Plugins]]

The MultiLang plugins introduces multiple language support in EPrints fields. The specific plugin replaces the '''title''' and '''abstract''' fields' types with their multilingual versions that calculate their values based on two newly added multilingual fields ('''ml_title''' and '''ml_abstract''' respectively). The wiki-page that explains how one can replace basic EPrints fields with multilang-fields -which basically explains how this plugin was created- is [[Adding multilang fields]].

== Installation Prerequisites ==
None.

== Installation ==

Install through the EPrints Bazaar.

== Warning for already populated repositories ==
Once this plugin is installed, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''! The users are requested to use EPrints API to copy the values from the old fields to the new ones.

== How to use the plugin ==
Once the plugin has been installed, the user needs to edit the workflow to contain the multilingual fields' versions instead of the default ones. This procedure is explained in the next section.

== Editing the workflow ==
In order for the plugin to be usable, the default field versions need to be commented out and the new multilingual ones need to be added. So, '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' has to be edited as follows:

<source lang="html4strict">


<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>

As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports the new '''ml_'''fields. Once the workflow has changed, the repository needs to be reloaded for the changes to take effect:

<source lang="bash">
./epadmin reload reponame
</source>

== Editing the phrase files ==
The plugin updates the EPrints system by providing an English and a Greek phrase file for the new fields it creates. The new fields are named '''ml_title''' and '''ml_abstract'''. So, if this plugin is used for repositories in other languages than these, the site administrator needs to provide the appropriate phrase files for their languages manually. What they can do is copy the phrases from the English language phrase file located in '''~eprints/archives/<reponame>/cfg/lang/en/phrases/local.xml''' and copy them to a new file located in '''~eprints/archives/<reponame>/cfg/lang/<your_lang>/phrases/'''. The file-name should have an '''xml''' extension (like '''local.xml''' in our example).

The phrases that need to be added are the following (excerpt from the Greek phrases file):
<source lang="html4strict">

<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>



<epp:phrase id="eprint_fieldname_ml_abstract">Περίληψη</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_abstract">Μία περίληψη των περιεχομένων. Εάν το τεκμήριο διαθέτει περίληψη, αυτή πρέπει να τοποθετηθεί εδώ. Υποστηρίζεται μόνον απλό κείμενο.</epp:phrase>

</source>

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

== How the plugin works ==
Basically, what the plugin does is that it replaces EPrints default '''title''' and '''abstract''' field types with a type that supports storing and retrieving information on multilingual fields. The procedure that we followed in order to achieve this functionality was the following:

* We introduced a new field's type, '''virtualwithvalue''', that would allow our fields to override EPrints fields' default behaviour.
* We added multilingual versions of the fields that are replaced (namely '''ml_title''' and '''ml_abstract''') to store multilingual content.
* We introduced our new fields' names, types and functionality to EPrints via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''
* We added the appropriate phrases in each language's phrase file.
* We added a custom lookup cgi script that uses the new '''ml_title''' field for the autocompletion feature.
* We changed EPrints basic and advanced search scripts to search our new multilingual fields instead of the original ones.

A screenshot showing how the two fields look when inserting a new document follows:
[[File:ruomo_multilang.jpg]]

MultiLang Fields Bazaar Package

2016-06-24T09:57:36Z

Mamalos@eng.auth.gr: /* Editing the phrase files */

[[Category:EPrints 3 Plugins]]

The MultiLang plugins introduces multiple language support in EPrints fields. The specific plugin replaces the '''title''' and '''abstract''' fields' types with their multilingual versions that calculate their values based on two newly added multilingual fields ('''ml_title''' and '''ml_abstract''' respectively). The wiki-page that explains how one can replace basic EPrints fields with multilang-fields -which basically explains how this plugin was created- is [[Adding multilang fields]].

== Installation Prerequisites ==
None.

== Installation ==

Install through the EPrints Bazaar.

== Warning for already populated repositories ==
Once this plugin is installed, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''! The users are requested to use EPrints API to copy the values from the old fields to the new ones.

== How to use the plugin ==
Once the plugin has been installed, the user needs to edit the workflow to contain the multilingual fields' versions instead of the default ones. This procedure is explained in the next section.

== Editing the workflow ==
In order for the plugin to be usable, the default field versions need to be commented out and the new multilingual ones need to be added. So, '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' has to be edited as follows:

<source lang="html4strict">


<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>

As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports the new '''ml_'''fields. Once the workflow has changed, the repository needs to be reloaded for the changes to take effect:

<source lang="bash">
./epadmin reload reponame
</source>

== Editing the phrase files ==
The plugin updates the EPrints system by providing an English and a Greek phrase file for the new fields it creates. The new fields are named '''ml_title''' and '''ml_abstract'''. So, if this plugin is used for repositories in other languages than these, the site administrator needs to provide the appropriate phrase files for their languages manually. What they can do is copy the phrases from the English language phrase file located in '''~eprints/archives/<reponame>/cfg/lang/en/phrases/local.xml''' and copy them to a new file located in ~eprints/archives/<reponame>/cfg/lang/<your_lang>/phrases/'''. The file-name should have an '''xml''' extension (like '''local.xml''' in our example).

The phrases that need to be added are the following (excerpt from the Greek phrases file):
<source lang="html4strict">

<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>



<epp:phrase id="eprint_fieldname_ml_abstract">Περίληψη</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_abstract">Μία περίληψη των περιεχομένων. Εάν το τεκμήριο διαθέτει περίληψη, αυτή πρέπει να τοποθετηθεί εδώ. Υποστηρίζεται μόνον απλό κείμενο.</epp:phrase>

</source>

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

== How the plugin works ==
Basically, what the plugin does is that it replaces EPrints default '''title''' and '''abstract''' field types with a type that supports storing and retrieving information on multilingual fields. The procedure that we followed in order to achieve this functionality was the following:

* We introduced a new field's type, '''virtualwithvalue''', that would allow our fields to override EPrints fields' default behaviour.
* We added multilingual versions of the fields that are replaced (namely '''ml_title''' and '''ml_abstract''') to store multilingual content.
* We introduced our new fields' names, types and functionality to EPrints via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''
* We added the appropriate phrases in each language's phrase file.
* We added a custom lookup cgi script that uses the new '''ml_title''' field for the autocompletion feature.
* We changed EPrints basic and advanced search scripts to search our new multilingual fields instead of the original ones.

A screenshot showing how the two fields look when inserting a new document follows:
[[File:ruomo_multilang.jpg]]

MultiLang Fields Bazaar Package

2016-06-24T09:56:48Z

Mamalos@eng.auth.gr: /* Editing the phrase files */

[[Category:EPrints 3 Plugins]]

The MultiLang plugins introduces multiple language support in EPrints fields. The specific plugin replaces the '''title''' and '''abstract''' fields' types with their multilingual versions that calculate their values based on two newly added multilingual fields ('''ml_title''' and '''ml_abstract''' respectively). The wiki-page that explains how one can replace basic EPrints fields with multilang-fields -which basically explains how this plugin was created- is [[Adding multilang fields]].

== Installation Prerequisites ==
None.

== Installation ==

Install through the EPrints Bazaar.

== Warning for already populated repositories ==
Once this plugin is installed, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''! The users are requested to use EPrints API to copy the values from the old fields to the new ones.

== How to use the plugin ==
Once the plugin has been installed, the user needs to edit the workflow to contain the multilingual fields' versions instead of the default ones. This procedure is explained in the next section.

== Editing the workflow ==
In order for the plugin to be usable, the default field versions need to be commented out and the new multilingual ones need to be added. So, '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' has to be edited as follows:

<source lang="html4strict">


<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>

As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports the new '''ml_'''fields. Once the workflow has changed, the repository needs to be reloaded for the changes to take effect:

<source lang="bash">
./epadmin reload reponame
</source>

== Editing the phrase files ==
The plugin updates the EPrints system by providing an English and a Greek phrase file for the new fields it creates. The new fields are named '''ml_title''' and '''ml_abstract'''. So, if this plugin is used for repositories in other languages than these, the site administrator needs to provide the appropriate phrase files for their languages manually. What they can do is copy the phrases from the English language phrase file located in '''~eprints/archives/<reponame>/cfg/lang/en/phrases/local.xml''' and copy them to a new file located in ~eprints/archives/<reponame>/cfg/lang/<your_lang>/phrases/'''. The file-name should have an '''xml''' extension (like '''local.xml''' in our example).

The phrases that need to be added are the following (excerpt from the Greek phrases file):
<source lang="html4strict">

<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>



<epp:phrase id="eprint_fieldname_ml_abstract">Περίληψη</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_abstract">Μία περίληψη των περιεχομένων. Εάν το τεκμήριο διαθέτει περίληψη, αυτή πρέπει να τοποθετηθεί εδώ. Υποστηρίζεται μόνον απλό κείμενο.</epp:phrase>

</source>

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

== How the plugin works ==
Basically, what the plugin does is that it replaces EPrints default '''title''' and '''abstract''' field types with a type that supports storing and retrieving information on multilingual fields. The procedure that we followed in order to achieve this functionality was the following:

* We introduced a new field's type, '''virtualwithvalue''', that would allow our fields to override EPrints fields' default behaviour.
* We added multilingual versions of the fields that are replaced (namely '''ml_title''' and '''ml_abstract''') to store multilingual content.
* We introduced our new fields' names, types and functionality to EPrints via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''
* We added the appropriate phrases in each language's phrase file.
* We added a custom lookup cgi script that uses the new '''ml_title''' field for the autocompletion feature.
* We changed EPrints basic and advanced search scripts to search our new multilingual fields instead of the original ones.

A screenshot showing how the two fields look when inserting a new document follows:
[[File:ruomo_multilang.jpg]]

MultiLang Fields Bazaar Package

2016-06-24T09:54:53Z

Mamalos@eng.auth.gr:

[[Category:EPrints 3 Plugins]]

The MultiLang plugins introduces multiple language support in EPrints fields. The specific plugin replaces the '''title''' and '''abstract''' fields' types with their multilingual versions that calculate their values based on two newly added multilingual fields ('''ml_title''' and '''ml_abstract''' respectively). The wiki-page that explains how one can replace basic EPrints fields with multilang-fields -which basically explains how this plugin was created- is [[Adding multilang fields]].

== Installation Prerequisites ==
None.

== Installation ==

Install through the EPrints Bazaar.

== Warning for already populated repositories ==
Once this plugin is installed, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''! The users are requested to use EPrints API to copy the values from the old fields to the new ones.

== How to use the plugin ==
Once the plugin has been installed, the user needs to edit the workflow to contain the multilingual fields' versions instead of the default ones. This procedure is explained in the next section.

== Editing the workflow ==
In order for the plugin to be usable, the default field versions need to be commented out and the new multilingual ones need to be added. So, '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' has to be edited as follows:

<source lang="html4strict">


<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>

As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports the new '''ml_'''fields. Once the workflow has changed, the repository needs to be reloaded for the changes to take effect:

<source lang="bash">
./epadmin reload reponame
</source>

== Editing the phrase files ==
The plugin updates the EPrints system by providing an English and a Greek phrase for the new fields it creates. The new fields are named '''ml_title''' and '''ml_abstract'''. So, if this plugin is used for repositories in other languages than these, the site administrator needs to provide the phrases file for their language manually. What they can do is copy the phrases from the English language phrase-file located in '''~eprints/archives/<reponame>/cfg/lang/en/phrases/local.xml''' and copy them to a new file located in ~eprints/archives/<reponame>/cfg/lang/<your_lang>/phrases/'''. The file-name should have an '''xml''' extension (like '''local.xml''' in our example).

The phrases that need to be added are the following (excerpt from the Greek phrases file):
<source lang="html4strict">

<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>



<epp:phrase id="eprint_fieldname_ml_abstract">Περίληψη</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_abstract_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_abstract">Μία περίληψη των περιεχομένων. Εάν το τεκμήριο διαθέτει περίληψη, αυτή πρέπει να τοποθετηθεί εδώ. Υποστηρίζεται μόνον απλό κείμενο.</epp:phrase>

</source>

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

== How the plugin works ==
Basically, what the plugin does is that it replaces EPrints default '''title''' and '''abstract''' field types with a type that supports storing and retrieving information on multilingual fields. The procedure that we followed in order to achieve this functionality was the following:

* We introduced a new field's type, '''virtualwithvalue''', that would allow our fields to override EPrints fields' default behaviour.
* We added multilingual versions of the fields that are replaced (namely '''ml_title''' and '''ml_abstract''') to store multilingual content.
* We introduced our new fields' names, types and functionality to EPrints via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''
* We added the appropriate phrases in each language's phrase file.
* We added a custom lookup cgi script that uses the new '''ml_title''' field for the autocompletion feature.
* We changed EPrints basic and advanced search scripts to search our new multilingual fields instead of the original ones.

A screenshot showing how the two fields look when inserting a new document follows:
[[File:ruomo_multilang.jpg]]

MultiLang Fields Bazaar Package

2016-06-24T09:44:54Z

Mamalos@eng.auth.gr: /* Warning for already populated repositories */

File:Ruomo multilang.jpg

2016-06-18T07:15:59Z

Mamalos@eng.auth.gr: Mamalos@eng.auth.gr uploaded a new version of "File:Ruomo multilang.jpg"

File:Ruomo multilang.jpg

2016-06-18T07:13:39Z

Mamalos@eng.auth.gr: Mamalos@eng.auth.gr uploaded a new version of "File:Ruomo multilang.jpg"

File:Ruomo multilang.jpg

2016-06-18T07:12:23Z

Mamalos@eng.auth.gr:

Adding multilang fields

2016-06-07T12:27:10Z

Mamalos@eng.auth.gr: /* Adding search support for the ml_title field */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek languages ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. So, for the English language we can add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases can be added in file '''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new '''multilang''' should be added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
EPrints' search functionality is only aware of the '''title''' field, so its search mechanisms -which directly search EPrints' database- has no knowledge about '''ml_title'''. To address this, we should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We have chosen not to change EPrints' default search configuration files, in order not to affect future EPrints' upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:26:39Z

Mamalos@eng.auth.gr: /* Adding search support for the ml_title field */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek languages ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. So, for the English language we can add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases can be added in file '''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new '''multilang''' should be added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
EPrints' search functionality is only aware of the '''title''' field, so its search mechanisms -which directly search EPrints' database- has no knowledge about '''ml_title'''. To address this, we should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We have chosen not to change EPrints' default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:26:06Z

Mamalos@eng.auth.gr: /* Adding search support for the ml_title field */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek languages ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. So, for the English language we can add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases can be added in file '''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new '''multilang''' should be added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
EPrints' search functionality is only aware of the '''title''' field, so its search mechanisms -which directly search EPrints' database- has no knowledge about '''ml_title'''. To address this, we should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:25:42Z

Mamalos@eng.auth.gr: /* Adding search support for the ml_title field */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek languages ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. So, for the English language we can add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases can be added in file '''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new '''multilang''' should be added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
EPrints' search functionality is only aware of the '''title''' field, so its search mechanisms -which directly search EPrints' database- has no knowledge about '''ml_title'''. To address this, we should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database'''Bold text'''. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:25:06Z

Mamalos@eng.auth.gr: /* Adding search support for the ml_title field */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek languages ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. So, for the English language we can add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases can be added in file '''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new '''multilang''' should be added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
EPrints' search functionality is only aware about the '''title''' field, so its search mechanisms -which directly search EPrints' database- will not search within '''ml_title'''. To address this, we should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database'''Bold text'''. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:22:21Z

Mamalos@eng.auth.gr:

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek languages ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. So, for the English language we can add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases can be added in file '''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new '''multilang''' should be added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:20:29Z

Mamalos@eng.auth.gr: /* Replacing the title field with ml_title field in the workflow */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek languages ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. So, for the English language we can add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases can be added in file '''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new '''multilang''' should be added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:19:49Z

Mamalos@eng.auth.gr: /* Adding the appropriate phrases */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek languages ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. So, for the English language we can add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases can be added in file '''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:19:35Z

Mamalos@eng.auth.gr: /* Adding the appropriate phrases */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek languages ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. So, for the English language we can add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases can be added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:18:51Z

Mamalos@eng.auth.gr: /* Adding the appropriate phrases */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek languages ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:18:28Z

Mamalos@eng.auth.gr: /* Adding the appropriate phrases */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:18:08Z

Mamalos@eng.auth.gr: /* Introducing ml_title field in EPrints and replacing title field's type */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality. Hence, doing so would cause EPrints to throw errors. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for our '''title''' field (using the data stored in '''ml_title''' field) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish when overriding these functions, as long as their code return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:14:50Z

Mamalos@eng.auth.gr: /* Introducing ml_title field in EPrints and replacing title field's type */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only a single value from the '''title''' (and '''abstract''') field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:14:06Z

Mamalos@eng.auth.gr: /* Introducing ml_title field in EPrints and replacing title field's type */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints' basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:13:34Z

Mamalos@eng.auth.gr: /* Introducing ml_title field in EPrints and replacing title field's type */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding '''multilang''' field ('''ml_title'''). So, each new record now has a '''multilang''' field which it can access via the interface provided by the calculated field of type '''virtualwithvalue''', which is no other than EPrints basic '''title''' field.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:11:21Z

Mamalos@eng.auth.gr: /* Introducing ml_title field in EPrints and replacing title field's type */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value via its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:10:22Z

Mamalos@eng.auth.gr: /* Introducing ml_title field in EPrints and replacing title field's type */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. The last statements of our example code show how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:08:00Z

Mamalos@eng.auth.gr: /* Introducing ml_title field in EPrints and replacing title field's type */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions -whose names imply their functionality- are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T12:06:49Z

Mamalos@eng.auth.gr: /* Introducing ml_title field in EPrints and replacing title field's type */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements the two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T11:03:14Z

Mamalos@eng.auth.gr: /* How to add a custom, multilingual field */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, and ''title''' and '''ml_title''' are used as our example fields. The code snippets are just for demonstration purposes - and proof of concept. If you want to see the final, working implementation, you should look at the source code of the [[MultiLang_Fields_Bazaar_Package]] plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T11:01:53Z

Mamalos@eng.auth.gr: /* How to add a custom, multilingual field */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* ''Static files need to be regenerated'' if our repository already contains data.
The following sections explain each step in detail, where we are using '''ml_title''' as our example field. The code snippets are just for demonstration purposes (and proof of concept). If you want to see the final, working implementation, you should look at the source code of the plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T11:01:26Z

Mamalos@eng.auth.gr: /* How to add a custom, multilingual field */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will needs a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* '''Static files need to be regenerated''' if our repository already contains data.
The following sections explain each step in detail, where we are using '''ml_title''' as our example field. The code snippets are just for demonstration purposes (and proof of concept). If you want to see the final, working implementation, you should look at the source code of the plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T10:59:34Z

Mamalos@eng.auth.gr: /* Multilang fields and EPrints */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

To address these limitations. this article explains how we can replace a basic field, the '''title''' field as an example, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will contain a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* '''Static files need to be regenerated''' if our repository already contains data.
The following sections explain each step in detail, where we are using '''ml_title''' as our example field. The code snippets are just for demonstration purposes (and proof of concept). If you want to see the final, working implementation, you should look at the source code of the plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T10:59:04Z

Mamalos@eng.auth.gr: /* Multilang fields and EPrints */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects a single output.

So this article explains how we can replace a basic field, the '''title''' field, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will contain a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* '''Static files need to be regenerated''' if our repository already contains data.
The following sections explain each step in detail, where we are using '''ml_title''' as our example field. The code snippets are just for demonstration purposes (and proof of concept). If you want to see the final, working implementation, you should look at the source code of the plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T10:58:41Z

Mamalos@eng.auth.gr: /* Multilang fields and EPrints */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages containing content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects one output.

So this article explains how we can replace a basic field, the '''title''' field, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will contain a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* '''Static files need to be regenerated''' if our repository already contains data.
The following sections explain each step in detail, where we are using '''ml_title''' as our example field. The code snippets are just for demonstration purposes (and proof of concept). If you want to see the final, working implementation, you should look at the source code of the plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T10:57:57Z

Mamalos@eng.auth.gr: /* Adding a new field type (VirtualWithValue) */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages which contain content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects one output.

So this article explains how we can replace a basic field, the '''title''' field, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will contain a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* '''Static files need to be regenerated''' if our repository already contains data.
The following sections explain each step in detail, where we are using '''ml_title''' as our example field. The code snippets are just for demonstration purposes (and proof of concept). If you want to see the final, working implementation, you should look at the source code of the plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' (see [[MetaField]] article for details) is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T10:56:50Z

Mamalos@eng.auth.gr:

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field -indirectly this means one language support. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with its subsystems.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages which contain content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects one output.

So this article explains how we can replace a basic field, the '''title''' field, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will contain a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* '''Static files need to be regenerated''' if our repository already contains data.
The following sections explain each step in detail, where we are using '''ml_title''' as our example field. The code snippets are just for demonstration purposes (and proof of concept). If you want to see the final, working implementation, you should look at the source code of the plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

MultiLang Fields Bazaar Package

2016-06-07T10:55:14Z

Mamalos@eng.auth.gr: /* How the plugin works */

MultiLang Fields Bazaar Package

2016-06-07T10:52:27Z

Mamalos@eng.auth.gr:

Adding multilang fields

2016-06-07T10:51:39Z

Mamalos@eng.auth.gr:

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with it.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages which contain content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects one output.

So this article explains how we can replace a basic field, the '''title''' field, with a multilingual one.

== Warning for already populated repositories ==
Once you follow this article's procedure, '''existing titles and abstracts''' will not be copied into their multilingual counterparts and '''access to them via the API will be lost'''!

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will contain a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* '''Static files need to be regenerated''' if our repository already contains data.
The following sections explain each step in detail, where we are using '''ml_title''' as our example field. The code snippets are just for demonstration purposes (and proof of concept). If you want to see the final, working implementation, you should look at the source code of the plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

MultiLang Fields Bazaar Package

2016-06-07T10:50:49Z

Mamalos@eng.auth.gr: /* Warning to already populated repositories */

MultiLang Fields Bazaar Package

2016-06-07T10:50:33Z

Mamalos@eng.auth.gr:

MultiLang Fields Bazaar Package

2016-06-07T10:24:46Z

Mamalos@eng.auth.gr:

MultiLang Fields Bazaar Package

2016-06-07T10:23:48Z

Mamalos@eng.auth.gr: /* How the plugin works */

MultiLang Fields Bazaar Package

2016-06-07T10:20:28Z

Mamalos@eng.auth.gr:

Adding multilang fields

2016-06-07T10:16:55Z

Mamalos@eng.auth.gr:

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with it.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages which contain content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects one output.

So this article explains how we can replace a basic field, the '''title''' field, with a multilingual one.

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will contain a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* '''Static files need to be regenerated''' if our repository already contains data.
The following sections explain each step in detail, where we are using '''ml_title''' as our example field. The code snippets are just for demonstration purposes (and proof of concept). If you want to see the final, working implementation, you should look at the source code of the plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

== Regenerating static files and abstracts ==
If our repository already contained records, we need to recreate static content such as static pages and abstracts. Hence, as eprints user we should run:
<source lang="bash">
$ ./bin/generate_abstracts reponame
$ ./bin/generate_static reponame
</source>
within eprints user's home directory.

Adding multilang fields

2016-06-07T10:14:34Z

Mamalos@eng.auth.gr: /* How to add a custom, multilingual field */

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with it.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages which contain content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects one output.

So this article explains how we can replace a basic field, the '''title''' field, with a multilingual one.

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will contain a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
* '''Static files need to be regenerated''' if our repository already contains data.
The following sections explain each step in detail, where we are using '''ml_title''' as our example field. The code snippets are just for demonstration purposes (and proof of concept). If you want to see the final, working implementation, you should look at the source code of the plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.

Adding multilang fields

2016-06-07T10:13:18Z

Mamalos@eng.auth.gr:

EPrints' builtin fields are not multilingual, in the sense that there is only one version of each field. This page explains how to add multilingual versions of existing fields in EPrints and how they can be integrated with it.

== Multilang fields and EPrints ==
EPrints supports the '''multilang''' field type (see [[Multilang]] for more details) which allows a user to insert different content for different languages. There are a few limitations with '''multilang''' fields though:

* When its value is printed it is shown in all languages which contain content.
* If a basic EPrints field's type is replaced to become '''multilang''' (like '''title''' and '''abstract'''), EPrints functionality breaks because it expects one output.

So this article explains how we can replace a basic field, the '''title''' field, with a multilingual one.

== How to add a custom, multilingual field ==
Replacing an EPrints basic field, like the '''title''' field, involves a few steps. First, a new field needs to be created that will be able to store information for different languages; this field will be of type '''multilang'''. Next, the basic field's type needs to be replaced with one that is able to use our newly created field as its storage place. This field type will use a function wrapper for storing and retrieving information from the '''multilang''' field, hence the '''title''' field will become a calculated field.

So, in order to add multilingual support for the '''title''' field, the following actions need to take place:

* A ''new field type needs to be created'' that will help our '''title''' field to implement some of EPrints logic. Our field type is called '''virtualwithvalue'''.
* A ''new '''multilang''' field needs to be created'' that will store our multilingual information. We will call this field '''ml_title'''.
* '''''ml_title''' field and '''title''' field's functionality need to be introduced to the EPrints system'' via a configuration file located in '''~eprints/archives/<reponame>/cfg/cfg.d/'''.
* ''EPrints' database needs to be updated'' to include the new field.
* ''The appropriate phrases need to be added'' for the '''ml_title''' field on each supported language.
* ''The '''title''' field needs to be replaced'' with '''ml_title''' field in the workflow.
* '''''ml_title''' field in the workflow will contain a custom lookup script''.
* ''The '''title''' field needs to be replaced with the '''ml_title''' field in the simple and advanced search scripts''.
* The ''repository needs to be reloaded''.
The following sections explain each step in detail, where we are using '''ml_title''' as our example field. The code snippets are just for demonstration purposes (and proof of concept). If you want to see the final, working implementation, you should look at the source code of the plugin.

== Adding a new field type (VirtualWithValue)==
In order to create a multiple-language field we have to create an appropriate field type. EPrints' '''MetaField''' is a perfect candidate for this, and we need to extend it and override its '''get_value''' and '''set_value''' functions for the field to work properly with the rest of EPrints API, as well as its '''get_property_defaults''' function to sort out warnings for default values. The following code could be a rough implementation of such a field:

<source lang="perl">package EPrints::MetaField::Virtualwithvalue;

use strict;
use warnings;

use EPrints::MetaField;

our @ISA = qw( EPrints::MetaField );

use strict;

sub get_property_defaults
{
my ( $self ) = @_;
my %defaults = $self->SUPER::get_property_defaults;
$defaults{get_value} = $EPrints::MetaField::UNDEF;
$defaults{set_value} = $EPrints::MetaField::UNDEF;

return %defaults;
}

sub get_value
{
my( $self, $object ) = @_;
if ( defined $self->get_property("get_value") )
{
return $self->call_property( "get_value", $object);
}
return undef;
}

sub set_value
{
my( $self, $object, $value ) = @_;
if ( defined $self->get_property("set_value") )
{
return $self->call_property( "set_value", $object, $value);
}
return undef;
}

</source>

We could save this file in '''~eprints/lib/plugins/EPrints/MetaField/Virtualwithvalue.pm'''.

== Introducing ml_title field in EPrints and replacing title field's type==
To inform EPrints about our new field (that will be of type '''virtualwithvalue'''), we should create a configuration file, eg: '''~eprints/archives/<reponame>/cfg/cfg.d/zz_multilang_field.pl''' with content like the following:

<source lang='perl'>
#define local fields
my $local_fields = [
{
name => 'ml_title',
type => 'multilang',
multiple => 1,
fields => [ { sub_name => "text", type => "longtext", input_rows => 3, make_single_value_orderkey => 'EPrints::Extras::english_title_orderkey' } ],
input_add_boxes => 1,
},

{
name => 'title',
type => 'virtualwithvalue',
virtual => 1,

get_value => sub
{
my ($eprint) = @_;
if ($eprint->is_set('ml_title'))
{
my $lang = $eprint->repository->get_langid;
my $lang_set = 0;
my $vals = $eprint->get_value('ml_title');
my $title = '';
if (!$lang)
{
$lang_set = 1;
}
else
{
# set the default lang's text as title
foreach my $v1 (@{$vals})
{
if ($v1->{lang} eq $lang)
{
$title = $v1->{text};
}
}
}
# if the language is not set or I can't find an abstract in the
# user's language, get the first object's text as abstract
if ($lang_set or $title eq '')
{
$title = $vals->[0]->{text};
}
return $title;

}
return undef;
},

set_value => sub
{
my ($eprint, $value) = @_;
my $lang = 'en';
#only use this on imports, NOT if the value is already set
if ($eprint->is_set('ml_title'))
{
return;
}
if ($value)
{
$eprint->set_value('ml_title', [{lang=>$lang, text=>$value}]);
}
}
},
];

#create lookup hash of local field names
my $local_fieldnames = {};

foreach my $f (@{$local_fields})
{
$local_fieldnames->{$f->{name}} = 1;
}

#merge in existing field configurations
foreach my $f (@{$c->{fields}->{eprint}})
{
if (!$local_fieldnames->{$f->{name}})
{
push @{$local_fields}, $f;
}
}

#overwrite original array of configured fields
$c->{fields}->{eprint} = $local_fields;
</source>

Where we can see that our new '''ml_title''' field is of type '''multilang''' and the '''title''' field's type has become '''virtualwithvalue'''. Moreover, the '''title''' field now implements two aforementioned functions: '''get_value''' and '''set_value'''. Both these functions are used by EPrints API, and their existence, as well as their return values, are critical for EPrints to work properly. Their names imply their functionality. In the end of our example code, one can see how a custom field can be added in the list of EPrints fields.

What has happened in effect is that the '''title''' field has become a calculated field that gets or sets its value in its corresponding multilingual field ('''ml_title'''). So, each new record now has a multilingual field which it accesses via the interface provided by the calculated field of type '''virtualwithvalue'''.

The reason we didn't set the '''title''' field to be of type '''multilang''' in the first place is that many EPrints builtin functions expect only one value from the title (and abstract) field, and '''multilang''' fields don't support such functionality, which causes EPrints to err. By using and extending calculated fields (like '''MetaFields'''), we can calculate and produce always a single output for the '''multilang''' field (based on some condition) via its '''get_value''' function; its '''set_value''' function is used for populating our '''ml_title''' field's values. Our sample code prints output based on the user's language settings, but the programmer can do whatever they wish in these functions, as long as they return values that comply with EPrints' API.

== Adding the appropriate phrases ==
EPrints phrases are not aware of our new field ('''ml_title'''), so we need to update them. In Our example we update the phrases for the English and for the Greek language ('''en''' and '''el''' respectively). We chose to add new files instead of changing the default ones so as to help EPrints upgrades. For the English language we add the file '''~eprints/archives/reponame/cfg/lang/en/phrases/local.xml''' that contains the following information:
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Title</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Text</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Language</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">The title of the item. The title should not end with a full stop, but may end with a question mark. There is no way to make italic text, please enter it normally. If you have a subtitle, it should be preceded with a colon [:]. Use capitals only for the first word and for proper nouns.
 Example: A brief history of time
 Example: Life: an unauthorised biography
 Example: Mathematics for engineers and scientists. 5th edition
 Example: Ecosystems of the world. Vol. 26. Estuaries of the world
</epp:phrase>
</source>

Greek phrases are added in file ''~eprints/archives/reponame/cfg/lang/el/phrases/local.xml''':
<source lang="html4strict">
<epp:phrase id="eprint_fieldname_ml_title">Τίτλος</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_text">Κείμενο</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_help">Το help τεξτ</epp:phrase>
<epp:phrase id="eprint_fieldname_ml_title_lang">Γλώσσα</epp:phrase>
<epp:phrase id="eprint_fieldhelp_ml_title">Ο τίτλος του τεκμηρίου. Ο τίτλος δεν πρέπει να τελειώνει με τελεία, αλλά μπορεί να τελειώνει με ερωτηματικό. Δεν υπάρχει τρόπος να γράψετε με πλάγια γράμματα, παρακαλώ χρησιμοποιήστε απλό κείμενο. Εάν έχετε έναν υπότιτλο, θα πρέπει να προηγείται η άνω και κάτω τελεία του υπότιτλου [:]. Χρησιμοποιήστε κεφαλαία γράμματα μόνο στην πρώτη λέξη και στα κύρια ονόματα.
 Παράδειγμα: Μια σύντομη ιστορία
 Παράδειγμα: Καβάφης: η βιογραφία
 Παράδειγμα: Μαθηματικά για μηχανικούς και επιστήμονες. 5η έκδοση
 Παράδειγμα: Οικοσυστήματα του πλανήτη. Τόμ. 26. Εκβολές του πλανήτη.
</epp:phrase>
</source>

== Replacing the title field with ml_title field in the workflow ==
In order to use our new '''ml_title''' field in the workflow, we need to replace the existing one ('''title'''). So, the '''title''' needs to be commented out and the new multilingual one need is added. This means that '''~eprints/archives/reponame/cfg/workflows/eprint/default.xml''' is edited as follows:
<source lang="html4strict">

<component><field ref="ml_title" required="yes" input_lookup_url="{$config{rel_cgipath}}/users/lookup/ml_title_duplicates" input_lookup_params="id={eprintid}&dataset=eprint&field=ml_title"/></component>
<component><field ref="ml_abstract"/></component>
</source>
As can be seen, the default lookup script is replaced by the plugin's lookup script ('''ml_title_duplicates''') which supports our new '''ml_title''' field.

== Updating EPrints database to include the ml_title field ==
EPrints database is updated when the user eprints executes (from his home directory):
<source lang="bash">
$ ./bin/epadmin update reponame
</source>

== Adding a custom lookup script for ml_title autocompletion ==
We copy the default lookup script to a new one ('''~eprints/cgi/users/lookup/ml_title_duplicates in our example'''):
<source lang="bash">
$ cp ~eprints/cgi/users/lookup/title_duplicates ~eprints/cgi/users/lookup/ml_title_duplicates
</source>
and edit the SQL statement to contain '''ml_title''' instead of '''title''' (line 70 in EPrints 3.3.14):
<source lang="php">
my $sql = "SELECT ep.eprintid, ml_title_text FROM eprint AS ep JOIN eprint_ml_title_text AS ml ON ep.eprintid = ml.eprintid WHERE ";
if ($dataset_name eq "eprint") {
$sql .= " $Q_eprint_status=" . $db->quote_value( "archive" ) . " AND ";
}
$sql .= "ml.ml_title_text IS NOT NULL" .
" AND ml.ml_title_text " .
$db->sql_LIKE() .
$db->quote_value( EPrints::Database::prep_like_value( $q ) . '%' );
</source>

== Adding search support for the ml_title field ==
We should allow our search scripts to be able to search into '''ml_title''' instead of '''title''', since '''title''' is now a calculated field and contains no information in EPrints database. To do so, we add two configuration files that add support for each search respectively. We chose not to change EPrints default search configuration files in order to to affect future EPrints upgrades. So, for the simple search we add the file '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_simple.pl''' with the content:
<source lang="perl">$c->{search}->{simple} =
{
search_fields => [
{
id => "q",
meta_fields => [
"documents",
"ml_title",
"abstract",
"creators_name",
"date"
]
},
],
# preamble_phrase => "cgi/search:preamble",
title_phrase => "cgi/search:simple_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

And for the advanced search we add the file: '''~eprints/archives/reponame/cfg/cfg.d/eprint_search_advanced_local.pl''' that reads:
<source lang="perl">
$c->{search}->{advanced} =
{
search_fields => [
{ meta_fields => [ "documents" ] },
{ meta_fields => [ "ml_title" ] },
{ meta_fields => [ "creators_name" ] },
{ meta_fields => [ "abstract" ] },
{ meta_fields => [ "date" ] },
{ meta_fields => [ "keywords" ] },
{ meta_fields => [ "subjects" ] },
{ meta_fields => [ "type" ] },
{ meta_fields => [ "department" ] },
{ meta_fields => [ "editors_name" ] },
{ meta_fields => [ "ispublished" ] },
{ meta_fields => [ "refereed" ] },
{ meta_fields => [ "publication" ] },
{ meta_fields => [ "documents.format" ] },
],
preamble_phrase => "cgi/advsearch:preamble",
title_phrase => "cgi/advsearch:adv_search",
citation => "result",
page_size => 20,
order_methods => {
"byyear" => "-date/creators_name/title",
"byyearoldest" => "date/creators_name/title",
"byname" => "creators_name/-date/title",
"bytitle" => "title/creators_name/-date"
},
default_order => "byyear",
show_zero_results => 1,
};
</source>

== Reloading our repository ==
In order for our changes to take effect, we should reload our repository by running:
<source lang="bash">
$ ./bin/epadmin reload reponame</source>
within eprints user's home directory as user eprints.