Upgrading

This page is intended to provide guidance for those upgrading between versions of EPrints 3.4. (e.g. 3.4.1 to 3.4.2). This is generally intended for those upgrading using a tagged release from EPrints 3.4 GitHub repository or upgrading using source tarballs available on files.eprints.org. However, new versions of EPrints 3.4 will no longer be released as Linux packages (i.e. RPMs or Debs).

General Advice

If you are running EPrints as a virtual machine, then it is worthwhile taking a snapshot that you could revert back to this in the event of an unsuccessful upgrade. Otherwise, you should take the following backups before starting an upgrade:

The EPrints database

This will likely have the same name as the archive.

The documents directory

This is the documents directory, directly under the archive's directory (i.e. /opt/eprints3/archives/<ARCHIVE_NAME>/).

The Xapian index

This is in the var directory of the archive under the directory named or starting xapian.

Your EPrints archive

All the directories and any files in your archive, except: documents (see above), html and var (except xapian, see above). If you have multiple archives be sure to backup each one.

The current EPrints codebase

In case any changes have been made to this. These may be needed if you have to rollback an upgrade or also apply these changes if they are needed in the upgraded version of EPrints. All directories in the EPrints path (e.g. /opt/eprints3/) should be backed up except: archives (see above), debian, license, pod, testdata, tests,tmp and var. No files in the top level EPrints path directory need to be backed up.

Again, if you have the ability to take a virtual machine snapshot of your EPrints repository, it is worth using this to produce a clone, (if you do not already have a test or pre-production server version of your EPrints repository). This will allow you to check if you will have any snags either with the upgrade process or with the upgraded repository itself. It should also allow you to refine your upgrade schedule for your live EPrints repository to reduce the amount of scheduled downtime required.

Upgrade Schedules

It would be expected that you have scheduled downtime (at least a day) and ideally try out the upgrade schedule on a cloned instance of your EPrints repository. If you tested the upgrade schedule on a clone, actual downtime (or more accurately at-risk period) will probably be less than an hour.

From GitHub release

N.B. These instructions are a work in progress.

Before staring check that no critical files have been modified in the core codebase.

cd /opt/eprints3/
git status

The only file that should have been modified (assuming you are running a publications flavour of EPrints) is flavours/pub_lib/inc.

There may be various new directories/files under the lib and ingredients directories. These will most likely have been created as a result of installing Bazaar plug-ins or EPrints ingredients. You should make a note of these and check to see that they are present after the upgrade schedule has been completed.

If there are modified files (according to git status), you should similarly make a note of these and the changes made, so it can be checked if either these changes have already been applied in the upgraded versions of these files or that they need to be re-applied after upgrading.

If git status finds missing files, then similarly these should be checked to make sure they are not present after completing your upgrade and remove them if they still exist. They were most likely removed to deliberately disable a piece of unwanted functionality. Although generally it should be possible to disable this with configuration.

If only the flavour's inc file is modified and the only other things git status shows are additional untracked files, then it is safe to do an In-place Upgrade. Otherwise, you will want to do a Clean Upgrade.

In-place Upgrade

All steps should be done as the eprints users, although some commands may prompt for a root password.

1. Stop the webserver and EPrints indexer.

apachectl graceful-stop
/opt/eprints3/bin/indexer stop

2. Make a copy of the flavour's inc file:

cp /opt/eprints3/flavours/FLAVOUR_lib/inc /home/eprints/inc.backup

3. Get the latest code version. This can be done with the following commands (substituting X for the latest version):

git fetch
git checkout v3.4.X  

4. Check if the flavour's inc file has been reverted, if it is move back the backup to re-update it

mv /home/eprints/inc.backup /opt/eprints3/flavours/FLAVOUR_lib/inc

5. Test the new configuration and fix ans issues reported.

/opt/eprints3/bin/epadmin test

6. Update the database schema for any changes between versions

/opt/eprints3/bin/epadmin upgrade ARCHIVEID

7. Regenerate the EPrints Apache configuration, just in case there have been any improvements to this in the new version:

/opt/eprints3/bin/generate_apacheconf --system --replace

8. Start the webserver and EPrints Indexer

apachectl start
/opt/eprints3/bin/indexer start

Clean Upgrade

Step 1 needs to be done as the root. Steps 2 and onwards should be done as the eprints users, although some commands may prompt for a root password.

1. Create a new directory for latest version of eprints (substituting X for the latest version):

mkdir /opt/eprints3.4.X
chmod 2775 /opt/eprints3.4.X
chown eprints:eprints /opt/eprints3.4.X

2. Clone the latest released version of EPrints 3.4 (again substituting X for the latest version):

git clone https://github.com/eprints/eprints3.4 /opt/eprints3.4.X
cd /opt/eprints3.4.X
git checkout v3.4.X

3a. If you are not using the flavours/pub_lib copy the flavour library across (substituting FLAVOUR_LIB and X as appropriate):

cp -rf /opt/eprints3/flavours/FLAVOUR_LIB/ /opt/eprints3.4.X/flavours/

3b. If you are using flavours/pub_lib just copy over the inc file to the newly checked out pub_lib (again substituting X as appropriate):

cp /opt/eprints3/flavours/pub_lib/inc /opt/eprints3.4.X/flavours/pub_lib/

4. Review any modified tracked files by comparing these with the new version of the file in /opt/eprints3.4.X/. If the change is present in the new version then this change can be ignored, if it is not you may need to figure out how to incorporate this change into the new version of EPrints.

5. Copy across any additional untracked files from /opt/eprints3/ to /opt/eprints3.4.X/. These will likely be either ingredients or separate files installed as part of a Bazaar plug-in.

6. Stop the EPrints indexer and webserver:

/opt/eprints3/bin/indexer stop
apachectl graceful-stop

7. Move around the old and new eprints directories and then move the archive directory across to the newly created archives sub-directory (substituting X and ARCHIVEID as appropriate):

mv /opt/eprints3 /opt/eprints3.old
mv /opt/eprints3.4.X /opt/eprints3
mv /opt/eprints3.old/archives/ARCHIVEID /opt/eprints3/archives/

8. Test the new configuration and fix any issues reported.

/opt/eprints3/bin/epadmin test

9. Update the database schema for any changes between versions

/opt/eprints3/bin/epadmin upgrade ARCHIVEID

10. Regenerate the EPrints Apache configuration:

/opt/eprints3/bin/generate_apacheconf --system --replace

11. Start the webserver and EPrints Indexer

apachectl start
/opt/eprints3/bin/indexer start

From a source tarball

1. Download the source tarball for both your current version of EPrints (e.g. 3.4.1) and your intended new version (e.g. 3.4.3) from files.eprints.org and unpack into a separate directory on your EPrints repository server. E.g.

tar -xzvf eprints-3.4.1.tar.gz /home/eprints/
tar -xzvf eprints-3.4.1-flavours.tar.gz /home/eprints/
tar -xzvf eprints-3.4.3.tar.gz /home/eprints/
tar -xzvf eprints-3.4.3-flavours.tar.gz /home/eprints/

2. Compare the code to see if any critical changes have been made to the core codebase:

diff -x archives -x cfg -x tmp -rq /opt/eprints3 /home/eprints/eprints-3.4.1 | grep -v "/\(ingredients\|var\)"

Depending on your level of customization you should also pay attention to the results of

diff -rq /home/eprints/eprints3.4.3/flavours/pub_lib/defaultcfg/ /home/eprints/eprints3.4.1/flavours/pub_lib/defaultcfg/

or the equivalent command for the other flavours, or for a Zero EPrints repository

diff -rq /home/eprints/eprints3.4.3/lib/defaultcfg_zero /home/eprints/eprints3.4.1/lib/defaultcfg_zero

and even

diff -rq /home/eprints/eprints3.4.3/flavours/pub_lib/cfg.d/ /home/eprints/eprints3.4.1/flavours/pub_lib/cfg.d/
diff -rq /home/eprints/eprints3.4.3/lib/cfg.d/ /home/eprints/eprints3.4.1/lib/cfg.d/ 
diff -rq /home/eprints/eprints3.4.3/lib/syscfg.d/ /home/eprints/eprints3.4.1/lib/syscfg.d/ 

One of the first two directories was used for the initial configuration of your archive, and the latters control your repository by default; each of them could be overridden by your customized versions ... Decide whether any personal version is still necessary, or meanwhile obsolete!

3. If there are any changes check to see if the change is present in the new version of EPrints. E.g.:

diff /opt/eprints3/flavours/pub_lib/inc /home/eprints/eprints-3.4.1/flavours/pub_lib/inc
diff /opt/eprints3/flavours/pub_lib/inc /home/eprints/eprints-3.4.3/flavours/pub_lib/inc

If the files in /opt/eprints3/ are the same as that in the latest version (e.g. /home/eprints-3.4.3/) then no change is required. If some are not present in the latest version you need to consider whether they can/should be copied across (e.g. into /home/eprints/eprints-3.4.3/flavours/pub_lib/inc). It is likely new ingredients will be added to a flavours inc file between versions, so this file is particular important to check so your repository's local modifications to this file and modifications in the latest version of EPrints can be reconciled.

4. If your repository has any Bazaar plug-ins installed go to the Eprints Bazaar page on your repository via the Admin menu and download each one individually from the Developer Tools tab.

5. Stop the webserver and EPrints indexer on your repository server and check they have stopped running:

systemctl stop httpd
ps aux | grep httpd
/opt/eprints3/bin/indexer stop
ps aux | grep indexer

httpd is for RHEL-based Linux OS, for Debian-based Linux OS use apache2.

6. As root, move the new version of EPrints (with any ported changes, from step 3) to /opt/eprints3-new

mv /home/eprints-3.4.3 /opt/eprints3-new

7. As the eprints user move the contents of the current archives into the new directory:

mv /opt/eprints3/archives/* /opt/eprints3-new/archives/

If /opt/eprints3/archives or any of its archives are on a different disk partition, this may take a long time and the partition you are moving it to may not be big enough. It may be better to figure out an alternative way of updating the other EPrints directories without moving the archive between disk partitions and then switching round the eprints directories (step 9 below).

8. As the eprints user copy across ingredients (except the bazaar and jquery ingredient) to the new version:

rsync -a --exclude=bazaar --exclude=jquery /opt/eprints3/ingredients/ /opt/eprints3-new/ingredients/

9. As root, switch round the old and new eprints directories:

mv /opt/eprints3 /opt/eprints3-old
mv /opt/eprints3-new /opt/eprints3

10. As the eprints user, test EPrints is in a good state:

/opt/eprints3/bin/epadmin test

If you have various Bazaar plug-ins installed, it is likely that this will report issues. If so you will need to disable these files until you can re-install the Bazaar plug-ins. E.g.

cd /opt/eprints3/archives/<ARCHIVE_NAME>/cfg/cfg.d/
rm ../epm/*
mv z_orcid_support.pl z_orcid_support.pl.disabled
mv zz_hefce_oa_fields.pl zz_hefce_oa_fields.pl.disabled
mv zz_rioxx2.pl zz_rioxx2.pl.disabled
mv z_meprints.pl z_meprints.pl.disabled 
mv kultur.pl kultur.pl.disabled

11. Assuming EPrints is (now) in a good state, regenerate EPrints' Apache configuration:

/opt/eprints3/bin/generate_apacheconf --system --replace

12. As root, check the Apache configuration is sound and restart the webserver:

apachectl configtest
systemctl start httpd

For Debian-based LinuxOS use:

apache2ctl configtest
systemctl start apache2

13. Load up your repository in a web browser, login and go to the EPrints Bazaar page via the Admin menu. Install all the Bazaar plug-ins you downloaded in step 4. You can do this by scrolling to the bottom of the Available tab and using the Upload and Install section. Once you have re-installed all the Bazaar plug-ins, you can move back all the files you disabled in step 10. Then use epadmin to re-test and restart the webserver:

/opt/eprints3/bin/epadmin test
systemctl start httpd

14. Now using your web browser do a thorough test of both the functionality and the appearance/layout of your repository and make tweaks to your local archive configuration as necessary (e.g. modify the CSS).

13. Finally, as the eprints user, restart the indexer.

/opt/eprints3/bin/indexer start

It is advised that you keep the /opt/eprints3-old directory for a while (at least a month) in case you spot any unexpected problems. Worse case scenario you can revert back to this version by doing steps 7 and 9 in reverse. However, typically it is useful as a reference if you forgot to port a bespoke core code change of your old version of EPrints or a change you did port does not work as expected.