Generic Reporting Framework

Introduction

The Generic Reporting Framework Bazaar plugin facilitates the creations of reports on records held within an EPrints repository that can be configured to return records based on certain criteria and to validate these records based on given test criteria.

For Users

The Generic Reporting Framework adds a new link to the key tools bar (alongside links such as 'Manage Deposits' and 'Admin') called 'Reports', that is only shown to administrator users.

Generating a Report

This link displays the Reports splash page from which any available reports may be selected.

The Reports splash page shows both 'Preset Reports' and 'Custom Reports' depending on the reports available. 'Preset Reports' display records that match predefined criteria and are typically added to the repository through the installation of other plugins such as REF CC or ORCID Support. 'Custom Reports' displays a dropdown of available reports, and a search form for the selected report. This can used to generate a report comprised of records which match the given search criteria. Generic Reporting Framework v3.0+ offers two custom reports by default, one for EPrints and another for Users.

Viewing a Report

Once a report has been selected the page will reload to display the results of that report. This process may take a short while if there are lots of records to process.

Next to each record is a coloured bar. For reports which indicate some form of validation (e.g. the REF CC compliance reports) this bar will indicate the status of the record, e.g. green for compliant, red for non-compliant. For reports which do not make use of compliance, this bar will be set to a default colour (which can be customised - see For Developers section below.

Each record may also be accompanied by some bullet points offering further information about the record (e.g. why it might not meet compliance criteria).

Reports may also be exported using the 'Export' button at the top of the page. Some reports support custom exports and where available a number of fields will be listed alongside checkboxes. Each checked field will be included in the exported report.

Reports may also be grouped and sorted by different fields using the 'Group by' and 'Sort by' links. Grouping recors has the affect of breaking down the page in to a number of different sections with a header to represent each group. Sorting records results in the records being reordered based on the selected field. If the records have already been grouped the records will be sorted within each group. The fields that are available for a given report's 'Group by' and 'Sort by' links may be set in the report's configuration - see For Developers for more details.

For Developers

GitHub: https://github.com/eprintsug/reports

Returning Report Bullet Points (v1.1+ only)

By default, any problems that are identified by the report's 'validate_dataobj' function are displayed as bullet points for that item in the report. However, if you wish to display bullet points for an item which does not report any problems when validating, a new 'bullet_points' function may be defined, e.g.

sub bullet_points
{
        my( $self, $eprint ) = @_;

        my $repo = $self->{repository};

        my @bullets;

        foreach my $creator( @{ $eprint->value( "creators" ) } )
        {
                if( EPrints::Utils::is_set( $creator->{orcid} ) )
                {
                        push @bullets, EPrints::XML::to_string( $repo->html_phrase( "creator_with_orcid", creator => $repo->xml->create_text_node(EPrints::Utils::make_name_string( $creator->{name} ) ), orcid => $repo->xml->create_text_node( $creator->{orcid} ) ) );
                }
                else
                {
                        push @bullets, EPrints::XML::to_string( $repo->html_phrase( "creator_no_orcid", creator => $repo->xml->create_text_node(EPrints::Utils::make_name_string( $creator->{name} ) ) ) );
                }
        }

        return @bullets;
}

The 'bullet_points' function should then be called from ajax_eprint when generating the report's JSON data, e.g.

sub ajax_eprint
{
        my( $self ) = @_;

        my $repo = $self->repository;

        my $json = { data => [] };

        $repo->dataset( "eprint" )
        ->list( [$repo->param( "eprint" )] )
        ->map(sub {
                (undef, undef, my $eprint) = @_;

                my $frag = $eprint->render_citation_link;
                push @{$json->{data}}, {
                        datasetid => $eprint->dataset->base_id,
                        dataobjid => $eprint->id,
                        summary => EPrints::XML::to_string( $frag ),
                        problems => [ $self->validate_dataobj( $eprint ) ],
                        bullets => [ $self->bullet_points( $eprint ) ],
                };
        });
        print $self->to_json( $json );
}

Returning Report Colours (v1.1+ only)

Version 1.0 of the Generic Reporting Framework plugin, can only display two colours for records displayed in the report. When validating a record using the a report plugin's 'validate_dataobj' function, if an array of problems is returned the item is displayed with a red bar, and if none are returned the item is displayed with a green bar.

To return other colours, define a 'get_state' function which should return a string representing a hex encoded colour. E.g.

sub get_state
{
        my( $self, $eprint ) = @_;

        my $repo = $self->{repository};

        if( $eprint->hasProblem() )
        {
            return "#E19141"; #orange
        }
        else
        {
            return undef;
        }
}

The 'get_state' function should then be called from ajax_eprint when generating the report's JSON data, e.g.

sub ajax_eprint
{
        my( $self ) = @_;

        my $repo = $self->repository;

        my $json = { data => [] };

        $repo->dataset( "eprint" )
        ->list( [$repo->param( "eprint" )] )
        ->map(sub {
                (undef, undef, my $eprint) = @_;
                
                my $frag = $eprint->render_citation_link_staff;
                push @{$json->{data}}, {
                        datasetid => $eprint->dataset->base_id,
                        dataobjid => $eprint->id,
                        summary => EPrints::XML::to_string( $frag ),
                        problems => [ $self->validate_dataobj( $eprint ) ],
                        state => $self->get_state( $eprint ),
                };
        });

        print $self->to_json( $json );
}

Custom Reports (v1.1+ only)

For a report to display in the 'Custom Reports' tab of the splash page, its custom property must be set. E.g.

#from within the local archive level
$c->{plugins}{"Screen::Report::REF_CC"}{params}{custom} = 1;

Custom Export Fields (v2.0+ only)

To specify which fields should appear in the custom export box for a specific report plugin, include an 'export_conf' parameter in the plugin's configuration that defines an ID for locating configuration relating to exports. E.g.

#in the report plugin's 'sub new' constructor
$self->{export_conf} = 'hefce_report';

The export fields themselevs should then be defined at the local archive configuration level, using this 'export_conf' ID.

Export fields are described using a hash of 1 or more arrays, allowing fields to be grouped under a particular heading. Note, that as per a search config, fields of other datasets may be defined also, providing the dataset has a connection with the parent dataset (e.g. document fields can be defined within an eprint context as documents.fieldname). E.g.

 
$c->{hefce_report}->{exportfields} = {
        ref_core => [ qw(
                eprintid
                documents.content
                type
                title
                abstract
                creators
                publisher
                divisions
                dates
                id_number
                isbn
                issn
                official_url                                            
        )],
        ref_rioxx => [ qw(
                rioxx2_free_to_read
                rioxx2_license_ref
                rioxx2_coverage
                rioxx2_source
                rioxx2_subject
                rioxx2_dateAccepted
                rioxx2_publication_date
                rioxx2_apc
                rioxx2_project
                rioxx2_version
                rioxx2_version_of_record
        )],
        ref_exceptions => [ qw(
                hoa_compliant
                hoa_problems
                hoa_ref_pan
                hoa_ex_dep
                hoa_ex_dep_txt
                hoa_ex_acc
                hoa_ex_acc_txt
                hoa_ex_tec
                hoa_ex_tec_txt
                hoa_ex_oth
                hoa_ex_oth_txt
        )],
};

For each of the fields defined above a checkbox will be displayed in the Report's 'Export Options' box. Some of these may be selected by default, by defining an "exportfield_defaults" value, also at the local archive configuration level, E.g.

$c->{hefce_report}->{exportfield_defaults} = [ qw( eprintid documents.content type title creators dvisisions dates hoa_compliant hoa_problems ) ];

Finally, functions may be defined that allow fields to be displayed in a bespoke manner. These functions are also defined at the local archive level via use of the 'export_conf' ID, with each field mapping on to a function. The functions take the DataObj and Report plugin as parameters. E.g.

$c->{hefce_report}->{custom_export} = {
        hoa_compliant => sub {
                my( $dataobj, $plugin ) = @_;

                my $compliance = "Compliant";

                my @problems = $plugin->validate_dataobj( $dataobj );
                if( scalar( @problems ) > 0 )
                {
                        $compliance = "Not Compliant";
                }
                $compliance = "Compliant pending open access" if defined $plugin->get_state( $dataobj );
                return $compliance;
        },
        creators => sub {
                my( $dataobj ) = @_;

                my @creator_names;
                my $creators = $dataobj->get_value( "creators" );
                foreach my $c ( @{$creators} )
                {
                        push @creator_names, EPrints::Utils::make_name_string( $c->{name} );
                }
                return join( ";", @creator_names );
        },

Custom Report Search Configuration (v2.0+ only)

Custom reports allow users to generate reports via a search config. This is stored in a report plugin parameter call "sconf" and is by default defined as "report":

$self->{sconf} = "report";

With default report config is defined as follows, matching the format used by search configs such as the Advanced Search:

$c->{search}->{report} =
{
        search_fields => [
                { meta_fields => [ "title" ] },
                { meta_fields => [ "creators_name" ] },
                { meta_fields => [ "date" ] },
                { meta_fields => [ "subjects" ] },
                { meta_fields => [ "type" ] },
                { meta_fields => [ "divisions" ] },
                { meta_fields => [ "publication" ] },
        ],
        order_methods => {
                "byyear"         => "-date/creators_name/title",
                "byyearoldest"   => "date/creators_name/title",
                "byname"         => "creators_name/-date/title",
                "bytitle"        => "title/creators_name/-date"
        },
        default_order => "byyear",
        show_zero_results => 1,
};

Bespoke search configurations can be defined, by overriding the 'sconf' parameter. E.g.

#in the report plugin's 'sub new' constructor
$self->{sconf} = 'hefce_report';

And then a corresponding search configuration may be defined at the local archive level. E.g.

$c->{search}->{hefce_report} =
{
        search_fields => [
                { meta_fields => [ "title" ] },
                { meta_fields => [ "creators_name" ] },
                { meta_fields => [ "date" ] },
                { meta_fields => [ "hoa_date_acc" ] },
                { meta_fields => [ "subjects" ] },
                { meta_fields => [ "type" ] },
                { meta_fields => [ "ispublished" ] },
                { meta_fields => [ "divisions" ] },
                { meta_fields => [ "publication" ] },
        ],
        order_methods => {
                "byyear"         => "-date/creators_name/title",
                "byyearoldest"   => "date/creators_name/title",
                "byname"         => "creators_name/-date/title",
                "bytitle"        => "title/creators_name/-date"
        },
        default_order => "byyear",
        show_zero_results => 1,
};

Custom Report Header (v2.1+ only)

The compliance information in the report header (the compliance bar visualisation and the percentage) can be removed by setting the 'show_compliance' value for a report. E.g.

#in the report plugin's 'sub new' constructor
$self->{show_compliance} = 0;

The word 'outputs' in the report header to show how many items are included in the report, may be customised by setting the 'labels' value. E.g.

#in the report plugin's 'sub new' constructor
$self->{labels} = {
                outputs => "users"
        };

Only the 'outputs' value can be changed at present, with the value stored in a 'labels' hash for any future changes that may be applied.

Change Log

1.0 (11 March 2015)

• Initial version of the Generic Reporting Framework plugin.

1.1 (14 June 2017)

• Added support for custom exports, where users can choose which fields to include in CSV exports from a predefined list in the report's plugin configuration.
• Added support for flexible reporting. Report plugins may be specified as permitting custom reports. Existing defined reports are displayed on the splash page in the 'Preset Reports' tab, a search form for custom reports is displayed in the 'Custom Reports' tab.
• Added support for allowing reports to return different colours, beyond the green=compliant, red=non-compliant states.
• Added support for bullet points to be added to any item in the report, not just non-compliant items.

1.1.1 (15 June 2017)

• Changed export interface to present field options in one div element, rather than as tabs.
• Removed custom report tab from the report splash page if none of the report plugins support custom exports.

1.1.2 (16 June 2017)

• Updates to export plugins. When custom export fields are available, the generic CSV exporter is used. When custom export fields are not available, any bespoke export plugins that come with the report are available instead. (In the event that custom export fields are not available and there are no bespoke export plugins, the export option is now shown, as per version 1.0.)

2.0 (8 September 2017)

• Custom export fields maybe defined at local archive level.
• Custom search configurations for reports may be defined at local archive level.
• Fixes to rendering of search forms and report item states (coloured bars) for IE.
• Allow default export fields to be defined.
• Allow export field custom rendering via functions defined at local archive level.

2.1 (3 August 2018)

• Allow reports to hide the compliance percentage where not relevant, and customise the word "outputs" so that it can represent different data objects, e.g. "users".
• Update Bazaar icon

3.0 (10 December 2018)

• Provides reports on EPrint and User datasets, allowing for searches to be conducted over these datasets with the user able to choose whihc fields should be included in the export.
• Provides an HTML report export, which can be used to generate PDF reports using a browser's HTML to PDF functionality.
• Grouping of items within a report based on fields specified in report config
• Sorting of items within a report based on fields specified in report config