Tweepository - Revision history

Af05v@ecs.soton.ac.uk: /* Exporting to a Web Observatory */

2015-12-18T12:22:48Z

‎Exporting to a Web Observatory

2015-12-18T12:19:51Z

‎Configuring a Web Observatory

2015-12-18T12:17:44Z

‎Configuring a Web Observatory

2015-12-18T12:17:05Z

‎Configuring a TweetStream

2015-12-18T12:15:02Z

‎Exporting to a Web Observatory

2015-12-18T11:54:31Z

2014-06-12T09:35:54Z

‎Add Twitter API Keys

2014-06-12T09:22:36Z

2013-01-02T14:24:28Z

‎Installation Prerequisites

2012-12-08T22:38:14Z

‎Setting Up Cron Jobs

@@ Line 210: / Line 210: @@
 ** More that one tweetstream can be inserted into a collection
 ** The URL of the tweepository will be prepeneded onto the collection ID.  If 'foo' is the collection name, then on the Web Observatory, the collection will be (e.g.) 'http://www.mytweepository.com/foo'
 == Troubleshooting ==

← Older revision		Revision as of 12:19, 18 December 2015
Line 196:		Line 196:

	perl -I/<eprints_root>/perl_lib /<eprints_root>/archives/<repositoryid>/bin/tweepository/web_observatory_test.pl <repositoryid> <webobservatoryid>		perl -I/<eprints_root>/perl_lib /<eprints_root>/archives/<repositoryid>/bin/tweepository/web_observatory_test.pl <repositoryid> <webobservatoryid>
		+
		+	==== authorised_users ====
		+
		+	A tweetstream is allowed to be pushed to a Web Observatory if its the person that created the tweetstream is in the list of authorised users.

	=== Configuring a TweetStream ===		=== Configuring a TweetStream ===

@@ Line 173: / Line 173: @@
 An example web observatory configuration is visible in the z_tweepository_cfg.pl file.  This should be copied to a new .pl file and an entry created for each Web Observatory database that is configured:
-<source lang="perl'>
+<source lang="perl">
 $c->{web_observatories} =
+{
@@ Line 196: / Line 196: @@
   perl -I/<eprints_root>/perl_lib /<eprints_root>/archives/<repositoryid>/bin/tweepository/web_observatory_test.pl <repositoryid> <webobservatoryid>
 === Configuring a TweetStream ===

@@ Line 205: / Line 205: @@
 * The ID of the web observatory -- In the above example, it would be 'webobservatory1'
 * Web Observatory Collection -- the MongoDB collection that this will be inserted into.  Note that:
 == Troubleshooting ==

← Older revision		Revision as of 12:15, 18 December 2015
Line 166:		Line 166:

	== Exporting to a Web Observatory ==		== Exporting to a Web Observatory ==
		+
		+	Tweet data can be exported nightly to a Web Observatory. This is accomplished by inserting tweet records into an external MongoDB database that has been set up by the administrators of the Web Observatory for this purpose.
		+
		+	=== Configuring a Web Observatory ===
		+
		+	An example web observatory configuration is visible in the z_tweepository_cfg.pl file. This should be copied to a new .pl file and an entry created for each Web Observatory database that is configured:
		+
		+	<source lang="perl'>
		+	$c->{web_observatories} =
		+	{
		+	'websobservatory1' =>
		+	{
		+	type => 'mongodb',
		+	server => 'foo.bar.uk',
		+	port => 1234,
		+	db_name => 'foo',
		+	username => 'username',
		+	password => 'password',
		+	authorised_users =>
		+	[
		+	'username1',
		+	'username2'
		+	]
		+	}
		+	};
		+	</source>
		+
		+	To test, use the web_observatory_test.pl script:
		+
		+	perl -I/<eprints_root>/perl_lib /<eprints_root>/archives/<repositoryid>/bin/tweepository/web_observatory_test.pl <repositoryid> <webobservatoryid>
		+
		+
		+	=== Configuring a TweetStream ===
		+
		+	When editing or creating a tweetstream, fill in the Web Observatory Export part of the workflow:
		+
		+	* Export To Web Observatory -- set to 'Yes'
		+	* The ID of the web observatory -- In the above example, it would be 'webobservatory1'
		+	* Web Observatory Collection -- the MongoDB collection that this will be inserted into. Note that:

	== Troubleshooting ==		== Troubleshooting ==

← Older revision		Revision as of 11:54, 18 December 2015
Line 164:		Line 164:

	These three roles have been assigned to repository administrators, editors and users respectively. This can be changed by modifying this part of the config.		These three roles have been assigned to repository administrators, editors and users respectively. This can be changed by modifying this part of the config.
		+
		+	== Exporting to a Web Observatory ==

	== Troubleshooting ==		== Troubleshooting ==

@@ Line 34: / Line 34: @@
 <pre>
+$c->{twitter_oauth}->{consumer_key} = 'xxxxxxxxxxxx';
+$c->{twitter_oauth}->{consumer_secret} = 'xxxxxxxxxxxxxxxxxxxxxxxx';
 </pre>

← Older revision		Revision as of 09:22, 12 June 2014
Line 28:		Line 28:

	Install through the EPrints Bazaar		Install through the EPrints Bazaar
		+
		+	== Add Twitter API Keys ==
		+
		+	Create a file in your repository's cfg.d directory for your twitter API keys (e.g. called z_tweepository_oath.pl). The content should look like this:
		+
		+	<pre>
		+
		+
		+	</pre>

	== Setting Up Cron Jobs ==		== Setting Up Cron Jobs ==

@@ Line 8: / Line 8: @@
 <pre>
 Data::Dumper
 Date::Calc
 Date::Parse
 Encode
 HTML::Entities
 JSON
 LWP::UserAgent
 Number::Bytes::Human
 Storable
 URI
 URI::Find

@@ Line 27: / Line 27: @@
 == Setting Up Cron Jobs ==
-There are three processes that need to be regularly run.  Due to the heavyweight nature of these tasks, they should be put into the crontab rather than being handled by the EPrints Indexer.  However, they have been created as event plugins for future inclusion in the indexer.   Scripts have been created in the plugin directory.
+There are three processes that need to be regularly run.  Due to the heavyweight nature of these tasks, they should be put into the crontab rather than being handled by the EPrints Indexer.  However, they have been created as event plugins for future inclusion in the indexer.  Scripts have been created to wrap the plugins
 Running these scripts will result in log files and a cache being created in the repository's 'var' directory.  These should be checked if there are any issues with harvesting.