Tag Archives: interoperability

The SWORD course slides now online

As part of the JISC-funded SWORD 3 project, I created ‘The SWORD Course’ and presented it during a two hour workshop at the recent Open Repositories 2010 conference in Madrid. The aim of the course was to empower repository managers and repository developers who knew what SWORD was, but who are not currently using it, to be able to go back to their institutions and start using it.
The course, entitled ‘Adding SWORD To Your Repository Armoury’ is made up of 5 modules:
  1. An Introduction to SWORD: Gives an overview of SWORD, the rationale behind its creation, and details of the first three funded SWORD projects
  2. SWORD Use Cases: Provides an introduction to use cases, and examines some of the use cases that SWORD can be used for
  3. How SWORD Works: A high level overview of the SWORD protocol, lightly touching on a few technical details in order to explain how it works
  4. SWORD Clients: The reasons for needing SWORD clients are shown, followed by a tour of some of the current SWORD clients
  5. Create Your Own SWORD Client: An overview of the EasyDeposit SWORD client creation toolkit, including the chance to try it out

The slides from each presentation have now been uploaded to Slideshare with a Creative Commons Attribution NonCommercial Sharealike licence. The workshop was video recorded too, and hopefully this will be posted online some soon too.

EasyDeposit – DOI integration with the CrossRef API

A few weeks ago I wrote about the EasyDeposit system we’ve created at The University of Auckland Library. In a nutshell, it allows you to easily create custom web-based SWORD deposit interfaces to enable the deposit of items into your repository. We’ve used it locally to create custom deposit interfaces for PhD theses, Masters theses, and Computer Science technical reports.

One of the key features of EasyDeposit is how custom interfaces can be built by selecting from a set of ‘steps’. Each step performs a task such as allowing the user to upload files, enter metadata, login, verify the details of the deposit etc. Steps can be easily added by creating two new files – one which contains the business logic (coded in PHP) and one which contains the view (HTML). We maintain some custom steps locally to manage the licensing of theses (to select embargo terms and Creative Commons options).

I recently became aware of the CrossRef API. This web service allows you to look up the metadata of an item that has a DOI. Requests are made using a URL in the form of:

http://www.crossref.org/openurl/?id=doi:*DOI*&noredirect=true&pid=*API_KEY*&format=unixref

The whole ethos behind EasyDeposit is to enable repository administrators to create custom SWORD deposit interfaces that make it easy for their users to deposit content into their repositories. Therefore it seemed a good idea to write a new EasyDeposit step that allows users to enter a DOI instead of manually entering metadata. A typical deposit interface could now be as simple as:

  • Do you have a DOI for your item? If so:
  • Login
  • Enter a DOI
  • Confirm the metadata is correct
  • Upload a file
  • Verify the deposit detail

I have now uploaded the ‘CrossRefDOILookup’ and ‘CrossRefDOIMetadata’ steps to the EasyDeposit site. You can use these steps (they must be used together) in your EasyDeposit interface. Here are some screenshots:

In order to use the API you must register with CrossRef. You can do this at: http://www.crossref.org/requestaccount/. Your API key will the email address that you used to sign up, and must be entered in the easydeposit.php configuration file:

// CrossRef API DOI lookup configuration
// Register for a key at http://www.crossref.org/requestaccount/
// Your API KEY is the email address that you used to register
$config['easydeposit_crossrefdoilookup_apikey'] = 'API_KEY';

As always, I’d be pleased to receive feedback about this feature.

EasyDeposit – SWORD deposit tool creator

The development of the SWORD (Simple Web-service Offering Repository Deposit) protocol has enabled repositories to start accepting deposits from remote systems and interfaces. If you’re unsure of the basics of SWORD, read one of the following:

However, to date there has not been a great deal of use of SWORD. One of the reasons is a lack of SWORD clients that can deposit items into repositories. Demonstration clients were created by the SWORD project, and a PHP SWORD library was created by the SWORD2 project, but no client that can easily be set up by web developers or repository administrators to be used by depositors has been created.

A bit of background:

Last year as part of my job at the University of Auckland Library, I had to create a SWORD deposit client to allow PhD candidates to submit an electronic copy of their thesis. We wanted to use SWORD to do this as it means the PhD students do not have to create a repository account, and learn how to submit in the repository. The SWORD client was written in PHP and made use of the SWORD PHP library. The client was made up of a very small number of pages: login, enter title of thesis, upload file, select embargo and licencing options, verify, submit.

I then had to create a second similar deposit interface to allow a department to archive a technical report series. This deposit interface was similar, but didn’t have the embargo option, asked for more metadata, and returned the URL of the deposited item in a format that could be inserted into their own web publishing system.

Developing and maintaining two similar but not identical systems seemed to be wasteful, therefore I decided to create a generic SWORD deposit interface toolkit that allowed new deposit systems to be easily created. EasyDeposit was born!

What is EasyDeposit?

EasyDeposit is a toolkit for easily creating SWORD deposit web interfaces using PHP. To start using EasyDeposit, follow the installation instructions.

How does EasyDeposit work?

EasyDeposit allows you to create customised SWORD deposit interfaces by configuring a set of ‘steps’. A typical flow of steps may be: login, select a repository, enter some metadata, upload a file, verify the information is correct, perform the deposit, send a confirmation email. Alternatively a deposit flow may just require a file to be uploaded and a title entered. A configuration file is used to list the steps you require.

EasyDeposit makes use of the CodeIgniter MVC PHP framework. This means each ‘step’ is made up of two files: a ‘controller’ which looks after the validation and processing of any data entered, and a ‘view’ which controls the web page that a user sees. This separation of concerns makes it easy for web programmers to edit the controllers, and web designers to tinker with the look and feel of the interface in the views.

What ‘steps’ come with EasyDeposit?

EasyDeposit comes with 14 different steps, including:

  • ldaplogin: Allows login to take place against an LDAP directory
  • nologin: Allows preset login inforamtino to be provided if you don’t wish users to have to login, then forwards the user on to the next step
  • depositcredentials: Sets credentials to be used for the deposit if you wish to use a generic set of credentials, then forwards the user on to the next step
  • selectrepository: Allows a user to select between multiple repositories
  • servidedocument: Displays a service document to the user to allow them to decide which collection to deposit into
  • title: Requires the user to enter a title for the item they are depositing
  • metadata: Requires the user to enter metadata for the item they are depositing
  • uploadfile: Allows the user to upload files to deposit
  • verify: Allow the user to verify their submission before the deposit
  • deposit: Performs the deposit, then forwards the user on to the next step
  • email: Sends an email confirmation of the deposit, then forwards the user on to the next step
  • thankyou: Displays a confirmation of the deposit to the user

Extra steps can be easily added just by adding a controller and a view for each new step.

Is EasyDeposit open source?

Yes! It is published with a modified BSD licence.

How do I use EasyDeposit?

Follow the installation instructions! If you have any questions, please leave comments on this blog entry, to get in touch with me directly.

If SWORD is the answer, what is the question?

I’ve just had a new collaborative paper published: ‘If SWORD is the answer, what is the question?’ (DOI: 10.1108/00330330910998057). It covers the most recent iteration of the SWORD repository deposit standard, looks briefly at some issues around the present lack of adoption of SWORD, and most usefully presents seven use cases of SWORD written by their developers:

Lewis, S., Hayes, L., Newton-Wade, V., Corfield, A., Davis, R., Donohue, T., Wilson, S., If SWORD is the answer, what is the question?: Use of the Simple Web-service Offering Repository Deposit protocol, Program: electronic library and information systems, 2009,  Vol 43, Issue 4, pp: 407 – 418, 10.1108/00330330910998057, Emerald Group Publishing Limited

Of course a copy is available open access in our repository: http://hdl.handle.net/2292/5315

Abstract:

Purpose – The purpose of this paper is to describe the repository deposit protocol, Simple Web-service Offering Repository Deposit (SWORD), its development iteration, and some of its potential use cases. In addition, seven case studies of institutional use of SWORD are provided.

Design/methodology/approach – The paper describes the recent development cycle of the SWORD standard, with issues being identified and overcome with a subsequent version. Use cases and case studies of the new standard in action are included to demonstrate the wide range of practical uses of the SWORD standard.

Findings – SWORD has many potential use cases and has quickly become the de facto standard for depositing items into repositories. By making use of a widely-supported interoperable standard, tools can be created that start to overcome some of the problems of gathering content for deposit into institutional repositories. They can do this by changing the submission process from a “one-size-fits-all” solution, as provided by the repository’s own user interface, to customised solutions for different users.

Originality/value – Many of the case studies described in this paper are new and unpublished, and describe methods of creating novel interoperable tools for depositing items into repositories. The description of SWORD version 1.3 and its development give an insight into the processes involved with the development of a new standard.

The seven case studies include a thesis submission system, a SWORD plugin for moodle, an automated laboratory data repository deposit tool, a desktop deposit tool, the BibApp repository integration module, a custom deposit tool for a technical report series, and the Facebook SWORD deposit tool.

SWORD PHP Library version 0.9 released + moved to github

I have just released version 0.9 of the SWORD PHP library. It has a few fixes and changes, the most noteworthy being:

  • Changed swordappservicedocument to build the servcedocument from the xml response rather than having the swordappclient do the work. This allows the service document to be parsed at a later time.
  • Changed the swordappclient deposit method to stream the file being deposited straight from disk rather than via memory to avoid using excessive memory and potentially exceeding the PHP memory limit. I’ve successfully tested this against DSpace with deposits of 600MB CD images.
  • Added some validation to the SWAP/METS packager to allow it to cope with filenames and metadata containing ampersands

These changes have come about following bug reports from other users of the code, and from some enhancements we needed for an exciting configurable SWORD web client that we’ll be unleashing on the world in the next week or two from the University of Auckland Library.

An important change is that the new code is now stored in a git repository at github (although it can still be downloaded from http://php.swordapp.org/):

I’m new to git, so if anyone who knows git better than I do notices that I’m not using in an optimal way, I’d love to hear about it.

Email your repository

What modern information handling system do we probably interact with most each day? For the majority of us, it is probably our email. We send and recive dozens of emails each day. So how about enabling repository deposit via email?

It has certainly been talked about from time to time, and a plugin to the Thunderbird email client has even been written that allows you to deposit attachments into repositories using SWORD (no mention is made of metadata). This plugin should work with any SWORD enabled repository, but only works with one email client. Wouldn’t it be great if there was a more general solution that worked with all repositories, and all email clients?

Now there is! The latest version of the SWORD PHP library (version 0.8) contains an example script showing SWORD and the PHP library in use. To make it work, just fill out a configuration file with your email address, password and IMAP mailbox details, and your repository login, password and deposit URL. After the configuration file has been filled in, all you need to do is run the script ‘imap-mail.php’ on the command line. The script will connect to your mailbox and look at each unread message. It will package each one up and deposit it into the repository.

How does it work?

It uses the standard PHP IMAP library to connect to your inbox. For each unread message it finds, it extracts the name of the sender of the email, the email subject, and the body of the message. It uses these for metadata:

  • From name -> Author
  • Subject -> Title
  • Message body -> Abstract

Along with the metadata, the script adds each email attachment to the deposited item. An example of an email deposited this way can be seen at: http://dspace.swordapp.org/jspui/handle/123456789/318

If you want to try it out, but don’t want to set it all up, for a limited time I’ll leave it running against the deposit@swordapp.org mailbox. Send an email to deposit@swordapp.org and when I periodically run the script, your email will be deposited into the test DSpace/SWORD repository at http://dspace.swordapp.org/. Please bear in mind that the repository is open access to the world, so anyone can see what your email and optional attachments contain! (Please consider working hours / timezones etc when working out when I am likely to next run the script! You will receive a confirmation email when your deposit has taken place.)

A few further thoughts:

  • What is the use of this script? I can think of a few. If you want to allow faculty members to deposit full texts easily, and you are happy to update the record with better metadata, this script could help you out. Alternatively if you have a system that for example generates weekly reports, and you want to archive these, you may find it easier to get the system to email the reports to the SWORD script than to develop a SWORD deposit interface to the system.
  • n for the price of 1: As far as I know, none of the major SWORD enabled repository platforms has an email deposit facility. This script shows one aspect of the real power of interoperability and SWORD’s part in creating an interoperable environment. Rather than developing an email deposit facility for just one repository, I have developed one for any SWORD compliant repository.
  • Extensions to the script: The following idea came up in a conversation with Kim Shepherd, the LCoNZ DSpace programmer who suggested that the ‘local part’ of an email address could be used to set further options. Email addresses can have extra tags applied following a ‘+’ or ‘-’ (for example username-XYZ@example.com). So if for example you wanted your users to be able to choose which collection in the repository they wanted to deposit an item into, they could change the deposit email to something like deposit+datasets@example.com or deposit-chemistrylearningobjects@example.com. These emails would all end up in the same mailbox, but the script could process them differently. Or of course the parameters could be used to set other options.

If I get time, my next extension to the PHP SWORD library will be a basic web client (similar to http://client.swordapp.org/ except written in PHP, and will create packages from files for you). If you have any other suggestions, please leave a comment!