Explore Collections

SERVICES > OAI DATA PROVIDER

 

IssueLab’s data sharing platform makes it easy for anyone to access our metadata and graphic content and use it in their own websites and applications at no cost. Our data provider complies with the Open Archives Intiative Protocol for Metadata Harvesting (OAI-PMH). Online since 2005, it uses tried and true technology to share data across systems and is an integral part of our distribution network.

 

---

Our OAI data provider delivers metadata records in XML — one XML file per resource record. This chart provides info on all the data points we collect. At a minimum, the data points that appear in bold type will be included in every XML file.

Coverage	<dc:coverage>	The geographic areas the research considers (uncontrolled list).
Creator	<dc:creator>	The author(s) of the research.
Date	<dc:date>	The date the research was published. Format YYYY-MM-DD.
Description	<dc:description>	The summary provided for the research work.
Format	<dc:format>	When available, the file format of a saved resource. Formats include: pdf; doc; xls; rtf; txt; ppt, etc. Note: If a record includes “<dc:format>scribd;xxxxx;xxxxx</dc:format>”, a Scribd.com document is available.More info.
Identifier	<dc:identifier>	The listings unique.
Language	<dc:language>	The language in which the research was authored/published.
Publisher	<dc:publisher>	The nonprofit organization(s) that made the research in question available to/through the IssueLab archive.
Rights	<dc:rights>	The copyright and usage instructions for the research work.
Subject	<dc:subject>	The issue areas that the research work falls under (see “Issue Areas and Sets” below). Research works can fall into up to three issue areas.
Title	<dc:title>	Title of the research work.
Type	<dc:type>	The research can be categorized as these types. Controlled list: CaseStudy; Dataset; Ethnography; Evaluation; FactSheet; InteractiveResource; Literature/Research Review; MovingImage; Policy/Issue Brief; Presentation/Slideshow; Report/Whitepaper; StillImage; Survey; Testimony; Toolkit/Guide.

XML Example

<?xml version="1.0" encoding="UTF-8" ?> 
<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
  <responseDate>2008-04-09T18:53:51Z</responseDate> 
  <request identifier="2003_2004_statewide_survey_of_immigrants_and_refugees" verb="GetRecord" metadataPrefix="oai_dc">http://harvest.issuelab.org/provider/oai</request> 
  <GetRecord>
    <record>
      <header>
        <identifier>oai:harvest.issuelab.org:2003_2004_statewide_survey_of_immigrants_and_refugees</identifier> 
        <datestamp>2008-04-09T17:26:47Z</datestamp> 
        <setSpec>human_rights_and_civil_liberties</setSpec> 
        <setSpec>education_and_literacy</setSpec> 
        <setSpec>immigration</setSpec> 
      </header>
      <metadata>
        <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
          <dc:title>2003-2004 Statewide Survey of Immigrants and Refugees</dc:title> 
          <dc:subject>education_and_literacy</dc:subject> 
          <dc:subject>human_rights_and_civil_liberties</dc:subject> 
          <dc:subject>immigration</dc:subject> 
          <dc:creator>Illinois Coalition for Immigrant and Refugee Rights</dc:creator> 
          <dc:publisher>Illinois Coalition for Immigrant and Refugee Rights</dc:publisher> 
          <dc:date>2007-08-01</dc:date> 
          <dc:description>This report weaves together demographic and field research conducted by ICIRR in 2003 to assess the needs of immigrants and refugees throughout Illinois and to recommend the formation of an Illinois immigrant integration policy.</dc:description> 
          <dc:identifier>https://www.issuelab.org/research/2003_2004_statewide_survey_of_immigrants_and_refugees</dc:identifier> 
          <dc:type>Text</dc:type> 
          <dc:language>eng</dc:language> 
          <dc:format>pdf</dc:format> 
          <dc:format>scribd;37067679;key-174p87aga4148zpw329a</dc:format> 
       </oai_dc:dc>
      </metadata>
    </record>
  </GetRecord>
</OAI-PMH>

Harvesting: The Six Verbs

OAI-PMH offers six case-sensitive commands, or “verbs” for querying purposes:

1. Identify – identifies the archive you are harvesting.
2. ListSets – provides a list of the sub-sets or groupings of data offered (see “Issue Areas and Sets” below).
3. ListMetadataFormats – provides info about the formats we offer our metadata in; we offer metadata in oai_dc.
4. ListIdentfiers – provides a list of research listing identifiers available to harvest.
5. ListRecords – a list that includes an individual metadata record for every research listing available in the IssueLab archive.
6. GetRecord – you issue this HTTP request in conjunction with a record identifier to obtain the metadata for the specified record only.

Here are the most basic HTTP requests you can issue. Note: a few verbs require that you include the “metadataPrefix” parameter as well as the “verb”; the “metadataPrefix” shows in green below:

http://harvest.issuelab.org/provider/oai?verb=Identify [Try it!]

http://harvest.issuelab.org/provider/oai?verb=ListSets [Try it!]

http://harvest.issuelab.org/provider/oai?verb=ListMetadataFormats [Try it!]

http://harvest.issuelab.org/provider/oai?verb=ListIdentifiers&metadataPrefix=oai_dc [Try it!]

http://harvest.issuelab.org/provider/oai?verb=ListRecords&metadataPrefix=oai_dc [Try it!]

http://harvest.issuelab.org/provider/oai?verb=GetRecord&identifier=record identifier&metadataPrefix=oai_dc [Seeded – try it!]

Harvesting: Parameters

You can do quite a bit with these verbs when they’re mixed with other allowed parameters. For example, you can ask for all records saved to the IssueLab repository between 2010-01-06 and 2011-01-01 within our “Health and Medicine” issue area like this (you must use date format: YYYY-MM-DDTHH:MM:SSZ):

http://harvest.issuelab.org/provider/oai?verb=ListRecords&metadataPrefix=oai_dc&from=2010-01-06T16:25:14Z&until=2011-01-01T23:59:59Z&set=health_and_medicine

Want records in a particular issue area stored from a particular date until the present? Just leave off the “until” parameter. Example:

http://harvest.issuelab.org/provider/oai?verb=ListRecords&metadataPrefix=oai_dc&from=2010-01-06T16:25:14Z&set=health_and_medicine

Harvesting: The “date” Parameter

The “from and “until” date parameters available to you pertain to the date a record became available in our data provider service and not the date that a research listing was originally published.

HTTP Request Parameters

Here are the arguments that you can use in your HTTP request (don’t forget — nothing happens unless you issue one of the six verbs!):

• verb – (required)
• metadata – (as needed; see examples above)
• from – the date you’d like to start with; format: YYYY-MM-DD
• until – the date you’d like to end with; format: YYYY-MM-DD
• set – the issue area you’d like to retrieve records from (see below)

Issue Areas or “Sets”

Interested in retrieving data about a particular IssueLab issue area? We provide the following “sets”. To use the set parameter, you would use a URL like this one (we’ll request all records indexed under the “animals” issue area):

http://harvest.issuelab.org/provider/oai?verb=ListRecords&metadataPrefix=oai_dc&set=animals

Here’s the complete list of issue areas with corresponding set name. You will use the set name in your HTTP request:

Technology and Thanks

We are using free data provider software created and shared by the University of Michigan. Enter verb=Identify and it’s UMich’s umich_oai_toolkit that returns the results.  Complete information is here.

Questions? Comments?

Don’t hesitate to get in touch with us should you require assistance. We’re happy to help! Send a note to oai at issuelab.org.