Difference between revisions of "Data Access"
Ryan Scherle (talk | contribs) (→= Searching the old Dryad (historical content only)) |
Ryan Scherle (talk | contribs) (→Web Browser User Interface) |
||
Line 1: | Line 1: | ||
== Web Browser User Interface == | == Web Browser User Interface == | ||
− | Primary access to Dryad is through its web interface, where users most commonly search on authors, titles, subjects and other metadata elements. Data files archived by Dryad may be downloaded one-by-one from their Dryad data package Web pages. | + | Primary access to Dryad is through its web interface at http://datadryad.org, where users most commonly search on authors, titles, subjects and other metadata elements. Data files archived by Dryad may be downloaded one-by-one from their Dryad data package Web pages. |
=== Searching the old Dryad (historical content only) === | === Searching the old Dryad (historical content only) === |
Latest revision as of 06:20, 10 September 2019
Contents
Web Browser User Interface
Primary access to Dryad is through its web interface at http://datadryad.org, where users most commonly search on authors, titles, subjects and other metadata elements. Data files archived by Dryad may be downloaded one-by-one from their Dryad data package Web pages.
Searching the old Dryad (historical content only)
DSpace, the platform on which the initial version of Dryad was built, supports several "hidden" ways to hack the system's URLs to get useful metadata from the Web interface.
Finding a data package page using the article DOI or PMID:
- http://v1.datadryad.org/discover?query="doi:10.1111/j.1558-5646.2007.00022.x"
- http://v1.datadryad.org/discover?query="PMID:17348941"
Viewing full metadata: add "?show=full" to the end of the URL
Viewing the raw DSpace representation of a page add "DRI" to the URL
Another way to view the raw DSpace markup is to add "?XML" to the end of the URL. This is less useful than the above method, though, because the page's content won't contain the externalized i18n strings.
Viewing metadata in machine-readable (METS) format. Can be performed using a DOI or a (legacy) handle:
Programmatic Data Access
In addition to the web interface, Dryad can be accessed programmatically through several APIs.
DataCite
Metadata for all Dryad objects is published through the DataCite system, and available in the DataCite search. This metadata is also available via the Scholix system.
OAI-PMH
The OAI-PMH API is no longer being updated, but the historical contents are still available from http://v1.datadryad.org/oai
OAI-PMH is a harvesting protocol that may be used to access Dryad's metadata. The specification is available, as are online tutorials, but we include a couple of examples of its use here for illustrative purposes.
Identify
- Used to learn about the service
ListSets
- Used to learn what sets of metadata are supported. Dryad offers a data package set and a data file set.
ListMetadataFormats
- Used to learn what metadata formats can be returned by the service. Dryad currently offers METS/MODS, OAI-DC (Dublin Core), OAI-ORE/Atom, and RDF/DC. The amount of information mapped into each format varies. For now, we recommend using the OAI-DC metadata format.
ListIdentifiers
- Used to list Dryad's OAI identifiers. It requires
from
andmetadataPrefix
parameters to know what range of identifiers to return and what format the metadata should be in (from the options returned by the ListMetadataFormats verb). We may modify this to return DOIs in the future.- NOTE: It is highly recommended that you use this call in conjunction with the "set" parameter, so you retrieve the records of interest. Otherwise, you may retrieve records that Dryad has harvested from other providers
ListRecords
- Used to list Dryad records. It requires
from
andmetadataPrefix
parameters so it knows the range of records to return. The records will be returned in the format associated with themetadataPrefix
requested. Available formats can be discovered by using the ListMetadataFormats verb.- NOTE: It is highly recommended that you use this call in conjunction with the "set" parameter, so you retrieve the records of interest. Otherwise, you may retrieve records that Dryad has harvested from other providers
GetRecord
- Used to return a single record. It requires the OAI identifier of the record (the
identifier
parameter) and the format in which the record should be returned (themetadataPrefix
parameter).
Using resumptionTokens with OAI-PMH
OAI-PMH requests may result in partial results lists being returned. In these cases, the results list will contain a resumptionToken that can be used to retrieve the next page of results.
For example, for a call like:
You will receive the first 100 records, ending with a resumptionToken of 2010-01-01T00:00:00Z/9999-12-31T23:59:59Z/hdl_10255_3/oai_dc/100
You can then retrieve the next 100 records with:
Note that when using a resumptionToken, OAI expects you to only repeat the verb, not any of the other parameters that were part of the original request.
DataONE API
The DataONE API currently not being updated, but we expect to update it again in the future. The historical contents are still available from http://v1.datadryad.org/mn
As part of Dryad's participation in the DataONE project, Dryad makes content available through a specialized API.
- List of objects
- Sample metadata call
- Sample file download -- URLs of this format can be used to download Dryad data directly from an external site, without forcing users to go through the Dryad interface.
- Technical documentation for the DataONE API
Programmatic access to data files using the DataONE API
- Obtain the DataONE ID of a Dryad object using the DataONE listObjects call: http://v1.datadryad.org/mn/object (e.g., dryad.1850/1)
- Retrieve the file: http://v1.datadryad.org/mn/object/doi:10.5061/dryad.1850/1/bitstream
- Retrieve system metadata about a file, including size and MIME type: http://v1.datadryad.org/mn/meta/doi:10.5061/dryad.1850/1/bitstream
- Retrieve descriptive metadata about a file: http://v1.datadryad.org/mn/object/doi:10.5061/dryad.1850/1
If you desire the full filename before downloading, obtain the METS document as described above. The filename is in the <mets:FLocat/> element in the xlink:href attribute.
Accessing Data Packages via Journal ISSN
Journals and their ISSNs can be accessed through a GET command:
http://datadryad.org/api/v1/journals
The corresponding ISSN can be used to get a list of packages in Dryad for that journal using the following GET command:
http://datadryad.org/api/v1/journals/{issn}/packages
If multiple pages of results are returned, the next and previous page links can be accessed from the link headers with `rel=next` and `rel=prev`.
There are additional query parameters that can be used to modify the results returned.
- `count` specifies the number of results per page.
- `date_from` and `date_to` can filter results to packages released in a date range.
- `cursor` can be used to specify the key used to start the results page.
Links to Data Packages/Files
Dryad uses DOIs (Digital Object Identifiers) to identify Dryad data packages and files. A few simple examples follow.
Data packages
- http://doi.org/10.5061/dryad.1664
- http://doi.org/10.5061/dryad.642
- http://doi.org/10.5061/dryad.1307
Data files
RSS Feeds
The RSS feed is no longer being updated, but the historical contents are still available from http://v1.datadryad.org/feed
There are a couple of feed options. Feeds are used by some browsers and all feed and news readers. They may also be used for programmatic access.
Everything -- data packages, data files, and metadata harvested from partner repositoriesData packages only
Data files only
Twitter Feed
Primary tweets from Dryad (typically not data)
SOLR search access
The SOLR interface is no longer being updated, but historical Dryad content can be searched using a SOLR interface.
- Basic query: http://v1.datadryad.org/solr/search/select/?q=Galliard
- Field-specific query: http://v1.datadryad.org/solr/search/select/?q=dwc.ScientificName:drosophila
- Search all text for a string, but limits results to two specified fields: http://v1.datadryad.org/solr/search/select/?q=Galliard&fl=dc.title,dc.contributor.author
- Dryad data based on an article DOI: http://v1.datadryad.org/solr/search/select/?q=dc.relation.isreferencedby:10.1038/nature04863+DSpaceStatus:Archived&fl=dc.identifier,dc.title_ac
- All terms in the dc.subject facet, along with their frequencies:
- Article DOIs associated with all data published in Dryad over the past 90 days:
- Data DOIs published in Dryad during January 2011, with results returned in JSON format:
For more about using SOLR, see the Apache SOLR documentation.
Widget API
The Widget API will become part of the "New" Dryad API (see below), but components are coming online. The Widget API provides simple images or dynamic iframes that link to content in Dryad and can be embedded into third-party sites.
Other access mechanisms
If you know of other community-developed services that can search or retrieve content that are not listed here, please alert us at help@datadryad.org
- ROpenSci rDryad package for search and retrieval of Dryad data and metadata within R.
Suggest Alternatives
We're interested in hearing what other forms of access people would like. If you have a suggestion for making Dryad's content more accessible, please let us know at help@datadryad.org.