Given a table of citations, how to reverse-lookup the Digital Object Identifier for each of the citations?

I don’t know of any complete packages or functions that do this already, but this is the general approach I would use. The Crossref DOI registration agency offers a Web-based approach for determining the DOI from bibliographic data at https://www.crossref.org/guestquery/.

On that page are several different ways to search, including the last one which takes an XML formatted search. The page includes information on how to create the appropriate XML. You would need to the submit the XML over HTTP (determining the details by picking apart the page to figure out form destinations and any additional information that needs to be included), and then parse out the response.

Additionally, you would need to verify that doing this in an automated manner does not violate the terms of service of the website in any way.

Below is the XML form for the Crossref free DOI lookup, where the searchable terms include article_title, author, year, journal_title, volume, and first_page:

<?xml version = "1.0" encoding="UTF-8"?>
<query_batch xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0" xmlns="http://www.crossref.org/qschema/2.0"
  xsi:schemaLocation="http://www.crossref.org/qschema/2.0 http://www.crossref.org/qschema/crossref_query_input2.0.xsd">
<head>
   <email_address>[email protected]</email_address>
   <doi_batch_id>test</doi_batch_id>
</head>
<body>
  <query enable-multiple-hits="false|exact|multi_hit_per_rule|one_hit_per_rule|true"
            list-components="false"
            expanded-results="false" key="key">
    <article_title match="fuzzy"></article_title>
    <author search-all-authors="false"></author>
    <component_number></component_number>
    <edition_number></edition_number>
    <institution_name></institution_name>
    <isbn></isbn>
    <issn></issn>
    <volume></volume>
    <issue></issue>
    <year></year>
    <first_page></first_page>
    <journal_title></journal_title>
    <proceedings_title></proceedings_title>
    <series_title></series_title>
    <volume_title></volume_title>
    <unstructured_citation></unstructured_citation>
  </query>
</body>
</query_batch>

This is an open problem. There are better and worse ways to attack it. Start by reading Karen Coyle’s summary of the problem. The bibliography at the end of that article is excellent.

In short, the problem of quantifying sameness between two bibliographic records is hard, and a substantial amount of machine-learning research has been done around this topic.


Here are two options

CSV upload

I have found another promising solution that does not work as well in practice as uploading a CSV directly and performing a text query here at http://www.crossref.org/stqUpload/.

However, only 18 of the 250 queries (≈7%) returned a DOI.

XML Query

Based on the answer by Brian Diggs, here is an attempt that does 95% of the work—toward writing the XML-based query. It still has a few bugs that need to be removed using sed. But the biggest problem is the “session timed out” errors I had encountered when the query was submitted.

The XML syntax includes an option to use fuzzy matching.

The doiquery.xml file contains the template text from Brian’s answer; the citations.csv file is linked above.

library(XML)
doiquery.xml <- xmlTreeParse('doiquery.xml')

query <- doiquery.xml$doc$children$query_batch[["body"]]

citations <- read.csv("citations.csv")

new.query <- function(citation, query = query){
  xmlValue(query[["author"]]) <- as.character(citation$author)
  xmlValue(query[["year"]]) <- as.character(citation$year)
  xmlValue(query[["article_title"]][["text"]]) <- citation$title
  xmlValue(query[["journal_title"]]) <- citation$journal
  return(query)
}

for (i in 1:nrow(citations)){
  q <- addChildren(q, add.query(citations[i,]))
}
axml <- addChildren(doiquery.xml$doc$children$query_batch, q )

saveXML(axml, file = 'foo.xml')

CSV to XML Converter

Creativyst software provides a Web-based CSV to XML converter.

The necessary steps to take are as follows.

  1. Enter the column names in the ElementIDs field.
  2. Enter document in the DocID field.
  3. Enter query in RowID field.
  4. Copy and paste the CSV file into the Input CSV file field.
  5. Click Convert.

See also a related question: Shell script to parse CSV to an XML query?

Tags:

Xml

Web Scraping

Doi

R

Mechanize

Recent Posts