Journal Integration Process

From Dryad wiki
Jump to: navigation, search

Integration checklist

Steps that Dryad must complete for a journal to be integrated:

Discussion phase (from initial contact --> journal decision to integrate) - ED oversees

Initial documentation

  • send basic info (wiki documentation, Integration Overview slides)
  • alert management and tech
  • add journal to Salesforce and Trello (formerly Partner spreadsheet)
  • (liaison/mgt) determine strategy; what issues are there, who can approve moving ahead etc.
  • arrange conference call, ask them to look at/fill out Pre-Integration Questionnaire (PIQ), to walk through integration implementation steps and answer questions

Get-to-know-you phone conversation (Executive Director and Operations Manager, occasionally Repository Architect)

  • answer questions from journal about workflows, optional customizations, etc.
  • get journal timeline for testing and rollout
  • obtain/confirm PIQ
  • goes to “in discussion” on Trello board

Implementation phase (from PIQ --> successful completed test) - Technical Specialist

  • Set up Journal Concept on the production server
    • journal.integrated should be set to false.
    • journal.subscriptionPaid should be set to false or "SHOULD BE TRUE" (even if the journal will sponsor payments, this field must not be set to true until the journal is ready to assume payment from individual submitters).
  • Liaise with journal to verify that metadata emails are received and processed correctly (see Testing).
  • Finalize integration:
    • Set journal.subscriptionPaid to true if the journal will sponsor payments for their authors
    • Set journal.integrated to true in the Concept on production
    • Add journal to list of integrated titles on website
    • Obtain cover image/logo; add to folder on Dropbox
    • Ensure cover image is added to Dryad, both on the data package pages and in the "recently integrated" box on the homepage
    • Notify curation team (staff email list)
    • Add journal to list of completed journals.</li>

Completion phase (after completed test --> rollout and ongoing maintenance)

  1. (Meredith) Concluding email with journal contact
    • tell journal they are officially integrated and we now expect to receive notices from them
    • tell journal about weekly curator reports; confirm recipients and add to Trello any updates since the PIQ was completed
    • check on policy for data references & updated journal instructions (Journal Instructions)
    • ask journal to inform us of any forthcoming changes to their manuscript system that will affect integration
    • invite journal representative to subscribe to the Announce and/or Users mailing lists
    • suggest journal use help@datadryad.org address for all urgent questions
    • offer statistics
    • explain Dryad membership, pricing plans, offer more information if not already done
    • ask permission to promote if applicable
  2. (Meredith) Follow-up
    • PR: tweet &/or blog post
    • (liaison/dir) maintain at least annual contact to confirm that notices conform to specifications, discuss journal options, update contact information, solicit feedback, and encourage participation in the Dryad organization

Testing overview

  • Set up a new journal concept on the Production server, with journal.integrated set to false. Journal settings come from the PIQ.
    • NOTE: if a journal is integrating, wants to sponsor submissions once the integration is complete, and does not have an existing sponsorship in place, do not set journal.subscriptionPaid to true on Production until the integration is launched. (Journals can be sponsored without being integrated, so this will cause them to appear as a sponsored journal in the Shopping Cart report when they are not—yet).
  • Dryad staff sends test metadata emails to journal-submit@datadryad.org and verifies success
    • Format specifications for automated email notices to Dryad: Journal Metadata (structure differs from different MS processing systems)
  • Journal sends test metadata emails to journal-submit@datadryad.org.
    • Format specifications for automated email notices to Dryad: Journal Metadata (structure differs from different MS processing systems)
    • sometimes journals use real articles in testing, sometimes they do fake ones
    • The Dryad production server runs a webapp, journal-submit, that checks the Gmail account every minute for a specific label (“journal-submit”) and processes any messages with the label (see Journal Metadata Processing Technology for specifics)
    • currently, all new emails are automatically tagged with "journal-submit".
    • The journal-submit webapp processes each message:
      • It looks for the Journal Code and looks up the matching Concept. If no Journal Code is provided, it attempts to look up the Concept by Journal Name.
      • It parses the email based on the parser provided in the Concept metadata field journal.parsingScheme.
      • If the email fails to parse properly, the message is tagged with "journal-submit-error" in Gmail.
      • Parsed emails are written out as metadata into the manuscript table in the database.
    • Dryad staff checks the server for the new metadata
      • Manuscript metadata can be looked up either directly in the database or via the REST API.
      • Pro tip: set up a Gmail filter on journal-submit-app to notify integration staff of emails received from the new journal.
    • Once metadata looks good, staff turns integration “on” on Production

    Sample data

    Sample journal concept

    When a new journal goes into integration testing, create a new journal concept with the following skeleton metadata:

    ID    Field Name    Value
    121    journal.journalID    {Dryad internal journal code}
    122    journal.fullname    {Full journal name}
    123    journal.metadataDir    /opt/dryad/submission/journalMetadata/{journalcode}
    124    journal.parsingScheme    manuscriptCentral
    125    journal.integrated    false
    126    journal.allowReviewWorkflow    false
    127    journal.embargoAllowed    
    128    journal.publicationBlackout    false
    129    journal.subscriptionPaid    false
    130    journal.sponsorName    
    131    journal.notifyOnReview    automated-messages@datadryad.org
    132    journal.notifyOnArchive    automated-messages@datadryad.org
    133    journal.notifyWeekly    automated-messages@datadryad.org
    134    journal.paymentPlanType    
    136    journal.canonicalManuscriptNumberPattern    
    137    journal.customerID    

    Sample email record

    Submit this message from your email account in plain text, addressed to journal-submit@datadryad.org. Indicate in the Subject line that Dryad is ready to receive test messages from this journal.

    Make sure the Journal Code field has the proper code and the MS Reference Number/MS Dryad ID have unique names/numbers.

    Journal Name: <JOURNAL NAME>
    Journal Code: <JOURNAL CODE>
    Journal Admin Email: test@testjournal.org
    Journal Editor: Test Editor
    MS Reference Number: MS-0000
    Article Status: accepted
    MS Title: This is a test article
    MS Authors: Smith, John; Doe, Jane
    Contact Author: John Smith
    Contact Author Email: johnsmith@test.edu
    Contact Author Address 1: 523 Evergreen Terrace
    Contact Author Address 2: 
    Contact Author Address 3:
    Contact Author City: London
    Contact Author State:
    Contact Author Country: United Kingdom
    Contact Author ZIP/Postal Code: WC1 0AY
    Keywords: computational social science, data science, social media, complex systems
    Abstract: Humans are inherently mobile creatures. The way we move around our environment has consequences for a wide range of problems, including the design of efficient transportation systems and the planning of urban areas. Here we gather data about the position in space and time of about 16,000 individuals who uploaded geo-tagged images from locations within the UK to the Flickr photo-sharing website. Inspired by the theory of Lévy flights, which has previously been used to describe the statistical properties of human mobility, we design a machine learning algorithm to infer the probability of finding people in geographic locations and the probability of movement between pairs of locations. Our findings are consistent with official figures reporting on the main populated areas in the UK and on travel flows between pairs of major cities, suggesting that online data sources may be used to quantify and model large-scale human mobility patterns.
    EndDryadContent

    This is saved as JSON data in the database:

    [ {
      "authors" : {
        "author" : [ {
          "familyName" : "Smith",
          "givenNames" : "John",
          "identifier" : null,
          "identifierType" : null
        }, {
          "familyName" : "Doe",
          "givenNames" : "Jane",
          "identifier" : null,
          "identifierType" : null
        } ]
      },
      "correspondingAuthor" : {
        "address" : {
          "addressLine1" : "523 Evergreen Terrace",
          "addressLine2" : null,
          "addressLine3" : null,
          "city" : "London",
          "zip" : "WC1 0AY",
          "state" : null,
          "country" : "United Kingdom"
        },
        "author" : {
          "familyName" : "Smith",
          "givenNames" : "John",
          "identifier" : null,
          "identifierType" : null
        },
        "email" : "johnsmith@test.edu"
      },
      "dryadDataDOI" : null,
      "keywords" : [ "computational social science", "data science", "social media", "complex systems" ],
      "manuscriptId" : "MS-0000",
      "title" : "This is a test article",
      "publicationDOI" : null,
      "publicationDate" : null,
      "dataReviewURL" : null,
      "dataAvailabilityStatement" : null,
      "optionalProperties" : {
        "Journal_Admin_Email" : "test@testjournal.org",
        "Journal_Editor" : "Test Editor"
      },
      "abstract" : "Humans are inherently mobile creatures. The way we move around our environment has consequences for a wide range of problems, including the design of efficient transportation systems and the planning of urban areas. Here we gather data about the position in space and time of about 16,000 individuals who uploaded geo-tagged images from locations within the UK to the Flickr photo-sharing website. Inspired by the theory of Lévy flights, which has previously been used to describe the statistical properties of human mobility, we design a machine learning algorithm to infer the probability of finding people in geographic locations and the probability of movement between pairs of locations. Our findings are consistent with official figures reporting on the main populated areas in the UK and on travel flows between pairs of major cities, suggesting that online data sources may be used to quantify and model large-scale human mobility patterns."
    } ]

    For these emails, what you're mostly looking for is odd formatting issues. Our system reads most emails and converts them appropriately, but sometimes the odd journal will format their author names weirdly.

    Once you are satisfied the emails are okay, then you can send an email to the client letting them know our system is ready to receive their test emails.

    Here’s a sample of an email I just sent to Jennifer at Physiological and Biochemical Zoology (you should send it to all contacts listed for journal integration questions on the PIQ)

    Hello Jennifer,

    Meredith Morovati asked me to reach out to let you know that I am handling setting up the test messages for Physiological and Biochemical Zoology. The Dryad system is now ready to test messages from Physiological and Biochemical Zoology so when you are able, feel free to send a few test messages to journal-submit@datadryad.org from the journal.

    As a reminder, the format specifications for the notices to Dryad are here:
    http://wiki.datadryad.org/Journal_Metadata

    The 6 mandatory fields are described there.

    The journal code specified in the Pre-Integration Questionnaire is PBZ, so we would expect to see your notices contain this field as follows:

    Journal Code: PBZ

    Because these notices are processed automatically by our submission system and are not read by a human, these format requirements are important. Please let me know if you have any questions. Also, if possible, let me know when you send a test so I can be sure to check on its status in our system and let you know the results. It would be great if you could provide a variety of emails—some with a single author, others with multiple authors, all with abstract and MS number.

    Please let me know if you have any questions or concerns.

    Best regards,
    Debra


    Once you’ve received actual test messages from journals, repeat the steps 5-10 above.

    Resolve issues if there are any. If not, you can contact journal and let them know we’re ready to go live with their integration. (Also ask for journal cover image if that hasn’t been handled before.)

    Related Documentation / resources