Difference between revisions of "Testing"

From Dryad wiki
Jump to: navigation, search
(Running Unit Tests)
Line 1: Line 1:
{{StatusBox|This page is a work-in-progress - dcl9, 2014-07-14}}
{{StatusBox|This page is a work-in-progress - --[[User:Dan|Dan]] ([[User talk:Dan|talk]]) 10:12, 14 July 2014 (EDT)dcl9, 2014-07-14}}
= Overview =
= Overview =

Revision as of 06:12, 14 July 2014

Status: This page is a work-in-progress - --Dan (talk) 10:12, 14 July 2014 (EDT)dcl9, 2014-07-14


Testing strategies were discussed in Developer Meetings on 2014-07-11. See Google Doc.

Dryad employs two broad testing strategies:

  1. JUnit tests run via Maven within the build system, using the surefire plugin
  2. Selenium tests run externally, connecting to any running Dryad server

Test Configuration

Requires a postgres database and dspace dir.

Running Unit Tests

Dryad uses the standard set of DSpace unit tests, based on JUnit. These tests are found in the src/test directory of each module. By default, testing is disabled for normal compilation.

Maven tests are run from the dryad-repo directory with:

mvn package -DskipTests=false -Ddefault.dspace.dir=/path/to/dryad-test

Issues with Unit Tests

Though surefire supports running a single test (and this works for our selenium tests), there is a DSpace configuration issue that prevents it from working: http://sourceforge.net/p/dspace/mailman/message/30149768/.

It is also documented that the tests should be runnable without building via

mvn test -P env-dev

But this fails with an error that org.dspace:dspace-xmlui-webapp:war:1.7.3-SNAPSHOT could not be found.


Selenium tests UI-specific functionality. Dryad maintains a set of Selenium tests in the selenium directory of the repository. These tests are run separately from the normal compilation process, because they require a running instance of Dryad.

Running Selenium Tests

These tests can be checked out and run from any host. Since they work through a web browser, they do not need any Dryad code present on the testing machine.

Within the selenium directory, there is a runtests.sh script that will run all the selenium tests. You must specify the URL of the server to test

./runtests.sh http://localhost:9999

The tests are run by maven and the surefire plugin. You can run an individual test by running a maven command like the following:

mvn package -DseleniumTestURL="http://localhost:9999" -Dtest=HomePageTest

Selenium IDE

  • Firefox plugin for automating tasks within firefox.
  • It internally stores scripts as HTML, but it can export scripts in Java/Ruby/Python code suitable for use with WebDriver/RC.
  • can record script through web page actions and then replay to test
  • can export script in variety of formats (JUnit, Ruby, C#)
  • Exported scripts are relatively brittle and must be modified for general use, but are a great starting point.

Selenium WebDriver

  • Tool that can launch a browser, execute a test script, and report results of the test.
  • Can be used with JUnit or other testing frameworks.
  • Rather than using JavaScript, it works directly with each browser's API for greater control of the browser.
  • Doesn't require running a separate server.
  • The "future" of Selenium.

Configuring Firefox: This is no problem on a desktop/laptop with Firefox installed. It will work automatically. On a server, you must:

  • ensure firefox is installed
  • in the root account, run "Xvfb :99"
  • the the account that is running selenium, "export DISPLAY=:99"

Best practices:

  • Create initial tests with IDE, then export to your target language/format and edit.
  • Use the HtmlUnitDriver instead of the default FirefoxDriver if possible -- it is faster, and runs without modification on more systems.
  • When creating a test in the IDE, use "assert" statements for all tests, because "verify" statements don't export properly to the code.
  • When modifying the testing code, use "verify" statements for any non-fatal errors. Use "assert" statements for errors that will cause the rest of the test to be meaningless.

Checklist for exporting a test from the IDE to Webdriver:

  1. Ensure the test has a name ending in Test (for JUnit compatability)
  2. Perform the export.
  3. Make the test extend JUnit's TestCase class.
  4. Modify the test to use HtmlUnitDriver.
  5. Modify all assertions/verifications (these will appear as comments with an error message) to use the existing webdriver api
  6. Ensure that a base URL is set.
  7. Add it to any appropriate test suites.

(Manual) Smoke Tests

These are the basic tests to ensure all components of Dryad are working. These tests are not covered by the Selenium tests. Once something is moved into Selenium, it should be removed from here.


  • A new user account can be registered (email works, clicking on the emailed token works)


  • DSpace home page
  • About page
  • Partners page

Item search:

  • Simple searches work.
  • browses work
  • faceting works
  • "view more" on the facets works

Item view:

  • Item view pages display correctly.
  • Bitstreams can be downloaded.
  • Statistics appear correctly.
  • Handles resolve properly.

Journal submit:

  • Does the journal URL work properly?

Basic submit:

  • Submit a test dataset, with at least one file to upload.
  • Make sure a proper Handle is assigned.
  • Make sure the item is searchable.
  • View the item summary page.
  • Remove the item.

"In Review" submit:

  • Submit a test dataset for an item with the status "in review" (need to set up a dummy journal, so real editors don't get the email)
  • Make sure the item is viewable when using the review URL.
  • Make sure the item is not viewable to normal users when not using the review URL.
  • Remove the item.

Relation to DSpace

Dryad uses the standard DSpace unit tests and integration tests. See DSpace Testing.