Test plan example

Introduction

The following document describes the overall testing process and deliverables necessary to test the Study Design and Study Participation messages (Messages).

The Study Design message answers the question, “What will be done in the study?” The Study Participation message answers the question, “Who will be involved in the study?”

In the September 2008 ballot, each message is being proposed as a Draft Standard for Trial Use (DSTU). Both messages will be tested during the trial use period to determine usability. During the DSTU period, issues, concerns will be captured and corrections to the standard will be made. The purpose of this Testing Plan is to provide an understanding of the approach, as well as an overview of the activities, deliverables, and testing strategy established to assess the usability of the intended message standard.

Objectives

The objectives of these tests are to prove that the information collected can be stored in a database (Janus) and meaningful analysis can occur from the database. As part of the test summary, the FDA will provide commentary on whether the messages have enough information on which to perform meaningful analyses. At a minimum, existing review artifacts (SDTM datasets) must be able to be created from the database.

The FDA’s test plan will be focused on loading the database and running reports off of the data to ensure the data can be accessed and used. Wherever possible, existing tools at the FDA will be used for testing (e.g. SAS, webSDM, etc.). Other organizations are encouraged to test the standard and provide evidence of the test effort.

To successfully test the standards we will need to create the Messages and the database to store the messages and have the ability to view the data in the database (to be determined in the test cases).

A secondary test would be for applicants to review their existing systems and determine if they can produce the Messages. This is an optional test.

Messages to be tested

• Study Participation
• Study Design

Scope

In Scope:

• The testing activity will utilize the balloted DSTU Messages as a starting point. It is possible that the messages will need to be modified in order to complete the test cases.
• The testing will be limited to FDA requirements.
• The DSTU testing period shall begin September 2008 lasting for a period of approximately 11 months
• Organizations who wish to participate in the testing activity will volunteer
• A test strategy based upon storyboards and executable Test Use Case will be documented as described in the Deliverable section of this project plan. The test scripts for the Message will consider but are not limited to:
  • Testing loading of the message into a database
  • Ensuring most state transitions can be accommodated
  • Testing of negative conditions such as loss of data or data integrity due to data entry errors, etc.

Out of Scope:

• Testing process and applications of each applicant and regulatory authority
• Verification of FDA’s database
• Requirements outsides of FDA’s requirements
• Any software used to create or read the Messages
• Any software used by FDA to extract, view, or analyze data loaded into the Janus database

Testing Process

Deliverables

Following summarizes the required deliverables by participating test entities: Document Purpose

Test Plan The plan for testing, including the environment that will be used, including tools and verification strategy

Storyboards Identify the test scenarios that could be tested

Test summary and interpretation Document the execution and results of each test

Timeline

Below summarizes the anticipated timeline for DSTU testing.

• September 2008 - DSTU Passes
• October 2008 a test version of Janus is hosted with all of the data elements needed for the Messages
• September – December 09 - Study Participation XForm developed, mock study design sample created manually
• January 2009 – database is loaded (study design, study participation)
• February 2009 – First availability of representing the study design and study participation in rudimentary ways that can prove the Messages

February – July 2009 – Refine capability and agree tests are successful

Verification

Acceptance criteria for the successful execution of a message shall result from controlled execution of storyboards or messages that are equivalent to the storyboard. Each test should capture the variable message input data necessary to communicate the content associated with the Messages described below within a ‘storyboard’ scenario. The storyboards comprise typical submission actions which require the use of the Messages.

Successful execution of storyboards shall result from documented observation of the expected result. The expected results would be included in the test scenario. To verify the test, it would be preferred if screen shots and annotated XML code is provided. If the secondary test is performed (applicant review of existing systems), the applicant will provide a list of gaps where they do not have the data to create the message and provide recommendations on how to change the standard so it is easier to be implemented. Ideally, two applicants would participate in this test.

Below lists the currently available ‘storyboard’ scenarios. Storyboards will be organized in to test cases. It is possible that more than one storyboard is satisfied by one test case. If more than one test case is needed to satisfy a storyboard then that storyboard will be separated out to more than one storyboard. If the aforementioned storyboard cannot be separated out in to more than one storyboard, then that requirement cannot be met by the current message.

Study Participation

• Investigator Information
• Updated Investigator Information
• Populate Clinical Investigator Registry
• Inspection Results
• Other Participating Organizations
• Subject Protection Approval
• IRB – withdrawal of approval
• Updated IRB Approval – Change in an Investigator
• Updated IRB Approval – Protocol Amendment
• Study Subjects Progress Report
• Final Study Subjects Disposition
• Participation of a group of subject
• Participation of a part of a subject

Design

• New Protocol – complex - drug
• New Protocol - Single arm - device
• New Protocol –cross-over design
• New Protocol – Repeat design, branching, biologic
• New Protocol – Cancer Drug + Radiation +/- Surgery
• New Protocol – Adaptive trial design
• Protocol Amendment – due to interim analysis
• Protocol Amendment – due to Safety recommendation
• Protocol Amendment –Unplanned change -eligibility
• Food Animal Study #1
• Aquaculture Study – Multiple species, derived baseline population size
• In vitro Toxicology Assay – Ames Test
• Embryo-Fetal Development Study – Parent-child relationships
• Multigenerational Study
• Stability Study
• Device Performance Study
• Observational (Cohort) Study

Coverage

To verify the standard at least 70% of the storyboards should be testing in at least one system. Two systems (if one program creates and reads the standard that would be considered two systems) should implement the standard to some degree. Data coverage and systems built will be provided in the test summary and interpretation document.

Requirements

To accurately perform the test we need an installation of the Janus database. We are planning for the FDA to host the test Janus database. We will need to extend the Janus database to house the new data elements represented in these Messages.

Risks

• Install Janus
• Modify/create tools to read off of Janus