Last updated: February 01, 2006
Validation and Insertion of DNA Sequencing Technologies
Validation and Insertion of DNA Sequencing Technologies
Cold Spring Harbor Laboratory
May 13, 1998
 | General Discussion Issues |
| Implementation Plan |
| Agenda |
| Invited Participants |
The purpose of the workshop was to discuss with a group of interested investigators how to ensure that technologies that have already been shown to have strong potential to decrease the cost of high-throughput DNA sequencing, can be implemented in production laboratories. The workshop arose out of the perception that the U.S. Human Genome Project (HGP) has done reasonably well at developing novel technologies, but is not moving these technologies effectively into production labs. This discussion is appropriate now because:
- Research on novel technologies, supported in the early years of the project, is now reaching the stage where some of these technologies are ready to move forward to the next stage of development.
- High throughput DNA sequencing laboratories have achieved production levels that make it relatively difficult for them to change their processes, so there must be good evidence that any new technology will present the opportunity for significant improvement.
- Providing that evidence requires running the technology under production-like conditions, both to harden the technology and to establish convincing evidence of its readiness for implementation, but most technology developers lack the resources and experience to conduct such tests.
- Production sequencing is moving beyond the pilot stage, into full production, and cost-effectiveness of the effort is now of paramount importance. It is therefore important to implement new, cost-effective technologies as soon as possible.
The recurring themes of the workshop were the need for cooperation, collaboration and communication; and establishing the principle that validation is time-consuming, at least as hard as, and considerably more expensive than establishing proof-of-principle. Furthermore, it was noted that the kind of effort needed to validate new technology is not the normal purview of National Institutes of Health (NIH) grant support. Therefore, it was suggested that new types of grant mechanisms and processes may be needed to accomplish this goal in a timely fashion.
General Discussion Issues
- The early stages of technology development will often occur in small laboratories which, because of their size, will usually only work on components of the entire sequencing process. This is useful only if the community can agree on standards for the inputs and outputs of such modules. With such agreement, new, more effective modules will be able to replace less effective ones in existing production labs, and modules from different sources will be able to be used together.
- Technology developers must understand the sequencing process as it is currently practiced. This will enable them to address real, rather than perceived, bottlenecks and cost factors. Such an understanding will also allow developers to anticipate and seek more effective ways to accomplish technology validation, which is an essential but under-appreciated step in the development process. Only by running new devices or using new methods at production scale, over a prolonged time, will it be possible to evaluate them for factors such as which parts will break or wear out, what aspects of the device or method are not compatible with other parts of the process and therefore must be changed or cannot actually be used, and what are the real costs of using the technology.
- Most technology development labs do not have the capacity to produce enough samples, or analyze enough results, to conduct validation tests. A number of alternative approaches are available to developers, and most of these require cooperation and integration between developers and users. For example, if the developer and user are geographically separated, personnel can be exchanged (from the user to the development lab early in the process, and from the developer to the user lab later in the process). Grant support mechanisms must be flexible enough to facilitate any of several models of collaboration. This will be particularly challenging when the private sector is involved, but the potential for important contribution by the private sector (in terms of both expertise and investment) dictates that effective mechanisms be established.
 | Top of page |
Implementation Plan
Workshop participants recommended a tripartite implementation plan to the National Human Genome Research Institute (NHGRI) for facilitating technology validation and insertion. This plan involves (1) establishing a specific funding program for this purpose, (2) changing the review process, and (3) facilitating communication.
- Technology validation/optimization is an essential task if newly developed technologies are to benefit DNA sequencing and other genome project goals, but it is expensive. NHGRI must decide if it can make the necessary financial commitment. If so, this should be identified specifically as a genome project goal and funds should be targeted specifically to it. These funds should be used to develop a program that favors partnerships and lays clear responsibility for success, and continued funding, on all members of the partnership. Flexibility, both in how the validation projects are structured and how the funds may be used, is key to success. Within a grant, there should be some funds available to be used in a flexible way to meet unanticipated needs. Alternatively, there should be ways to supplement the grant quickly when such needs arise.
The success of the program will be dependent on the development of incentives to encourage participation in technology validation. Incentives are needed, for example, to enlist the participation of the private sector in cases where the market may be small. Different incentives would be necessary for user labs to establish and maintain personnel dedicated to the validation/insertion process, with separate accounting (relative to production sequencing) of the expenses involved.
- The criteria for evaluating technology validation proposals are very different from those used to evaluate research proposals. It is unrealistic to expect panels that review research proposals to completely change their approach to accommodate the review of validation proposals. For example, while innovation and novelty are key to research proposals, they are not of primary importance for technology validation, which instead has to accomplish a prescribed task with intense focus and without diversion. Therefore, the workshop participants recommended that NHGRI establish specific review panels to evaluate proposals for support of validation projects.
Appropriate criteria for the review of such proposals include:- The state of the art in those laboratories conducting research on the particular technology under consideration.
- How long the development project has been under way and what it has accomplished.
- The time course likely to be required for the technology under development to achieve its capabilities, and whether validation testing can be initiated aggressively for those aspects that are ready to move forward, with additional capabilities being incorporated as they come on-line.
- How the technology under development relates to the high-throughput sequencing process as a complete system.
- The requirements for validation and hardening of the technology under development.
- The track record of the applicants in achieving technology validation and insertion.
- The barriers to validation/implementation (e.g., intellectual property, availability of personnel with appropriate experience, market factors [e.g., variety of types of users, market size]) that are likely to come into play for the technology under consideration.
- The importance of distributed risk and portfolio diversification for projects of the scale of the HGP.
- NHGRI needs to enhance communication among the relevant communities of developers and users. To formulate individual projects, technology developers must have access to information about the production sequencing process in various production centers. At the same time, the results of technology development research must be rapidly disseminated to the production users, so that they, in turn, can provide feedback to developers as guidance to further research, and so that users can anticipate upcoming developments in their production process planning.
The workshop participants, as members of these communities, agreed that the success of the entire enterprise will require their participation in this kind of information sharing. Information exchange can occur through meetings and workshops, Internet postings, and individual collaborations. NHGRI should take the lead in facilitating this information exchange. An important benefit of establishing this communication infrastructure will be the additional training of individuals who already have expertise relevant to some part of the technology development/validation/implementation process, about other parts of the process, and to foster effective collaboration. The establishment of such a communication network (a technology development/validation/implementation consortium) would form the basis for planning how the technology validation grants program should proceed and how proposals should be reviewed.
| Top of page |
Agenda
Development of a sustained capacity for sequencing 500 megabases of DNA/year would insure significant capability to sequence regions of the human genome from many individuals, and the genomes of numerous other organisms that are important in the study of biology and medicine. But if we hope to be able to afford the sequencing that is needed beyond the completion of the very high-value human genome, the cost of sequencing must be significantly reduced. This will require faster, more cost-effective DNA sequencing technologies, which must be transferred from their developers to the laboratories of users. This transfer is becoming increasingly difficult, as sequencing labs scale up and require more convincing evidence of the robustness of new technologies that have been tested at substantial scale.
|
| Top of page |
Invited Participants
Mark Adams
The Institute for Genomic Research
David Barker
Molecular Dynamics, Inc.
Elbert Branscomb
Lawrence Livermore National Laboratory
David Burke
University of Michigan
Ronald Davis
Stanford University
Glen Evans
University of Texas Southwest Medical Center
Richard Gibbs
Baylor College of Medicine
Steve Gordon
Intelligent Automation Systems
Eric Lander
Whitehead Institute
Richard Mathies
University of California, Berkeley
Richard Myers
Stanford University
Phil Green
University of Washington
Bruce Roe
University of Oklahoma
Gerry Rubin
University of California, Berkeley
Lloyd Smith
University of Wisconsin
Steve Soper
Louisiana State University
Marvin Vestal
PerSeptive Biosystems
Robert Waterston
Washington University
Staff members from several of the laboratories also participated in the workshop.
NHGRI Staff
Francis Collins
Adam Felsenfeld
Mark Guyer
Elke Jordan
Jane Peterson
Rudy Pozzatti
Jeff Schloss
DOE Staff
Charles Edmonds
Marvin Stodolsky
| Top of page |
Last Reviewed: February 2006