Format of the Conference

The conference will consist of plenary sessions and workshops. The plenary sessions will present approaches currently used by the U.S. Department of Energy and other departments, and will discuss a straw person methodological framework for identifying benefits of R&D programs. The conference will seek to build upon work that has been done by many in this field, including the previously-cited report by the National Research Council.

The NRC study developed a methodological framework that is represented by a matrix. The basic types of impacts are listed along one dimension of the matrix: economic, environmental, and security. The other dimension of the matrix reflects the degree of commercialization, certainty, and time-frame. Along this dimension, the benefits are classified as being realized, options, or knowledge. In the following figure, we have modified the NRC matrix by adding the "prospective" column. It was not part of the NRC framework because that study considered only retrospective benefits.

Modification of National Research Council Committee's Framework
for Identifying Benefits of Energy R&D

This framework, together with ideas about some issues that this study did not fully address, will serve as a starting point for the conference. The four workshops will focus on methodological questions whose answers have been particularly elusive.

The four workshops will largely be self-contained, with most conference participants encouraged to attend just one workshop, rather than "hopping from one to another," so that workshop participants will have time to "sink their teeth" into the key issues that are the main topics of each workshop:

1. Prospective benefits -- cross-cutting issues in estimating the prospective benefits of R&D programs,

2. Options which R&D programs provide -- defining and estimating their value,

3. Knowledge -- defining and estimating the value of the scientific and engineering knowledge generated by R&D programs, and

4. Security, economic, and environmental benefits of energy technologies and systems.

The plenary sessions and the workshops will also have common issues they will address. These include: developing agreed-upon definitions; developing measures for performance and evaluation of energy R&D, both qualitative and quantitative; assessing data needs and identifying appropriate sources; and developing methodologies to estimate the benefits of energy R&D that can be used prospectively for program development as well as retrospectively for program evaluation.

Workshop A: Prospective Benefits -- Cross-Cutting Issues

Workshop A will address issues that cut across the problems of estimating the prospective security, economic, and environmental benefits of energy technologies associated with government R&D programs. The key question is how to estimate the "delta" or incremental contribution of a government-sponsored R&D whose goal is to develop particular energy technologies. To address this question, two important issues demand special attention:

Developing a Baseline, Absent a Technology Developed in Part by a Government Program

To estimate the prospective benefits of a government program, a baseline must be established. The baseline is the case without the government effort that affected the development of the technology. It is the energy market, environmental impacts, and energy security that would exist, absent the energy technology that is to be developed through government-sponsored R&D. The task in the first part of Workshop A is to identify methodologies for developing both retrospective and prospective baselines for the market adoption of different energy technologies under the most likely scenario, absent the technology under consideration. Several questions are germane to this issue. How are plausible scenarios developed for technological progress (including that of existing technologies), economic conditions, and regulatory and policy changes; and how would these affect the projected baseline absent a government R&D program? Should alternative baselines be used to reflect uncertainty about future conditions? How does one consider the potential effects of other government-sponsored R&D programs that are being conducted when evaluating one related program? What are the different issues and potential approaches for considering these questions prospectively as contrasted with retrospectively?

Impact of the Government Program

Given some estimate of the benefits of an energy technology, relative to the baseline, a separate but related issue is that of estimating the relative contribution of the government R&D program to the total benefits of the technology. That is, what is the impact of the government relative to the overall beneficial outcome as a result of the technology? Typically, a government program might lead to the eventual commercialization of a technology, or accelerate its introduction into market; but the private sector also plays a critical role. Looking prospectively, this session in Workshop A will focus on identifying a method to estimate the effect of a government program — what is its impact on the innovation, development and design of a technology, and how is the market penetration of the technology be affected? Would the technology never be developed absent a government program? Does the government program just accelerate the development of the technology, and if so by how much? Does the program affect its projected market penetration — by how much? Also, how does the definition of the baseline condition affect how the impact of the government program should be calculated? Does the program have other beneficial effects and how are they weighed? How does one relate future streams of benefits to past, present, and future R&D investment? Should one measure the government's impact on the basis of individual R&D programs or across a portfolio?

Workshop B: Option Value

Options provide discretionary choices to deal with deviations from planned scenarios. In a prospective sense, option value helps justify R&D in technologies that are being developed intentionally not to enter the market under the most likely conditions, but that would provide economically viable solutions under alternative plausible conditions. Physical or intellectual assets (not necessary fully developed technologies as defined in the NRC study) that might be deployed in the future (not just under what are now considered to be improbable conditions) may also contain significant option value. In this sense, options provide insurance in the face of market uncertainties, yet retain the ability to capture the upside benefits should improbably scenarios be realized. In a retrospective sense, technologies that are already developed, but that are unlikely to be commercialized under current or anticipated market conditions, may yet contain option value. This value is derived from the uncertainty surrounding future market conditions. These technologies were developed to enter the market but did not do so because of changes in conditions -- the technologies remain available for the market in the future should conditions change to make them commercially viable. Workshop B will define more precisely the concept of options and will identify methodologies to evaluate the value of their benefits.

Workshop C: Knowledge Value

R&D produces scientific knowledge. It might be either specific information or general insight about a field of study. The very nature of science programs is that they have no immediate commercial objective. Even technology-oriented R&D programs that fail might contribute to the state of knowledge about the field in terms of what does not work. Workshop C will focus on how to evaluate the benefits of scientific knowledge — what indicia could be used and how could one assess them? The primary task of Workshop C is to identify methodologies to assess knowledge benefits both prospectively and retrospectively. In a prospective context, the goal of a science program is primarily to produce knowledge. In a retrospective context, whether intended or not, a program might lead to results that have commercial value and to economic, environmental, or energy security benefits. Workshop C will also address the relationship between knowledge and technology innovation.

Workshop D: Security, Economic, and Environmental Benefits

Energy security in the past has typically focused on vulnerability to oil imports in a cartel market. Since September 11, 2001, the vulnerability of energy infrastructure to catastrophic disruptions has become a greater concern. In general, energy security refers to the adequate and reliable delivery of energy products and services with reasonably stable prices in a competitive market. In addition to oil security and energy infrastructure security, energy security includes electric power reliability. Workshop D will focus on identifying the nature of energy security, how it can be measured and valued, and how improvements in it can be determined and evaluated. Workshop D will also address economic and environmental benefits. These latter topics have already been the subject of extensive study (e.g., global climate change). Sessions in Workshop D will identify the economic- and environmental-benefit literatures and, possibly, the range of values which could be used. These sessions will also identify other economic and environmental considerations, such as the economy-wide impacts of R&D, and suggest methods for estimating their relative value.

If you have any questions, please contact:
Ms. Stephanie Floyd
Oak Ridge National Laboratory
P.O. Box 2008, Oak Ridge, TN 37831-6207
Phone: 865-574-5517; Fax: 865-574-8272; E-mail: floydsd@ornl.gov
or
Russell Lee
Phone: 865-576-6818; Fax: 865-574-8272; E-mail: leerm@ornl.gov

Home  |  Conference Information  |  Presentations  |  Rapporteurs' Summaries  |  Synthesis Report
Knowledge Center  |  Steering Committee  |  Speakers and Panelists  |  Pre-Conference Reading

Updated: February 28, 2002