Abstract
For someone from the United States, addressing the theme of the “frontiers of evaluation” is a pleasure. Frontiers are an important part of the American self-concept. We have always seen ourselves moving west, into new territory that is expected to bring freedom and prosperity. This paper will trace that moving frontier in research evaluation in the United States, starting with a little history, moving towards the present, and peeking into the future. It begins with the transition from program evaluations to accountability systems, then turns to the current effort to establish a Science of Science Policy that will provide the research base for future evaluation techniques. I will illustrate current practice with several examples: complex rating systems, logic models, economic estimations, and mapping, before describing a new dataset still in preparation. I will conclude with some observations about where U.S. research evaluation has been and is going.References
AUSTIN, D.; MACAULEY, M. Estimating future consumer benefits from ATP-funded innovation: the case of digital data storage. Gaithersburg, MD, National Institute of Standards and Technology, 2000.
COMMITTEE ON EVALUATING THE EFFICIENCY OF RESEARCH AND DEVELOPMENT PROGRAMS AT THE U.S. ENVIRONMENTAL PROTECTION AGENCY, N. R. C. Evaluating research efficiency in the U.S. environmental protection agency. Washington, DC, National Academies Press, 2008.
COMMITTEE ON THE EVALUATION OF THE SEA GRANT PROGRAM REVIEW PROCESS, N. R. C. Evaluation of the Sea Grant Program Review Process. Washington, D.C.: National Academies Press, 2006.
COMROE, J. H.; DRIPPS, R. D. Scientific basis for support of biomedical science. Science, v. 192, n. 4.235, p. 105-111, 1976.
COZZENS, S. E. Expert review in evaluating programs. Science and Public Policy, v. 14, n. 2, p. 71-81, 1987.
COZZENS, S. E. The Knowledge pool: measurement challenges in evaluating fundamental research programs. Evaluation and Program Planning, v. 20, n. 1, p. 77-89, 1997.
COZZENS, S. E. Results and responsibility: science, society, and GPRA. In: A. TEICH, H.; NELSON, S. Science and technology policy yearbook, 1998.. Washington, D.C.: American Association for the Advancement of Science, 1999.
DENG, Y. The Value of knowledge flows: evidence from patent citations data. Computing in Economics and Finance 2005. Society for Computational Economics, 2005.
ENGLAND, J. M. A patron for pure science: the National Science Foundation’s formative years, 1945-57. Washington, D.C.: National Science Foundation, 1982.
HARDEN, V. A. Inventing the NIH: federal biomedical research policy, 1887-1937. Baltimore, Md.: Johns Hopkins University Press, 1986.
IITRI – Illinois Institute of Technology Research Institute. Technology in retrospect and critical events in science. Washington, D.C.: National Science Foundation, 1968.
JAFFE, A. B.; LERNER, J. Reinventing public R&D: patent policy and the commercialization of national laboratory technologies. Rand Journal of Economics, v. 32, n. 1, p. 167-198, 2001.
LANE, J.; COZZENS, S. A deeper look at the visualization of scientific discovery in the federal context by NSF. Washington, D.C.: National Science Foundation, 2008. Available in: http://scienceofsciencepolicy.net/media/p/188.aspx.
LING, J. T. et al. Evaluative study of the materials research laboratory program. McLean, Va., The MITRE Corporation, 1978 (Summary report).
MANSFIELD, E. Estimating social and private returns from innovations based on the advanced technology program: problems and opportunities. Gaithersburg, MD: National Institute of Standards and Technology, 1996.
MANSFIELD, E. et al. Social and private rates of return from industrial innovations. Quarterly Journal of Economics, v. 91, n. 2, p. 221-240, 1977.
MARBURGER, J. Speech to AAAS Forum on Science and Technology Policy. Washington, D.C., 2005. Available in: http://www.scienceofsciencepolicy.net/media/p/59.aspx.
MCLAUGHLIN, J. A.; JORDAN, G. B. Logic models: a tool for telling your program’s performance story. Evaluation and Program Planning, v. 22, n. 1, p. 65-72, 1999.
NARIN, F. Evaluative bibliometrics: the use of publication and citation analysis in the evaluation of scientific activity. Cherry Hill, New Jersey: Computer Horizons, Inc., 1976.
RAFOLS, I. et al. Science overlay maps: a new tool for research policy and library management. Journal of the American Society for Information Science and Technology, v. 61, n. 9, p. 1.871-1.887, 2010.
ROESSNER, J. D. Outcome measurement in the USA: state of the art. Research Evaluation, v. 11, n. 2, p. 85-93, 2002.
RUEGG, R.;FELLER, I. A toolkit for evaluating public R&D investment: models, methods, and findings from ATP’s first decade. Washington, D.C.: National Institute of Standards and Technology, 2003. Available in: http://www.atp.nist.gov/eao/gcr03-857/contents.htm.
SAPOLSKY, H. M. Science and the navy: the history of the Office of Naval Research. Princeton, N.J.: Princeton University Press, 1990.
SHERWIN, C. W.; ISENSON, R. S. Project Hindsight: Defense Department Study of the Utility of Research. Science, n. 156, p. 1.571-1.577, 1967.
SMALL, H. Cocitation in scientific literature – new measure of relationship between 2 documents. Journal of the American Society for Information Science, v. 24, n. 4, p. 265-269, 1973.
The content of the articles and reviews published in RBI are of absolute and exclusive responsibility of their authors.
Downloads
