Notes on Related Concepts: Technology Assessment

American academic radiologists have developed a parallel, but more traditionally oriented paradigm, the "Technology Assessment" paradigm. This is very focused on improving the quality of primary radiology research studies. It arose, as far as we know, from the activities of the Association of University Radiologists [1] a ‘think-tank’ that has, over many years, sponsored major initiatives in pursuit of better research, as well as better education.

The ‘Technology Assessment’ paradigm also stresses the relationship of radiology to the totality of patient care. It uses the concepts of efficacy, effectiveness and efficiency. A generic definition of efficacy is "the probability of benefit to individuals in a defined population from a medical technology (such as diagnostic imaging) applied for a given medical problem under ideal conditions of use" [2]. Effectiveness "reflects performance of a medical technology under ordinary, rather than ideal, conditions" [2]. Differences in local expertise or local disease profiles may mean that the technology does not perform as well as it did under ideal circumstances. Efficiency studies address the cost-effectiveness of procedures. Cost can be measured as money spent per procedure, money spent per satisfactory outcome etc. It is at this level that non-clinical outcomes such as quality of life are measured [2].

The traditional primary goal of most radiologists has been to obtain the highest quality images that provide the most accurate diagnostic information possible. This is a localised, limited goal. It does not take into account the eventual consequences of results of imaging examinations [3]. In the broadest sense, much of the "clinical research" published in the diagnostic imaging literature is "technology assessment". It is at the most basic end of the assessment spectrum – anecdotes and observational series. Because of severe case selection, observer and gold standard biases, the tendency is to overestimate how effectively the innovation will perform in a general population of patients and physicians [4].

Discussion and analysis have taken place in the diagnostic radiology literature, which now generally supports a structured approach to the evaluation of diagnostic tests. The "Conceptual Continuum for Efficacy", coined by Thornbury and Fryback in 1990, refers to a hierarchical model for technology assessment in diagnostic radiology. The key feature of this model is the understanding that for a procedure to be efficacious at a higher level in the hierarchy, it must be efficacious at lower levels. The reverse is not true. Increases in the efficacy at a lower level (e.g., in diagnostic imaging, technical image quality) do not guarantee commensurate improvement at higher levels (e.g. patient outcome) [5, 6]. We have incorporated it into this website’s interactive critical appraisal of Diagnostic Radiology studies. For interventional studies, we have incorporated a recently published continuum based on similar principles [7].

Research initiatives arising from the ‘Technology Assessment’ paradigm include the GE/Radiology Research Fellowships (GERRAF Program) [8], the American College of Radiology Imaging Network (ACRIN) (a co-operative group that manages clinical trials of imaging technologies as they relate to cancer) [9] and the Society for Health Sciences Research in Radiology [10]. Readers interested in improving their ability to design and conduct high-quality clinical trials should visit the AUR, ACRIN and RAHSR websites.

References

1. The Association of University Radiologists.
February 19, 2002. [ link ]

2. Brook RH ,Lohr KN, Efficacy, effectiveness, variations, and quality. Boundary-crossing research. Med Care 1985; 23 (5):710-722. [ link ]

3. Thornbury JR, Why should radiologists be interested in technology assessment and outcomes research? AJR Am J Roentgenol 1994; 163 (5):1027-1030. [ link ]

4. Hillman BJ, New imaging technology and cost containment. AJR Am J Roentgenol 1994; 162 (3):503-506. [ link ]

5. Fryback DG ,Thornbury JR, The efficacy of diagnostic imaging. Med Decis Making 1991; 11 (2):88-94. [ link ]

6. Thornbury JR, Intermediate outcomes: diagnostic and therapeutic impact. Acad Radiol 1999; 6 Suppl 1 :S58-65; discussion S66-58. [ link ]

7. Malone DE, MacEneaney PM, Applying 'technology assessment' and 'evidence based medicine' theory to interventional radiology. Part 1: Suggestions for the phased evaluation of new procedures. Clin Radiol 2000; 55 (12):929-937. [ link ]

8. Radiology Research Academic Fellow Program (GERRAF). [ link ]
February 19, 2002.

9. American College of Radiology Imaging Network. American College of Radiology and the National Cancer Institute, USA, [ link ]
February 19th, 2002.

10. Radiology Alliance for Health Services Research . [ link ]
February 19th, 2002.