The Top American Research Universities

You don't have the latest version of Adobe Flash Player.

Please update your flash player.

Please login or register to make a comment!

...Description...... more. less.

methodology, and distribute those in which they rank high to their alumni. The Myth of Number One A mericans love the eternal pursuit of the mythical number one. First Place, Número Uno, Best of Class ... . We have many ways to express our enthusiasm for placing things in ordered lists: The best wine, the best dressed executive, the best cities, the best cars, and the best movies. This pursuit of the best carries with it a significant commitment to defining and measuring the quality that underlies the ranking and a recognition that competition tends to drive individuals and organizations towards higher performance. Yet, with all of our enthusiasm for identifying number one, there is a remarkable amount of controversy over exactly what we can measure that will define the best. We often qualify our understand- ing of the cbest d and talk about the best minor league team, the best small cities, the best of show, or the personal best. The Rankings Game W e who live in America 9s research universities also worry about which one is the best. When the various surveys and rankings appear from time to time, we eagerly consume them in search of the best colleges, the best American universities, the best business schools, the best MBA pro- grams, or the best medical colleges in an ordered and numbered list. In almost every case, universities decry the commercialism of the rankings, attack the methodology of the ranking process, and proudly distribute to their alumni those rankings in which they appear high. The most famous 4and perhaps most controversial 4of the rankings come from US News & World Report , whose annual issue ranking colleges and universities carries the same suspense for some academics that the final college football polls have for sports fans . University administrators, public relations officials, and fundraisers wait expectantly for the rankings, and institu- tional research officials fill out the forms for US News with great care and attention in hopes of improving their rank. The compil- ers of the US News rankings modify the criteria and weightings that drive their rankings with considerable frequency in an effort to improve the reliability of the results. Each change in methodology, however, changes the rankings of individual universi- ties, creating an illusion that universities rise Page 6 ! The literature on ranking, including critiques and alternative ranking methodologies, is extensive. By far the best guide to these resources is a web page maintained by the University of Illinois library. For those interested in following the debate, the on-line and printed sources available here are kept current and provide a comprehensive and annotated resource. College and University Rankings , (Education and Social Science Library, University of Illinois at Urbana- Champaign, March 2000) at [http://www.library.uiuc.edu/edx/rankings.htm]. A complete discussion of the US News & World Report methodology is available in a report published on TheCenter web site by Denise S. Gater at [http://thecenter.ufl.edu/usnews.html]. and fall in their relative significance from year to year. This illusion of rapid and dramatic institutional change has some perhaps unintended benefits. From the magazine 9s perspective, it creates great interest, for if the rankings change from year to year, the newest issue attracts a larger audience. When a university rises in the US News lists, the administration promotes the new ranking widely as an example of superb manage- ment and high quality faculty productivity. When an institution falls in the lists, the administration highlights the errors and inappropriate methodology. Sometimes it simply ignores the rankings altogether. The variability of the US News methodology generates the interest that sustains the process. ! While those of us who study the rankings know their faults well, we also know that underneath the hype lies a fundamental and important truth. American universities exist in a highly competitive marketplace, com- peting for the people and money that deliver excellence. All major American research universities compete for their share of a relatively limited supply of highly productive research faculty. These faculty, through their discoveries and writing, create the knowledge that drives our economy and defines our era. The larger the number of highly productive research faculty at a university, the more intellectually powerful the institution becomes. The academic and public reputation of research institutions closely follows their success in acquiring research faculty, al- though reputations rise and fall much more slowly and uncertainly than the reality they reflect. Universities that seek to rise into the ranks of the nation 9s elite research institu- tions need reliable measures of performance that will reflect their success in the competi- tive higher education marketplace. Characteristics of Universities M ost of the currently available rankings, focused as they are on an ordering of institutions from number one on down, obscure some of the fundamental characteristics of university change and the university marketplace. Over the past several years, TheCenter has developed a structure for identifying some key characteristics of top research universities in America. This structure helps institutions to understand the characteristics of the marketplace and the opportunities for improvement. TheCenter clusters universities into groups defined by their relative performance on a variety of research university characteristics: research, private support, faculty, doctorates, postdoctoral appointees, and undergraduate quality. While issues of scope (land-grant Characteristics of Universities A ll major American research universities compete for their share of a relatively limited supply of highly productive research faculty. The Top American Research Universities Page 7 mission, health and engineering programs, affiliated laboratories and hospitals, and professional schools) provide a context within which research universities function, they do not determine the success of the research university. Institutions of quite different scope and scale (student, faculty, budget size) appear at all levels among America 9s top research institutions. Any definition of university quality will provoke controversy and disagreement. This is both healthy and expected. For the purposes of this study, we use measures that identify institutional performance relevant for a top research university. We could imagine other measures as well, but in most cases, the data for more complex evaluations do not exist in a reliable form. Indeed, for all the intellectual sophistication of universi- ties, they resist accurate, consistent, and standardized measurement of almost every- thing they do. Accounting practices, definitions of such fundamental concepts as teaching and research, and the methodolo- gies for calculating measures of faculty productivity vary significantly from institu- tion to institution, from state to state, and from private to public ownership. As a result, systematic evaluation of research universities must rely on surrogates, data elements with some degree of consistency and face-validity in the academic community that provide direct or indirect measures of institutional performance. Universities of the highest quality tend to do most things very well. Other institutions will perform very well on some elements but not as well on all. Many institutions do not participate in the research competition at high levels, and for that reason the indicators used to characterize research institutions do not apply to them. While it is possible to proliferate measurements, we believe that for research universities a relatively few indica- tors provide sufficient evidence of overall quality. In most cases, the use of more indicators contributes little additional information. This is so because the differ- ence among research universities with high levels of performance is not great. Ranking Berkeley, Michigan, and Wisconsin or Harvard, Stanford, and Chicago from one to three tells us very little more than if we ranked them in a different order. These institutions are different in many ways, but these six represent premier American public and private research universities. By using multiple indicators and combining them with different weights and formu- las, we could produce rankings with these institu- tions in many different sequences. For this reason, we use the fewest measures needed to identify groups of outstanding institutions and make no effort to rank the institutions within groups. Defining the Research University American public and private universities come in a bewildering variety of institutional forms, embedded in political arrangements and governance structures of remarkable diversity. Some universities consist of multiple campuses, each governed indepen- dently with its own curriculum and student body. Others have geographically diverse campuses that function as a single institu- tion. Although this often appears in the form of a single geographic campus at Ann Arbor, Palo Alto, or Seattle, for example, it can also appear in multiple geographical locations in F or all the intellectual sophistication of universities, they resist accurate, consistent, and standardized measurement of almost everything they do. Page 8 Baltimore and Washington D.C. The key element is the organizational focus that permits the university to operate as a single institutional entity. To take an example, the University of North Carolina has many campuses but only one president. For the purposes of our analysis, TheCenter considers the University of North Carolina at Chapel Hill as one research university and does not include the productivity of the faculty at other UNC campuses as part of the Chapel Hill data. This study defines the research univer- sity as the main campus of multi-campus universities, and we use the institutional definition of the main campus in adjusting the data. Most private universities do not present as many definitional difficulties as do the complex political structures of public institutions, but The Johns Hopkins Univer- sity is an instructive example. This univer- sity consists of various schools scattered over a wide geographic area from north Baltimore to Washington, D.C. Hopkins, nonetheless, operates as one institution with one gover- nance and institutional structure, and the productivity of the faculty in all of the university 9s schools form part of The Johns Hopkins institutional data. Hopkins offers an additional illustration of the difficulty of defining the scope of a university. It currently includes the research productivity of its Applied Physics Labora- tory (APL) as part of the university 9s work. This rests on the recognition that APL 9s staff has a variety of teaching and academic missions that connect this laboratory organically to the university, even though the primary funding of APL derives from special appropriations from the federal government. An alternative model occurs for the Department of Energy labs managed by the University of California system. Although The Lawrence Livermore Laboratory, for example, exists in close geographic and intellectual connection to the University of California campus at Berkeley, this institu- tion does not include the research funding of the Lab in its totals. Perfection in classification is difficult to achieve. Fortunately, while the Hopkins case creates an outlier in the research data, removing the APL component would not affect its inclusion within the top group, illustrating one benefit of the clustering methodology. Universities also have complex and differing relationships with their teaching hospitals. In some cases, clinical research done by faculty physicians with appointment and tenure in the sponsoring university appears in the totals for the hospital that is the host for this research. In other cases, the clinical research flows through the university and appears in the university totals. These differences in organization affect both public and private institutions and led to the clustering strategy that puts high performing institutions in groups rather than in precise numerical rank order. Often multi-campus public universities or university systems report data for the larger collection of campuses rather than for the research campus. In those cases, TheCenter staff worked with the campus institutional research offices and used data available from institutional and national sources to determine what portion of the reported data Characteristics of Universities A merican public and private universities come in a bewildering variety of institutional forms, embedded in political arrangements and governance structures of remarkable diversity. The Top American Research Universities Page 9 ! Federal research and postdoctoral appointees correlate at .544 for all universities in this group; for federal research and doctorates, the correlation is .464. However, federal research and SAT scores correlate at only .287, a level that is not significant for either private or public universities at the .01 level. we should assign to the research campus. This process serves to make the research universities comparable for the purposes of this analysis of institutional performance. An alternative research project might well choose to review the productivity of university systems composed of multiple campuses, but that is not the purpose of this project. A complete description of the adjustments made to the officially reported data for individual institutions appears in the A ppendix and on TheCenter web site [http://thecenter.ufl.edu]. Indicators of Performance T he identification of performance indicators is the most important task facing any project that hopes to assess comparative institutional performance. Academics can identify a wide range of useful indicators, but only a few have reliable data available. Fortunately, there are enough measures with reliable data to support a clustering of universities by quality. The indicators of university performance used here permit the development of reliable comparative data that have face validity as reasonable references for research university performance. No available data can accurately capture the totality of a university 9s quality and productivity. No available indicator can measure the complete performance of these complex and diverse institutions. At the same time, some measures provide quite reliable indicators of institutional perfor- mance, even when they do not capture all of that performance. This is particularly true of research universities, whose core competency and competitiveness in research define the institution 9s character. While the measures we use bear some relationship to each other (for example, institutions with high research volume tend to have a significant number of doctorates and postdoctoral appointees), ! the relation- ship is not particularly strong. This is partly because research volume captures only a portion of a university 9s research productiv- ity, while the doctorates indicator includes all disciplines: arts, humanities, social sciences, and professions, as well as the sciences. SAT scores for the undergraduate entering class bear almost no relationship to the research volume of the institution, but high quality undergraduates form an important part of America 9s premier research universities. The following nine measures provide us with the reference points for identifying the top research universities: " Total research expenditures; " Federal research expenditures; " Endowment assets; " Annual giving; " Faculty members in the National Academies; " Faculty awards; " Doctoral degrees; " Postdoctoral appointees; and " Entering freshmen SAT scores. T he measures used here provide quite reliable indicators of institutional performance even when they do not capture all of that performance. Page 10 TheCenter evaluates public and private universities in the same way, using exactly the same data. We present the performance of public and private universities separately, however, because the public and private research universities operate in significantly different contexts by virtue of their governance and funding structures. Private universi- ties tend to have much larger endowments than public universities, while public institutions enjoy a much higher level of tax-based public support. Public universities tend to serve much more diverse constitu- encies in ways that affect their size and organization. Private universities often focus their efforts more closely and define their missions more precisely. The goal of this analysis is to identify research university performance, not to analyze relative funding or governance. Public and private institutions compete for the same research grants, the same faculty talent, the same high quality students, and in a similar fashion for private annual giving. The top categories of American research universities include both public and private institutions, and TheCenter conducts the evaluation of top universities without regard to ownership, although it presents the results for public and private universities separately. Because we believe that the top universi- ties have strength in research, private sup- port, faculty, graduate and post-graduate programs, and quality undergraduates, the methodology we use for the evaluation considers all nine indicators described above. At the same time, the precise ranking of a university on these indicators is less impor- tant than their inclusion within the top groups. For this analysis, we defined the top category in terms of the performance of the top 25 public and the top 25 private institu- tions on each indicator. To create the groups of universities, we identified the universities that ranked among the top 25 on each of the nine measures, again taking public and private institutions separately. We then grouped the institutions by the number of indicators for which their performance put them in the top 25. Obviously, the choice of 25 as the top quality cohort is somewhat arbitrary. A smaller definition of the top cohort would have included fewer institu- tions and would also have left out some clearly significant research universities. A larger cohort would have created groups that, upon closer inspection, do not always share reasonably equivalent levels of quality. The top category in the public and private lists, then, includes universities that rank in the top 25 on all nine of the indicators. These institutions have high levels of research funding (total and federal), substan- tial endowments and strong programs of annual giving, excellent faculty in the sciences and in the humanities and social sciences, strong doctoral and postdoctoral programs, and outstanding undergraduate students. The second group includes universities with eight of the nine indicators in the top 25, and so on for the rest of the groups in the public and private lists. For the purposes of this analysis, TheCenter includes only research universities with at least $20 million in federal research expenditures per year. This number is somewhat less than the Carnegie Classifica- tion cutoff for Research I ($40 million) and somewhat more than Carnegie used for Research II ($15.5 million). Indicators of Performance P ublic and private institutions compete for the same research grants, the same faculty talent, the same quality students, and in a similar fashion for private annual giving. The Top American Research Universities Page 11 Forty-seven public universities and thirty- five private institutions have at least $20 million in federal research and appear in the top 25 on at least one of the measures. These 82 institutions meet our criteria and thus appear in the lists. Each of the criteria, described in detail below, contributes to an understanding of the breadth of performance needed for a top research university. Total and Federal Research Expenditures Even with research, however, we must settle for something less than a measurement of an institution 9s total research and creative productivity. The only comparable and reliable indicators of university research measure the dollars spent by the institution from research grants and contracts. These measures, while expressed in mostly compa- rable terms for all institutions, are less a complete measurement of the university 9s research than they are representative of that research. The reason for this distinction is that the dollar numbers for total and federal research expenditures ( TheCenter uses both measures) do not reflect many other kinds of significant university research. The data used come from the NSF annual Survey of Scientific and Engineering Expendi- tures at Universities and Colleges. They explicitly exclude non-science and engineer- ing research in such fields as law, education, humanities, business, fine arts, and journal- ism. While historians, poets, literary scholars, some social scientists, and most artists and composers, for example, produce exceptional research and creative products, these activities do not appear in the indica- tors of total or federal research because of the methodology defined by NSF 9s survey. An additional element involves the mix of disciplines even within the externally funded marketplace of science and engineering. Research in experimental physics, for example, requires large grants to deliver modest results. Research in theoretical physics or mathematics, in contrast, may well produce significant results with rela- tively small grants. Meanwhile, federal preferences for physical or biological science research may shift funding opportunities differentially among institutions. Finally, some forms of research in professional and other programs compete in an external marketplace that does not involve the university. For example, faculty in a business or engineering school may develop Median Total Research, 1998 Private and Public Universities (x $1,000) Private Public 9 8 7 6 5 4 3 2 1 $0 $50,000 $100,000 $150,000 $200,000 $250,000 300,000 Number of Times in Top 25 Total Research 9 8 7 6 5 4 3 2 1 $0 $50,000 $100,000 $150,000 $200,000 $250,000 $300,000 $350,000 $400,000 450,000 Number of Times in Top 25 T he only comparable and reliable indicators of university research measure the dollars spent by the institution from research grants and contracts. Page 12 9 8 7 6 5 4 3 2 1 $0 $500,000 $1,000,000 $1,500,000 $2,000,000 $2,500,000 $3,000,000 3,500,000 Private Public Number of Times in Top 25 Endowment Assets Median Endowment Assets, 1999 Private and Public University Groups (x $1,000) research products within the context of relationships that pay consulting fees and reimburse expenses, which do not become part of the university 9s accounting system for grants and contracts. Although these issues make the total and federal research numbers incomplete repre- sentations of research competitiveness, they nonetheless serve as good measures of an institution 9s overall commitment to and success in research. The numbers help us to understand the strength of research universi- ties and provide two of the elements for grouping institutions. TheCenter 9s approach to identifying top universities creates groups of institutions that demonstrate equivalent strength rather than sorting the institutions on a composite, weighted numerical scale. While federal research expenditure is a relatively straightforward measure, the total research number requires some explanation. Total research includes all those expenditures on research reported by the university to NSF, including corporate, state, and local as well as federal sources. This number creates some potential for differential reporting by institution depending on the definition of local and state expenditures for research, but for the purposes of this clustering approach, the possible error does not appear too great. This research measure captures an important element of research for many institutions that have a large corporate support structure for their research or a mission that includes agricultural research funded by the state through a land-grant system. Private Support The total financial resources of universities prove difficult to measure accurately given the wide diversity of mission and the varying structure of public and private funding sources in American research universities. Endowment Indicators of Performance 9 8 7 6 5 4 3 2 1 $0 $50,000 $100,000 $150,000 $200,000 $250,000 Number of Times in Top 25 Federal Research 9 8 7 6 5 4 3 2 1 $0 $50,000 $100,000 $150,000 $200,000 $250,000 Number of Times in Top 25 Median Federal Research, 1998 Private and Public Universities (x $1,000) Private Public The Top American Research Universities Page 13 9 8 7 6 5 4 3 2 1 $0 $50,000 $100,000 $150,000 $200,000 250,000 Private Public Number of Times in Top 25 Annual Giving assets capture a stable and common element in the financial resources of all research universi- ties, both public and private. While private universities tend to have an economy that relies significantly on tuition revenue and endow- ment income, and public institutions receive significant tax-based support, all research universities devote considerable effort to raising private dollars. The endowments of public universities do not yet approach the level of private institutions, but within the context of public higher education, a university 9s endow- ment represents a significant source of revenue in support of research and quality education. This source of revenue is even more significant in the context of private research institutions. When looking at public and private universi- ties separately, endowment serves as a useful indicator of an institution 9s available resources. Although endowments represent stable resources, their value at the end of each fiscal year also reflects the investment wisdom of managers and the portfolio composition of institutional endowment funds. Endowment reflects generations of gifts and the investment growth of those gifts, not necessarily the current work of the university. TheCenter , then, also includes annual giving as one of its measures. All research universi- ties commit themselves to the task of raising private money, and success in this competi- tion serves as a useful indicator of the institution 9s ability to mobilize financial support from its many constituencies. While these two measures serve as good references for institutional financial strength, they do so only within the separate contexts of public and private universities. Private universities with large endowments may appear better supported than they actually are in comparison to public universities with large tax-based contributions. Further complicating an evaluation of total financial strength, public and private universities often have very different mechanisms for acquiring capital investment for buildings and for funding the depreciation cost of those physical assets. The measures of private support identify the success of the university in persuading its various constituencies that its programs represent a good investment. Faculty If research and private resources provide key measures for identifying America 9s top research universities, some other characteris- Median Annual Giving, 1999 Private and Public University Groups (x $1,000) 9 8 7 6 5 4 3 2 1 0 10 20 30 40 50 60 70 80 90 Private Public Number of Times in Top 25 Faculty in National Academies Median Number of Faculty in National Academies, 1999 Private and Public University Groups F aculty quality, of course, is the primary source of the institution 9s strength as a competitive academic enterprise. Page 14 tics offer additional evidence of institutional quality for this analysis. Faculty quality, of course, is the primary source of the institution 9s strength as a competitive academic enterprise. While the research numbers offer an indication of the faculty 9s ability to compete for grants and contracts, the honors and awards of the faculty provide a somewhat different perspective on the institution 9s distinction and capture some elements of quality not reflected in the data on research expenditures. TheCenter uses two measures of faculty quality: membership in the three National Academies (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine); and the number of faculty receiving a range of academic awards in the sciences, social sciences, humanities, and health professions. The A ppendix lists the awards included in this analysis. Advanced Training Research universities not only produce research, they also make a major contribu- tion to the education and training of the next generation of researchers. As an indicator of a university 9s participation in this activity, TheCenter counts the number of doctorates awarded and the number of postdoctoral positions supported. These measures serve as indicators of the strength of an institution 9s graduate and post-graduate education and research training activities. The number of postdoctoral appointees also reflects the strength of medical school research programs that often support many postdoctoral positions. 9 8 7 6 5 4 3 2 1 0 100 200 300 400 500 600 700 Private Public Number of Times in Top 25 Doctoral Degrees Median Number of Doctorates, 1998 Private and Public University Groups 9 8 7 6 5 4 3 2 1 0 10 20 30 40 50 60 Private Public Number of Times in Top 25 Faculty Awards 9 8 7 6 5 4 3 2 1 0 100 200 300 400 500 600 700 800 Private Public Number of Times in Top 25 Postdoctoral Appointees Median Number of Faculty Awards, 1999 Private and Public University Groups Median Number of Postdocs, 1998 Private and Public University Groups Indicators of Performance The Top American Research Universities Page 15 entering class. Of these, the median SAT scores of the entering class offer the best general indicator of undergraduate quality. The number of National Merit and National Achievement Scholars varies depending partly on the size of the undergraduate population and partly on institutional policies that award special financial aid and scholarships to these students. The median SAT, while not a complete measure of student quality, is relatively standard because most institutions use it as part of the admis- sions process, and it is also less influenced by differences in undergraduate population size or financial aid practices. The median SAT scores for the top private universities are much higher than the scores for the top publics, reflecting the mission of public universities to provide access to a greater number of students. The Purpose of The Top Universities T heCenter 9s interest in this topic comes from the experience of observing universities and their supporters as they pursue improvement programs. Many universities want to get better, to improve their standing among their research univer- sity colleagues, and they have a keen interest in the variables that determine institutional performance. Traditional rankings that put universities in order by some weighted index of prestige, resources, or other categories do not help institutions to understand what makes research universities succeed. Some- times the rankings fail to serve a useful purpose because they use inappropriate criteria. Primarily, however, the difficulty comes from the ranking and weighting process that, in its complexity, obscures the Median SAT Scores, Fall 1999 Private and Public University Groups 9 8 7 6 5 4 3 2 1 950 1,050 1,150 1,250 1,350 1,450 Private Public Number of Times in Top 25 SAT Scores Undergraduates While almost all of America 9s most successful research universities serve under- graduate student populations, the variation on this dimension is large. Public land-grant universities, for example, may have 30,000 undergraduates; smaller private universities may have 1,500 to 3,000; and specialized academic medical centers may have no undergraduates at all. Although TheCenter includes specialized medical centers in its evaluations since they are major competitors for faculty and research support, we make the judgment that a quality undergraduate program is an essential feature of America 9s top research universities. The quality of undergraduate programs proves difficult to measure directly. The data on placement rates, persistence rates, and the like are often unreliable and difficult to acquire in consistent ways. These and other calculations, such as graduation rates, also fluctuate as a function of size, mission, geographic location, and ownership rather than as a function of quality or effectiveness. We considered two possible measures of undergraduate quality: the median SAT scores and the number of National Merit and National Achievement Scholars in the Page 16 relative strength of the institution 9s many elements. In addition, weighted rank ordering 3 while it presents an easily referenced list 3 does not capture the complexity of American research university mission and performance. These rankings give the false impression that the precise order of institutions reflects precise differences. The very best universities excel in almost everything; very good universities excel in some things and perform less effectively in others. Aspiring research universities do well, but not at a level close to the top performers. Successful research universities must have a constant, continuing commitment to competition and performance. Assertions about performance aspirations rarely have any effect unless accompanied by some sense of where an institution fits into the competi- tive structure of American higher education and unless supported by measurable indicators of comparative performance. TheCenter 9 s Top Universi- ties provide that context and offer universities a reference for measuring their own achievement and clearly understanding the nature of the competition. When over-enthusiastic people assert institutional goals, such as reaching the top ten of American research univer- sities by some not-too-distant date, they usually do so without understanding what this achievement actually entails. Research universities live in a highly competitive marketplace, and none of those in the top categories is likely to cease improving. This means that to get relatively better, a univer- sity must match and then exceed the growth of its competitors. This is a major challenge, and the indicators in these tables provide explicit reference points to measure this kind of success. Although universities improve and decline in performance relative to each other, the patterns of change are much different in the top group than in the groups nearer the bottom of the table. In terms of federal research, for example, over a ten-year period, universities in the top groups change posi- tion infrequently. Members of these groups may move up or down by one position at most. In the bottom groups, however, universities change position by much larger margins. This pattern reflects the increasingly greater intensity of the competition towards the top. Universities with $20 million of research can receive a few major grants and increase their spending by one or two million dollars over ten years and still improve their position, while other universi- ties at similar levels of funding can easily lose the same amount of funding and decline. Institutions at the top, with $300 million or 1-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100 0 2 4 6 8 10 12 14 16 Clusters of Universities by Rank in 1990 Rank Change (up or down) Average Absolute Change in Rank in Federal Research between 1990-1998 Top 100 Universities in 1990 The Purpose of Top Universities S uccessful research universities must have a constant, continuing commitment to competition supported by measurable indicators of comparative performance. The Top American Research Universities Page 17 Top Group 24.6% All Other Universities 27.7% Other Top Universities 47.8% Market Share of All Federal Research Private and Public Universities: Top Group, Other Top Universities, All Other Universities more of research, have so many people engaged in the research enterprise at such a high level that they rarely rise or decline much more than the other institutions in their group. This is partly because the scale of their research operations is so large that failures to win grants balance the successes in the acquisition of new grants. This group of universities also controls a large share of the federal research market. The relatively few universities identified by TheCenter as the very top group of universi- ties (14 private and public universities) have 24.6% of the total federal research expendi- tures of all universities receiving federal funds. The other top universities in this study (68 private and public institutions) control 47.8% of the market, leaving all other private and public universities with a 27.7% market share. From another perspec- tive, the 82 top universities included in this study have a 72% share of the total federal research expenditures reported by NSF for all universities in the country. The size of this group 9s participation in the research market- place creates significant barriers to challenges from rising institutions, whether from outside the group included in this study or from the institutions included here but located at a considerable distance from the top group of institutions. Another way of looking at this barrier is to isolate the federal research dollars among these very competitive institutions. The number 10 private university has about $187 million and number 25 has $80 million. To move from number 25 to number 10 in research performance would require the number 25 institution to more than double its research base. This would have to come, of course, from the market share of other institutions. On the public side, the number 10 public institution has $169 million and number 25 has $97 million. For the number 25 public university to move into the top ten will require an increase of 174%, again an increase that would have to come at the expense of other highly competitive institu- tions. N > 450 N = 14 N = 68 In the case that all universities are equally successful in gaining grants (which means that they all increase their grant volume by the percentage increase of the total pool), the top group of universities will continue to grow faster in total volume than the bottom groups. However, many universities in the Federal Research Public Universities " " v v #10 #25 $187 Million $80 Million 234% growth to move to #10 " " v v #10 #25 $169 Million $97 Million 174% growth to move to #10 Private Universities Page 18 lower brackets grow faster in percentage terms than those in the upper brackets. This narrows the gap somewhat between the top institutions and those substantially below them in federal research. Recent increases in federal research spending have hovered around 8% per year. The rate of change required for a number 25 institution to make it into the top ten within ten years ap- proaches 28% per year for a private univer- sity and 24.5% per year for a public institu- tion. This represents a very challenging task and also explains the continued success of the top performers among research universi- ties and the relative stability of American research university reputations. If the competition at the top level seems daunting, movement at lower levels of the hierarchy is also challenging, despite the smaller margins of change. Among the private universities in this analysis, the institution with the least amount of federal research expenditures has about $23 million, and the number 25 institution has $80 million. For the last institution to reach the level of the number 25 institution, the faculty would need to more than triple their research productivity. For public institu- tions, the smallest federal research volume in this group is about $29 million and the 25 th is $97 million, presenting the faculty of the public institution with a similar challenge of a more than threefold increase in research productivity. Although large changes in the rank ordering of universities on many of these criteria appear difficult, smaller changes of one to three or four places on the list are well within the competitive capabilities of most institutions. Thus, a university that moves up from 25 to 23 in the federal research list has beaten some formidable competition. The university that sets a goal of moving from 25 to number 10 is probably engaged more in public relations than in academic competition or planning. Similar calculations would produce similar results for other indicators in this study, although the dynamics differ. For example, the data show considerable volatil- ity in the annual giving category as universi- ties launch and complete successful capital campaigns. Even so, the range separating the fundraising capabilities of the top universi- ties in this category from those in the middle is even larger than the range for research. For the most recent year, the number 10 private institution raised about $208 million and number 25 brought in $78 million; the number 10 public institution gained about $148 million with number 25 raising about $77 million. For the 25 th private university to achieve the fundraising success of the 10 th most successful private institution, it would need to increase its annual giving by over two and a half times. For their public counterparts, the increase would need to be just under two times. Here, as in the case of research funding, the leading institutions do not stand still, but increase their annual giving every year. As a result, competitors The Purpose of Top Universities Annual Giving Private Universities Public Universities " " v v #10 #25 $208 Million $78 Million 266% growth to move to #10 " " v v #10 #25 $148 Million $77 Million 193% growth to move to #10 The Top American Research Universities Page 19 need not only to improve their own perfor- mance, they need to improve it by a factor larger than the improvement of their com- petitors. We believe that universities have an organizational model that emphasizes self- replication. Institutions with large numbers of competitive faculty and students tend to replace these faculty and students with individuals of equivalent competitiveness. Those with less competitive faculty also replace themselves with less competitive faculty. Overall, and absent a strong drive for change, most institutions stay more or less the way they are: stable, competitive at their level, but unlikely to move dramatically without significant and unusual impetus. A bsent a strong drive for change, most institutions stay more or less the way they are: stable, competitive at their level, but unlikely to move dramatically without significant and unusual impetus. This project to identify the top American research universities provides a frame of reference and the data to understand the structure of this segment of American higher education. This publication captures the current condition of these institutions, and subsequent editions will update the data as they become available. No observer is limited to the decisions and evaluations used here, for TheCente r 9s web site provides all the data so others can construct and analyze the information for their own purposes. As the work of TheCenter continues, additional publications will look at the process of change over the past decade that has produced the structure of research institutions outlined here. Page 20 California Institute of Technology 9 $ 185,066 14 $ 177,748 11 $ 1,333,229 21 Columbia University 9 $ 267,007 10 $ 229,723 6 $ 3,636,621 8 Duke University 9 $ 282,388 7 $ 172,532 12 $ 1,678,728 17 Harvard University 9 $ 306,100 6 $ 251,876 4 $ 14,255,996 1 Johns Hopkins University 9 $ 853,620 1 $ 752,983 1 $ 1,520,793 19 Massachusetts Institute of Technology 9 $ 413,098 2 $ 310,741 3 $ 4,287,701 6 Northwestern University 9 $ 223,235 12 $ 127,911 15 $ 2,634,850 13 Stanford University 9 $ 410,309 3 $ 342,426 2 $ 6,005,211 4 University of Pennsylvania 9 $ 333,477 5 $ 247,914 5 $ 3,281,342 9 Yale University 9 $ 262,680 11 $ 205,046 7 $ 7,197,900 2 Cornell University 8 $ 363,511 4 $ 204,187 8 $ 2,869,103 11 New York University 8 $ 156,452 18 $ 101,426 22 $ 1,035,900 24 Princeton University 8 $ 115,996 25 $ 69,005 27 $ 6,469,200 3 University of Chicago 8 $ 151,635 19 $ 125,982 16 $ 2,762,686 12 University of Southern California 8 $ 268,806 9 $ 190,547 9 $ 1,589,833 18 Washington University 8 $ 269,550 8 $ 187,173 10 $ 3,761,686 7 Vanderbilt University 7 $ 135,214 22 $ 106,325 19 $ 1,831,766 15 Case Western Reserve University 6 $ 176,330 15 $ 132,274 13 $ 1,434,200 20 University of Rochester 6 $ 174,617 16 $ 130,773 14 $ 1,119,027 23 Baylor College of Medicine 5 $ 216,528 13 $ 110,610 18 $ 1,029,156 25 Boston University 5 $ 130,054 23 $ 104,428 20 $ 652,161 46 Carnegie Mellon University 5 $ 137,450 20 $ 95,046 23 $ 719,320 39 Emory University 5 $ 172,884 17 $ 118,045 17 $ 4,475,755 5 Brown University 4 $ 73,977 35 $ 44,412 36 $ 1,181,514 22 Dartmouth College 4 $ 64,964 37 $ 45,053 35 $ 1,710,585 16 Georgetown University 3 $ 116,611 24 $ 84,801 24 $ 684,193 40 Rice University 3 $ 41,067 42 $ 34,772 40 $ 2,936,622 10 Rockefeller University 3 $ 115,494 26 $ 43,845 37 $ 1,007,600 27 University of Miami 3 $ 136,972 21 $ 101,492 21 $ 428,571 64 University of Notre Dame 3 $ 28,873 48 $ 23,053 47 $ 1,984,256 14 Brandeis University 2 $ 44,589 41 $ 28,098 43 $ 355,012 78 Tufts University 2 $ 92,130 30 $ 61,167 29 $ 464,107 61 Yeshiva University 2 $ 99,000 28 $ 80,000 25 $ 674,833 44 George Washington University 1 $ 74,481 34 $ 45,072 34 $ 673,589 45 Thomas Jefferson University 1 $ 69,460 36 $ 51,728 33 $ 384,973 71 Top Private Universities Research Universities by Number of Measures in Top 25 of All Private Universities (Alphabetically within Groups) No. of Measures in Top 25 Total Research 1998 x $1,000 Rank Total Research Federal Research 1998 x $1,000 Rank Federal Research Private Support Endowment Assets 1999 x $1,000 Rank Endow Assets Top Private Universities Page 22 University of California - Berkeley 9 $ 420,435 5 $ 171,135 9 $ 1,654,557 3 University of California - Los Angeles 9 $ 447,367 2 $ 233,702 5 $ 1,103,038 7 University of Michigan - Ann Arbor 9 $ 496,761 1 $ 311,450 2 $ 2,424,588 2 University of North Carolina - Chapel Hill 9 $ 235,296 19 $ 171,505 8 $ 925,746 10 Pennsylvania State University - University Park 8 $ 319,126 11 $ 163,921 14 $ 712,967 17 University of Florida 8 $ 274,862 15 $ 106,510 23 $ 601,813 21 University of Illinois - Urbana-Champaign 8 $ 338,841 10 $ 168,871 10 $ 612,430 20 University of Minnesota - Twin Cities 8 $ 345,910 9 $ 204,741 7 $ 1,283,934 5 University of Washington - Seattle 8 $ 432,383 4 $ 336,748 1 $ 745,217 16 University of Wisconsin - Madison 8 $ 443,695 3 $ 240,513 4 $ 909,834 11 Texas A&M University 7 $ 393,720 7 $ 144,938 16 $ 3,596,759 1 University of California - San Francisco 7 $ 379,970 8 $ 219,912 6 $ 701,933 18 University of Iowa 7 $ 199,063 25 $ 115,312 20 $ 476,800 22 University of Texas - Austin 7 $ 244,843 18 $ 165,082 13 $ 894,113 13 Georgia Institute of Technology 6 $ 259,233 16 $ 113,643 22 $ 948,600 9 Ohio State University - Columbus 6 $ 301,518 13 $ 124,177 19 $ 1,086,350 8 Purdue University - West Lafayette 6 $ 216,479 23 $ 92,844 27 $ 1,222,411 6 University of Arizona 6 $ 302,328 12 $ 161,999 15 $ 272,950 42 University of California - Davis 6 $ 288,796 14 $ 114,912 21 $ 300,828 39 University of California - San Diego 6 $ 418,790 6 $ 262,303 3 $ 200,552 64 University of Pittsburgh - Pittsburgh 6 $ 213,842 24 $ 168,511 11 $ 854,840 14 University of Virginia 6 $ 133,049 46 $ 93,328 26 $ 1,398,068 4 University of Maryland - College Park 5 $ 223,190 21 $ 129,198 18 $ 314,183 38 University of Utah 5 $ 142,956 37 $ 100,722 24 $ 269,430 43 Rutgers the State University of NJ - New Brunswick 4 $ 137,937 43 $ 48,880 57 $ 350,741 30 University of Colorado - Boulder 4 $ 186,211 27 $ 137,241 17 $ 195,585 66 University of Texas SW Medical Center - Dallas 4 $ 153,711 32 $ 97,200 25 $ 406,415 26 University of Alabama - Birmingham 3 $ 227,720 20 $ 166,830 12 $ 205,860 59 Indiana University - Bloomington 2 $ 68,702 80 $ 38,336 69 $ 400,000 27 Michigan State University 2 $ 193,611 26 $ 81,146 35 $ 265,238 45 North Carolina State University 2 $ 254,254 17 $ 79,533 37 $ 275,532 41 University at Stony Brook 2 $ 141,766 38 $ 91,531 28 $ 22,383 171 University of California - Irvine 2 $ 130,415 47 $ 65,902 44 $ 100,276 96 University of Georgia 2 $ 217,945 22 $ 54,712 53 $ 334,534 35 University of Nebraska - Lincoln 2 $ 118,857 50 $ 41,888 66 $ 416,340 25 Iowa State University 1 $ 156,766 31 $ 51,196 55 $ 266,348 44 University at Buffalo 1 $ 151,650 34 $ 76,037 39 $ 438,002 23 University of California - Santa Barbara 1 $ 96,034 57 $ 68,408 43 $ 100,276 96 University of Cincinnati - Cincinnati 1 $ 159,695 30 $ 90,307 29 $ 898,976 12 University of Colorado Health Sciences Center 1 $ 121,624 49 $ 89,022 30 $ 97,793 99 University of Delaware 1 $ 69,896 79 $ 33,688 73 $ 777,349 15 University of Illinois - Chicago 1 $ 151,739 33 $ 73,797 40 $ 204,143 60 University of Kansas - Lawrence 1 $ 66,756 82 $ 28,823 80 $ 613,338 19 University of Texas MD Anderson Cancer Center 1 $ 141,260 39 $ 63,074 45 $ 162,566 71 University of Texas Medical Branch - Galveston 1 $ 86,488 67 $ 48,588 58 $ 243,849 51 Virginia Polytechnic Institute and State University 1 $ 167,118 28 $ 82,734 33 $ 340,244 33 Washington State University - Pullman 1 $ 95,422 58 $ 44,510 61 $ 421,402 24 Top Public Universities Research Universities by Number of Measures in Top 25 of All Public Universities (Alphabetically within Groups) No. of Measures in Top 25 Total Research 1998 x $1,000 Rank Total Research Federal Research 1998 x $1,000 Rank Federal Research Endowment Assets 1999 x $1,000 Rank Endow Assets Top Public Universities Private Support Page 24 Other Data Elements T he purpose of this study is to talk about research universities. The issue that matters for us is to discover measures that identify America 9s strongest research universities. The emphasis on this perspec- tive is critical, for universities have many other values and products. Some of us may want to know which universities have the most effective and ethnically diverse under- graduate programs, which ones place most of their students in high paying jobs, which institutions have the best programs in community service, which institutions do the best job of teaching values. Those are separate questions not addressed here. TheCenter recognizes that the measures in the preceding tables do not fully capture the range of institutional characteristics that may be of importance to some audiences. For that reason, although we use the nine measures for the identification of the Top Research Univer- sities, we also collect additional data on a wide range of other institutional characteristics related largely to questions of the size and composition of the various institutions. TheCenter provides the data used in the Top Research Universities evaluation and the other data elements in files, accessible from TheCenter web site [http://thecenter.ufl.edu], so that colleagues can develop their own evaluations and emphasize issues of importance for particular institutional priorities. Within this context, TheCenter provides data online for all universities with over $20 million of federal research expenditures. The data in this set include the following indica- tors, in addition to the nine measures: institutional ownership (private or public); medical school (yes or no); enrollment (undergraduate, professional, graduate by full- and part-time); and National Merit and National Achievement Scholars. Size and Composition TheCenter does not use the institution 9s absolute size as an indicator, whether in terms of budget or undergraduate student population or total faculty and staff. Such institutional characteristics are important in other contexts, but our focus is on elements of research quality and productivity. This is a complex issue, of course, because large universities often have resources unavailable to smaller institutions to compete for quality faculty. At the same time, large universities, and particularly public institutions, often perform a wide range of functions that do not contribute to the research distinction of the institution. They may do agricultural extension; they may perform community 9 8 7 6 5 4 3 2 1 0 10,000 20,000 30,000 40,000 Private Public Number of Times in Top 25 Total Enrollment Median Total Enrollment, Fall 1998 Private and Public University Groups Median Percent Graduate and Professional Fall 1998 Enrollment Private and Public University Groups 9 8 7 6 5 4 3 2 1 20% 30% 40% 50% 60% Private Public Number of Times in Top 25 Graduate and Professional Enrollment Other Data Elements The Top American Research Universities Page 25 service; they may teach large numbers of undergraduates; and they may have signifi- cant programs in various forms of continu- ing or distance education. All these repre- sent important and useful services, and although they often require faculty to support the effort, their existence does not necessarily increase or decrease the academic research quality of the institution. In addition, while there is clearly a difference between the percentage of graduate and professional students in private and public universities, the differ- ences, once the institution is above the mid 20% range for publics and the 40% range for privates, most likely reflect disciplinary emphasis and distribution rather than differential success as research institutions. Schools and Colleges Other institutional characteristics attract our attention as well. Some research institu- tions have medical schools and own hospitals, others have medical schools only, and some have no medical school at all. While a medical school can be a source of outstanding research faculty and produce a considerable volume of high quality research, not all universities that have medical schools become significant research institutions and some universities without medical schools compete very effec- tively on academic quality and productivity. In addition, the distribution of disciplinary focus will vary depending on how a university organizes its faculty and delivers its instruction and research. Institutions with strong research programs in agriculture, engineering, and medicine tend to have many of their research faculty in the life and other sciences located in these programs rather than in colleges of arts and sciences. Institutions without those pro- grams will see much more research in colleges of arts and sciences. In some institutions, economics research takes place in business schools; in others it occurs in the economics department of arts and sciences colleges; and in some, it takes place in all of these as well as in certain programs in agriculture. American research universities vary consid- erably in their internal organization so that metrics focusing on faculty size, student size, specific college productivity, and the like, will often prove less useful than anticipated. Sources of Funding Universities also vary considerably in the sources of funding and the size of institutional budgets. Large public research universities can often have total budgets of approximately $1 to $2 billion while outstanding private research universities may have budgets that do not reach a billion dollars. Again, these differences represent many things, only some of which affect the research productivity and the quality of the institution. Universities may support extensive athletic programs with budgets that reach into the $30 million range and beyond. Public institutions may carry on agricultural extension work or support statewide library services, and in many cases, they teach very large undergraduate student populations. All of these functions increase the budget and faculty size, but they often do not enhance the university 9s research capability. If an institution owns and operates a hospital, that budget may form part of the institutional resources. Public universities often receive substantial income from public funds, while their private counterparts draw more heavily on income from large endow- ments and high tuition. Even within the category of public universities, the distribu- R esearch universities vary considerably in their organization so that metrics focusing on faculty size, student size, and college productivity will often prove less useful than anticipated. Page 26 tion of funding sources between public funds, private gifts and endowments, and student tuition varies widely. Comparisons of such indicators as budget per student or budget per faculty member, because they combine many dissimilar entities and purposes, produce data of marginal useful- ness and deceptive meaning. Another source of revenue comes from earnings on patents, licenses, and royalties. While some institutions have a diversified portfolio of patents, licenses, and royalties that reflects the breadth of their research, many other institutions with significant revenue in this category rely on a very few exceptionally successful items. Moreover, in some cases, large income reflects the value of a trade name, on a royalty basis, rather than the value of a scientific invention based on a patent. Patents, of course, have a limited lifespan, but royalties on trademarks last as long as a market exists for the branded product. The research strength of the institution appears to be more accurately reflected in the total and federal research expenditure indicators. Institutional Reputation Then there is the question of reputation. Many university rankings rely on various forms of reputation assessment. Usually based upon survey data, the reputation of a university comes from the opinions of presumably informed academic observers. In most cases, the reputations of institutions in the top rank of American universities have a basis in performance, often with reference to undergraduate student quality and undergraduate program prestige that may very well match objective data. However, when a reputation survey includes more than twenty or so institutions, we should view the validity of the ranking with considerable skepticism. Most academ- ics do not know much about more than ten or fifteen universities. They may have a good idea about Berkeley, Michigan, Yale, Harvard, Hopkins, and Illinois (and a variety of small prestigious liberal arts colleges such as Grinnell, Pomona, or Swarthmore). Many observers, however, may not have a clear understanding of the differences between the University of California at Santa Barbara and the University of California at Santa Cruz or between Brown and Rice. Furthermore, much opinion about university quality reflects wisdom acquired at one point in time and rarely reassessed. For example, once people have a clear sense that Berkeley is a great institution, they rarely reassess this judgment over time by reviewing any objective data about Berkeley 9s subse- quent performance. Is Berkeley getting better, is it declining, and if so, on what basis do we make such a judgment? In such surveys, the opinions generated often do not reflect the actual current performance of the institution. Reputation rankings are not necessarily wrong; they are just unreliable and insensitive to institutional change. For America 9s best research universities, this may not matter, since these institutions do not change much over time. However, for the institutions below the top ten or fifteen public or private institutions, and for the many more quality universities not included in this study, reputation does not accurately reflect either performance or improvement. Average Faculty Productivity Some have approached the issue of evaluat- ing universities from the perspective of average productivity; that is, how much research or other productivity does an institution generate per faculty member. Such analysis, while attractive in theory, fails in practice. As outlined above, universities differ dramatically Other Data Elements I n reputational surveys, the opinions often do not reflect actual current performance. They are not necessarily wrong; they are just unreliable and insensitive to institutional change. The Top American Research Universities Page 27 in their size and composition; in the functions they perform; and in the ways they define faculty and non-faculty. Reliable data for an approximation of average faculty productivity by institution simply do not exist. The major source of data for faculty numbers is IPEDS (Integrated Postsecondary Education Data System, U.S. Department of Education), and institutions should theoretically report their faculty numbers using the same criteria. Unfortunately, institutional definitions of faculty categories differ so much that not only are the data inconsistent between institutions, but the definitions used vary within each institution by program and by year. Some subsets of undergraduate colleges with similar student populations and similar disciplinary distributions might find such an analysis of value, but for America 9s research universities, this kind of analysis obscures more than it reveals. By using inappropriate measures, such a methodology catapults unexpected institu- tions to high ranks by virtue of a different system of counting faculty than is used by other institutions in the same cohort. To give but a simple example, imagine two universities with $100 million in research grants and contracts. Both report 2,000 faculty members, and in productivity terms, they each generate $50,000 of research per faculty member. Are they equally produc- tive? No, because one university counts librarians in its faculty and the other one does not. This example is but a one-dimen- sional illustration of the complex reality that underlies the deceptively simple data element: number of faculty. National Merit and Achievement Scholars The National Merit Scholarship Corpora- tion (NMSC) is an independent, non-profit organization that awards scholarships to the nation 9s outstanding high school seniors based on their academic achievement; qualifying test scores; high school principal and counselor recommendations; and their activities, inter- ests, and goals. The NMSC names approxi- mately 14,000 National Merit Finalists each February. Of these, about one-half will receive a National Merit

less