Critical Review (Brown University)

In 1976, just seven years after Brown's "New Curriculum" had been announced, The Other View: A Consumer's Guide to Courses at Brown was started by a group of Brown undergraduates as a publication of the Undergraduate Council of Students. In this annual magazine, students wrote reviews of courses based on responses to both the organization's own questionnaires and also based on departmental questionnaires which were made available by several departments. The written evaluations were short, and the numerical ratings sections, when presented, consisted of only averages. Department Undergraduate Groups were essential to the process because they distributed and collected questionnaires from classes and helped to develop customized questionnaires for their respective departments.

There was no set process for putting the magazine together, and since the questionnaires were different for different classes, the course summaries varied significantly in terms of what information they presented and emphasized. "There must be a better way than having three people slave over questionnaires in the UCS office for two summer months," wrote the editors on the cover page of one of the first editions, "A more rational, uniform process must be developed." Sometime between 1978 and 1981, the name was changed to The Critical Review: A Consumer's Guide to Courses at Brown, and then to simply The Critical Review a few years later. The number of courses reviewed in each edition varied considerably from year to year. The editors changed the procedure for compiling the magazine annually. Co-Editors-in-Chief Tom Mashberg '82 and Ian Maxtone-Graham '82.5 led one such overhaul.

In 1984, recently appointed Editor-in-Chief Rob Markey sought feedback from students and faculty and then overhauled the system in time to create a 1985–1986 Semester I edition, the first issue of the Critical Review to cover a single semester. The editors introduced the now standard two-reviews-per-page format, with bar graphs, which were meant to depict the general distribution of student responses. The bar graphs were implemented in order to improve the accountability of the Critical Review by balancing the written summaries with objective data from the student surveys and by presenting sample sizes, so that professors would know the reviews were not based on the comments of just one or two students. The editors also created customized software to make it easy to produce the bar graphs. They developed a matching questionnaire to generate the student input on the courses in quantitative form. However, at this point the Critical Review still relied on professors to distribute its questionnaires, which some refused to do or did haphazardly. The publication also accepted departmental evaluation forms as an alternative to its own questionnaire. As a result, bar graphs were not available for many classes. Nevertheless, having found a satisfactory and manageable system to which Brown students and faculty responded favorably, subsequent editors changed relatively little about the format, process, and policy of the Critical Review over the next few years.

Mark Popofsky became Editor-in-Chief in 1989 and developed the bar graph system that the Critical Review still uses today. He established new policy for the organization which created lasting precedent: "Because we solely represent student opinion, the Critical Review no longer prints any department-sponsored articles and cannot use departmental evaluations." The editors rewrote the questionnaire and developed new procedure that reduced the time required for the production cycle. As Popofsky wrote, "This issue marks the culmination of what has been a personal crusade of mine for the past year: to publish the Critical Review in time for preregistration." Later editors said that he inspired them to make every effort to turn the Critical Review into a more professional-looking book.

In 1991 the Critical Review began to publish "Insights from Student Surveys," which quickly evolved into a regular feature called "Funny Quotes." The level of faculty participation gradually rose from the 1980s to the early 1990s. The Critical Review began using an instructor questionnaire as well in 1992, "in order to gain the perspective of the faculty." A few semesters after switching publishers to the company TCI Press in order to print higher quality magazines, the number of courses reviewed reached a peak of 425 in the 1993–1994 Semester I edition. The Critical Review began using the program Adobe PageMaker to compose its page layouts at this time, which further improved upon the look of the book.

Despite the efforts and hard work of the editors, the editions of the Critical Review that were published at this time were not flawless. The accelerated production cycle, combined with limited computing resources available to editors, resulted in a few editions of the magazine that contained significant errors. Some instructors stopped participating in the Critical Review because they saw it as unprofessional or sloppy. The gradual upward trend in the number of courses reviewed per semester came to a halt, and the number of reviews per semester gradually decreased until 1998, at which point it flat lined at about 280 reviews per edition.

Christopher Anderson joined the Critical Review around 1993 and rapidly rose through its ranks as the computer consultant. He systematized many aspects of the production process when computing assistance was urgently needed and editors before him and after him identified him as a tremendous help. As editor-in-chief in 1995, he worked with faculty and Critical Review staff to perform a comprehensive analysis and revision of the organization's questionnaires. Despite all the help with the computer aspects that he offered over the years, he identified his new questionnaire as his greatest contribution to the Critical Review.

Rebecca More, Professor of history at Brown and Director of the Harriet W. Sheridan Center for Teaching and Learning, was one of the faculty members who worked with Anderson on this project. "Many of the questions still in use today on the form date from that effort," she says. "Our collective goal at that time was to develop questions which would be constructive and help faculty improve their teaching and student learning." Chris Anderson envisioned the Critical Review as more than just a resource for students. He saw it as a constructive feedback service for faculty as well, and worked toward that end by organizing a "teaching forum" for faculty in the spring of 1995. Unfortunately, the frequency with which the Critical Review published errors at this time, to some extent due to understaffing, prevented many instructors from taking a renewed interest. Nevertheless, Anderson's new questionnaire was used from 1996 to 2001, at which point it was further modified with the help of faculty.

Even as early as 1993, the editors of the Critical Review recognized that the organization should eventually establish a web-based resource. As they wrote, "The goal of putting together an on-line CR connected to the Brown On-line Course Announcement (BOCA) should be discussed." In 1997, the organization finally achieved that goal, thanks to its first website architect, David Tom '98. There was only marginal interest in the Critical Review's website during its first year of existence, however, as printed copies were still available for all undergraduates.

Just as the website was constructed, the Undergraduate Finance Board decided to slice the Critical Review's annual budget from $25,890 to $11,490, the largest budget cut for any single student group that year. Facing a funding decrease themselves of more than $100,000 because of their reliance on diminishing holding funds, UFB deliberately targeted the Critical Review in order to eliminate its printed form.

The Critical Review editors were devastated by this decision. Evan Snyder '00 served as editor-in-chief for the next edition, and he outlined his case for the importance of a printed form of the Critical Review in it. He argued that with just an online format, the Critical Review would not be able to attract as many writers, students and faculty would have more limited access to it, readers would no longer be able to browse it leisurely, and thus it would be "virtually impossible" for readers to discover great courses inadvertently. He added, "In our hyper-frantic rush to produce this book from scratch in a three-week period, the last thing we wanted to be doing was fighting for our existence. And yet this is precisely what circumstances dictate we do." The editors said that faculty would probably have to pay for their own copies from then on, and that they would no longer be able to produce enough copies for every undergraduate.

The Undergraduate Finance Board eventually returned most of the funding that it had taken away. After Evan Snyder graduated in 2000, however, UFB cut the Critical Review's printing budget again, and made the cut effective immediately. With the reduced budget, the circulation of the next edition was 2500 copies, down from 5500 copies three years earlier. In this edition, the editors again presented their argument for the importance of printed copies. "We asked each student who filled out a [questionnaire] to tell us whether she believed that the hard copy of The Critical Review should continue to exist, and the reply was 'yes' in overwhelming numbers [...]. But sadly, this view has not been that of those with the most financial control over the publication."

Interest in the organization rose after 2001, as the number of courses reviewed per semester gradually increased. The number of Critical Review staff members nearly tripled from 2001 to 2004, to the point that for the first time in its history, the Critical Review was able to introduce an application process to become a writer. The organization's website also underwent major revisions from 2002 to 2004, and at the beginning of the spring 2005 semester, thousands of students visited it each day. Students and instructors from other universities, including UC Berkeley, Boston University, Clark University, and the University of Rhode Island, began to contact the Critical Review for advice about how to set up equivalent services at their respective schools.

In a 2004 independent study to examine ways that the organization and service could be improved, co-Editor-in-Chief Nicholas Schade '05 studied available research on student ratings of instruction and sought suggestions from survey methodology experts. He also contacted Critical Review alumni, interviewed several Brown professors, and surveyed hundreds of students to compile a "Report on the Critical Review." Among other things, this document brought to light concerns about the overall goals of the organization, institutional memory, faculty and student perceptions, and persistent internal difficulties. Partly in response to these issues, the editors hosted a "Faculty Forum" in December 2004 to seek suggestions and feedback from instructors about Critical Review policy and the questionnaires themselves. Some professors who had not participated in the Critical Review for years began to participate again.[1] The editors formally rejected the perception that the publication is a service to only the student body by ratifying a group constitution in February 2005, outlining the objectives of the organization with regard to serving both students and faculty of Brown in a professional manner. Schade supplemented his report with other documents to produce a Critical Review "Chief Editor's Resource" in April 2005 so that these records could be further augmented, edited, and passed on to future editorial teams.

Meanwhile, funding remained a major concern as the editors tried repeatedly to convince the Undergraduate Finance Board that the Critical Review must remain in print form. There were some gains, but after 2002, the Critical Review's budget increased only marginally. In 2007, despite the efforts of Editors-in-Chief Ariana Cannavo '08 and Dara Steinberg '09, UFB eliminated the entire printing budget, instructing the Critical Review to be a web-only publication.[2] Since 2007, the organization has published course evaluations on its web page only.