Peer review has been around since I can remember, but that doesn’t shed light on its history that features interactions with the technology and forms of sharing information, with censorship, the rise and fall of generalists and polymaths and concerns about marketing. To ignore peer review’s past is to miss both an exciting tale of connections and a understanding of the conflicted views we have about peer review today.
When Guttenberg’s press created the opportunity for scientific information to be more widely shared it was followed, eventually, by one of the earliest forms of peer review – scientific criticism by the Church. Copernicus and Galileo both underwent an early form of peer review, and their work was even banned, Copernicus for four years and Galileo's work until 1758, when telescopes better proved heliocentricity (it was proved beyond doubt in 1838.) Their peer review ordeal was relatively tame compared to Miguel Servatus, who was burned at the stake for his beliefs about the Trinity. He was later applauded for his work understanding pulmonary circulation, the idea that blood from the right side of the heart traveled through the lungs to the left side of the heart.
Servatus is described as a polymath, scientists without formal training, amateurs in the best possible sense, pursuing knowledge out of a passionate curiosity. The Enlightenment saw the rise of polymaths and while Guttenberg’s press facilitated the discourse between these individuals a face-to-face discussion was still more reliable and quicker than only sharing of manuscripts. As with many things, these informal discussions were creative, productive and susceptible to taking on a more formal nature. Informality gave way to Royal Societies, granted a license by the King for the legal printing of books - the crown’s means of preventing sedition. Censorship remained an important mechanism.
Regardless of government motivation, these formalized groups sprang up everywhere. In Scotland, England, and France groups were widely disseminating information by publishing collections of articles. The Royal Society of Edinburgh is given credit for the first peer review with its first volume of medical reports, “the editor distributed the submitted essays for review to individuals he considered to be ‘most versed in these matters.’” [1] And in an echo of today, the purpose was the promulgation of new knowledge, accuracy and truthfulness were not guaranteed. The Royal Society of London for Improving Natural Knowledge began publishing the works of their members in Philosophical Transactions in 1663. Censorship remained on their minds.
“No book be printed by order of the council, which hath not been perused and considered by two of the council, who shall report, that such book contains nothing but what is suitable to the design and work of the society” [2]
In 1731 having taken fiscal responsibility for the Transactions, and presumably now having a greater stake in their publication, they adopted a review system similar to Edinburgh’s that bore an uncanny resemblance to today’s peer review.
“Memoirs sent by correspondence are distributed according to the subject matter to those members who are most versed in these matters. The report of their identity is not known to the author Nothing is printed in this review which is not stamped with the mark of utility.” [3]
In 1831, William Whewell, another polymath and member of the Royal Society of London proposed that reports be made on all submissions to Philosophical Transactions, saying that the reports were “often more interesting than the memoirs themselves,” [4] and could be used to popularize science. Whewell, by the way, is credited with creating the term "scientist." His attempt at the first report is interesting, instructive and echoes a divisive issue still present. Having agreed to write the first report, Whewell took a collaborator, John Lubbock, and together they considered a paper on astronomy. Unsurprisingly, they quickly came to very different conclusions about the article. The spent many months in a back and forth over what these reports should be.
“Whewell wanted to spread word of the discovery and to place it in the bigger picture … ‘I do not think the office of reporters ought to be to criticize particular passages of a paper but to shew its place,’ he told Lubbock. If they picked out flaws, he warned, authors would be put off. Lubbock had other priorities: ‘I do not see how we can pass over grievous errors,” he wrote.’” [4]
Whewell was an “authoritative generalist,” Lubbock, a specialist asked to review a direct competitor. And while Whewell’s generalist approach won out, the long time necessary for peer review and the idea that peer review could introduce conflicts of interest was established. Whewell’s position changed with time, reviewers became “defender of a society's reputation, working behind the scenes to exclude publications that do not belong.” [4] It was considered unseemly for a gentleman to criticize a colleague explicitly. Many of the criticisms of the day were by anonymous authors. Anonymity allowed “the individual is merged in the court which he represents, and he speaks not in his own name, but ex-cathedra (with full authority).” [4]
Anonymity also provided a playground for scoundrels, “perhaps through undetectable acts of piracy — at the expense of helpless authors,” to advance their own interests [4] as some say was the case of a very great scientist, Charles Darwin. In the fashion of the day, authors would frequently submit their thoughts to members of a society seeking feedback and introduction of their paper to referees. Alfred Russel Wallace sent the wealthy, more socially recognized Darwin his manuscript discussing natural selection, a topic that Darwin had long thought about but had not written down. He quickly did and Darwin's friends organized a presentation at the Linnean Society where both papers were released. They got joint credit for the theory of evolution by natural selection but Darwin wrote On the Origin of the Species and shot to worldwide fame. To be fair, Darwin was torn about Wallace's relative lack of recognition and helped Wallace both academically and financially but fame showed the die was cast. Conflicts of interest were here to stay.
The 1800’s saw the rise of newspapers as the quickest form of communication and a growing connection between science and its application in the industrial revolution. As more and more voices sought to be heard, scientific reporting became modeled upon newspapers. “One of the models for medical editing was, therefore, the crusading newspaper writer, or, at least, the publication that was a personal vehicle for the editor.” [5] The journal edited by Graham (of cracker fame) comes readily to mind. Two consequences followed adopting a newspaper model; first, the editor was the arbiter of what was and was not published and second, there were deadlines. Newspapers were scheduled, and there was a need for copy to fill the space. This inequality, between the small supply of articles and the significant demand to fill space, made for journals that judged articles by length rather than quality. Journal staffs were employed to solicit materials, especially accounts of medical meetings.
Another model for information sharing were official publications of academic or other institutes, for example, John Kellogg and his Battle Creek Sanitarium or the proceedings of the Massachusetts Medical Society which we now know as the New England Journal of Medicine. Since the editor was frequently the institute’s head and expert in the institute’s subject matter, “no outside referees were needed to judge the merits of the articles.” [5] Occasionally a topic did occur, the work of a specialist, which the editor might feel uncomfortable in reviewing, but this could be sent out in an ad hoc manner. After all, an MD degree was considered to be qualification enough to judge a medical article.
It was the development of both specialization and a new form of dissemination that lead to the next development in peer review. The typewriter and carbon paper facilitated the copying of articles. At the same time, specialists were developing technical expertise that the powerful but generalist editor did not have and could not safely comment upon, at least while guaranteeing the reputation (and competitiveness) of their journal. But recognizing the inadequacy of an editor’s skills was a personal insight, and adoption of outside review remained sporadic. The first of the editors to recognize the inadequacies was Ernest Hart, editor of the British Medical Journal.
“The only system that seems to me adequate to the real needs of professional readers is that in which every unsigned editorial paragraph is written by a specially selected expert. That is the principle on which I have modelled the journal I have the honor to conduct. Every letter received, every paragraph, every cutting editorially dealt with, is referred to an expert having special knowledge and being a recognized authority in the matter. ... It is a laborious and difficult method, involving heavy daily correspondence and constant vigilance to guard against personal eccentricity or prejudice or—that bugbear of journalism—unjustifiable censure. But that method may, I venture to think, be recommended as one that gives authoritative accuracy, reality, and trustworthiness to journalism.” [5]
Many different formats to support editors appeared, including committees and editorial boards. Prizes were offered for articles that might be published. But as the supply of studies remained far less than available space, these editorial boards often rewarded active contributors and served to solicit rather than review articles. Editors began to see themselves as educators, with reputations to protect; and authors were asked to consider “biography, expression, and logic of their contributions resulting in revision.” [5] For some editors, peer review required the employment of in-house staff, staff more easily controlled than outside reviewers. JAMA used in-house review almost exclusively in the 1930’s.
While specialization had raised concerns about editorial expertise, a dramatic increase in the number of scientists and their publications overwhelmed editors during the 40’s and afterward. No one editor or even in-house staff could adequately review submissions; outside help became increasingly necessary. So, the twin engines of increasing specialization and volume took peer review from an ad hoc, casual practice into the institutionalized version we have today.
The controversies that have plagued peer review from its earliest days, censorship, conflict of interest, the tension between early reporting and veracity, the need to fill space, the desire for prestige and income remain with us today. They may have assumed different forms, but at their core are flaws in a system designed by flawed humans. It may not be the best system, but it is the one we use. Changing peer review is doomed to fail if we do not consider and understand its evolution. Peer review is both old and new. It is only our inability to remember history that makes us simultaneously believe that peer review has a long, long history and that it is a modern means of curating and shepherding knowledge.
References:
[1]Ups and downs of peer review Dale Benos et al.
[3] Peer review in the 18th century David Kronick
[4] Peer review troubled from the start Alex Csiszar
[5] Evolution of editorial peer review John Burnham
Chuck Dinerstein, MD, MBA
Director of Medicine
Dr. Charles Dinerstein, M.D., MBA, FACS is Director of Medicine at the American Council on Science and Health. He has over 25 years of experience as a vascular surgeon.
