« Discovery of Bar Kokhba era Scroll Fragment
Main
A Short Note on Pope Benedict and the Secular West »

July 15, 2005

What is a Scholarly Consensus?

The Stuff of the Earth, PaleoJudaica, Ralph the Sacred River, Anduril, Higgaion, Biblical Theology and perhaps others who study the Bible have been discussing the nature of consensus. It all seems to have started when NT Gateway Weblog took issue with a claim by N. T Wright concerning how the "great majority of scholars" date the gospels. Wright says that the "great majority of scholars" claim that the gospels were "written by the 80s of the first century at the latest." Mark Goodacre of NT Gateway Weblog, counters,

I don't think the majority, let alone the great majority, date all four canonical gospels to the 80s, and certainly not the 80s "at the latest." My reading of the consensus would be that Mark is written by the 80s -- it's usually dated somewhere in the late 60s to early 70s -- but that Matthew, Luke and John are variously dated anywhere from the 80s to the early second century.

While I think Goodacre is certainly correct, I have given the above information only to set the stage for what I find every bit as interesting and perhaps more important to a layman, in my case, a layman in almost every field that interests me: What is a consensus? As laymen, we come to understand most of the scholarly world by a reliance on consensus. If we are intellectually honest, we will and want to draw-upon that consensus to formulate our own ideas we will probe the consensus at a few points and see if it holds up. If it does then we take it as more or less a fact. Life just isn't long enough to get dozens of PhDs and do research in hundreds of fields to become knowledgeable at the level that might be desirable.

Michael Pahl of Stuff of the Earth, asks four questions that Higgaion summarized as follows:

  • What is consensus? [I.e., what percentage of people must agree to make a "consensus"?]
  • Second, who gets to be part of the polling sample?
  • Third, how does one actually go about doing the polling to assess consensus?
  • What does consensus prove?

I will not attempt to gives each participant's answers to these four questions. You can go read their answers and make your own judgment. If you do, you will see that I largely agree with Jim Davila of PaleoJudaica. At least I hope he is mostly correct. I also am using Davila because he was first to put an oar in the water.

On the first question, "What is consensus? [I.e., what percentage of people must agree to make a 'consensus'?]," Davila give a rather trite answer but one I believe to be near the truth.

I take consensus to mean that more people think so than not, but not everyone thinks so. Call it 51%-99%.

We'll get to consensus of what later. For me, any percentage is a little hard to defend. 51% would not make me feel very good if the other 49% had an equally unified, coherent opinion based on the same evidence. On the other hand, I just might feel good about a less that 50% "consensus" if the "majority" were highly fragmented in their opinions. Of course the greater numbers of scholars that are in general agreement, the more confident I become. If scholarly opinion is highly fragmented I, as a layman, have no right to speak on the subject without prefacing my remarks with something like, "this is my ignorant personal opinion" unless I have studied it myself enough to critique meaningfully all the options that are currently being considered by scholars. And that's a lot of work.

On the second questions, " Second, who gets to be part of the polling sample?," Davila says,

I think most people measure consensus on the basis of published work in scholarly monographs and collections of essays, articles in peer-review journals, etc. I would weigh this more heavily than what a scholar thinks but hasn't published, since publication in itself requires more work and more commitment than just studying. When one refers in a publication to a consensus about something, normally a footnote follows with specific references to previous publications.

[snip]

"Scholars" and "authorities" are people who are familiar with this [scholarly] conversation; who have the necessary training (languages, knowledge of primary sources, methods of a field, etc.) for their opinions to be worth something; and who carry on the conversation in the venues that have developed to continue it.

With this, I agree without any real quibble. The so-called fallacy of appeal to authority often comes up in the context of a discussion such as this. Joe McFaul of Law Evolution Science and Junk Science has a very good discussion of this supposed problem. A snipped from it is relevant here.

The Fallacy of the Appeal to Authority is something I happen to know quite a lot about because I work with some version of it every day. It’s perfectly legitimate to appeal to appropriate authority. Most people don’t realize that the full name of the fallacy is Appeal to Inappropriate Authority.

I call this the "Advertiser’s Fallacy" because it’s so prevalent in commercials, such as the one where a famous baseball slugger gives medical advice on erectile dysfunction (that should pick up the hit count!). No. See a properly qualified doctor for ED, see Rafael Palmiero only if you want to improve your baseball swing.

On the third questions, "how does one actually go about doing the polling to assess consensus?" Davila writes,

So don't be simplistic. Read carefully and widely, particularly the published work of people who disagree with you, and be able to summarize the views of people you read with nuance and accuracy. It's hard work.

But this is, as he says, "hardwork." If you are outside the field, how do you know that you are dealing with such an authority? What I have done is go to University websites and see what textbooks are being used; one or two that are widely used will often reflect the consensus. Also well edited popular magazines like Scientific America, if read carefully, are also good sources of authorities' positions on the issues they cover. Another "trick" that I have found helpful is to look to see the extent to which a person cites evidence rather than other people's opinions. They are more likely an appropriate authority if they rely on evidence rather that others opinions. Of course, a thorough review of the literature is also often another mark of an authority.

Now for the last question, "What does consensus prove?" Davila writes in part,

The consensus is the thing you need to improve on if you want to advance the state of the question -- which is what scholarly research is all about. No one should pretend that a consensus is the final word. It's the starting point for your research. Apply new evidence or new methods or better reasoning to give us a better understanding of the problem. People who challenge Sanders's work don't want to go back to what was before it, they want to show that the problems are more complicated than he realized.

[snip where he gives some good examples of consensus]

If there isn't a clear consensus, your mission, should you choose to accept it, is to find an answer that is more persuasive than the ones currently available, and that then becomes a consensus.

I would add that if a scholar, looking at the evidence, old and new, who is trying to improve on the consensus, finds it to be wrong that may be an improvement. If he or she can convince a significant majority of other scholars who are qualified to evaluate the evidence then it is clearly an improvement.

When I was in graduate school we would joke that if someone said, "most scholars agree" these words should always be understood as "I think." A similar, but somewhat more sophisticated view, is expressed by Jim West at Biblical Theology . If you are inclined in this direction, give it a read. While I strongly agree that "truth is not subject to voting." I do believe that a poll among the knowledgeable can lead to a good first order approximation of the truth (with a small "t") as it is known at the time.

The fact is the issue is not a no-brainer. It may not even apply to all fields of study and if so, the nonprofessional is lost but so are those who toil there. For this reason, it would be interesting to see how biologist and physicists for example deal with "consensus."

Posted by DuaneSmith at July 15, 2005 04:13 PM | Read more on Odds and Ends |

Trackback Pings

TrackBack URL for this entry:
http://WWW.telecomtally.com/cgi-bin/blog/mt-tb.cgi/235

Comments

Sorry, comments are closed for this post.
Send me an email if it is important.