My undergraduate degree was in history and one of the required courses was a course in the methodology of research. Our primary textbook was The Modern Researcher by Jacques Barzun and Henry F. Graff; the sixth edition paperback is rather expensive—far more expensive than it was when I got it in college.

Another book I read was Historian’s Fallacies: Toward a Logic of Historical Thought, by David Hackett Fischer. It is much more reasonably priced. It demonstrates some of the mistakes that can be made in historical research, and uses actual works by historians to illustrate the problems.

Both books, together with the class, gave me a good foundation in how to conduct research. The principles are not difficult or complicated, and once learned, seem so obvious that it’s a wonder we have to be instructed. But instruction we indeed need.

Together with a course in logic, and an understanding of both the scientific method and Occam’s Razor, an understanding of basic research methodology could go a long way in limiting the problems that people would have in figuring out whether something is true, or if the evidence is lacking. Once you get the principles inside of you, even if you don’t know what the truth really is, it becomes pretty easy to recognize lies, stupidity, and general misinformation.

I am frequently annoyed by what I read on Facebook, what I hear from politicians, or what I endure from the poorly researched articles created by journalists and pundits.

Some Basic Principles of Research

Various basic principles have become so generally established, so tried in the fires of experience, that the scholars concerned hardly ever feel the need to even mention them in print. They include the following:

1. The primary importance of facts

Priority must always be given to tangible, objective data, and to external evidence over subjective theory or speculative opinions. Facts must control theory and not vice-versa. Source material must always be scrutinized in this light.

2. The importance of primary sources

In research, one must always seek out the original source material—seek to discover the origin of a tale or legend, or incident. For instance, the story of George Washington chopping down a cherry tree originates in a book written by Parson Mason Locke Weems entitled The Life of Washington that was published in 1800. The story appears nowhere prior to Weems’ book. He attributes it to “…an aged lady, who was a distant relative, and, when a girl, spent much of her time in the family…” He does not name her so it is impossible to verify.

3. A positive attitude toward source material

You could summarize this briefly as: the source material is innocent until proven guilty. It is normal practice to assume the general reliability of statements in our sources, unless there is good, explicit evidence to the contrary. Unreliability, secondary origins, dishonesty of a writer, or tendentious traits — all these must be clearly proved by tangible evidence, and not merely inferred to support a theory. So, given the axe he had to grind, there is reason to have some doubts about everything that Josephus wrote: he was attempting to justify his behavior at the battle where he surrendered to the Romans, and he was attempting to explain why the Romans should not dislike the Jewish nation. It is unlikely that Josephus flat out lies, but it seems probable that he selected his material and describes it in the best possible light for his purposes. It is similar to a fictional tale about a two man race between an American athlete and a Soviet athlete during the Cold War. The American won. Pravda reported the race as follows: “The imperialist American came in next to last, while the glorious Soviet worker came in second.” Nothing precisely inaccurate in the account, but the impression is another thing altogether.

3. The Inconclusive Nature of Negative Evidence

Negative evidence is commonly not evidence at all, and is thus usually irrelevant. If some person, event, etc. is mentioned only in documents of a later age, the absence of any directly contemporary document referring to such a person or event is not in itself a valid or sufficient ground for doubting the historicity of the person, event, etc.

It must always be remembered that the absence of evidence too often merely reflects the large gaps in our present day knowledge of historical periods. The gaps in our knowledge of even of the relatively well-documented culture of, say, Ancient Egypt are significant. Much relavent evidence still awaits discovery or decipherment, or else it has simply been lost. Although cuneiform tablets and fragments in the world’s museums are numbered in the hundreds of thousands, they are but a fraction of all that were written — perhaps ninety-nine percent are still in the ground. In the words of Cyrus Gorden, “for every mound excavated in the Near East, a hundred remain untouched.” For instance, the fact that there are no contemporary documents besides the Bible that mention Jesus is not surprising: Palestine was a minor province in the Roman Empire, and far from Rome. Jesus was not important or significant (at that time) to the Romans or anyone outside a small group of people in Palestine, most of whom would have been considered insignificant.

4. A Proper approach to apparent discrepancies

The basic harmony that ultimately underlies extant records should be sought out, even despite apparent discrepancy. Throughout ancient history, our existing sources are incomplete and elliptical. We must weigh and take into account all relevant sources, and make allowance for missing or ill-interpreted factors. Finally, in speaking of error or inconsistencies, one must distinguish clearly between primary errors (mistakes committed by the original author of a work) and secondary errors (not in the original, but resulting from faulty textual transmission or the like).

5. Secondary sources should not all come from one political, economic, cultural or religious point of view.

Likewise, it is important in analyzing secondary sources to make sure that they aren’t simply quoting from one another, or all saying exactly the same thing in the same way.

An Example of Bad Research

A good illustration of a failure of proper research methodology and its devastating consequences can be illustrated by the bestselling book The Coming Economic Earthquake by Larry Burkett, which appeared in 1991 and was published by Moody Press (that it won praise and awards demonstrates the failure of editors, fact checkers at the publisher, reviewers and readers alike to have any idea of how to do proper research). Someone gave me the book and told me it was a good book. I read it expecting that it would be good.

It wasn’t.

The factual errors in the book were so obvious and so bad that I was flabbergasted. Of course, the mistakes were obvious to me only because my undergraduate degree was in modern European history and because I had taken an introductory course in economics. The average, uninformed reader would not necessarily notice the problems that easily. The errors were not minor; they were so serious that they entirely undermined the point of the book and cast its conclusions into doubt.

Five examples (there were many more problems than this) can serve to illustrate the failure of his research. Remember, these are not matters of opinion. They are matters of plain fact.

1. p. 27:

“Their spokesman for this New Deal was an articu¬late aristocrat with a household family name: Roosev¬elt. Franklin Roosevelt was born to wealth, raised to wealth, and educated in wealth at Harvard, where he was exposed to the phi¬losophies of Dr. John Maynard Keynes of England. Keynes, an avowed socialist, had long advocated the use of government control over banking and business to ensure prosperity for all. This phi-losophy was not new. Karl Marx had advocated essen¬tially the same doctrine, only to a more radical group — the poor.”

a. John Maynard Keynes was not a socialist. According to the Encycopaedia Britannica:

In Cambridge, to which Keynes now returned, his reputation was rather different. He was quite simply esteemed as the most brilliant student of Alfred Marshall and A.C. Pigou, the two Cambridge economists who between them had produced the authoritative expla¬nation of how competitive markets functioned, business firms operated, and consumers spent their incomes.

Although the tone of Keynes major writings in the 1920’s was occasionally skeptical, he did not directly challenge that conventional wisdom of the period that held laissez-faire, only slightly tempered by public policy, the best of all possible social arrangements.

(Encyclopaedia Britannica, volume 10, p. 447, 1984)

b. It is impossible that Roosevelt was influenced by Keynes in Harvard because Keynes was born on June 5, 1883. Roosevelt was born January 30, 1882. Roosevelt was older than Keynes, and they were both in college about the same time. It seems unlikely that Roosevelt would be studying the philosophy of someone who was himself taking classes at the same time in Cambridge, from firmly laissez-faire capitalist economic teachers — especially when you consider that Keynes had yet to develop the economic philosophy about which Burkett is so critical.

c. Keynes’ book, in which he propounded his economic theory of unemployment (Larry Burkett terribly misrepresents and apparently doesn’t understand Keynesian economics in the first place) was called The General Theory of Employment, Interest and Money, which appeared in England at the very end of 1935. Roosevelt had been elected president in 1932.

This is how the Encyclopaedia Britannica summarizes Keynes argument in his book:

The central message is readily translated into two powerful propositions. The first declared the existing theory of unemployment nonsense. In a depression, according to Keynes, there was no wage so low that it could eliminate unemploy¬ment. Accordingly, it was wicked to blame the unemployed for their plight. The second proposition proposed an alternative explanation about the origins of un¬employment and depression. This centered upon aggregate demand — i.e., the to¬tal spending of consumers, business investors, and public agencies. When aggre¬gate demand was low, sales and jobs suffered. When it was high, all was well.

From these generalities there flowed a powerful and comprehensive view of economic behaviour. Because consumers were limited in their spending by the size of their incomes, they were not the source of business cycle fluctuations. The dynamic actors were business investors and governments. In depressions the thing to do was either to enlarge private investment or to create public substitutes for private investment defi¬ciencies. In mild economic contractions, monetary policy in the shape of easier credit and lower interest rates just might stimulate business in¬vestment and restore the aggregate demand caused by full employment. Severer contractions required as therapy the sterner remedy of deliberate public deficits ei¬ther in the shape of public works or subsidies to afflicted groups.

(Encyclopaedia Britannica, volume 10, p. 448, 1984.)

Whether Keynes is right or not is a separate issue. But Burkett’s presentation of him is far from accurate, therefore rendering Burkett’s conclusions very suspect.

2. p. 72:

“It was assumed that by injecting a modest amount of new currency into the economy, only a modest amount of inflation would follow. Advocates of this plan assured the Kaiser that a modest amount of inflation would be manageable and would actually allow producers to reap more profits, thus helping to repay the Weimar Repub¬lic’s debts with cheaper currency.”

a. Germany did not have a Kaiser after World War I. Therefore, how could there be advisors to this by then non-existent person? Following World War I, the Kaiser abdicated and moved to Holland, together with his family. He had no power or influence on Germany after that. Before the rise of Hitler, Germany had a popularly elected, democratic government that the Kaiser had nothing to do with–since he was in exile. In Holland. Where he died in 1941.

3. p. 165:

“This is what George Orwell described as ‘government speak’ in his novel 1984.”

Orwell called it “Newspeak” in Orwell’s 1984, not “government speak.”

4. p. 166:

“Then in the sixties President Nixon substituted the use of base metal coins for silver coins effectively removing all fixed asset value from U.S. currency.”

The coins were changed from silver to nickel/copper sandwiches in 1965. I’m a coin collector. I have these coins. And Nixon did not take office as president until January, 1969. Thus, he didn’t have anything to do with eliminating silver from the coins, since he didn’t become President until four years after the deed was done.

5. p. 198:

“Once the word was made public, investors outside the U.S. rushed to convert their U.S. dollars into the E.C. Eurodollar, adopted as the official world currency by virtually all members of the World Economic Council, excluding the United States of course.”

Although Burkett was describing a fictionalized account of a possible future crisis in 1999, what he is described would be a remarkable trick indeed, considering what Eurodollars are, according to the Encyclopaedia Britannica:

Eurodollars, deposits of United States dollars in foreign banks obligated to pay in U.S. dollars when the deposits are withdrawn.

(Encyclopaedia Britannica, volume III, p. 998, 1984.)

What does that mean? Eurodollars are simply U.S. dollars that happen to be sitting in European banks. If Burkett’s scenario took place, I suspect the Europeans would find their Eurodollars just as worthless as the U.S. dollars — since they are the same thing.

The fundamental problem with Burkett’s book can be traced back to his research methodology or lack thereof. First, facts were apparently not of primary importance for driving his theory. Second, he did not look at primary source materials. He didn’t even doublecheck the information he’d gotten. Say, in a basic reference book. Like an encyclopedia. Second, and perhaps most significant, the list of the sources he used according to his surprisingly short bibliography all came from the same economic and political point of view. Third, his source texts (which are secondary or tertiary sources) are essentially really only one source—and an unreliable one at that—because they all quoted from one another and from one author who apparently came up with the misinformation in the first place.

I wrote Larry Burkett a letter pointing out these problems. He was unconcerned. One of his associates told me that yeah, there were some typographical errors in the book, but so what? After all, the book was selling well and garnering awards.

I’ve seen similar behavior on Facebook; people will post untrue and false statements. Even after learning that their statements are false (not a matter of opinion; demonstrably, factually false, like arguing that 2 and 2 equal 73), they and their friends ignore the criticism and continue praising, sharing, and reposting the misinformation.

It puzzles me, though it no longer surprises me. If a high percentage of politicians, journalists, pundits, economists and other people in positions of power and influence are unconcerned with reality, why should I expect the percentages to be any different among any other groups of human beings?

I’ve learned that many people simply are not particularly interested in the truth. They are interested in having what they think to be the case confirmed. I’ve had people come up to me and ask me what I thought about a particular interpretation of the Bible, or theological position. When I start to show a different way of looking at things, or demonstrate that what they’ve told me simply can’t be true, nine times out of ten the reaction is not particularly positive. Instead, they quickly disengage from the conversation and walk away. Many people are not really looking for the truth; instead they are looking for a hug and confirmation. If it doesn’t fit, then they will ignore it or explain it away, or in some way justify their continued hold on illusion. Likewise, in my experience, most people aren’t interested in learning new things, nor are they willing to alter their beliefs or opinions, regardless of the data or new information–which they resist hearing or learning about and sometimes actively avoid.

People seem to approach much of reality in the same way sports fans approach their favorite teams: the umpire is always wrong, the ball was always in play, the batter was out, and their team was robbed when it loses.

Send to Kindle