steamer

Papers which influence public policy almost always seem to come to the same pre-ordained conclusions as was originally held by those commissioning those papers. Just as a laywer can always find an "expert" to satisfy his requirements, it seems that there are studies which are performed by these same kinds of "experts" which always give the desired answer.

How skeptical ought we be of those "scientific" papers which influence public policy? It seems to me that we ought to more skeptical of these than ones which are politically neutral. This brings up another question. Which is more important, the truth or a political solution that brings about a positive social change, but which is based on what is essentially a lie or an unknown?

Jehanne

I am sure that you have an example or two! For starters, how many different religions are there in the World today? Hundreds of thousands? Millions? Back at an American Astronomical Society convention six or so years ago, five groups showed up with research on the age of the universe. Three of the groups were not aware of what the other groups were doing, and the two groups that did know about each other's research did not have any collaboration or contact with each other. Nevertheless, all five groups, using different techniques, got the same age for the Cosmos! Science, above all else, is global, which limits the ability of politics and religion to influence the process.

scigirl

Very.

But I would argue, we should be equally skeptical of all scientific studies: of their methods as well as their conclusions. Irregardless of whether or not the science may be used for political purposes.

Perhaps you could provide us an example of how a specific scientific study, published in the peer reviewed literature, has flawed methodology and/or methodological conclusions, presumably based on their political beliefs.

I certainly can:

http://www.ncbi.nlm.nih.gov/entrez/q...89&query_hl=15
Psychol Rep. 2003 Dec;93(3 Pt 2):1173-94. Homosexual parents in custody disputes: a thousand child-years exposure. Cameron P, Harris DW. Family Research Institute, Inc, Colorado Springs, CO 80962, USA.

Note that Paul Cameron works for Focus on the Family, whose mission is to "to cooperate with the Holy Spirit in disseminating the Gospel of Jesus Christ to as many people as possible, and, specifically, to accomplish that objective by helping to preserve traditional values and the institution of the family."

Does this bias automatically make Cameron's studies flawed? Of course not. To prove a study is flawed, you have to . . . well prove that the study is flawed.

This is not hard to do with Cameron's research.

In the above study, Cameron evaluates custody appeals, and presents statistics regarding homosexuals who harm children. His study is flawed in several ways: 1) he lumps parents engaging in criminality or homosexuality together, 2) he only includes couples who have one partner who is homosexual, implying that the homosexual acts were likely adultery, and 3) he does not adequately compare heterosexual couples involved in custody appeals. Also realize what Cameron is doing: he’s starting with a population of bad parents by looking at custody disputes, and then finding anything that’s more common to the homosexual than heterosexual population. How do we know that the rates of molestation, murder, alcohol and drugs, were perhaps the same, or even higher in the heterosexual group? We don’t.

So steamer, if you want to dispute a particular study, you are free to do so, by this type of analysis. Go for it. We give you permission.

scigirl

chapka

We should be dubious of a single study, certainly--especially if it conflicts with a lot of other research.

We should also insist on being told about the methodology. For example, the "porn causes violent thoughts" studies that have been quoted at me used neutral images (bunnies, flowers) and violent pornography (hardcore S&M stuff), but no nonsexual violent images such as war movies or college football. Needless to say, the people who draw conclusions from these studies don't tend to consider whether the methodology supports their conclusions.

scigirl

Very astute point, Jehanne.

No doubt, if studies from different societies and countries come to similar conclusions, and the studies themselves appear valid, you can say that political influences are not substantially influencing the conclusions.

scigirl

steamer

I don't think we should be equally skeptical of all studies. It has occurred in purely scientific circles that data has been falsified to support a particular conclusion, but this only earns the researcher scorn.

Myself, I find those studies commisioned by agencies with agendas to be more suspect than those where no there is no agenda. Others may have motive for massaging the data but without a motive to force a particular outcome most don't bother to. Cigarette companies for example commisioned some studies where the results showed cigarettes to actually be healthful.

Most of us do not pay very close attention to when a particular paper has been debunked. The internet has no such means of classifying papers that have been debunked. When later studies cast serious doubt on a particular study there isn't really much means to view both studies except in pay-per-view libraries. In other words, Bible-like, I can pretty much find a study that will support whatever view I happen to hold.

There is a real danger that we can be manipulated in our political thinking by any study whose conclusion is bought and paid for by anyone with the dollars to do it. When later studies come about, debunking the new claim, I hold it as no certainty that we will ever even hear about that new paper especially when public policy has already been made based on that obsolete paper.

Example

Lil' Jerry Seinfeld

Why not? I don't think any study should get an easy pass through.

Doesn't everyone have an agenda when it comes to commissioning studies? On what basis do you determine when an agency has an agenda when commissioning a study and when they don't? This seems quite arbitrary to judge some more skeptical then others based on a completely subjective notion of them having an agenda or if they don't.

There's really no way of stopping that, is there?

So applying a completely arbitrary and subjective judgement when it comes to being skeptical about some papers and then not so much on others is going to solve this problem?

steamer

I'm not saying any paper ought to get an easy pass, just that there is more motive for manipulation in some topics than in others.

I doubt you believe this. How much more dubious is a study of the effects of smoking when commisioned by a tobacco company?

Probably not, but if we agree that this is the case, how much value should we place on our links to any studies?

Given the stated criteria of viewing with skeptism those papers which make obvious political claims, one would be hard-pressed to call it arbitrary, but it is subjective and does require personal judgement.

Problems only move towards solutions when people become aware of them. To many of us, these studies are tellling us what to believe as surely as the bible tells those believers.

scigirl

steamer,

Thank you for that example. I agree with you that using science falsely to promote an agenda is unethical. Right now, I'm currently unhappy about the abortion rights organization NARAL for quoting Roberts out of context regarding that abortion doctor murder case. Roberts argued that racketeering laws didn't apply to the case (and he was right, they didn't), he did not condone killing abortion doctors like one of NARAL's ads suggested. I think NARAL's agenda should stand on its own without resorting to tactics such as this.

I'll have to read that article more thoroughly when I have the time. On the whole, it seems legitimate. However, this passage sent up a red flag (bold added by myself):
I don't doubt that groups do these things - I myself have accused Christian advocacy groups of being guilty of these tactics. Click here for some examples.

However, I am a bit skeptical of the bolded claim.

In my experience both as a long-time poster of IIDB as well as a scientist who used to work in a laboratory, it seems that most advocacy groups are rarely doing the primary research, but rather quoting it. There are exceptions to this rule. But it's much easier for advocacy groups, who are merely quoting science to get away with such unethical tactics, than it is for actual scientists who are held accountable by the brutal peer review and grant renewal processes.

Actual science that is published in scientific journals (not just posted on a web site somewhere) is held to fairly high standards of proof. Of course, exceptions abound, and the peer review process is far from perfect. However, scientists use science to argue with each other, and in doing so, tend to ferret out the scientists who are blatantly publishing studies "as if they were impartial research, by planting newspaper stories, publishing in journals whose referees are as biased as the studies' authors, getting corporations to fund advertising that masquerades as a "documentary", etc"

An example would be Paul Cameron who I talked about earlier. His research was denounced by both the American Psychological Association and the American Sociological Association. Not because he was controversial - scientists thrive on controversy and proving each other wrong. Because he was found guilty of misusing social science data and methods to push an agenda. Here is a website which discusses it further. So, I doubt that scientific theories which have broad international support from a wide variety of countries and cultures are going to survive very long if they are blatant lies made up by activists.

The solution? Teach people to think critically, and improve the scientific literacy of the population. If you have noticed, steamer, I rarely use web sites hosted by advocacy groups to argue a case. Instead, I go to the original research.

scigirl

Lil' Jerry Seinfeld

I disagree, I see the motive for manipulating things in almost any topic.

Then you doubt wrong. I don't believe any agency who commissions a study isn't doing so without an agenda.

I'd treat it no less skeptical then I would of an organization that was anti-smoking. It appears by your statement of being more skeptical of studies commissioned by an "agency with an agenda" is more close to "agency with a viewpoint differing with mine".

You determine the value of said studies on a case by case basis. I don't see how being less skeptical of some papers over others due to some subjective standard is going to fix the problem you state.

So taking your example of the tobacco company commissioning a study, would you be equally as skeptical of an agency who's purpose was to campaign against smoking? If not, then yes I see that as a very abitrary standard.