What exactly is peer review

Discuss autism theories, media stories, and efforts to put ASD on the government agenda here.

Moderator: ModeratorBill

Forum rules
Please limit quotes from articles to five paragraphs. Also, researchers may post study information here.
srinath
Posts: 941
Joined: Tue Dec 20, 2005 12:33 pm

What exactly is peer review

Postby srinath » Thu Jun 08, 2006 12:17 pm

For example - If Dr N comes up with a treatment protocol and other doctors review it and start following it ... isn't that peer review.
Now why this "lack of peer review" keeps popping up then.
I guess they want a peer review by someone outside of the DAN ...
Now of course there was no DAN several years ago, some doctor started a treatment protocol and others saw it and started doing it and it grew from there to what now is the community of DAN doctors.
Of course DAN docs are called quacks ... even though its a group that peer reviewed what was one of more doctors treatment methods and adpoted it. So if a doctor comes in from outside to review and endorses the findings of a DAN doctor I'll guess they will quickly be added to the roster of quacks. So I'd venture out to say peer review of any DAN material with a positive outcome ... is career suicide for any doctor. now you understand the reluctance and hence the lack of peer review.
Cool.
Srinath.

livsparents
Posts: 1629
Joined: Thu Jun 23, 2005 10:29 pm

Postby livsparents » Thu Jun 08, 2006 12:41 pm

There was a quack once who told doctors, maybe you should wash you're hands before moving on to the mothers giving birth, something you can't see is killing those babies and mothers. He was laughed into suicide because he did not have the DATA, the exact reason for what was going on. It didn't make it any less valid. Because it does not follow the exact 'processes and procedures' on the AMA it MUST be quackery, lets wait 10-20-40 years until we get around to a PROPER study, hence the 'N' in DAN!

littlebopeep
Posts: 1387
Joined: Wed Feb 15, 2006 5:33 pm

Postby littlebopeep » Thu Jun 08, 2006 12:50 pm

There was also a quack once who said, "Lay on these leeches and ye shall be healed!!"

Peer review is a necessary part of valid research. I'll let others elucidate, as they can explain it better than I.
Fred, 7, NT
Barney, 5, autism

livsparents
Posts: 1629
Joined: Thu Jun 23, 2005 10:29 pm

Postby livsparents » Thu Jun 08, 2006 1:30 pm

They use leeches to prevent clotting now ya know....really
Bill

I'm a member of LEECHWEB.com!

srinath
Posts: 941
Joined: Tue Dec 20, 2005 12:33 pm

Oh really

Postby srinath » Thu Jun 08, 2006 1:43 pm

littlebopeep wrote:There was also a quack once who said, "Lay on these leeches and ye shall be healed!!"

<snip>


OK then what happened ...
Cool.
Srinath.

livsparents
Posts: 1629
Joined: Thu Jun 23, 2005 10:29 pm

Postby livsparents » Thu Jun 08, 2006 1:58 pm

Remember, leeches were part of mainstream treatment, it was the guys saying "here eat this mold, it'll kill the demons inside" that were considered the quacks :wink:

MOLDWEB.com
Bill

srinath
Posts: 941
Joined: Tue Dec 20, 2005 12:33 pm

Remmeber

Postby srinath » Thu Jun 08, 2006 2:01 pm

livsparents wrote:Remember, mercury were part of mainstream treatment, it was the guys saying "here eat this DMSA, it'll kill the demons inside" that were considered the quacks :wink:

MOLDWEB.com
Bill


With your permission ...
Cool.
Srinath.

mom_of_an_autie
Posts: 734
Joined: Sat Apr 01, 2006 6:18 pm

Postby mom_of_an_autie » Thu Jun 08, 2006 3:26 pm

Thanks for the laugh Bill.... :lol:

littlebopeep
Posts: 1387
Joined: Wed Feb 15, 2006 5:33 pm

Postby littlebopeep » Thu Jun 08, 2006 4:13 pm

:?
Fred, 7, NT

Barney, 5, autism

srinath
Posts: 941
Joined: Tue Dec 20, 2005 12:33 pm

Yes ... serious

Postby srinath » Thu Jun 08, 2006 4:20 pm

Its dead serious. I dont joke. Unless Icons are trailing ... or leading ...

So answering me with one of the Kevinleitch type "Keep quiet ... adults are talking" or "you must be trolling" is evidence that he doesn't have a leg to stand on ... when we are discussing facts ...

And In opinoins ... there is no right or wrong ... so dismissives just show you are undecided ...
Cool.
Srinath.

Mary
Posts: 472
Joined: Fri Feb 25, 2005 8:28 am

Postby Mary » Thu Jun 08, 2006 5:25 pm

If you were searching for a definition, I found this:

peer review

Peer review is a process used for checking the work performed by one's equals (peers) to ensure it meets specific criteria. Peer review is used in working groups for many professional occupations because it is thought that peers can identify each other's errors quickly and easily, speeding up the time that it takes for mistakes to be identified and corrected. In software development, peer review is sometimes used in code development where a team of coders will have a meeting and go through code line by line (even read it aloud possibly) to look for errors. Generally, the goal of all peer review processes is to verify whether the work satisfies the specifications for review, identify any deviations from the standards, and provide suggestions for improvements.

respect
Posts: 2402
Joined: Mon Sep 19, 2005 3:52 am

Postby respect » Thu Jun 08, 2006 5:53 pm

I await discussions on the "efficacy" of peer review.

respect
Posts: 2402
Joined: Mon Sep 19, 2005 3:52 am

Postby respect » Thu Jun 08, 2006 5:55 pm

If you have 10 mad scientists peer reviewing each others work, it really dosent mean much except that they are using their own common "madness" to scrutinise each other.

Alex's mom
Posts: 1626
Joined: Sat Mar 18, 2006 2:11 pm

Postby Alex's mom » Thu Jun 08, 2006 6:28 pm

If you have 10 mad scientists peer reviewing each others work, it really dosent mean much except that they are using their own common "madness" to scrutinise each other.


Except that it is far less likely to find 10 mad scientists with converging madness who also happen to conceal their madness well enough to become leaders/experts in their respective fields (these are usually the reviewing "peers"), than it is to find one mad scientist who is trying to validate his pet theory. Look around at how much garbage websites the internet provides. Personally, I feel better knowing that something has passed the test of peer review than the test of Google. It is really unfortunate that there is so little funding and research for autism treatments, that we don't have the benefit of the kind of data we should be seeking.
Alex's mom

BTDT
Posts: 3492
Joined: Tue Dec 20, 2005 8:02 pm

Postby BTDT » Thu Jun 08, 2006 6:50 pm

I await discussions on the "efficacy" of peer review.


What's to discuss?

MCA
Posts: 2124
Joined: Fri Feb 25, 2005 9:21 pm

Postby MCA » Thu Jun 08, 2006 7:02 pm

I wonder too who the peers are for a study to be considered peer reviewed. That's a good point.

Speaking of Google... what gets me worried more is Wikipedia... if I understand correctly, all the articles are just submitted by whoever? No fact check?

I am curious what wikipedia says about autism related subjects.

Winnie
Posts: 4227
Joined: Sat Mar 18, 2006 2:48 pm

Postby Winnie » Thu Jun 08, 2006 7:38 pm

Personally, I feel better knowing that something has passed the test of peer review than the test of Google


That...is the funniest quip I have read... since the iron curtain riddle :lol: :lol: :lol: I hope you won't mind me borrowing it sometime...

Peer review is fundamental to science. C'mon folks, this is not some remote concept...one could look it up in far less time than it takes to type a half dozen posts insinuating that it is a practice used by conspiracists.

Follows is some info on peer review and reasons why studies are rejected for publication:

Excerpted from:
Trisha Greenhalgh, "How to read a paper: The basics of evidence-based medicine" (1997, BMJ Publishing Group, p.35).

Most good scientific journals send papers out to a referee for comments on their scientific validity, originality, and importance before deciding whether to print them. This process is known as peer review, and much has been written about it. Following are common reasons why papers are rejected for publication by peer-reviewers:

:!: the study did not examine an important scientific issue
:!: the study was not original, i.e. someone else has already done the same or very similar study
:!: the study did not actually test the authors' hypothesis
:!: a different type of study should have been done
:!: practical difficulties (e.g. in recruiting subjects) led the authors to compromise on the original study protocol
:!: the sample size was too small
:!: the study was uncontrolled or inadequately controlled
:!: the statistical analysis was incorrect or inappropriate
:!: the authors have drawn unjustified conclusions from their data
:!: there is a considerable conflict of interest (e.g. one of the authors as a sponsor might benefit financially from the publication of the paper and insufficient safeguards were seen to be in place to guard against bias)
:!: the paper is so badly written that it is incomprehensible


also:

How to read a paper: Assessing the methodological quality of published papers

http://bmj.bmjjournals.com/cgi/content/ ... 5/7103/305

Here is another interesting commentary on the topic of peer review from Nature:

Perspective: Does peer review mean the same to the public as it does to scientists?

Even reviewed literature can be cherry-picked to support any argument.
The research community understands that scientific information that has not been peer reviewed should not be taken seriously. As scientists, we discriminate between what is put out on blogs or in press releases and what is published in the formal scientific literature. We also know the difference between a peer-reviewed primary paper or review, and an unreviewed letter to the editor or opinion piece. In other words, we understand the peer-review system, and use it as a filter to sort the wheat from the chaff.

As other contributors to this forum will no doubt opine, the peer-review system has its flaws. My own view is that it's the least-bad system that can be devised, and that, although it might need tinkering with, its fundamentals should remain intact. One way to consider its operating principle is: 'Judgement of the scientists, by the scientists, for the people.' But do the people understand the limitations of the process? I suspect not. Even science writers and journalists who should act as important links between scientists and the public sometimes seem not to appreciate what peer review means.

It's been peer reviewed, so it must be right, right? Wrong! Not everything in the peer-reviewed literature is correct. Indeed, some of it is downright bad science. Professional scientists usually know how to rate papers within their own fields of expertise (all too often very narrow ones nowadays). We realize that some journals are more stringent than others in what they will accept, and that peer-review standards can unfortunately be too flexible. A lust for profit has arguably led to the appearance of too many journals, and so it can be all too easy to find somewhere that will publish poor-quality work.

The public doesn't understand this, how could it? But the term 'peer review' is often equated with 'gold standard'. Hence, the politically motivated, lazy or unscrupulous can use the peer-reviewed literature selectively, to make arguments that are seriously flawed, or even damaging to public policy. Chris Mooney, in The Republican War on Science (Basic Books, 2005), provides several examples of how this operates in the political world.

Professional scientists can see through this tactic. We know that scientific truth evolves on the basis of a mounting consensus, not through an isolated paper that adopts a maverick position, even if it has been 'peer reviewed'. In contrast, politicians all too often cherry-pick the 'facts' they find most convenient to their party's agenda. And politicians are not alone.

In my own field of AIDS research, a small clique of scientists and scientifically ignorant laymen promotes the bizarre view that HIV does not cause AIDS, or, in a particularly dubious variant of the genre, that HIV does not actually exist. These AIDS denialists are experts at selectively using the peer-reviewed literature to superficially bolster their positions. I think they lack the training – or if trained, the integrity – to appreciate two things that are understood by professional scientists. First, that peer-reviewed literature develops over time, so that what was legitimately uncertain 20 years ago is fully understood today. This means that citing decade-old papers and ignoring more recent ones is an unscrupulous tactic. Second, that ignoring every paper bar the one that most conveniently suits a preconceived position could be considered scientific misconduct.

Similar practices can be found in other science-related areas. For example, advertisements claiming that vitamin pills can cure cancer and infectious diseases selectively cite the peer-reviewed literature.

One problem the public faces when trying to understand science is that the peer-reviewed literature is still not generally accessible, despite the efforts of the open-access movement (see 'Access to the literature'). I am often asked to email scientific papers to AIDS activists who cannot easily access publication databases. I shudder to think how frustrating it must be for the true layperson entering an area of research for the first time, without the professional connections to acquire information, let alone interpret it. The publishers need to wise up; the public has a right to see the papers its tax dollars pay for. Otherwise, the public may resort to the Internet to inform them – to the blogs and websites that all too often promote strange, pseudoscientific ideas.

And so, despite professional scientists laughing at the notion that HIV does not cause AIDS, some vulnerable, newly infected people, who would like to believe that they have not just contracted a deadly virus, end up surfing the web for answers. Sadly, so do some science writers and their editors (see www.aidstruth.org for recent examples). This kind of fiasco might be avoided if the public had better access to the peer-reviewed literature, and if bona fide scientists were willing to give the public more assistance in interpreting it properly.

http://www.nature.com/nature/peerreview/debate/op5.html

Wait no more... :wink:
Winnie
"Make it a powerful memory, the happiest you can remember."

livsparents
Posts: 1629
Joined: Thu Jun 23, 2005 10:29 pm

Postby livsparents » Thu Jun 08, 2006 8:38 pm

Winnie wrote:
Peer review is fundamental to science. C'mon folks, this is not some remote concept...one could look it up in far less time than it takes to type a half dozen posts insinuating that it is a practice used by conspiracists.



The only conspiracy I see with the process is that well funded, pharmaceutical-ly backed (many times to ready a new multi-billion dollar-revenue drug for market) study will move through quickly and magically be reviewed favorably, while 'orphan' drugs and diseases will languish because they don't have the muscle to get pushed up "the Hill" (although I can't for the life of me figure out 1 out of 166 is an orphan! :?).

The conspiracy, as I have posted on another thread, is that there seems to me to be not enough money to be made on autism studies by the large corporations controlling much of the federal funding of 'legitimate' peer reviewed projects. Face it, the pharmeceutical lobby on Capital Hill is enormous; I'm not even sure there IS a lobby for the Speech or ABA or Occupational therapists; a homeopathic lobby would be lucky to get an audience with the Pope of Liberal Causes Ted Kennedy, let alone anyone in the Red Zone...

In short, I'm through talking about the conspiracies that we will probably never be able to prove, there are enough hiding in plain sight that require my energy...
Bill

Gabesmom
Posts: 643
Joined: Mon Mar 27, 2006 10:35 pm

Postby Gabesmom » Thu Jun 08, 2006 8:41 pm

Thanks for saving me the trouble on that one Winnie. That is why calls to consider the source are indeed important. It's also frustrating when you go to internet sites and you have to pay ridiculous fees to access information that should be available to all. One thing for those of you with library cards. If your library has a website, find out if they're a member of Ebsco (I think that's the right spelling) If they are, you can sometimes use your library card number to access their catalog of journals right from the comfort of your own home without paying a dime. I have used this free service many times, including when I first started my search about autism. There are also Eric documents, publications regarding education, including special education.
Jennifer, Mother to Anna, 5 (NT), Gabriel, 4, Autistic

srinath
Posts: 941
Joined: Tue Dec 20, 2005 12:33 pm

Ah well ...

Postby srinath » Thu Jun 08, 2006 9:53 pm

So a peer review should be done by a group of "non mad in the specific slant of study but be in the same field" scientists ...
Isn't that rather microscopic ...
As in we should find a group of metal toxicologists not slanted to autism causation one way or another to review one man's work on metal toxicology and autism causation ... How is that example ...
Of course ... say 5 out of 10 of the peer's agree on it ... and they get shunned and suffer from career anaemia ... we have effectively eliminated 1/2 our panel for the next study that comes along ... and say these 5 get together and form themselves a support group and say that this Idea deserves further investigation :lol: :lol: :lol:
... and say they are from Danmark, Australia, and North america ... and they decide to start a multi national organisation ... and say they call it The DAN (for you know danmark Australia and North america) scientists research group ... What would that be like ... (son of the beach type bikini clad bimbo's with big ... eyes ... in cheesy dream sequence) :lol:
How would that be guys ... :lol:
Cool.
Srinath.


Return to “Autism Articles, Studies & Politics”