COVID-19 | science, damage limitation, NO POLITICS

[churly]

As far as I've been able to gather, the paper was withdrawn because of perceived unreliability of the source data; that's not something peer review is designed, or indeed able, to detect.

Jeremy

Having done many peer reviews  for journals (not in the medical area), I agree that it is often very difficult to evaluate the reliability of data unless there is a clear methodological flaw or systematic error in the gathering of the data.  Keep in mind that folks doing peer review are busy people and the time spent doing peer review is essentially volunteered in the sense that the 4-5 hours required to do a good peer review just takes time away from all of the other things that need to be done.

[Alan Goldhammer]

Having done many peer reviews  for journals (not in the medical area), I agree that it is often very difficult to evaluate the reliability of data unless there is a clear methodological flaw or systematic error in the gathering of the data.  Keep in mind that folks doing peer review are busy people and the time spent doing peer review is essentially volunteered in the sense that the 4-5 hours required to do a good peer review just takes time away from all of the other things that need to be done.

+1; this was my experience as well back in the day when I was still an active researcher.  It is the same time commitment for reviewing research grant applications.

[Robert Roaldi]

Exactly! We're all biased to some degree, and earning a living, maintaining the cash flow, and increasing one's prosperity, is a major cause of bias. However, I wouldn't describe that as 'without much consequence'.

Here's an informative article from the 'Journal of the Royal Society of Medicine' which addresses the problems of the Peer Review process.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1420798/

"Peer review is at the heart of the processes of not just medical journals but of all of science. It is the method by which grants are allocated, papers published, academics promoted, and Nobel prizes won. Yet it is hard to define. It has until recently been unstudied. And its defects are easier to identify than its attributes. Yet it shows no sign of going away. Famously, it is compared with democracy: a system full of problems but the least worst we have.


Yes, there are all kinds of ways that things can go off the rails. I hope you don't think that this is news.

I used to work in scientific publishing. At every level, guarding against all the things that can go wrong is standard operating procedure every single day. Now and then, something slips through the cracks, we fix it and move on. That's how everything works the world over. You did not just discover something that everyone else missed.
That is why Robbie Fox, the great 20th century editor of the Lancet, who was no admirer of peer review, wondered whether anybody would notice if he were to swap the piles marked `publish' and `reject'. He also joked that the Lancet had a system of throwing a pile of papers down the stairs and publishing those that reached the bottom. When I was editor of the BMJ I was challenged by two of the cleverest researchers in Britain to publish an issue of the journal comprised only of papers that had failed peer review and see if anybody noticed. I wrote back `How do you know I haven't already done it?'   

I'll add a few more quotes from the article, for the benefit of those who can't be bothered to read it.

"But does peer review `work' at all? A systematic review of all the available evidence on peer review concluded that `the practice of peer review is based on faith in its effects, rather than on facts'. But the answer to the question on whether peer review works depends on the question `What is peer review for?'.
One answer is that it is a method to select the best grant applications for funding and the best papers to publish in a journal.

Stephen Lock when editor of the BMJ (British Medical Journal) conducted a study in which he alone decided which of a consecutive series of papers submitted to the journal he would publish. He then let the papers go through the usual process. There was little difference between the papers he chose and those selected after the full process of peer review.

Peer review might also be useful for detecting errors or fraud. At the BMJ we did several studies where we inserted major errors into papers that we then sent to many reviewers. Nobody ever spotted all of the errors. Some reviewers did not spot any, and most reviewers spotted only about a quarter.

So we have little evidence on the effectiveness of peer review, but we have considerable evidence on its defects. In addition to being poor at detecting gross defects and almost useless for detecting fraud, it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.

People have a great many fantasies about peer review, and one of the most powerful is that it is a highly objective, reliable, and consistent process. I regularly received letters from authors who were upset that the BMJ rejected their paper and then published what they thought to be a much inferior paper on the same subject. Always they saw something underhand. They found it hard to accept that peer review is a subjective and, therefore, an inconsistent process."

[Alan Klein]

But think of how white your teeth become.

"Americans are gargling with bleach and drinking household cleaners to prevent coronavirus: CDC report"
https://www.msn.com/en-us/health/medical/americans-are-gargling-with-bleach-and-drinking-household-cleaners-to-prevent-coronavirus-cdc-report/ar-BB156p7v

[hogloff]

That's not the way in which the words are used in the UK, Bart.

And, for what it's worth, the virus is infectious but it is not contagious (spread by touch).

Jeremy

Where did you hear this? It's information ( bad info ) spreading around that does more harm than good. Yes, it is mostly spread through the air...but I haven't read one item saying it is not spread from object to human. Do you have your source on this information ( misinformation ) ?

[Ray]

Yes, there are all kinds of ways that things can go off the rails. I hope you don't think that this is news.
I used to work in scientific publishing. At every level, guarding against all the things that can go wrong is standard operating procedure every single day. Now and then, something slips through the cracks, we fix it and move on. That's how everything works the world over. You did not just discover something that everyone else missed.

Robert,
Of course we all make mistakes. That's obviously not news. For example, you have just made the mistake of inserting your reply to my post in the middle of my italicized quotes from the article, causing the confusion that your reply is part of the quotes. 

The purpose of the article was to point out that there is an over-rated confidence and belief in the efficacy of the Peer Review system. Consider the relevant quotes below. You seem to have missed the main point of the article, perhaps because of your own biases. 

"At the BMJ we did several studies where we inserted major errors into papers that we then sent to many reviewers. Nobody ever spotted all of the errors. Some reviewers did not spot any, and most reviewers spotted only about a quarter.

So we have little evidence on the effectiveness of peer review, but we have considerable evidence on its defects. In addition to being poor at detecting gross defects and almost useless for detecting fraud, it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.

People have a great many fantasies about peer review, and one of the most powerful is that it is a highly objective, reliable, and consistent process.

From the Conclusion:

So peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative, and scientists and editors have a continuing belief in peer review. How odd that science should be rooted in belief."

[BernardLanguillier]

As far as I've been able to gather, the paper was withdrawn because of perceived unreliability of the source data; that's not something peer review is designed, or indeed able, to detect.

Jeremy,

If you look at the details, you will see that there were very clear inconsistencies that should have triggered major red flags had the paper been peer reviewed, which it apparently wasn't, but also at the level of editor.

The Lancet first corrected some of these issues before deciding to withdraw the paper.

Cheers,
Bernard

[Alan Klein]

"Hydroxychloroquine is not dead yet
Trials for its use against COVID-19 continue, with some researchers still hopeful it can serve as a preventative in people not yet exposed to the virus."
https://www.nbcnews.com/health/health-news/hydroxychloroquine-not-dead-yet-n1224586

[Jeremy Roussak]

Jeremy,

If you look at the details, you will see that there were very clear inconsistencies that should have triggered major red flags had the paper been peer reviewed, which it apparently wasn't, but also at the level of editor.

The Lancet first corrected some of these issues before deciding to withdraw the paper.

Bernard, my interest is insufficient. I gave up reading The Lancet as a medical student, when I realised that I couldn't understand even the titles of half the articles (the BMJ was easier to cope with). Horton's record for withdrawing misleading papers before they do significant harm is far from stellar: it took him something like a decade to retract Wakefield's spectacularly damaging paper on MMR and autism.

Jeremy

[Alan Goldhammer]

"Hydroxychloroquine is not dead yet
Trials for its use against COVID-19 continue, with some researchers still hopeful it can serve as a preventative in people not yet exposed to the virus."
https://www.nbcnews.com/health/health-news/hydroxychloroquine-not-dead-yet-n1224586

I wonder if any of these trials will be completed.  The Duke study is designed to enroll 15,000 patients (the statistician who helped design the study is a former colleague).  A lot of healthcare workers are not going to take the drug but may participate in the control arm (this was mentioned in the article).  Most of my friends would not take it because of the adverse event profile and long duration in the body (half life is 24 days). It is also much more difficult to prove prevention than it is treatment for statistical reasons.  If the virus wanes during the summer, proof of prophylaxis will be very difficult and it's unclear what the correct dosage ought to be.  For malaria it was a once a week dose regimen but the drug is not used much these days because of resistance.  I am not holding out much hope that we will find out whether it works or not.  Meanwhile there are other drugs which do have a plausible mechanism for prevention that are being understudied. 

[Alan Klein]

Having done many peer reviews  for journals (not in the medical area), I agree that it is often very difficult to evaluate the reliability of data unless there is a clear methodological flaw or systematic error in the gathering of the data.  Keep in mind that folks doing peer review are busy people and the time spent doing peer review is essentially volunteered in the sense that the 4-5 hours required to do a good peer review just takes time away from all of the other things that need to be done.

Bernard, my interest is insufficient. I gave up reading The Lancet as a medical student, when I realised that I couldn't understand even the titles of half the articles (the BMJ was easier to cope with). Horton's record for withdrawing misleading papers before they do significant harm is far from stellar: it took him something like a decade to retract Wakefield's spectacularly damaging paper on MMR and autism.

Jeremy

So how is the average layman suppose to accept scientific reports and studies when scientists themselves have little respect for the science journals and studies? When we laymen refuse to follow Covid recommendations or Climate Control, we're called to task.  Well, it seems that we're right not to respect these studies.  They only sew more confusion and distrust of the whole process. 

[Alan Goldhammer]

So how is the average layman suppose to accept scientific reports and studies when scientists themselves have little respect for the science journals and studies? When we laymen refuse to follow Covid recommendations or Climate Control, we're called to task.  Well, it seems that we're right not to respect these studies.  They only sew more confusion and distrust of the whole process.

the overwhelming majority of papers that are peer reviewed are solid.  Some areas of medicine and science are inexact.  A classic example to point to are the antidepressants.  We don't have a simple test to determine whether someone suffers from depression.  We know a lot about brain chemistry but not enough.  Drugs get developed using subjective measures, does the patient feel better; is he/she able to better handle every day tasks; do they have fewer suicidal thoughts, etc.  It would be great if we could do neural scans and say for certain the drug is helping you.  Most antidepressants work on maybe 60% of the patients and often different drugs need to be tried to find one that is optimal for that patient.  You can read all the papers on this topic and still not have a firm idea of what drug works for which patient.

Following COVID-19 recommendations is quite different from climate change.  Climate change may not affect your life over the next 5 or so years but if you go out to a crowded market without a mask and start talking to people for more than five minutes or so you put yourself at risk.

[Robert Roaldi]

So how is the average layman suppose to accept scientific reports and studies when scientists themselves have little respect for the science journals and studies?

What?  Who says they have no respect for them. Knowledge in a subject area is built up over time when studies are replicated and results confirmed by more than one researcher. It is pretty rare that one solitary study leads to immediate policy, if ever, so why do you pay attention.

The answer to your first question is that the average layman is not supposed to understand what is written in specialized technical papers. Why would you expect to? Do you expect to understand papers in mathematics or physics journals? Do you read IEEE journals to understand how new computer chips work?

I'm sorry but your premise makes no sense. And anyway, you asked the same thing yesterday when I answered that what you do is rely on doctors and public health authorities to keep abreast of these things and act as consultants when it comes time to enact policy.

[faberryman]

So how is the average layman suppose to accept scientific reports and studies when scientists themselves have little respect for the science journals and studies? When we laymen refuse to follow Covid recommendations or Climate Control, we're called to task.  Well, it seems that we're right not to respect these studies.  They only sew more confusion and distrust of the whole process.

It is really pretty straight forward: you believe the articles you want to believe and call the others fake news. Then when asked how you tell the difference, you just say "discernment". At least that is what you told me when I asked you how you distinguished true stories from fake news in the New York Times. Discernment. Should work for scientific articles as well.

[Chris Kern]

So how is the average layman suppose to accept scientific reports and studies . . .

(1) Wait for an expert consensus to develop.
(2) Accept the consensus since, as "an average layman," you do not possess the credentials to dispute it.
(3) If the expert consensus changes as new evidence is acquired, accept the new consensus.

[Ray]

(1) Wait for an expert consensus to develop.
(2) Accept the consensus since, as "an average layman," you do not possess the credentials to dispute it.
(3) If the expert consensus changes as new evidence is acquired, accept the new consensus.

That would be fine if we were all objective and unbiased people. But we are not. Even scientists can belong to a group that promotes their own self-interest. I find the issue of 'climate change' quite fascinating because it presents so many examples of 'untruths' or 'partial truths' that are promoted in the interests of getting a certain type of political and economic action.

I'm reminded of the famous 'double ethical bind' quote from the late Dr. Stephen Schneider who used to work in the field of climate science at Stanford University. I found it a very revealing explanation for the confusion and continual disagreements about humanity's effect on the current changing climate.
Here's the quote:

"On the one hand, as scientists we are ethically bound to the scientific method, in effect promising to tell the truth, the whole truth, and nothing but — which means that we must include all the doubts, the caveats, the ifs, ands, and buts. On the other hand, we are not just scientists but human beings as well. And like most people we’d like to see the world a better place, which in this context translates into our working to reduce the risk of potentially disastrous climatic change. To do that we need to get some broad based support, to capture the public’s imagination. That, of course, entails getting loads of media coverage. So we have to offer up scary scenarios, make simplified, dramatic statements, and make little mention of any doubts we might have. This ‘double ethical bind’ we frequently find ourselves in cannot be solved by any formula. Each of us has to decide what the right balance is between being effective and being honest. I hope that means being both."

Regarding the problem of the 'average layman' understanding complex, scientific issues, a quote from Albert Einstein might be relevant.

"If you can’t explain something simply, you don’t understand it well.
Most of the fundamental ideas of science are essentially simple, and may, as a rule, be expressed in a language comprehensible to everyone.
Everything should be as simple as it can be, yet no simpler."

Crikey! I've mentioned the word 'political' in my post. Sorry! 

[Alan Klein]

If we have to blindly trust the experts, what need would there be for a second opinion? 

[Robert Roaldi]

If we have to blindly trust the experts, what need would there be for a second opinion?

As always, you're free to seek out as many opinions as you like. No one is stopping you.

[Ray]

Wow!! I'm very impressed.

"New Zealand Prime Minister Jacinda Ardern says the country has officially eradicated COVID-19 and will return to normal after the last known infected person recovered.
Isolation and quarantine for those arriving from abroad will continue.

Large public gatherings, such as concerts and sports events, will be allowed for the first time since March 23, when Ardern announced a nationwide lockdown amid a rising number of daily cases. Restaurants and public transport will also be allowed to resume normal operations."
https://www.npr.org/sections/coronavirus-live-updates/2020/06/08/871822321/with-no-current-cases-new-zealand-lifts-remaining-covid-19-restrictions

[Alan Goldhammer]

Wow!! I'm very impressed.

"New Zealand Prime Minister Jacinda Ardern says the country has officially eradicated COVID-19 and will return to normal after the last known infected person recovered.
Isolation and quarantine for those arriving from abroad will continue.

Large public gatherings, such as concerts and sports events, will be allowed for the first time since March 23, when Ardern announced a nationwide lockdown amid a rising number of daily cases. Restaurants and public transport will also be allowed to resume normal operations."
https://www.npr.org/sections/coronavirus-live-updates/2020/06/08/871822321/with-no-current-cases-new-zealand-lifts-remaining-covid-19-restrictions

This is not surprising at all.  Countries that took the right precautions early on have seen success.  The same thing is happening in Israel where they had an extreme lockdown and now hardly any cases and kids are back in school.  Israel has a mandatory quarantine on visitors as well.  the US could have done much better but people disregard public health measures.  Here is an instructive survey of epidemiologists in the US about activities they would or would not undertake over the next year:  https://www.nytimes.com/interactive/2020/06/08/upshot/when-epidemiologists-will-do-everyday-things-coronavirus.html