Source: France Soir
A letter to the editor of the journal CMI (Clinical microbiology and infection), Nicolas Védrines, Anne-Typhaine Bouthors PhD and doctors Stéphane Gayet and Alexis Lacout marred the Fiolet meta-analysis. The conclusions of the authors of the letter are final:
Overall, our review raises serious concerns about the relative risk (RR) choices made for study inclusion and selection. The main results reported by Fiolet et al. (contributors) are based on four false statements of the original work.
In light of these elements, we believe that the authors should at least reconsider their conclusions.
On the Fiolet meta-analysis, questioned by this letter
Recall that young researchers, non-doctors, had published with great media support at the end of August a meta-analysis which concluded to the dangerousness of hydroxychloroquine (HCQ) and which recommended to stop all studies on the subject, with a violent certainty and disconcerting.
Professor Raoult had very quickly sent a “warning letter” to the newspaper about this study which seemed to him incomplete and including too much bias.
On this new letter to the editor
The four authors of this letter to the editor come back to this meta-analysis in detail by doing a systematic review of the Fiolet meta-analysis. However, the answer is limited in number of characters; it is a constraint of the editor. They do not declare any link of interest. Doctor Lacout tells us: “ This analysis took into account the biases that had been omitted by the young researchers, surely by inexperience in the medical field. “
Joined today Nicolas Védrines told us: “ we are a little disappointed because the publisher has somewhat modified our conclusions. The journal CMI did not wish to publish the corrected statistical analysis because of their inability to set up a proofreading process for the letters, too bad. This analysis is currently published on the pub-peer site, a site specializing in study reviews. Fiolet will be able to answer them ”. And he adds: “the cited corrections can only be refuted in bad faith . However there are experts in this field ”.
It appears, that Fiolet’s meta-analysis and Fiolet’s possible refutation of this letter to the editor, that science could be considered subjective. The anti-Raoult position taken by the authors of the Fiolet meta-analysis would therefore be correlated and causal to the result of their meta-analysis.
FS: What consequences does your letter have for hydroxychloroquine (HCQ)?
NV: “Four corrections result in a significant effect of HCQ and an absence of a negative conclusion on the dual therapy HCQ + AZT (Azythromycin) . Adding Arshad would have amplified a significant effect. Arshad was not included so that Fiolet et al. cannot refute our conclusions about the mere addition of Arshad. “
FS: Are your corrections only in favor of hydroxychloroquine?
NV: The corrections have not always been in favor of the HCQ. For example we did not include Arshad in the statistical analysis. In addition, we corrected the ratio of the Ayerbe study because Fiolet was based on a document in pre-publication. The final publication is less in favor of the HCQ. Note that Fiolet et al made two errors on this point: one on the fact that they described this study as having already been published when it was a pre-publication, and another on the erroneous bibliographic reference concerning this study. . The proofreading does not seem to have been of very high quality. Other criticisms could be made such as cohort duplication (Rivera and Kuderer) which makes it impossible to accurately figure out the number of patients in the control group.
Note also that the Cochrane recommends the use of adjusted data, which Fiolet did not do for Cavalcanti. For this study, we took the Relative Risk (RR) least in favor of the HCQ by respecting the choice of Fiolet concerning the analysis on the HCQ. The effect would therefore have been more in favor of HCQ. And since we wanted to be honest with the methodology, we took the adjusted RR for dual therapy because it is less favorable to the RR without adjustment. In short, we took the worst RRs of Cavalcanti and the effect should have been more in favor of HCQ. A previous FS article mentioned this problem (see the article “A meta-analysis and the return of the veterans”).
In conclusion, we have deliberately set aside three points, the taking into account of which would have amplified the positive effect of HCQ (Arshad, Ayerbe, Cavalcanti).
FS: Did you wonder about the RECOVERY study overdose?
NV: Yes, the effect is greater excluding the specificity of the RECOVERY study where patients are treated with a very high dosage of HCQ. A statistical analysis specific to the SOLIDARITY and RECOVERY data would point to this hypothesis of a harmful effect in the event of overdose.
HCQ has been criticized for its antiviral effect, but it also acts at the immunomodulatory and anti-inflammatory levels and significantly accelerates the development of adaptive immunity (seroconversion much greater than D14 observed in Mitja PPE). In this case, it would not be necessary to resort to high doses. It should also be noted that the combination of HCQ and Bromhexine appears to be effective at the anti-viral level. Theoretical analysis is very good and two RCTs give good results. The Russians have just recommended this dual therapy. We should also note the antiviral efficacy of the HCQ and zinc bitherapy, observed in a RCT in 500 patients (p-value of 0.001 on day 7).
Regarding the National Agency for Health and Medicine (ANSM), she says she is thinking about resuming the studies stopped at the end of May following the LancetGate. HYCOVID specifies that it was not able to resume because there were not enough patients. But today this is no longer the case and the dosages of these studies are not overdose. Especially the known results of these studies are encouraging. They could confirm our statistical analysis, the methodology of which uses the criteria of opponents of HCQ such as Fiolet and his collaborators. It is absurd not to take over DISCOVERY and HYCOVID. Opponents of HCQ are present in the teams of these studies and they will certainly lobby for this research on HCQ not to resume. The IHU should supervise the resumption of these studies.
FS: what about the other studies in Fiolet’s meta-analysis?
NV: We observed a problem of selection bias from Fiolet et al. While the ANSM and the High Council of Public Health validate the Arshard study as a reference, it was not included in their statistical analysis. This positive study on the efficacy of HCQ should have been included. However, our statistical analysis does not include this study. The positive effect is therefore greater than that which we observe in the statistical analysis. It is important.
Fiolet et al. asserted that Fried’s study could have been included if this publication had taken place before the end of their selection, stating that the workforce is large and that the conclusion is negative. We conclude to media manipulation because apart from many other biases, this study only gives a crude percentage of mortality, without any multivariate analysis, and the authors claim that the patients treated were in a more severe state. This constitutes a red flag and Fiolet should reread Cochrane. If Professor Raoult had acted in the same way, he would have received insults.
Conversely, a posteriori, a Belgian study and an Italian study could have been included. These publications concern a very large workforce and their conclusions are positive. Anti-HCQ claim that the groups had too large age differences. We observe subgroups stratified according to age in the Belgian study and a delta of 7 years in the Italian publication. At the same time, Fiolet included Sbidian where we observe a delta over the age of 8 years.
But there have been other recent publications, including SOLIDARITY. Through a recent meta-analysis in pre-publication, anti-HCQ jokes that HCQ kills when the problem comes from overdose. By removing the overdose of RECOVERY, the positive effect of HCQ would have been greater in the Fiolet meta-analysis. The studies guided by WHO (RECOVERY and SOLIDARITY) are very specific. They do not correspond to the usual dosages of international experts.
FS: What conclusions do you draw from your analysis?
NV: The Fiolet team gives the feeling of biasing the analyzes, by orienting them against the HCQ. But nothing surprising when you know the hateful context towards Professor Raoult.
What is serious is that their reference is often cited in international publications. If the review does not require a modification of the conclusions, one might wonder about a problem of corruption. We agree to a review of our statistical analysis.
The scientific approach is essential in my opinion, but I am disappointed by this science filled with certainties or with the desire to create a buzz. We should learn from the manipulation seen with the LancetGate.
Note that at the same time as our letter was published, the journal CMI also published a document linked to Fiolet’s meta-analysis on “the difficulty of editorial decisions”. It justifies its decision to publish Fiolet in particular because of the particularity of the conclusions (“… an alarm bell, and it could have influenced our decision”). Rigor was visibly lacking; the desire to create a buzz is a problem.
FS: What is the result of the revised and unpublished meta-analysis by CMI?
NV: The results and conclusions of the Fiolet meta-analysis are called into question.
The journal CMI did not publish the table of the redesigned analysis presented in the initial letter to the editor, as a review was necessary. This letter format does not allow this review. After taking into account the four biggest errors, the statistical analysis was redone and the conclusions are modified. Even when adding the Geleris and Arshad studies, the results remain in favor of hydroxychloroquine.
An example of error, we do not know the final outcome (cure or death) for 40% of patients not included in the Magagnoli sub-cohort. Do you prefer to base yourself on as many unknowns or on a group where 100% of the outcomes are known? Fiolet prefers strangers. We find it desirable to prefer to avoid bias, especially since the treated patients are in a more severe condition, so they tend to die sooner, which completely biases the estimate of the Hazard Ratio.
The results are in favor of hydroxychloroquine with a relative risk RR of 0.79 and a confidence interval below 1. HCQ would therefore be effective.
Regarding the dual therapy, it is not possible to conclude that the result is not significant.
FS: Did you get any responses from the authors of the Fiolet meta-analysis?
NV: We were able to note a few responses given by authors of the Fiolet meta-analysis that are more like insults. These same people complained some time ago to be victims of insults. Here are some examples of the messages that we receive in response. Nothing constructive.
Rebeaud : “it makes them happy to jerk off in a circle”, “it’s not terrible” “Kilsonkons”
Peiffer-Smadja: “Their analysis is frankly zero. On pubpeer because no newspaper will accept it. “
Another anonymous comment “I wouldn’t like to be treated in Aurillac” (note: workplace of one of the authors of the critical letter of the meta-analysis.)
Revisited analysis of the Fiolet study
The authors of the letter insisted that the revised variant of their analysis be published in order to remove any doubt about the conclusions of the statistical analysis transmitted to PubPeer,
Our conclusions on the Fiolet meta-analysis are not affected if the Geleris, Rivera and Arshad studies are included.
Statistical analysis of the efficacy of hydroxychloroquine with the included Geleris and Arshad studies shows that hydroxychloroquine is effective ( with a significant relative risk ratio of 0.74).
Regarding the statistical analysis of the efficacy of the HCQ and AZ dual therapy with the included Rivera and Arshad studies: There is no possible conclusion of a harmful effect.
Dominique Costagliola, research director at Inserm and member of the Académie des Sciences, praised the seriousness of the work carried out by the Fiolet study, declaring that ” they do not include studies at risk of critical bias, which is recommended . But in the appendix, we still have the results that are obtained if we include these studies . ” We really have everything we need to be able to judge the relevance of what is done and the results. “” We can quite see that observational studies that are not too badly done do not find things very different from clinical trials “, she notes. Conversely, notes the epidemiologist,” when we have studies with critical biases, we find more favorable things,
The review CMI must act and imperatively and request the modification of the erroneous conclusions with the serious consequences of the study Fiolet et al.
The other option is the retraction of this meta-analysis, no matter what the opinion of it. author Fiolet. This meta-analysis, which had received significant media coverage on the various television channels of Dr Nathan Peifer Smadja, will also have to be the subject of a contradictory one.
Inserm should undoubtedly carry out an audit and take sanctions for what could be considered at least as a scientific error ( a reasoning or a procedure not respecting a set of rules recognized by the scientific community, defined as involuntary ) or scientific fraud (action intended to deceive in the field of scientific research, which constitutes a violation of the research ethics and professional ethics in force within the scientific community)
Mr Fiolet contacted today by the editorial staff was not available.
Annex, detail of the letter
The detail of the letter is reproduced below
We read with interest the article by Fiolet, which examined the effect of the hydroxychloroquine (HCQ) regimen alone or in combination with azithromycin (AZI) on COVID-19 mortality. For the effect of HCQ alone on mortality, we noted the following inaccuracies:
1) Fiolet et al. misinterpreted the relative risk (RR) in the Geleris study as that of mortality. The original work used the composite criterion “ intubation or death ”.
2) Fiolet et al. reported the relative risk associated with the entire patient cohort in the Magagnoli study. This ratio was affected by biased sampling depending on the duration of the sampling, and moreover a large number of parameters were missing which led to an imbalance between the groups (18% of patients on HCQ, 12% of patients on HCQ + AZI and 26% of controls were still hospitalized). Magagnoli performed a sub-analysis to account for “the issue of time-biased sampling and differential rates of right-censored observations among groups” and provided a better estimate of the RR.
For the effect of HCQ + AZI on mortality, we noted:
3) Fiolet erroneously interpreted the relative risk in the Rivera study by combining it with HCQ + AZI treatment. The original work indicates that this RR was associated with “HCQ plus any other treatment”, which was not limited to AZI since approximately 22% of these patients were not treated with AZI.
4) For the reasons mentioned in 2., the RR of the Magagnoli study sub-analysis. should have been reported.
In addition, it should be noted that Fiolet excludes the results reported by Arshad et al. which provided additional evidence in favor of the use of HCQ. Fiolet argued that patients in the HCQ arm were twice as likely as controls to receive steroids. However, the Arshad study corrected for this imbalance using propensity score matching and found the RR of corticosteroid-associated mortality to be 0.8, which is remarkably similar to the effect of dexamethasone on mortality. evidenced by the Recovery trial (RR = 0.83). On the other hand, the Rivera study. was included in the meta-analysis even though the original work reported a relative ratio of 3 associated with steroid use, which strongly suggests that indication bias does not have not been sufficiently corrected. We recognize that Arshad’s results are not without bias or limitations, visible in their multivariate regression, but they did provide detailed information about multivariate regression scores and match tables, which for example were absent in Magagnoli.
Such changes in the meta-analysis could reverse the conclusions of the original article.
In observational studies, the groups of patients are heterogeneous and vary considerably from one study to another, that is to say therapeutic management (type of oxygen therapy, corticosteroid therapy, anticoagulation, time to implementation of these different treatments), different population (eating habits, smoking). Such an analysis does not describe an often more complex reality, which is also a major bias.
Overall, our review raises serious concerns about the choices made for study inclusion and selection. The main results reported by Fiolet et al. are based on four errors or misinterpretations of the original work.
In light of these elements, we believe that the authors should at least reconsider their conclusions.
Related: