Kel073 1154.1157

How evidence-based are advertisements in journalsregarding the subspecialty of rheumatology? P. van Winkelen, J. S. van Denderen, C. Y. Vossen, T. W. J. Huizinga1 andF. W. Dekker for the SEDUCE study group Objectives. In rheumatology, five different billion-dollar drugs have emerged in recent years, making this subspecialty the focusof extensive advertising campaigns. Considering this development and the fact that the scientific content of advertisements hasbeen questioned, we initiated a study to determine how evidence-based advertisements are in four leading journals ofrheumatology.
Methods. Advertisements were extracted from the journals Arthritis and Rheumatism, Rheumatology, Seminars in Arthritisand Rheumatism and the Journal of Rheumatology, and were evaluated using a standardized score form.
Results. In total, 353 advertisements were obtained, of which 84 were unique. Of the 300 references provided by these uniqueadvertisements, 53 (18%) were considered ‘supporting’. In addition, 87 (29%) of the 300 references referred to randomizedcontrolled trials (RCTs), of which 49% supported the claim. The vast majority of RCTs (97%) were sponsored by theadvertising company. In the 84 unique advertisements 288 claims were made, of which 190 provided one or more references. Ofthese 190 claims, 33 (17%) were judged ‘well supported’. In total, only four (5%) of the 84 different advertisements were judged‘well supported’ and 11 (13%) of the 84 were judged ‘misleading’ because they contained one or more misleading claims,i.e. claims with references to articles contradicting the claim.
Conclusions. This study indicates that few advertisements in rheumatology journals are entirely evidence-based.
KEY WORDS: Advertisements, Evidence-based, Rheumatology.
Medical journals are the major source of information for doctors, claim more impressive, so other researchers focused on the through both peer-reviewed articles and advertisements. The references accompanying the claims. Cooper described the kinds scientific content of advertisements, trying to meet commercial of documents cited in support of claims and assessed the and scientific standards at the same time, has been questioned availability of these documents [7]. Lankinen, Villanueva and [1–9]. The World Health Organization (WHO) formulated Loke examined references in more detail. The study of Lankinen ethical and scientific criteria for pharmaceutical drug promotion et al. indicated that 21% of references were either totally irrelevant in 1968. According to these criteria, advertisements should to the claim, or at least did not provide scientific support to it [1].
provide ‘scientifically based, balanced factual information without In the study of Villanueva et al., the promotional statement was ambiguities’ [10]. Another attempt to establish norms for promo- not supported by the reference in 41% of claims [3]. Loke et al.
tional quality was made by the International Federation of found that, of 855 quantitatively substantiated claims, 55% referred to a randomized controlled trial or a systematic review The code became operational in 1981 and the accompanying as their source of information [4]. Three studies [5, 6, 8] assessed complaints procedure in 1982. Information is judged according to the methodological quality of articles that were referred to, but whether it is up to date, meets ethical standards, is not misleading, only Lexchin and Holbrook used an official score to assess the reveals its commercial nature, and provides qualification of methodological quality of articles cited [8]. Further, of all studies keywords such as ‘safe’. The promotional material should be only two focused on a single medical discipline [5, 9].
‘based on an up-to-date evaluation of evidence’. A number of None of the researchers focused on the discipline of rheuma- pharmaceutical companies have adopted the IFPMA code as tology. This subspecialty, however, has been the subject of their guideline for advertising [11].
important changes during the last 6 yr. According to the 2005 Several studies have tried to substantiate the value of pharma- president’s speech of the American College of Rheumatology, ceutical advertisements as a source of information for doctors, five different billion-dollar drugs have emerged in rheumatology thereby focusing on a wide range of issues. Some researchers in recent years, making this subspecialty the focus of the ‘most focused on the presence of important information such as drug Considering this development and the results of the studies References supporting the promotional claims can make the mentioned above, it is interesting to evaluate the quality of Department of Clinical Epidemiology and 1Department of Rheumatology, Leiden University Medical Center, Leiden, The Netherlands.
Submitted 22 November 2005; revised version accepted 7 February 2006.
Correspondence to: F. W. Dekker, Department of Clinical Epidemiology, Leiden University Medical Center, PO Box 9600, 2300 RC Leiden, The Netherlands. E-mail: [email protected] ß The Author 2006. Published by Oxford University Press on behalf of the British Society for Rheumatology. All rights reserved. For Permissions, please email: [email protected] How evidence-based are advertisements in journals? information given in these advertising campaigns. Therefore, the with nothing but well supported claims were labelled ‘well aim of the present study was to measure the quality of evidence supported’. Advertisements with one or more misleading claims behind advertisements in four leading journals of rheumatology.
were considered ‘misleading’. Score forms were given to two This was done by focusing on claims and evaluating the researchers, who entered the data in SPSS (Chicago, Illinois, USA).
accompanying references. References to trials were evaluatedwith regard to their methodological properties and potentialto support the claim. For the methodological properties of randomized controlled trials, the modified score of Chalmerswas used.
From the sample of four journals, 353 advertisements wereobtained, representing 31 different products. After exclusion ofidentical advertisements, 84 advertisements remained. These 84advertisements provided a total of 300 references. These references were evaluated; 53 (18%) were considered ‘supporting’, indicating Advertisements used in this study were extracted from four that the article referred to was a traceable, methodologically journals: Arthritis and Rheumatism, January to October 2004; correct RCT (n ¼ 43), a systematic review (n ¼ 2) or other research Rheumatology, 2003; Seminars in Arthritis and Rheumatism, 2002 (n ¼ 8) with sufficient methodological quality (neutral or positive and 2003; and Journal of Rheumatology, 2004. A team of four Chalmers’ score), which dealt with population, intervention and researchers searched the above journals and included all drug- control characteristics comparable to the claim. Of the 247 ‘non- promoting advertisements with one or more claims. Identical supporting’ references, 181 referred to ‘data on file’, ‘prescribing advertisements were copied only once but the frequency of information’ and ‘other sources (e.g. web sites, proceedings, appearance was noted. The advertisements found were distributed research not traceable on PubMed), 44 referred to RCTs and among the research group and evaluated using a standardized 22 to other research. Of the 300 references, 87 referred to RCTs.
score form. Meetings to discuss definitions and qualifications were However, 84 (97%) of the 87 RCTs were sponsored by the organized to diminish inter-observer variability.
advertising company. In none of the 87 cases did the RCT referred Each claim was evaluated. A claim was defined as all statements to get a negative Chalmers score: 57 RCTs got a positive score and concerning the safety or efficacy of a drug, and other statements the remainder a neutral score. Forty-nine per cent of the references if they included a reference. Prescribing information in advertise- to RCTs were supporting the claim, indicating that the Chalmers ments (always included in separate textboxes) was not checked score was neutral or positive and no discordance was found when for claims because this was regarded as non-promotional informa- comparing the advertisement and the article referred to with tion. The quality of evidence-based advertising was evaluated respect to the populations, controls and interventions.
by determining the number of claims with references, their The 353 advertisements obtained represented 31 different products from the whole spectrum of drugs used in rheumatology.
query.fcgi?db¼PubMed), and by comparing the claim with the A total of 116 advertisements (33%) were published on three factual information in the references. Points of comparison were different TNF- blockers. Four coxibs formed the second largest those essential for the validity of the reference serving as evidence group, with 59 advertisements (17%), followed by cholinergic for the claim. Thus, we asked whether the population, the controls agonists (10%), bisphosphonates, viscosupplements and NSAIDs and the intervention in the advertisement corresponded to the (7% each) and a group ‘other’, representing 19% of all advertise- article referred to. Furthermore, all references were divided into ments. Misleading advertisements were found for all the different the following categories: systematic review; randomized controlled drug types except coxibs and biphosphanates.
trial (RCT); research other than an RCT or systematic review; In the 84 unique advertisements, 288 claims were found. Ninety- prescribing information; data on file; and ‘other’ (e.g. web sites, eight claims did not refer to any source, so they were categorized as proceedings, articles not traceable on PubMed). The methodo- ‘poorly supported’. One hundred and ninety claims had one or logical properties of the RCTs referred to were also assessed.
more references. Of these, 16 were judged ‘misleading’, 33 ‘well For this purpose, a score described by Chalmers et al. [13] was supported’ and 141 ‘poorly supported’. Ninety-nine of 141 poorly used, which is a more detailed variant of the more widely used supported claims were so judged because they provided only one Jadad score [14]. Both scoring systems rate the methodology of thestudy by focusing on the reports of randomization, blinding and the handling of withdrawals. The Chalmers score rates these three items on a scale of 0–3. The sum of the three subscores formsthe total score, which can be grouped into the following categories:<3, negative; 3–5, neutral; and 6þ, positive.
Conclusions were drawn using the following definitions.
When references were of neutral or positive methodologicalquality, and in accordance with the claim, they were considered ‘supporting’. When the evidence for the claim was poor (differences in populations, controls or interventions), they werecalled ‘non-supporting’. Because only articles available onPubMed were considered accessible and scientifically interpretablefor practitioners, references were also called ‘non-supporting’when the reference category was ‘prescribing information’, ‘data on file’ or ‘other’. At the claim level, the outcome could be ‘wellsupported’, ‘poorly supported’ or ‘misleading’. Claims referringto traceable systematic reviews or trials, which supported the • 77 data on file (63x being only ref.) claim, were labelled ‘well supported’. Claims without a traceable • 35 prescribing inf. (23x being only ref) reference or with an insufficiently supporting reference were labelled ‘poorly supported’. Claims that referred to an article that contra- dicted the claim were labelled ‘misleading’. Advertisements were • 13 other research (6x being only ref) rated ‘well supported’, ‘poorly supported’, ‘misleading’ or ascontaining both well and poorly supported claims. Advertisements reference, referring to data on file, prescribing information or other RCTs got a neutral or positive Chalmers score. The major problem data (the three poorly supporting categories of information) was that the advertisements tended to extrapolate trial results to other patient groups, dosages or treatments. In addition, Five per cent of the 84 different advertisements were the advertising companies sponsored 97% of the RCTs referred labelled ‘well supported’, meaning that their claims were all well to. This is relevant as the results from RCTs differ according to supported by their references. Thirteen per cent of the 84 different whether they are or are not company-sponsored trials [15].
advertisements were labelled ‘misleading’ because they contained This study included four rheumatology journals. We have not one or more misleading claims, i.e. claims with references to formally tested whether the advertisements in these journals are articles contradicting the claim (Table 1).
a proper reflection of all journals regarding rheumatology, butthere are no data to indicate that advertisements differ in thedifferent journals. Three of the four journals examined were from the American side of the Atlantic. Because both the Americanand the European pharmaceutical industry have adopted the This is the first study that has assessed the quality of information IFMPA Code of Pharmaceutical Marketing Practices as their provided by advertisements in journals covering the subspecialty of advertising guideline [11], we do not expect large differences rheumatology. Remarkably, only four (5%) out of 84 advertise- between American and European advertising practices as a result ments were fully supported by high-quality evidence. Many of the of differences in regulatory requirements. The volumes of the references accompanying claims did not refer to a proper paper journals we used formed a relative small sample, but an up-to-date traceable on PubMed. Three categories—data on file, prescribinginformation, and other (e.g. web sites and readers)—accounted one. The assessment itself was subjective in its core, but researchers for 60% of the references; the category most referred to was data used standardized score forms and discussed difficulties in on file (30%). There is a possibility that data on file will be interpretation. A small pilot study was performed to train published later and will become available for validation of the researchers in using the score form, and to discover shortcomings claim. Therefore, the term ‘non-supporting’ should not be inter- preted as a definitive judgement for this category of references.
In the present study, a rather small percentage (5%) of the Moreover, the relatively high proportion of references to data on advertisements proved to be evidence-based. In a comparable file may be caused by the tendency of pharmaceutical companies study, Gutknecht classified 18% of distinctive advertisements to advertise for new indications for which the data have been (n ¼ 187) as evidence-based. In his study, ‘evidence-based’ meant: presented to the regulatory authorities but have not yet been ‘citing numerical data from controlled clinical trials, and pre- published in the public literature. However, as a way to distribute sumably seeking to influence readers through provision of that evidence’ [2]. We used a wider definition (not restricted to the citing questioned. Cooper and Schriger investigated the availability of of numerical data) and still found a smaller percentage, probably cited documents: only 20% of the requests for data on file because we made a comparison between the claim and the produced a document, showing that data on file are not easy information in the article and found many incompatibilities.
available [7]. Thus, it may be more logical to refer to the formal Only when a referred-to trial supported a claim was the claim label of the drug instead of data on file, because this information is regarded evidence-based. In conclusion, this study indicates that there is room for improvement in the quality of information in Companies do not present a balanced list of references representing pro and contra articles in their advertisement. Thiscould lead to a biased opinion when practitioners do not realize this enough. In the present study we did not assess the potentialinfluence on how evidence-based an advertisement is.
We only assessed the various forms of research (RCT, journals are entirely evidence-based.
systematic review and other research methods) available onPubMed as possible evidence to support claims, because thesearticles give the opportunity to draw conclusions based on theresults given. However, in 44 (51%) of 87 references referring to an RCT, the RCT did not support the claim properly.
The problem was never the methodological quality, as all the TABLE 1. Assessment of advertisements with and without referenced No conflict of interest has been declared by the authors.
1. Lankinen KS, Levola T, Marttinen K, Puumalainen I, Helin- Salmivaara A. Industry guidelines, laws and regulations ignored: quality of drug advertising in medical journals. Pharmacoepidemiol 2. Gutknecht DR. Evidence-based advertising? A survey of four major journals. J Am Board Fam Pract 2001;14:197–200.
3. Villanueva P, Peiro S, Librero J, Pereiro I. Accuracy of pharma- ceutical advertisements in medical journals. Lancet 2003;36:27–32.
4. Loke TW, Koh FC, Ward JE. Pharmaceutical advertisement claims in Australian medical publications. Med J Aust 2002;177:291–3.
5. Bhattacharyya T, Tornetta P 3rd, Healy WL, Einhorn TA. The validity of claims made in orthopaedic print advertisements. J Bone aClaims concerning efficacy or safety, without references.
How evidence-based are advertisements in journals? 6. Wilkes MS, Doblin BH, Shapiro MF. Pharmaceutical advertisements 13. Chalmers TC, Smith H Jr, Blackburn B et al. A method for assessing in leading medical journals: experts’ assessments. Ann Intern Med the quality of a randomized control trial. Control Clin Trials 7. Cooper RJ, Schriger DL. The availability of references and the 14. Jadad AR, Moore RA, Carroll D et al. Assessing the quality of sponsorship of original research cited in pharmaceutical advertise- reports of randomized clinical trial: is blinding necessary? Control 8. Lexchin J, Holbrook A. Methodologic quality and relevance of 15. Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts references in pharmaceutical advertisements in a Canadian medical of interest in biomedical research: a systematic review. JAMA 9. Sutherland AG, Craig N, Maffulli N, Brooksbank A, Moir JS.
Accuracy of references in the orthopaedic literature. J Bone Joint SurgBr 2000;82:9–10.
10. WHO. Resolution WHA41.17 adopted by the Forty-first World Health Assembly, 13 May 1988. Geneve: World Health Organization, The SEDUCE study groups consists of 14 medical students of the Leiden University Medical Center: P. van Winkelen, 11. IFPMA. IFPMA code of pharmaceutical marketing practices.
J. S. van Denderen, R. A. van Adrichem, M. M. de Boer, Geneva: International Federation of Pharmaceutical Manufacturers M. S. Bruijn, D. J. Bruijn, P. D. Dijk, B. P. C. Hoppe, C. J. Kortmann, S. van der Marel, A. G. van de Merwe, J. van Rhenen, S. Saadatmand and V. Vennemann; and of C. Y. Vossen, T. W. J. Huizinga and F. W. Dekker.

Source: http://icto.weblog.leidenuniv.nl/files/2008/11/artikel_advertenties.pdf