Bone Spur Research

Evaluating the Quality of Internet-Derived Information on Plantar Fasciitis

Ali Moshirfar, MD; John T. Campbell, MD; Fardin A. Khasraghi, MD; and James F. Wenz, Sr., MD

Patients seeking medical advice increasingly turn to the World Wide Web. To test the hypothesis that Internetprovided medical information often is inaccurate and biased, we selected a common musculoskeletal condition. Three common Internet search engines were used to search “plantar fasciitis” and “plantar fascitis.” Combining the first 50 web addresses from each search engine for both phrases and eliminating duplicate sites resulted in a database of 152 websites. Each website then was analyzed for authorship, sponsoring agency, and presence of financial incentive. The informational value score of each site then was graded using a 10-point scale evaluating the following parameters: basic definition of plantar fasciitis, causes and risk factors, clinical symptoms, diagnostic tests, differential diagnosis, conservative and surgical treatment options, and complications. The overall mean information value score for all 152 sites was 3.8 points; 36% of the sites showed financial bias. The highest mean scores among all authorship and sponsorship categories belonged to orthopaedic surgeons and academic institutions with mean scores of 7.1 and 6.1 points, respectively. Sites without obvious financial bias had significantly higher information value scores than sites with financial interests. Healthcare professionals should consider these findings when advising patients.

Patients seeking medical advice increasingly turn to the World Wide Web. The World Wide Web has become a common source of information in the field of medicine and many other disciplines.&sup8; The use of the Internet with respect to various musculoskeletal conditions has been studied previously, and much of the information so derived was inaccurate or misleading.²&sup7;&sup9; Medical professionals rely on medical databases such as the National Institutes of HealthÂ’s (NIH) PubMed site, which catalogs many peerreviewed articles and indexed journals. The lay public, however, often resorts to using common search engines where the information is not subject to independent verification or review. Therefore, the information that the lay public accesses may be inaccurate, misleading, or undermined by financial conflict of interest.

The information available on the Internet in the field of foot and ankle surgery has been studied previously in a limited fashion.&sup7; However, to our knowledge, no previous study has investigated the relationship between the quality of information and potential financial or commercial bias. To investigate this link, we chose to study the quality of Internet-derived information on plantar fasciitis. Plantar fasciitis is a common musculoskeletal condition treated by orthopaedic surgeons, rheumatologists, primary care physicians, and other allied health professionals. Although most patients respond to standard treatment modalities, in some cases the symptoms become chronic and cause the patient pain, frustration, and anxiety. In addition to seeking treatment from various physicians and medical professionals, such patients may be particularly susceptible to advertisements for novel products or treatment methods that are scientifically unproven.

We hypothesized that much of the information available on the World Wide Web regarding plantar fasciitis is inaccurate and misleading and that many potential sites accessible to a layperson would contain obvious commercial or financial bias. These hypotheses were tested by analyzing the content of common Internet sites dealing with plantar fasciitis.

MATERIALS AND METHODS

CLINICAL ORTHOPAEDICS AND RELATED RESEARCH

Number 421, pp. 60–63 © 2004 Lippincott Williams & Wilkins&sup6;&sup0; (www.yahoo.com, www.msn.com, and www.lycos.com) found 13,100, 10,047, and 12,996 universal resource locator (URL) web addresses for “plantar fasciitis,” respectively, and 4550, 3080, and 1890 addresses for “plantar fascitis, ” respectively. The first 50 addresses from each search were combined to form the initial website database. After eliminating the 148 sites that were duplicates, had restricted access, provided only links to other sites, or otherwise were unavailable, the final database consisted of 152 websites.

The 152 websites then were analyzed for authorship, sponsorship, financial incentives or conflict of interest, and informational content. Authorship of each site was classified as an orthopaedic surgeon, nonorthopaedic physician, podiatrist, chiropractor, physical therapist, layperson, or unidentified. The sponsorship was defined as academic (sponsored by a nonprofit university, institution, or research organization), commercial (for-profit entities selling a product or service), private practice (those that promoted particular physicians, allied health professionals, or groups), governmental (those developed by governmental agencies such as the FDA), bulletin board (those designed to allow user communication with others in a dialogue format), or other (nonidentified sponsorship). An information value score (range, 0 – 10 points) was devised to analyze the quality of information provided on each site. Websites that explicitly advertised a particular product or service, or that advertised a particular physician or allied health professional for the specific treatment of plantar fasciitis, were categorized as having financial bias or conflict of interest.

The following parameters were evaluated: basic definition of plantar fasciitis, causes and risk factors, clinical symptoms, diagnostic tests, differential diagnosis, nonoperative treatment options, surgical treatment options,³ and adverse outcomes of treatment or complications (Table 1). The two junior authors (AM and FAK) independently scored these parameters for each site. If both junior authors agreed with the assigned score for each parameter of the information value score, then that score was used for that respective parameter. Any disagreement in the scoring of any parameter was resolved by the decision of the senior author (JTC). Although this process was subjective, the appropriate score was given as long as a particular website satisfied any of the conditions (Table 1) for any parameter.

Student ’s t test analysis was used to compare the information value scores between groups, two at a time. Probability values less than or equal to 0.05 were considered statistically significant. Values are reported as mean ± SD.

To assess intraobserver reliability, the first author determined the information value score on 10 randomly selected websites in the database on three separate occasions and at least 2 days apart. The interobserver reliability was assessed by having the first three authors assess the information value score on 10 randomly selected websites from the final database and then calculating the kappa values with linear weighting (Stata Version 7, Stata Corp, College Station, TX).

RESULTS When the data were analyzed for authorship (Table 2), the largest group of sites consisted of those for which no author was identified (39%). Sites authored by orthopaedic surgeons had a significantly higher mean information value score (7.1 points ± 1.2 SD) than any other authorship group.

Table 3 catalogs the types of sponsorship and their information value scores. Most sites were classified as commercial sponsorship (81 of 152, 53%); 55% of those (45 of 81 sites) had obvious advertisements or financial bias. Academic sponsorship sites had a mean information value score of 6.1 points ± 2.2 SD, a significantly higher value than sites sponsored by commercial entities, bulletin boards, and the government but not significantly higher than sites sponsored by private practices or others not categorized elsewhere. The overall mean information score for all sites was only 3.8 points ± 2.7 SD. Fifty-five sites (36%) had clear financial incentive or bias, with a mean information value score of 2.8 points ± 2.9 SD. Conversely, 97 sites (64%) without clear financial bias had a mean information value score of 4.4 points ± 2.5 SD. These results were significantly different.

TABLE 1. Information Value Score (0 to 10 points) Parameter Possible Points Criteria Basic definition 0–1 1 point for explaining plantar fasciitis Causes or risk factors 0–1 1 point for any mention of causes or risk factors Symptoms 0–1 1 point for any mention of potential symptoms Diagnostic tests 0–1 1 point for any mention of any diagnostic tests Differential diagnoses 0–1 1 point for any mention of differential diagnosis Nonoperative treatment 0–2 1 point for mention of 1 or 2 treatment options 2 points for mention of 3 or more treatment options Surgical treatment 0–2 1 point for mention of any surgical option 2 points for mention of 6-month trial of nonoperative treatment before surgery and that open release is the gold standard³ Adverse outcomes 0–1 1 point for any mention of adverse outcomes Number 421 April 2004

Internet Information: Plantar Fasciitis 61 The intraobserver reliability was 100%: the same information value score was assigned to each of the 10 randomly selected websites by the same author on three different occasions. The interobserver reliability was greater than 95% in all cases, with kappa values of at least 0.85 (Table 4). This score signifies an almost perfect strength of agreement with the kappa statistics, as shown by Landis and Koch.&sup6;

DISCUSSION The results of this study indicated that readily accessible Internet sites for plantar fasciitis, a common musculoskeletal condition, provide a large quantity but poor quality of information. This is of concern because increasing numbers of patients turn to the Internet to obtain information about medical conditions and treatment options. The information available from the World Wide Web is plentiful but in many ways is unregulated. Search engines allow an individual to input a particular phrase or word and generate a list of relevant websites related to that topic, but the search engines provide no information on the validity, quality, or timeliness of the identified sites. Duplicate, inactive, or restricted-access websites are common and because the majority of such information is not subjected to any formal review process, much of it is potentially biased, misleading, or erroneous. In addition, the content of many sites is obscured by advertisements for products or services, links to private practice sites of medical professionals and groups, and other extraneous information that is not directly educational or helpful to the patient. To investigate the source and quality of Internetderived information, we chose plantar fasciitis as a test subject based on the following rationale. Plantar fasciitis, a common cause of heel pain, often is resistant to treatment. Although surgical options include open and arthroscopic plantar fascia release, published information indicates that nonoperative treatment is effective in more than 90% of the patients.&sup4; However, the most effective modality has not been identified, and the response to treatment can be delayed, with symptoms lasting for as long as 1 year before improvement is seen.&sup4; Many patients become confused and frustrated with this chronic condition and therefore turn to the Internet for additional information, novel treatment methods, or emerging products.

We found that the overall mean information value score, used as a measure of the quality of information, was only 3.8 points. Most sites (53%) were commercially sponsored and many had obvious financial bias or conflict of interest (36%). In 39% of cases, the identity and professional qualifications of the author were not made clear.

The highest mean information value scores belonged to sites authored by orthopaedic surgeons and sponsored by academic institutions. This finding may represent bias in the defined scoring system for the current study, namely an instrument designed by orthopaedic researchers with similar training, clinical experience, and common perspective. However, it also indicates that many Internet authors and sponsoring entities do not focus on the topics considered relevant by contemporary musculoskeletal scientists and practitioners. Another statistically significant finding in our study was that sites with a clear financial bias or conflict of interest, regardless of authorship or sponsorship, had a significantly lower mean information value score than those that did not (4.4 versus 2.8). To our knowledge, this is the first study of its kind to show this negative correlation between financial bias and the quality of information on a particular orthopaedic condition. Orthopaedic surgeons, other physicians, and allied health professionals need to be aware of this correlation and caution their patients accordingly.

Our study Â’s information value score instrument may have been limited in that it assessed several parameters with an all-or-none binary method, with one point scored for any mention of a particular topic versus no points in the absence of such information. An instrument with graded point scoring for each parameter or a larger data point set might provide finer distinctions of information quality. However, the current system was designed to award points to websites for even the most cursory mention of the various topics, in effect biasing the results in the direction of greater quality scores. It is likely that more extensive and critically designed scoring instruments could detect more subtle distinctions among websites. In addition, the authors did not analyze the frequency with which the 152 final websites were used by patients. However, the belief was that the use of three common search engines and the inclusion of the first 50 websites from each search would provide an adequate selection of sites that a layperson most likely would visit. This method, for study purposes, was systematic and unbiased. Finally, our focus on plantar fasciitis and the findings cannot be generalized to the quality and source of medical information on other conditions. However, previous studies have shown that the quality of information regarding various musculoskeletal conditions also may be inadequate.²&sup7;&sup9;

During the current study, we found numerous websites that recommended unconventional or anecdotal treatments, including yoga exercises, acupuncture, vitamins, magnetic insoles, and immediate endoscopic plantar fascia release. Most of these sites fell into the category of commercial sponsorship with obvious financial bias. We did not record the incidence of such unconventional opinions, but future research could explore this topic and the financial bias regarding such claims.

Currently, there are nonprofit agencies that evaluate health-related sites. One such agency is the Health on the Net Foundation (HON), which also has established guidelines to standardize the quality of information provided on the World Wide Web.&sup5; However, as Beredjiklian et al² reported, because the process of posting information on the Internet is unregulated and fluid, it is unclear whether such organizations will have a substantial impact on the quality of health-related content on the Internet. Nevertheless, patients may be well served if reputable national professional organizations, such as the American Academy of Orthopaedic Surgeons or the American Orthopaedic Foot and Ankle Society, establish guidelines or criteria for the quality of information on commonly used medical sites. By creating a seal of approval designation, such professional bodies might motivate website authors and sponsoring agencies to provide only the highest quality information to patients and consumers. For example, the American Dental Association has long established such a distinction that is used by manufacturers and service organizations.¹

This is one approach that might allow reliance on the Internet for medical information to evolve from an often incomplete or misleading endeavor to a more reliable educational resource.

Analysis of internet information on plantar fasciitis showed a low mean information value score (3.8 of a possible 10 points) and a statistically significant negative correlation between the quality of information provided and obvious financial bias by the site Â’s sponsor. Orthopaedic surgeons and other professionals treating patients with plantar fasciitis need to be aware of the questionable material that their patients may be receiving from this resource.

References
1. American Dental Association: Standards and guidelines for dental materials, instruments, and equipment. www.ada.org/prof/prac/stands, 2002.

2. Beredjiklian PK, Bozentka DJ, Steinberg DR, Bernstein J: Evaluating the source and content of orthopaedic information on the internet: The case of carpal tunnel syndrome. J Bone Joint Surg 82A:1540 –1543, 2000.

3. Ferkel RD: Arthroscopy of the Foot and Ankle. In Coughlin MJ, Mann RA (eds). Surgery of the Foot and Ankle. Ed 7. St Louis, Mosby 1257 –1297, 1999.