One of the persistent challenges facing professionals and regulators alike is that of obtaining information regarding adverse events potentially associated with herbs and then, sharing this information in a meaningful way with the public. Despite the fact that many of these substances have been used for centuries and a substantial body of historical data exists to support their relative safety, there is still a need to ensure that current usage in terms of types of preparations, strength and concentration, doses, indications and use patterns do not change a given safety profile. For these reasons there is still a need to monitor the clinical experience related to the use of these substances, especially as it pertains to adverse effects.

There is a variety of venues whereby adverse events involving herbs might be discussed. Health professionals often rely on information contained in scientific reports that appear in the medical literature. In these instances, adverse effect information may be included in the results of a clinical trial or represent the experience of a random sampling of patients taking a given herb. Another more common scientific reporting method involves that of anecdotal or case reports. In these cases, independent practitioners evaluate and summarize information regarding a given adverse effect or experience involving a specific patient and the clinical observations that may suggest a temporal relationship between that patient’s condition and other events or external factors which might include the taking of an herbal product.

In addition to these types of scientific reports, there are also situations whereby incidents involving adverse effects or injury involving herbs may be "spontaneously reported" to a variety of entities. These reports might be received by individual health professionals, regulators or other governmental officials, the media, organized medical information services such as poison centers, and manufacturers or distributors of dietary supplements. These reports are often tabulated into a database and further subjected to analysis and interpretation. The nature of these reports is often quite different from other scientifically documented incidents. Although this type of information is vital in monitoring the safety landscape, it must be recognized that the reports are typically not the same as those contained in prospective clinical trials or even isolated anecdotal reports from independent practitioners where readers may examine all (or at least most) of the available facts regarding a given case and draw their own conclusions. Spontaneous reports have both strengths and weaknesses, both of which must be acknowledged and considered when attempting to evaluate the significance of individually reported incidents or aggregated incidents.

Ephedra Under Scrutiny

Recently, the herb ephedra (Ephedra sinica Stapf, Ephedraceae) has received substantial scrutiny regarding its adverse effect profile. While some of the concern has been generated by legitimate clinical trials where certain types of adverse effects have been documented, much of the information comes from data contained in spontaneously reported incident data collection systems. In these instances, aggregated spontaneously reported incident data has been interpreted to suggest that a safety concern exists. As an example, in March of 2003 a paper was published in the Annals of Internal Medicine¹ that reviewed the relative incidence of adverse effects "related" to ephedra that were reported in the American Association of Poison Control Centers (AAPCC) incident database known as the Toxic Exposure Surveillance System (TESS). The article received relatively widespread media attention and was cited as a basis for the need for increased warnings or a ban on ephedra dietary supplements. Before looking more closely at this article, a brief review of TESS as well as the nature of spontaneously reported incident data should be considered.

The TESS Database

Since 1984, the experience of poison centers has been reported through the publication of an annual report of TESS.^2-19 Currently, these reports appear annually in the September issue of the American Journal of Emergency Medicine. These data have been quoted in numerous publications addressing issues of toxicology and the epidemiology of poisoning in the United States.

The focus of certified regional poison center operations is telephone-based poison information. There is no pre-registration, payment or any other requirement of the callers who use the service and calls may be made anonymously although phone numbers may be identified by the "Caller ID" service provided by local phone companies. Calls or reports to the center are voluntary, as there are no local or national requirements that any given incident must be reported. The center’s phone lines are considered "hotlines" and some centers align themselves or co-locate service with nurse "advice line" programs, drug information centers, industry product surveillance services, or community 911 emergency phone operations.

"Information" calls are separated from "exposure" calls and have not been tabulated in TESS annual reports. That is, there does not have to be an actual case of "poisoning" for an individual to contact a poison center. An individual need only perceive that an adverse effect or poisoning related threat may exist or has occurred. The poison center specialist assesses the incident and determines the most appropriate method to mitigate injury, if injury is likely. This may include advising the caller on appropriate treatment options or referring a patient to a local healthcare facility for further medical evaluation. Recognizing the limitations of telephone patient assessment, it is usually the practice of poison specialists to consider a worst-case scenario regarding the incident in question. This may result in referral of individuals without the presence of any adverse effects in questionably toxic situations so that healthcare providers can provide a more in-depth hands-on patient assessment.

It is important to realize that not all calls represent exposures and not all exposures represent poisoning. In more than 72% of exposure related inquiries, no adverse effects resulting from the exposure are reported. When including those cases in which no more than minor effects are reported, the total represents more than 87% of the cases.¹⁹ It is also important to recognize that not all reported incidents are created equal, especially those calls classified as "adverse effects" secondary to drugs, chemicals, or other agents. Some reported effects might be as minor as self-limiting brief irritation from a simple irritant in the eye whereas other incidents may represent serious life-threatening effects such as arrhythmias or respiratory arrest.

Each case is documented and typically includes:

Each case is assessed as to its overall outcome. The options are "no effect"; "minor effect"; "moderate effect"; "major effect"; "death"; "no follow-up, nontoxic"; "no follow-up, minimal toxicity", "no follow-up, potentially toxic"; and "unrelated". Unless specifically coded as "unrelated", the severity rating implies that the overall outcome is the result of the exposure to the substance(s) identified in the incident. Although a "causal link" is suggested with this type of classification system, many epidemiologists do not believe that these reported incidents are amenable to determination of "cause-effect" relationships and at best, may only suggest an association between a given substance and a certain effect or outcome.

Another aspect of TESS data is that in the vast majority of reported incidents, information is simply taken at face value from individuals who may not be knowledgeable of all the exposure circumstances, clinical effects and treatments that have been rendered. Often, patients may relay what was explained to them by medical practitioners regarding diagnosis and treatments rendered. The specialist must then translate this information into corresponding TESS data fields in order to document the incident. There is no attempt or ability to "authenticate" these incidents in terms of their validity or the integrity of the information that is collected. Furthermore, in those few cases where there is some attempt to do so, it is impossible to segregate those incidents from aggregately reported data contained in the overall database. It is important to acknowledge the strengths and limitations of this and other databases representing spontaneously reported information. This is especially true when drawing safety conclusions from aggregated data represented in the database.

Problems with the Recent Report on Ephedra Adverse Event Reports

In the Bent et al. article¹ which compared the number of reported incidents involving ephedra-containing products to other herbs, a number of concerns have been raised over the scientific methodology used and unsupported conclusions regarding safety being drawn.²⁰

The article was written by Stephen Bent, MD (University of California San Francisco), Thomas N. Tiedt, PhD (Med-Tox Group, Longboat Key, FL), and Michael D. Shlipak, MD, MPH (San Francisco Medical Affairs Medical Center, San Francisco, CA). First, the authors in this report imply that the reported incidents coded as "adverse reaction" and contained in TESS are adjudicated as attributable to a given substance. As mentioned earlier, spontaneously reported incidents contained in the TESS database are not investigated or validated in such a way so as to allow a cause and effect determination. Many of these reports are made anonymously, and all are taken at face value without the benefit of independent confirmation by a licensed healthcare practitioner.

Another significant limitation in reviewing these data, is that calls classified as adverse reaction are infrequently reported to poison centers as compared to incidents involving single, acute exposures to potentially toxic substances. Calls classified as representing "adverse reaction" represent only 2.2% of the 1,931,841 calls received by poison centers nationwide.¹⁹ [And, with respect to all botanical calls, cases classified as "adverse reaction" represent only 2,115 incidents or 0.1% of all cases reported in the 2001 TESS annual report.] This suggests that calling poison centers is either an infrequent option for consumers and healthcare providers for the reporting of adverse reactions involving botanicals, or that the relative frequency of these incidents is extremely small. This may further suggest that the "sensitivity" of this database may not be sufficient to identify any significant trends one way or the other.

An additional and substantial flaw in the methodology used by Bent et al. pertains to the assumption of outcome and adverse clinical effects regarding all incidents listed in the "adverse reaction" column for ephedra-containing substances. It was assumed that all of the 1,178 incidents represented cases in which adverse effects were either reported or believed to be associated with these botanical substances. The authors were apparently unaware that of all incidents involving ephedra-containing products, approximately 58% were coded in a number of outcome categories that either suggested there were no documented adverse effects or that the effects were believed to be unrelated to the substance in question. These outcome categories are not listed in the table that was relied upon by the authors and not readily apparent from the tables contained in the TESS annual report.* Furthermore, it is assumed that for those cases where adverse clinical effects are reported, all of these effects are deemed to be of equal concern and severity. Again, there is no way of determining how the individual cases were coded according to the nine available options.

It is also curious as to why Bent et al. chose to compare the effects of ephedra-containing substances to the wide variety of other botanicals, the effects of which are not only diverse, but often times subtle and noted over an extensive period of time. Ephedra, on the other hand, demonstrates clinical effects that may be readily detected and quite noticeable to the user within a relatively short period of time. These expected effects, even when minor in nature, may be interpreted as unpleasant to one user as opposed to another. If the authors were attempting to compare the relative incidence of reported adverse effects of these substances, a more meaningful comparison might have been to compare against other agents that possess either similar pharmacology or indications.

Lastly, the authors used inaccurate sales data to suggest that ephedra-containing dietary supplements are disproportionately responsible for many botanical adverse event reports (AERs): they erroneously stated that ephedra sales constitute less than 1% of total supplement product sales in the botanical market. More accurate sales data suggests that ephedra-containing dietary supplements comprise an estimated 20-30% of the botanical supplement sales.** Considering its shortcomings, for the Bent et al. paper to have received such considerable attention with so little knowledgeable peer review and scrutiny, objective scientists might reasonably question whether or not ephedra and other dietary supplement botanicals may be the subject of a double standard (or perhaps inexact scientific scrutiny) as compared to the evaluation of the safety of pharmaceutical drugs. As an example of the importance given to this paper, the erroneous conclusions from the Bent et al. article were cited by the U.S. Food and Drug Administration in the Federal Register in February when the Agency made reference to it as part of its rationale for proposing strong warnings for the labels of ephedra supplements.²¹ [Note: The authors of the present article are not attempting to diminish the safety concerns concerning ephedra by making these comments. However, as demonstrated in this article, there appears to be a problem of potentially significant proportions in attempting to rationally interpret and draw meaningful conclusions surrounding AER data on herbal products from the existing reporting mechanisms.]

In defense of their article, upon publication of letters raising questions about their methodology by the senior author of this article and his colleague Stephen W. Borron MD, MS (International Toxicology Consultants, Washington, D.C.),²⁰ Bent et al. respond that, "The data from the national system of Poison Control Centers provide a critical safety valve for consumers. Health researchers, government agencies, and product manufacturers rely on these data to warn the public about product dangers and to design safer products."²³ There is no question that such groups do utilize the TESS database. As noted in the present article, misinterpretations can, and do, occur even when well-intentioned or authoritative entities analyze information contained in the TESS annual report. Bent et al. also responded with another calculation without grounds. They respond to the point that an average of 15.4% of all AERs in the database (herb and non-herb) are later determined to be unrelated to the substance in question by using this figure to once again compare ephedra to other "herbs." The salient point that was brought to their attention was that it is impossible to calculate this relative incidence, because the numbers are not present in the data that Bent et al. had available for evaluation. The percentage of AERs implicating ephedra, and then subsequently determined to be unrelated, could theoretically range from 0—100%, rendering their calculation meaningless.

With respect to a letter from Annette Dickinson, PhD, president of the Council for Responsible Nutrition, that questions the market statistics used by the authors to conclude that ephedra supplements constitute only 1% of the total herbal supplement market but in actuality constitute approximately 33% of the market in 2001,²⁴ Bent et al., respond, "We are perplexed at Dr. Dickinson’s statistic of ephedra sales from the Nutrition Business Journal. We did not rely on this journal’s data because the journal uses an unspecified combination of sales data and information from surveys of herbal manufacturers and also uses dollar sales rather than unit sales (the relatively expensive cost of ephedra products biases the dollar sales estimates)."²³ Despite this response, it is the opinion of the authors of this article that the data suggesting that ephedra supplements are only 1% of the total herbal market upon which Bent et al. relied is misleading in relation to the actual volume that ephedra previously enjoyed in the U.S. market.

Monitoring of adverse effects potentially associated with herbs and other natural medicinals will continue to be a challenge. While the use of multiple methods to track and report incidents is important, the strengths and limitations of all available reporting options must be considered. A significant strength of monitoring spontaneously reported data is that it is well-suited to generate hypotheses on safety or toxicity. It is also extremely useful in defining a particular product’s safety profile, especially when a given method of adverse event reporting is well positioned to be utilized by those who may experience an adverse effect and report it to a given reporting system.

As the future of herbs and other dietary supplements unfolds, there must also be a concerted effort on the part of healthcare providers and other professionals, regulators, manufacturers, and the public at large to participate in the process of dietary supplement post-market surveillance. Collection of spontaneously reported botanical AERs in a systematic and consistent manner is a vital part of product stewardship and safety assurance. Defining and monitoring the safety of dietary supplements is a dynamic process, and cooperation among multiple stakeholders using reliable methods of surveillance and analyzing collected data in proper context will aid in this process.

Richard (Rick) Kingston, PharmD, is Co-Founder, Vice President and Senior Clinical Toxicologist with the PROSAR International Poison Center and an Associate Professor in the Department of Experimental and Clinical Pharmacology, University of Minnesota, College of Pharmacy. He has over 26 years experience in poison control and clinical toxicology, previously serving as co-founder and former Director of the Minnesota Poison Control System. At the University of Minnesota, Dr. Kingston serves as Course Director for "Therapeutics of Herbal and other Natural Medicinals," a course dedicated to the clinical application of science related to dietary supplement use. The PROSAR International Poison Center is the largest product safety focused poison center in the United States, serving over 200 corporate, governmental, and NGOs in the areas of health and safety call center support, post-market surveillance, incident investigation, and health information management.

Mark Blumenthal is founder and executive director of the American Botanical Council and editor and publisher of HerbalGram.

References: