Liz Roffe1 BSc (Hons), Katja Schmidt2 BSc (Hons), MSc, C Psychol, Edzard Ernst2 MD, PhD, FRCP, FRCP, Ed
1 School of Nursing and Midwifery
University of Southampton
Highfield
Southampton
S017 1BJ
2Complementary Medicine
Peninsula Medical School
Universities of Exeter & Plymouth
Institute of Health & Social Care
25 Victoria Park Road
Exeter EX2 4NT
United Kingdom
Tel: +44 (0) 1392 424839
Fax: +44 (0) 1392 427562
E-mail: katja.schmidt@pms.ac.uk
Abstract
Objective: To assess whether the use of the 10 most frequently recommended CAM therapies for cancer was supported by the experiences of UK, USA and Canadian practitioners of naturopathic medicine.
Design: An e-mail survey was carried out, in which practitioners were sent an introductory email, and a survey consisting of 3 questions.
Participants: All practitioners of naturopathic medicine with an email address registered on one of 3 websites of naturopathic medicine organisations were contacted (n = 1028).
Results: We received a total of 32 responses (response rate = 3.1%). The very low response rate made it impossible to analyse the data in any meaningful way.
Conclusion: The low response rates to this survey highlight a need to bridge the gap between researchers and practitioners. Increased collaboration is required to overcome the problems identified in this report.
Key words: naturopathic medicine, e-mail survey, response rate
Introduction
Cancer patients are increasingly turning to complementary and alternative medicine (CAM) practitioners. A postal survey in 2002 found that 37% of naturopathic doctors contacted in the USA and Canada had provided naturopathic care to women with breast cancer in the past 12 months. The researchers in this survey achieved a 47% response rate to their initial screening form and a 12% response rate to their 13-page survey.1
We conducted a short email survey of practitioners of naturopathic medicine regarding their use and experience of 10 CAM therapies with cancer patients. The therapies included in our survey were recently identified as the 10 therapies most frequently recommended on the Internet for cancer: CoQ10, shark cartilage, Laetrile, vitamin C, Gerson’s diet, mistletoe, visualisation, meditation, acupuncture, green tea and vitamin E.2 Previous email surveys of a similar nature were carried out at our unit in the last 3 years, and achieved response rates between 20-72% in populations of reflexologists, medical herbalists, homeopaths, chiropractors and acupuncturists.3-7
In order to assess whether the use of the 10 mostly recommended CAM therapies for cancer (as identified by the previous project) was supported by the experiences of practitioners of naturopathic medicine, we asked practitioners in the UK, USA and Canada the following three questions:
- Which of the following therapies have you recommended to cancer patients? (from a list of 10 CAM therapies)
- What benefits can you report for each therapy in cancer patients?
- What adverse effects can you report for each therapy in cancer patients?
Method
The survey was carried out with the ethical approval of the Peninsula Medical School. Our target population was practitioners of naturopathic medicine who had undergone sufficient training to become accredited members of organisations. Email addresses of such naturopathic doctors were obtained from websites of 3 organisations, in the UK (http://www.naturopathy.org.uk/), USA (http://www.naturopathic.org/) and Canada (http://www.naturopathicassoc.ca/).
The survey was carried out between November 2003 and February 2004. An introductory email, which included a brief explanation of the project, was sent to each of the 1312 email addresses (Appendix 1). It informed the participant that they had been selected to take part in the survey and would shortly receive the survey in a second email.
The survey was sent by email to each address the following day, entitled “Naturopath survey” (Appendix 2). The survey questions were included in the body if the text, rather than as an attachment. The emails were sent in plain text format to avoid any changes to the format of the text occurring when delivered to different webservers. Anonymity and confidentiality were assured and the contact details, affiliation and website of the research group were included in the email to provide reassurance that we were conducting genuine research. No deadline was stated in the email for the return of the survey, but respondents were given two weeks to reply, as experience in previous research has shown that most responses are being delivered in the first few days after sending out an email.
Results
Two-hundred and eighty-four emails were returned as undeliverable (i.e. email accounts closed).
A total of 1028 emails were successfully delivered and received. Table 1 shows the response rates, including the number of general comments received, according to each country.
The most frequent reasons for refusal to participate included:
- Believing that the information about treatments should be retrieved from the literature rather than querying practitioners of naturopathic medicine directly.
- Not being comfortable with responding by email.
- Requesting financial compensation for time to participate.
- Feeling that the questions were structured according to allopathic medicine to address a different paradigm, and would therefore not reflect the practice of naturopathic medicine (i.e. holistic approach of treating whole person as opposed to localised disease).
- Not having the time to participate.
A further 21 people responded with a variety of questions regarding our research group, participation in the survey, the aims of the study, and made recommendations of other contacts who may be in a better position to help, or who they would refer cancer patients on to (e.g. naturopaths working in specialist cancer clinics). These comments and questions were addressed in a standard letter, which was sent to the relevant respondents (Appendix 3).
Discussion
The reported response rates are based on the responses we received after the two-week time period. They are remarkably low.
So what can be deduced from our survey? No conclusions can be drawn regarding our initial research questions due to the very low response rate. However, the survey does raise several issues regarding why so few practitioners of naturopathic medicine participated. We would like to discuss some of these points.
1. Misunderstanding of the aims of the study?
We specifically asked which treatments had been, not would be recommended to cancer patients, to enable respondents to draw on their previous, tangible experiences rather than hypothetical situations. However, it seemed some practitioners thought the research was attempting to find out which treatments were appropriate or effective for treating cancer.
2. Tired of emails/SPAM or concerned about a risk of a virus?
The Standish et al survey1 was a postal survey, rather than an email survey which might account for some difference in response rates. There seems to be an ever-increasing amount of junk or SPAM emails being circulated, and despite efforts to block them, there is no entirely satisfactory means of filtering emails. Due to the vast quantity of emails people now have to deal with and the risk of computer viruses, emails from unrecognised addresses are often deleted. However, if practitioners advertise their email address on the Internet, it is assumed they must be prepared to receive emails regarding their practice, from unrecognised sources. Our survey was sent within the main text of the email rather than as an attachment to avoid any suspicion regarding viruses.
3. Poorly designed questionnaire?
Some practitioners commented that the questions were phrased according to an allopathic approach, which did not reflect their practice. However, the intention was that the simplicity and conciseness of the survey would encourage a greater response.
4. Was the survey irrelevant to practitioners? – Were few practitioners working with cancer patients?
We asked practitioners who had any previous experience with cancer patients to participate. 37% of practitioners responded to the Standish et al postal survey1, stating that they had treated women with breast cancer alone in the last 12 months. Given the far narrower criteria of the Standish survey, our survey should theoretically have been relevant to a larger group of practitioners of naturopathic medicine.
5. Resistance to research (particularly from a unit such as ours, which has a reputation for scientific rigour)?
The Standish et al survey1 was carried out by practitioners of naturopathic medicine – could this provide a further explanation for the difference in response rates? Previous surveys have found much lower response rates if practitioners are approached by researchers rather than by researchers posing as clients or relatives.3,5,7 The fact that researchers, who were allied to a medical school carried out this survey rather than peers from a naturopathic medicine group may have contributed to the very low response rate.
6. Practitioners fear ‘negative results’
Practitioners often only want to participate in research that will provide ‘positive’ evidence to back up their practice and reinforce the beliefs they already hold about the treatments they use. However, a ‘negative’ result can be beneficial if it saves patients (and practitioners) time and money on ineffective treatments or reveals an important safety issue. We were not attempting to assess the efficacy or safety of treatments with the survey, but we are aware of one practitioner who approached their organisation asking if it was possible to predict whether the survey will come out positive or negative for naturopathic medicine. Did practitioners not reply because they were worried that we might report naturopathic medicine to be an ineffective or unsafe?
Solutions
The survey offered an unexpected vehicle with which to explore some of the difficulties encountered with this type of research, the primary issue being the communication between researchers and practitioners. We offer some solutions to bridging this gap below, together with some ideas for carrying out any future email surveys. With these points in mind, a greater awareness and understanding between practitioners and researchers may hopefully be achieved. The following approaches may also help to monitor and understand response rates from email surveys.
Technical & Methodological improvements
When carrying out email surveys researchers should:
– Use the ‘request reply’ facility when sending emails to determine how many emails were actually opened and how many were deleted without being read.
– Include a question to account for the number of people who do not feel the survey is relevant to them (i.e. if they had not treated cancer patients before).
Collaboration
Increased collaboration between researchers and practitioners is needed for a more open exchange of information, to overcome the problems highlighted in this report.
- Researchers must work more closely with organisations and practitioners in the early stages of planning research. This will help promote openness, avoid misunderstandings and encourage members to participate in research.
- Researchers and practitioners must work together to increase researchers’ understanding of approach and practice.
- Researchers and practitioners must work together to improve practitioners’ understanding of objective research and reduce mistrust
- Practitioners and researchers must work together in developing suitable tools for research, which are both objective and appropriately reflect practice.
Table 1. Response rates to naturopathic medicine survey
| Country of Naturopathic Medicine Organisation | Total number of emails delivered | Completed surveys
returned |
No cancer patients / not practising | Refusals | Total responses including comments and questions |
| USA
|
586 | 15 (2.6%) | 19 | 4 | 57 (9.7%) |
| UK
|
66 | 3 (4.5%) | 1 | 3 | 8 (12%) |
| Canada
|
376 | 14 (3.7%) | 8 | 2 | 35 (9.3%) |
| Total | 1028 | 32 (3.1%) | 28 | 9 | 100 (9.7%) |
Appendix 1
Dear naturopath,
My name is Liz Roffe. I found your email address on the website http://www.naturopathic.org/. I am working at the Peninsula Medical School (www.pms.ac.uk), and my research areas cover the field of complementary and alternative medicine (www.ex.ac.uk/sshs/compmed/new_title.htm).
We are currently carrying out a project investigating various complementary medicines and therapies for the treatment of cancer and palliative cancer care.
We would like to know your personal experiences and recommendations you can share with us regarding the safety and efficacy of different complementary and alternative therapies and medicines for cancer.
Only the research team will have access to the information you give me in your response email. All response emails will receive a unique number to guarantee anonymity. We will use descriptive statistics to summarise all responses. There will be no publication of any personal data, but rather a collective summary, thus making it impossible to identify individuals. Once we have finished our project we intend to publish the findings in a peer-reviewed medical journal.
The Ethics Committee of the Peninsula Medical School has reviewed and approved this project.
A second email, containing our questionnaire, will follow shortly.
Please do not hesitate to contact me if you have any queries.
Many thanks and best wishes,
Liz Roffe
Research Assistant
Complementary Medicine
Peninsula Medical School
Universities of Exeter & Plymouth
Institute of Health & Social Care
25 Victoria Park Road
Exeter, EX2 4NT, UK
Appendix 2
Dear naturopath,
I contacted you previously regarding a survey we are carrying out at the Peninsula Medical School, Complementary Medicine.
If you have had clients with cancer or clients who have been recovering from cancer and are happy to participate in our survey, please answer the following questions.
We would like to know your recommendations to cancer patients and experiences you can share with us regarding the safety and efficacy of the following complementary therapies and medicines for cancer: CoQ10; Shark Cartilage; Laetrile; Vitamin C; Gerson’s diet; Mistletoe; Visualisation; Meditation; Acupuncture; Green tea; Vitamin E.
Please click on “reply” then type your answers to the following questions alongside the relevant treatment:
1. I have recommended this therapy for the following conditions in cancer patients…
CoQ10
Shark Cartilage
Laetrile
Vitamin C
Gerson’s diet
Mistletoe
Visualisation
Meditation
Acupuncture
Green tea
Vitamin E
2. I can report the following benefits of this therapy with cancer patients:
CoQ10
Shark Cartilage
Laetrile
Vitamin C
Gerson’s diet
Mistletoe
Visualisation
Meditation
Acupuncture
Green tea
Vitamin E
3. I can report the following adverse events associated with this therapy:
CoQ10
Shark Cartilage
Laetrile
Vitamin C
Gerson’s diet
Mistletoe
Visualisation
Meditation
Acupuncture
Green tea
Vitamin E
By answering this email you will be consenting to participate in the study and giving permission to include your data.
The security of the data is no different from the security of emails generally. Only the research team will have access to the information you provide. All responses will receive a unique number to guarantee anonymity.
Many thanks and best wishes,
Liz Roffe
Research Assistant
Complementary Medicine
Peninsula Medical School
Universities of Exeter & Plymouth
Institute of Health & Social Care
25 Victoria Park Road
Exeter, EX2 4NT, UK
Appendix 3
Dear Naturopath,
Thank you to all of you who have already returned our survey, your time and help is much appreciated. For those of you who have not yet returned the survey IT IS NOT TOO LATE! We will be collecting the surveys up and including Friday 19th December 2003 and we would be very grateful for your participation.
Thank you also for the questions, comments and advice I have received regarding the survey. I apologise for not responding individually to you at this point but I hope that this email covers the points raised.
Who we are:
We are a Complementary Medicine research group, linked to Peninsula Medical School, Universities of Exeter and Plymouth. I am a research assistant, not a student or a medical practitioner.
The project we would like your help with:
Investigating Complementary and Alternative Medicines (CAM) for cancer is one of our areas of research. We are also interested in the use of the internet for providing health information and have identified the top ten complementary therapies promoted on the internet for cancer. For the current project, we are researching these ‘top ten’ modalities only.
We are of course conducting extensive literature searches on these modalities but we are also approaching practitioners to determine if your use of therapies matches our findings. We have a strict methodology and are currently sending the survey to practitioners from one organisation, the American Association of Naturopathic Physicians, so please don’t forward the survey to anybody else or it will change the population we have selected for our study!
I recognise that the questionnaire will not demonstrate the complexities of naturopathic practice, but hope it will provide a context for our findings in the literature and on the internet. I am only sending surveys out by email and unfortunately, we do not have funds to reimburse respondents.
If you work with cancer patients and have not already replied to me, I hope very much you can help by completing and returning the survey. Please email me if you would like me to send it to you again.
Many thanks and best wishes,
Liz Roffe
Research Assistant
Complementary Medicine
Peninsula Medical Schoo
Universities of Exeter & Plymouth
Institute of Health & Social Care
25 Victoria Park Road
Exeter, EX2 4NT, UK
Email: liz.roffe@pms.ac.uk
References
1. Standish LJ, Greene K, Greenlee H, Kim JG, Grosshans C. Complementary and alternative medical treatment of breast cancer: a survey of licensed North American naturopathic physicians. Altern Ther Health Med 2002;8:68-75.
2. Schmidt K, Ernst E. Assessing websites on complementary medicine for cancer. Ann Oncol 2004;15:733-42.
3. Schmidt K, Ernst E. Internet advice by acupuncturists – a risk factor for cardiovascular patients? Perfusion 2002;15:44-50.
4. Ernst E, Schmidt K. Health risks over the Internet: Advice offered by ‘medical herbalists’ to a pregnant woman. Wien Med Wschr 2002;152:190-92.
5. Schmidt K, Ernst E. Are asthma sufferers at risk when consulting chiropractors over the Internet? Respir Med 2003;97:104-5.
6. Schmidt K, Ernst E. MMR vaccination advice over the Internet. Vaccine 2003;21:1044-7.
7. Schmidt K, White A, Ernst E. Reflexologists’ responses to a patient with abdominal pain – Internet advice. Complement Ther Med 2003; 11:98 – 102.