What is the evidence for effectiveness of WHO guidelines for the care of children in hospitals in developing countries?

¹ Monash Medical Centre Clayton Victoria Australia
² College of Medicine, Blantyre 3, Malawi

The World Health Organization has produced guidelines for the management of common illnesses in hospitals with limited resources. This series reviews the scientific evidence behind WHO's recommendations. The WHO guidelines, and more reviews are available at
http://www.who.int/child-adolescent-health/publications/CHILD_HEALTH/PB.htm

This review addresses the question: What is the evidence for effectiveness of WHO guidelines for the care of children in hospitals in developing countries?

Introduction:

A great deal of time, resources and expertise is required to develop and implement World Health Organization (WHO) clinical practice guidelines for the care of children. This investment is made with the assumption that WHO guidelines improve the care of children, particularly in developing countries. This review intends to answer the question: What is the evidence for effectiveness of WHO guidelines for the care of children in hospitals in developing countries?

Methodology

The search strategy employed was: (infant[MeSH] OR child[MeSH] OR adolescent[MeSH] OR child* OR infant* OR pediatric* OR paediatric*)

AND

("World Health Organization"[MeSH] OR world health organisation OR world health organization OR world health organi*)

AND

("Guideline"[Publication Type] OR "Guidelines"[MeSH] OR "Guideline Adherence"[MeSH] OR "Practice Guideline"[Publication Type] OR guideline* OR protocol*)

561 potentially relevant articles were found.

All abstracts were read and if there was any doubt as to the relevance of the article, the complete article was sourced. Articles examining the effect of implementation of WHO guidelines for the care of children in hospitals in developing nations were included. We excluded studies in developed nations and studies in adults. We also excluded narrative reviews and editorials as well as studies which did not include a comparison group.

Results

Full text was retrieved of 31 articles. Five were included. No relevant RCTs were found. Each included study compared a group of children cared for before implementation of the guidelines, with a group of children cared for after implementation of WHO guidelines.

All included articles were type 4 – poor quality cohort studies, as defined by the Oxford Centre for Evidence-based Medicine Levels of Evidence (May 2001), meaning that they met at least one of these criteria:

• Failed to clearly define comparison groups

• Failed to measure exposures and outcomes in the same (preferably blinded), objective way in both exposed and non-exposed individuals

• Failed to identify or appropriately control known confounders

• Failed to carry out a sufficiently long and complete follow-up of patients.

The included studies were undertaken in 2 hospitals in Eastern Cape Province, South Africa [1], the Nutrition Rehabilitation Centre in Ceará, Brazil [2], the Department of Paediatrics, Tirana, Albania [3], one hospital in Northern Province South Africa and one in the Volta Region of Ghana [4] and a district hospital in rural eastern Zambia [5]. The studies investigated the impact of WHO guidelines for severe malnutrition [1,2,4], diarrhoea [3] and pneumonia [5].

Mortality was the primary outcome for four of the five studies [1,3,4,5]. There was a statistically significant fall in mortality rates after guideline implementation in two of the six comparisons (OR for death=0.31, 95%CI 0.01, 0.98 p=0.023 [1] and OR for death=0.51, 95%CI 0.27, 0.95 p=0.022 [5]). In three of the six comparisons there was a non-significant fall in mortality rates [3,4] and in one of the six comparisons [1] there was a non-significant rise in the mortality rate. (Two studies [1,4] evaluated the impact of implementation at two different hospital sites.)

Three studies reported the impact of guideline implementation on length of stay [2,3,4]. Two studies reported a reduction in average length of stay, from 4-5 months to less than 1 month in one study [2] and from 5.5 days to 4.8 days in the other [3], however no further data were provided so it is not possible to tell whether these reductions are statistically significant. The third study [4] reported that average length of stay rose from 2 weeks at one hospital and 3-4 weeks at the second hospital pre-implementation, to 4 weeks at both hospitals post-implementation. The authors do not discuss whether this is a positive or negative outcome, and again it is not possible to tell whether the change is statistically significant.

The study by Cavalcante et al [2] reported that after implementation of the guideline average weight gain rose from 2.4g/kg/day to 10.0g/kg/day. The study also reported a reduction in cost of diet from R$2.5/R$2.8 to R$0.5/R$0.9 per litre (no further data provided so statistical significance uncertain) [2].

Two other studies reported measures of process change [4,5]. The study by Deen et al [4] reports that malnutrition management practices (including early frequent feeding, management of hypoglycaemia, hypothermia and infection) were strengthened however this was not quantified. The authors also note that some recommended practices were not feasible in the local setting and problems were encountered in implementing others. Smyth et al [5] report that the post implementation group was more likely to receive oxygen (OR 4.74 , 95%CI 2.68, 8.51), more likely to receive intravenous antibiotics (OR 9.89, 95%CI 2.94, 51.67) and more likely to receive intravenous fluids (OR 4.11, 95%CI 1.68, 11.48).The proportion of patient receiving fluids (intravenous or nasogastric) remained steady as did the proportion of patients receiving parenteral (intravenous or intramuscular) antibiotics.

Discussion

No high quality studies were identified. The little evidence which is available is limited by a lack of methodological rigour, particularly in that the control and intervention groups may differ markedly in aspects other than receipt of the intervention. The amount of data reported in the identified trials is also minimal.

The results of the studies identified suggest that the effect of implementation of WHO guidelines may be positive, however the low quality of the study designs, and paucity of data reported mean this cannot be stated conclusively. This review has a number of limitations. Importantly, the scope of the search was restricted to evidence published in English and indexed in PubMed. Searching only PubMed, which is freely available, ensures that the search is able to be repeated in contexts with only limited access to evidence. This restriction also means that the body of potentially relevant research only available in other databases such as Embase or CINAHL (to which access is not freely available) was not identified.

Restricting the search to studies published in English is another substantial limitation, particularly with a topic of this nature where it is likely that research has been published in other languages. The restriction was made as a result of resource limitations.

Accurately classifying documents as World Health Organisation guidelines was a challenge for this search. Many guidelines are produced with some funding from WHO, or in partnership with WHO, however these documents are not necessarily WHO endorsed, or implemented. For the purposes of this review we only included those guidelines explicitly identified within the study as being developed by WHO.

Searching for WHO guidelines was also challenging as many publications do not use the phrase “World Health Organisation” but only the acronym WHO, which databases cannot distinguish from “who”. Searching with the term “WHO” resulted in retrieval of an overwhelming number of irrelevant citations. In light of this and in consultation with the steering group, we restricted our search to articles retrieved using the search term “world health organi*” and appropriate category headings, realising this means we are likely to have missed some relevant articles.

It is possible that the results of this review are open to publication bias. Small, methodologically weak studies which show no impact, or a negative impact of WHO guidelines may be unlikely to be published as the results might be dismissed as the outcome of poor research design. However small, methodologically weak studies demonstrating a positive effect may well be published in spite of the poor research design. This effect would be likely to exaggerate the perceived benefit of WHO guidelines.

Summary

There is very limited evidence available to determine whether in children in hospital in developing countries, care delivered according to guidelines developed by the World Health Organisation, compared to standard care, improves clinical outcomes or the process of care.

The results of the studies identified suggest that the effect of implementation of WHO guidelines may be positive, however the low quality of the study designs, and paucity of data reported mean this cannot be stated conclusively.

More evidence is required to determine whether the implementation of WHO guidelines in hospitals in developing countries improves care or outcomes for children. The resource-poor settings in which these guidelines are implemented often means that the impact of the implementation is not evaluated.

Small, well conducted cohort studies could provide useful evidence where resources are not available to fund randomised controlled trials. Funding should be provided to undertake these studies in order to demonstrate the benefit (or otherwise) of the extensive and potentially expensive implementation process.

References

1. Ashworth, A., Chopra, M., McCoy, D., Sanders, D., Jackson, D., Karaolis, N., Sogaula, N. and Schofield, C. (2004) WHO guidelines for management of severe malnutrition in rural South African hospitals: effect on case fatality and the influence of operational factors Lancet, 363, 1110-5.[Medline]

2. Cavalcante, A. A., Pinheiro, L. M., Monte, C., Guimaraes, A. R. and Ashworth, A. (1998) Treatment of malnutrition in Brazil: simple solutions to common problems Trop Doct, 28, 95-7.[Medline]

3. Centuori, S., Mati, L., Foto, E., Gellili, L. and Tamburlini, G. (1998) Success and constraints in the implementation of WHO guidelines for the management of diarrhoea in Albania. Minerva Pediatr, 50, 57-61. [Medline]

4. Deen, J. L., Funk, M., Guevara, V. C., Saloojee, H., Doe, J. Y., Palmer, A. and Weber, M. W. (2003) Implementation of WHO guidelines on management of severe malnutrition in hospitals in Africa. Bull World Health Organ, 81, 237-43. [Medline]

5. Smyth, A., Ridwan, R. and Cairns, J. (1998) Impact of a case management protocol for childhood pneumonia in a rural Zambian hospital. Ann Trop Paediatr, 18, 155-60. [Medline]