Main content

Moving the goalposts in research, Involving parents in the care of premature babies, Feedback

Fiddling figures in research and why it matters that outcomes aren't switched or goal posts moved; involving parents in the care of premature babies to improve recovery; feedback on the current series

Available now

28 minutes

Programme Transcript - Inside Health

Downloaded from www.bbc.co.uk/radio4

 

THE ATTACHED TRANSCRIPT WAS TYPED FROM A RECORDING AND NOT COPIED FROM AN ORIGINAL SCRIPT.  BECAUSE OF THE RISK OF MISHEARING AND THE DIFFICULTY IN SOME CASES OF IDENTIFYING INDIVIDUAL SPEAKERS, THE BBC CANNOT VOUCH FOR ITS COMPLETE ACCURACY.

 

 

INSIDE HEALTH – Programme 12.

 

TX:  26.03.19  2100–2130

 

PRESENTER:  MARK PORTER

 

PRODUCER:  ERIKA WRIGHT

 

 

Porter

Hello. Coming up in the next half hour:

 

Caring for premature and sick babies – we visit a neonatal unit that is actively encouraging parents to be more hands on.  To everyone’s benefit.

 

Clip

Kate

It was isolating and scary at times, I’m not going to lie, but I think due to the experience I’d had on the unit I almost felt like a nurse myself.

 

Porter

What our listeners can’t see, of course, is that Constance is fast asleep on your chest right now, behaving extraordinarily well.  She looks a fairly good size.

 

Kate

She is, she’s done really well.  It’s a dance between the baby and the mother and I think if you really get to know your baby, and with the help of this care model you really do, these things become less frightening.

 

Porter

It may seem obvious but involving parents to that extent is far from universal. 

 

And how long before an operation should you stop drinking water to ensure that your stomach is empty?  Not as long as long as most doctors think.

 

But first a new study suggesting that some researchers are fiddling figures to make their findings look better.  And if you can’t trust a peer reviewed trial published in an eminent journal, then what can you trust?  It’s all down to outcome switching – what you and I might call moving the goalposts.

 

Margaret McCartney is in our Glasgow studio to explain more. Margaret.

 

McCartney

Well an outcome is just anything you choose to measure.  So, it could be, for example, blood pressure or a score of how depressed or anxious you are or how many days you stayed in hospital, it could be anything.  These are called pre-specified outcomes.  So, the idea is before you start to do the trial you register your trial and you say what it is that you’re going to measure.

 

Porter

Now what happens if people switch outcomes?

 

McCartney

Okay, so, Mark, you know the card game 21s and this is where you try and get three cards that add up to 21.  And let’s say, Mark, that you and I are playing this game of 21s and I deal the cards and then I have a look at my cards and I say actually, Mark, we’re not playing 21s we’re playing 19s and oh look, I’ve won.  That’s outcome switching.  I changed the rules after I dealt the cards.  And it’s easy to see that that is cheating.

 

Goldacre

So, if you allow yourself to selectively only report the results that go in your favour then you’re cheating at running a clinical trial, you’re not playing by the rules.

 

Porter

Ben Goldacre is senior clinical research fellow at the Centre for Evidence Based Medicine at the University of Oxford and one of the team behind the COMpare study.

 

Goldacre

Now these are longstanding well recognised rules around clinical trial reporting, in fact there’s something called the Consort Trial Reporting Standard which is universally recognised, it’s been endorsed by over 500 academic journals and it says, very simply, when you do a clinical trial you’ve got to report the results that you said you were going to report, you’ve got to report your pre-specified outcome measures at the end of the trial.  And if you don’t you do at least have to disclose that you’ve switched your pre-specified outcomes. 

 

Now we knew from all the previous research that’s been done that academic journals who’ve signed up to these very strict trial reporting guidelines have also been breeching those guidelines.  We decided that we would do a live correction service.  So, for every single clinical trial published in the big five academic medical journals – that’s JAMA, the Lancet, the British Medical Journal, Annals of Internal Medicine and New England Journal of Medicine – for every single trial published in those journals over a six week period we assessed every clinical trial on the day it was published, we looked to see if it had correctly reported its outcomes, if it had engaged in outcome switching then we sent a correction letter to the journal.  And we weren’t interested in measuring how often people switched their outcomes because this is a longstanding and well documented problem, we were interested in what the journals would do when they got a correction letter, which we sent in good faith and also completely standardised and very, very, very polite, I must say, because we wanted to avoid any unhelpful argument.

 

Porter

And did they know, Ben, at this stage that it was coming from you and your unit or were these anonymised in some way?

 

Goldacre

No, no we submitted these letters in the same way that anybody would submit a letter from an academic team.  So, we were astonished.  First of all, the majority of letters were not published.  Out of 67 trials that we assessed 58 had a discrepancy.  Of those 23 in total were published.  Most of those were published in two journals – in the Lancet and in the British Medical Journal.  The New England Journal of Medicine didn’t publish a single letter from us.  JAMA – the Journal of the American Medical Association – didn’t publish a single letter.

 

Porter

Where’s the responsibility here?  Obviously, the researchers who are doing the trials have a responsibility to act in good faith and stick to the outcomes that they were originally supposed to do but also the journals, if they’re signed up to Consort, have a duty to check that that’s exactly what happened.

 

Goldacre

What was really interesting about sending these correction letters is we thought initially we would be engaged in a very simple project, we thought we were just going to send some correction letters and we would see whether or not people published them.  What actually happened next was that we got an extraordinary series of responses from the journal editors which really shed a lot of light on the reasons why this problem has persisted for so long.  First of all, we had very senior academic journal editors telling us, very openly, that they were happy for researchers to misreport their clinical trial outcomes, they were, in some cases, openly disclosing that they actively encouraged people to switch their pre-specified outcomes without disclosing it.  So, because I think nobody had ever really asked editors about these issues before in such a direct way, everybody had passively assumed that editors understood what the Consort guidelines were, why they were important and how they worked.  But what our study shows, very clearly, is that while these journals are all signed up as endorsing the Consort guidelines, in many cases the editors themselves don’t actually understand what those Consort guidelines require of them. 

 

Porter

Well, Elizabeth Loder is Professor of Neurology at Harvard and Clinical Editor of the British Medical Journal.

 

Loder

At the BMJ we’re on the same side as Dr Goldacre and his team of people, we really are committed to making sure to the extent possible that the trials that we publish do report the pre-specified outcomes or if they don’t explain why and explain where those outcomes will be published.  And I was pleased to learn in this paper that the BMJ had published all of the letters that the researchers sent to us.  Having said that, I would not characterise every instance of departure from the pre-specified outcomes as cheating.  It’s a bit more complicated than that and in practice it can be very difficult to decide exactly what constitutes outcome switching.  For example, if researchers have said on the trial registry that they are going to study depression and later they say they’re going to study depression at six months and 12 months and two years, is that to be counted as three instances of misreporting or just one problem?  I think you can get an idea of exactly how easy it is to consider things in the worst light, rather than the best light.  And that gives rise, I think, to a perception on the part of some of the triallists and perhaps some of the journal editors that Dr Goldacre and his colleagues are not being entirely fair to them when they count up instances of transgressions.

 

Porter

But Elizabeth the BMJ may have come out of this quite well but Ben there was suggesting that perhaps some editors and I don’t want you to name and shame any, perhaps some editors don’t fully understand the Consort guidelines or that they’re actively encouraging outcome switching.  I mean your take, as somebody in this field, do you think that’s happening?

 

Loder

I do think it’s happening, yes.  Not as much at the large major journals that were studied in this case but in the situation of small journals that have volunteer editors are run by professional societies and where there is often a mixing of people who evaluate the research with the same people who do the research.  There are fewer safeguards and there’s less understanding of these rules.  I’ve not seen a case where a journal editor has actively encouraged switching but I certainly see cases in the smaller journals, where I work as a volunteer editor, where other editors really do not understand the Consort guidelines or don’t take them as seriously as perhaps they should.  So, it’s a major problem.

 

McCartney

Elizabeth is correct, there are many legitimate reasons for changing the outcomes after the study has started and that’s absolutely fine, if it’s done in an open and transparent way and the authors refer to that in their study, so that everybody knows why that’s done.  And you can imagine a study might take 10 years, for example, a better test might come out during the course of that study that people want to incorporate in to try and make it as high quality as possible, that’s absolutely fine.  And actually, the COMpare Project themselves say that outcomes can change for good reasons after a trial has started but they just must be declared and explained in the trial report.  And actually, the COMpare Project don’t declare this to be outcome switching if it’s actually been done in a transparent and open manner.

 

Porter

Margaret, this is a complicated story but why does it matter, why should it worry listeners?

 

McCartney

This is complicated but it’s really worth knowing about.  So, back in 2001 a study was published that said the antidepressant Paroxetine was – and I quote – “generally well tolerated and effective for major depression in adolescence” and this led to millions of prescriptions being given out by doctors before it was realised that this study actually was flawed and really seriously flawed.  When new independent researchers went back and looked at the data from this study in 2015 what they found was that the pre-specified outcomes, that they said they were going to look at before they started this trial, actually weren’t positive.  But during the trial 19 extra measures were added on during the study and that only four outcomes that actually were positive, to show the drug worked, had been added in during the trial, none had been pre-specified.  And further, they found that actually this drug had serious side effects in young people as well.  So, the bottom line is this drug had been being prescribed for millions of people worldwide without the evidence that it was resting on being accurate and that’s a really big problem, this wasn’t clear when the study was published at all.

 

Porter

And the accusation is that the researchers purposefully manipulated their data so it looked better than it actually was.

 

McCartney

Yes, so let’s go back to our game of 21, Mark.  So, if I’m dealing the cards, I have a look at my cards and I don’t like the look of them, so I decide to help myself to another dozen cards and then, Mark, well look I’ve won, it’s exactly the same.  Yeah, I won but I also cheated to get that result by adding in more and more outcomes it gives you more chance to get the right answer.  And that’s going to happen randomly, it’s going to happen by chance, if you add in enough outcomes eventually you’ll get some that turn out to be positive just because of the play of chance.  And if you don’t tell anyone that you took all those extra cards or chances or outcomes it will look as though your study won every time when it didn’t.

 

Porter

And perhaps, more importantly, in the case of Paroxetine, the trial would have (inverted commas) “failed” and we wouldn’t have used the drug quite so much.

 

McCartney

Yeah and that would have been really useful information to know about, it meant that you could have avoided something that would have otherwise done harm to people.  And the fear is that there might be other medications or drugs out there that there are similar problems with but we just don’t know.  And that’s why we really need to, when we’re thinking about new drugs or treatments, think about has this study been replicated somewhere else by different researchers, do we have all the data about this, has someone searched through the literature for all the unpublished studies, for example, that might give us better information about it?

 

Porter

But isn’t that part of the reason why we have so much respect for eminent journals, in that they’re supposed to be doing this on our behalf?

 

McCartney

Well I think it’s asking major questions and I’m really glad that this COMpare Project is up and working.

 

Porter

Margaret McCartney, and there is a link to Ben Goldacre’s research on the Inside Health page of the Radio 4 website.

 

Neonatal unit noises

 

The neonatal units at Queen Charlotte’s and Chelsea Hospital and St Mary’s at Paddington look after some of the most premature babies in London.  Tiny, fragile, vulnerable babies that can be born four months early and weigh well under a kilo – less than 2lbs.  But despite the complex care they receive and need the staff are actively encouraging parents to get more involved – from changing nappies, to giving medication.  The programme is called Integrated Family Delivered Care and has proved popular with all involved.  

 

Consultant neonatologist Dr Aniko Deierl.

 

Deierl

When we have a baby admitted to a neonatal unit it’s a very highly medicalised environment, they are placed in an incubator connected to wires, tubes and machines which can be emotionally very, very overwhelming.  And obviously, the incubator and the highly technicalised environment is a barrier between parents and the baby.

 

Porter

And I mean I remember from my time working in the unit 30 plus years ago it was a huge barrier, I mean sometimes there’s even glass between the parents and their baby, there was certainly an incubator which is basically like a big fish tank.  What’s changed?

 

Deierl

I would say that practices in the UK are still very variable.  So, we have designed a new care model called Integrated Family Delivered Care, in our units it’s actually needed a cultural change.  And actually, we wanted to improve this extremely stressful journey for the parents with this new care model where we allow them to become active caregivers to their babies through some competency-based training. 

 

Porter

Because still the norm would be in most units, would it not, that most of the standard care – the feeding, the changing of the baby – all of these things, would be done by the staff on the unit?

 

Deierl

I would say generally yes.

 

Katie

My name is Katie and I am mother to Constance.  I had a very complicated identical twin pregnancy and sadly one of our girls didn’t make it but I gave birth to Constance on 11th October last year at roughly three months premature. 

 

Porter

So, she was just under 30 weeks.

 

Katie

She went straight to the intensive care unit at Queen Charlotte’s and Chelsea Hospital where she remained for six weeks.  It’s a massively overwhelming experience.

 

Porter

Tell me how you were approached to take part in this trial.

 

Katie

I think when we were about to have our emergency C-section Aniko came to visit us to talk about what coming on to the unit would mean and gently showed us a new model.

 

Porter

Did it appeal to you from the outset, the idea that you’d be more upfront in the care?

 

Katie

Very much so.

 

Deierl

When you work in a neonatal unit and you speak to parents you realise that they enter a world which they didn’t know it exists – you know a neonatal intensive care.  From inside they found it difficult to bond to their babies, they often say they feel as a spectator or a visitor, they have a complete loss of control.  I think units really involve parents at a later stage, usually just few weeks before they’re going home, at the discharge preparation phase but that’s too late, you know some families will spend six, 10 weeks on the intensive care nursery.  So, we started as soon as possible, whenever they have the confidence to do very small tasks, which is like a nappy change or holding the feeding tube and we have designed a competency-based training for them and all the support and resources to get there.

 

Katie

From the outset I was able to hold my baby, she was covered in wires and she was ventilated which could have created problems but due to the model they really encourage parents to hold their baby from the very beginning and in order to do it successfully you actually strip the baby down and you are naked on your bare chest and the interaction between the skin is proven to be incredibly beneficial, both to mother or father and to baby.

 

Porter

But having Constance on your chest while she was being ventilated, did you find it scary?

 

Katie

I did, I found it really scary to begin with.  One of the easiest ways I got round that was actually watching consultants and nurses handle our baby because they look quite alien, as well as the environment around you being alien, the baby looks quite alien, it’s not at all what you’d expect and watching the care that she received totally normalised an otherwise very abnormal situation and it gave me the confidence to handle my baby in the same way.

 

Deierl

All the resources and material we prepared were co-designed with parents, we call them veteran parents, so they had a previous pre-term baby on the unit.  And all the material was read, written, designed, edited, formatted and finalised…

 

Porter

By parents for parents?

 

Deierl

Yes.

 

Porter

What about outcomes, what sort of impact has this had on the babies and their parents and how do you measure that?

 

Deierl

What we found that especially the babies who are born very prematurely, less than 30 weeks, they are going home about two weeks earlier, so we have reduced the length of stay significantly.  It has improved parent experience greatly and also we have a very high rate of breast milk feeding at discharge, about 90% of them receiving maternal milk and the exclusive breastfeeding rate is about 70% in this population, which is well above the national average.

 

Porter

Have you noticed any difference in what happens when the babies go home during those first few weeks because sometimes you might get people coming back in or contacting the unit if they’re concerned, have you noticed a change there?

 

Deierl

I don’t really have any formal data collection but we feel that parents will have an increased confidence and able to assess their babies’ problems or when the baby is sick in a better way.  So, we would expect that actually GP attendances, any A&E attendances have reduced following these care model, just because they know their baby better.

 

Porter

What about potential harms, I suppose one of the concerns might be that if you have parents who aren’t as expert as the carers who normally work in the unit that mistakes might be made, accidents might happen?

 

Deierl

Obviously, we follow up any possible incident on the unit but I’m not aware of any even to which would cause any problem or harm.  I think there is growing evidence that this new care model in neonatal settings is hugely beneficial.  Research in Canada has shown that parents have improved mental health, their anxiety and stress cause are reduced.

 

Katie

Psychologically I think it instigated a really strong bond between Constance and I which would have been impossible.  I found it very difficult to sleep away from my baby, so I came in quite often at 3 am, 4 am, I’d drive in, and I’d start with skin to skin, I would do all her care, so I learnt how to change her nappy in the incubator, which obviously initially is not something you learn at NCT or any other class.

 

Porter

No, lots of wires and things around.

 

Katie

Lots of wires.  It is overwhelming but you quickly get used to it.  I sung to her, read books to her, really just tried to interact with her like any newborn.

 

Porter

Did you have people watching you while you were doing this?  Because I can imagine one of the problems might be that there’s a very experienced nurse standing there thinking ooh, I’ll just step in and help out here.

 

Katie

Do you know, I think, they really believe with that model that the parents are the most important figure and they know their baby the best.  So, nurses and doctors very much trusted, from the outset, that I could do it and that gave me a huge amount of confidence as well.  And although she was heavily medicated at the beginning and ventilated, she progressed in quite an upward curve and as the weeks progressed I became more confident, she got to know me better, she kind of demedicalised her own little incubator.

 

Porter

Bit by bit the wires came off?

 

Katie

Yeah.

 

Porter

There’s your baby.

 

Katie

Totally.  It was isolating and scary at times, I’m not going to lie, but I think due to the experience I’d had on the unit I almost felt like a nurse myself.

 

Porter

What our listeners can’t see, of course, is that Constance is fast asleep on your chest right now, behaving extraordinarily well.  She looks a fairly good size.

 

Katie

She is, she’s done really well.  It’s a dance between the baby and the mother and I think if you really get to know your baby and with the help of this care model you really do these things become less frightening.

 

Porter

Katie and a contented Constance.  And there are more details on the programme on our website. Where you can also find out how to get in touch.

 

Our recent item on moves to prevent patients becoming weaker and frailer in hospital prompted a number of you to ask why we interviewed doctors, nurses and patients on this de-conditioning, but no physios and occupational therapists.  Or dietitians.

 

Brito-Ashurst

I’m Ione de Brito-Ashurst.  I am a dietician by background and I’m currently Head of Therapies for the Marsden.  I love your programme, Inside Health, it’s really good that somebody’s looking at patient mobility and functioning in the hospital.  But you didn’t talk about nutrition, so it was a bit frustrating really.  One of the classic things that people think are the best foods for patients in hospital are the soups, the jellies but really we should think in terms of key nutrients for that patient, to support minimising that loss of muscle that happens when you have bed rest.  And it’s protein, dairy products, meats, cheese, fishes.  But if your patient is already malnourished, probably have a poor appetite, not eating much, you need to consider adding nutritional supplements, so really important to them.  Contact the dieticians that is going to advise you on the best nutritional supplement.  So, you do the exercise and we think that 30 minutes afterwards you would give a high protein food.

 

Porter

What proportion of patients coming into hospital are already nutritionally challenged?

 

Brito-Ashurst

Malnutrition in admission, it’s around – it depends on patients’ age, it depends on clinical condition but about 50% of patients on admission, that have chronic illnesses, tends to be malnourished.  And in the hospital, as well, when I see the patient, I’ll tell them when they have the plate in front of them that they should start with the food that gives them the protein sources and then they move on to the rest of the meals.  And actually, those vegetables, they are good sources of fibres and good sources of vitamins, they should come in after the protein.  So, we want that protein first to make sure that the fibre’s not filling up the patient and preventing him from eating that protein source.  Also, there is some data showing that given a high protein food or drinks, the supplements, before bedtime, really supports the increment in muscle to continue happening during the night.

 

Porter

Thank you, Ioni. We do know that preventing deconditioning is s a team effort, but there are only so many people we can cram into an item.

 

Quite a few of you emailed in after my interview with the team behind the PREPARE programme at St Mary’s – which prehabs patients – mentally and physically – in the lead up to major cancer surgery.  John has been through it and was keen to sing its praises.

 

John

I am in my sixties and was diagnosed with cancer of the oesophagus in 2015.  I’m blind and depend on a guide dog so recovery from major surgery was likely to be a challenge.  But I was determined to beat the disease and the programme helped me do just that.

 

Porter

Keen cyclist and runner Yvonne wasn’t the least bit surprised that prehabilitation can improve outcomes.

 

Yvonne

I had a double mastectomy 15 years ago followed by reconstruction.  I was told I’d be off work for five months but, thanks to being in such good shape, I was back within five weeks.

 

Porter

And David wanted to stress the importance of mental preparation.

 

David

Prior to my triple bypass surgery four years ago I was assigned a designated “heart nurse” who answered all my questions and allayed my pre-op nerves.  I am sure this was a major factor in my speedy recovery.

 

Porter

And talking of advances in preparing for surgery we came across an interesting tweet from anaesthetist Graham McCracken who has been involved in research at Torbay Hospital into whether you really need to restrict clear fluids before surgery.  Current guidelines advise no fluids for at least two hours before an anaesthetic to reduce the risk of aspiration – that’s inhaling vomit.  But Torbay’s experience, after allowing patients to drink water up until they are taken to theatre, suggests that is unnecessarily restrictive.

 

McCracken

The anaesthetic department aligned its practice with our current understanding of how the stomach works, gastric physiology, which is that the stomach empties half its content of fluid around every 12-15 minutes.  And really, they combined this fact with the inherent delays in sending for a patient to theatre between the patient transferring to theatre and the beginning of anaesthesia.  They combined these two to provide a better experience for the patient.  We had water fountains, water dispenser in the waiting areas, encouraging patients just to drink as they felt fit prior to surgery. 

 

Porter

So, there was no restriction at all on their clear fluids.  What did the results show?

 

McCracken

Results were very encouraging.  We conducted a study that looked back at the change in policy, we looked at a year before and a year afterwards, we looked about eleven and half thousand patients over that whole time period and we saw a reduction in post-operative nausea and vomiting.

 

Porter

That’s counterintuitive that more fluid would lead to less nausea and vomiting afterwards, many of our listeners might find that surprising.

 

McCracken

Yeah, I see that as counterintuitive.  The mechanism for post-operative nausea and vomiting may be a result of improved hydration.  But certainly, it doesn’t seem to increase aspiration and it does seem to improve post-operative nausea and vomiting.

 

Porter

So, win win situation?

 

McCracken

Very much so.

 

Porter

Graham McCracken.  

 

And finally, Andrew tweeted after listening to the discussion on how vaping has polarised public health opinion.

 

Andrew

Plaudits to Dr Mark Porter and Inside Health.  I was seriously impressed by the in-depth interview on the vaping debate.  Encouraging and allowing people to advance complex arguments is a challenging business.  Top job.

Porter

Thank you, Andrew – I hope Mum is listening.

 

We will back in the summer for another series.  Until then goodbye.

 

ENDS

Broadcasts

Podcast