BBC BLOGS - Open Secrets
« Previous | Main | Next »

Marking teachers

Post categories:

Martin Rosenbaum | 08:45 UK time, Tuesday, 15 March 2011

How good are you at your job? Does your boss know? If your boss is the British public - in other words, if you're a public sector worker - do they know?

The transparency agenda of this government and its predecessor has recently opened up a lot more information about the details of public spending. Some of this has been about the salaries and expenses of identifiable individuals, with more promised for the future.

But to assess value for money, cost is only half the equation - the other half is achievement. Plenty of data about targets and indicators for public services has been issued over the past few years - but much less about specific staff. Freedom of information requests have only occasionally (such as for heart surgeons) produced records about the performance of individuals.

These thoughts occurred to me after I came across a project in the Los Angeles Times, which rated the success of 6,000 of the city's teachers by name. This was actually published last August, but I only became aware of it last month when it won the top award at the conference of the US National Institute for Computer-Assisted Reporting.

The teachers' results were based on the "value-added" progress made by their pupils from year to year in standardised English and maths tests. The LA Times included a list of the 100 teachers who scored best according to this system, although it doesn't seem to have publicised the lowest performers in the same way.

Of course not all those covered by this analysis are happy, and this kind of methodology for measuring teacher effectiveness has been criticised by some.

Now other newspapers in different parts of the US have tried to get the same information. A judge in New York has ruled that the interests of parents and taxpayers should outweigh the privacy rights of public employees.

It's very unlikely that the same kind of data about the performance records of individual teachers would be released in the UK, even if the information existed in that form. As well as the privacy concerns, questions would be raised about how well one numerical measure could encapsulate an individual's achievements. But the LA Times didn't find it easy to obtain the material either.

When I told Jason Felch, one of the leading reporters on the story, that there would be enormous resistance to the publication of such data here, he replied:

"'Enormous resistance' is a fair description of what we faced".

Is that resistance justified?

Comments

  • Comment number 1.

    Value added sounds like a sensible measure but it can deliver results which are quite unexpected. For a number of years my wife taught Spanish A level - a subject where English mother tongue students compete with many native speakers. Her very best added value was always with Spanish mother tongue students. Her very best English mother tongue students with excellent GCSEs struggled to deliver a positive residual - most of them showed a negative residual even if they scored an A grade at A level (then the highest grade available).

    That kind of oddity is less likely to be a problem in mathematics and English across a broader population of students in a large city like Los Angeles, but even there I would hesitate to use value added as a prime measure of teacher performance.

  • Comment number 2.

    Training as a teacher and I'm undecided. We are public servants and so should be accountable (if you don't like it - get another job). I know the US teaching unions, where outraged by it. I'm not sure exactly why it's seen as so insulting?

    Yes it is not a perfect measure but as far as I'm aware it's the best measure we have. There are lots of examples of problems that arise with target based systems (least of which is more stress in the classroom). Humans do not behave as rational beings and there can be lots of unintended consequences that lead to kids learning less.
    http://danariely.com/2011/01/10/a-gentler-and-more-logical-economics/

    As a scientist I would try do a few good empirical studies, publish the scores of teachers of several schools and see if there's any improvement compared to a control group.

  • Comment number 3.

    Yes, the resistance is justified. There are a huge number of factors affecting any performance metric you can think of. If this blog's readership growth were to slow, would that imply that its author has become less informative? Or maybe the author has become less populist but more informative. Maybe some other blog has attracted lots of readers. Maybe it's just that all those people who might ever start reading this blog are already reading it. It's impossible to disentangle.

    Similarly, if a the performance of a class increases less than last year's class, that could mean that the teacher was less good this year, but it could be that a disruptive pupil joined the class, or that a few of the kids had problems at home, or that the classroom they were allocated was noisier or draughtier, or that or that the previous years' test had been marked a bit generously, etc etc.

    This value-added measure probably is the best measure we have, but sadly it's still not a useful one. We need to find a better way of monitoring and incentivising performance. We also need to accept that simple measures like this are only actually simple in role-playing games, not in real life.

  • Comment number 4.

    Martin, this is a VERY dangerous way forward. I expected better from someone with your level of knowledge and experience.

    There is a vast amount of evolutionary and behavioural science based evidence that selection based on individual performance as opposed to population or group performance is a patently stupid idea. Basically all this would do is stop individual teachers helping weaker colleagues as it would now be in their interests for other teachers to be worse than them. All you would end up with is selfish back-stabbing teachers and a poor working environment.

    Any action like this should focus on 'teams'. This then encourages collaborative effort and means that the good teachers have an interest in either supporting or 'outing' poor colleagues.

    This is the worst type of 'bright idea' that behavioural scientist should be jumping on and denouncing. Anyone who thinks this is sensible is an idiot.

    PS The same applies to bonuses and performance pay - individual rewards are just divisive. Team, department or faculty reward and promote teamwork and collective responsibility.

  • Comment number 5.

    Martin - Bonus/performance information of senior public sector employees can and often is requested in this country under the Freedom of Information Act.

    The Information Commissioner’s guidance on disclosure of employees’ salaries states that public authorities need only disclose salary information within a £5,000 band unless any of the following exceptional circumstances arise.

    These are where:
    • there are current controversies or credible allegations;
    • there is a lack of safeguards against corruption;
    • normal procedures have not been followed;
    • the individual in question is paid significantly more than the usual salary for their post; or,
    • the individual or individuals concerned have significant control over setting their own or other’s salaries.

    http://www.ico.gov.uk/upload/documents/library/freedom_of_informatio n/practical_application/salaries_v1.pdf

    He has taken the same stance in relation to staff bonuses. However there is a more cautious approach when it comes to performance information.

    The First Tier Tribunal (Information Rights) decision in Davis v IC and Olympic Delivery Authority (EA/2010/0024) concerned a request for bonus payments, performance targets and the targets levels achieved in relation to senior staff at the ODA. In coming to its decision, the Tribunal distinguished between bonus information and performance assessment information.

    It ordered disclosure of certain information relating to the bonuses of senior employees of the ODA: the maximum performance-related bonuses to which the chief executive and communications director were contractually entitled, and the percentage of the maximum available bonus actually paid to certain other members of senior management.

    However it decided, that details of the performance targets which individuals failed to hit to 100% satisfaction should not be disclosed. It said that in each case disclosure would involve an intrusion into an element of the individuals’ lives which, while work-related, has such a direct impact on career progression and personal self-esteem that it would only be warranted if, in addition to the matters of public interest identified, the operation of the remuneration scheme justified significant criticism.

    If this is the stance the Tribunal is taking in relation to senior employees' performance, one can expect an even less "open" stance in relation to more junior/front line staff. I think we are a long way off the New York model.

  • Comment number 6.

    I echo comments #3 and #4 above. This data is not only misleading for the reasons stated, but also apparently dangerous.

    The Huffington Post covered an extreme response to the LA Times coverage (http://www.huffingtonpost.com/2010/09/28/rigoberto-ruelas-suicide-_n_742073.html%29, where a teacher reportedly took his own life on the back of the article (the LA Times disputes this, of course). Such a reaction will be the exception but it underscores the burden that misleading the public can place on this particular front-line community profession.

    I don't advocate for no accountability. I believe the buck stops with management. It's management that should be publicly accountable for the year-on-year performance of their staff and in turn, teachers are accountable to their managers within a private space that effective organisations [should] afford.

    Bringing this full circle to FOI, thankfully British jurisprudence recognises this relative notion of accountability, as comment #5 above sets out in detail.

  • Comment number 7.

    When I worked in a public sector, my senior manager was totally ignorant in science. However, the sector was working on very dangerous materials.

    She loved to make technical decisions, but all the decisions based on her own feelings. She had no Bachelor's degree, but had been working there for more than thirty-five years.

    I was very disappointed. I thought that such working conditions in a science sector could have only happened in developing country.

 

More from this blog...

Latest contributors

BBC © 2014 The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.