How good are the weather forecasts?
The BBC weather service
offers countless weather forecasts on their web site. One of the
most useful features being their localised weather reports. This
gives a five day prediction of the weather for any location in
Britain. For example, here's what the weather in
is prophesised to be.
Problem is, just how good are these predictions? After once using the
BBC's weather page to book myself a nice sun-drenched Friday off work,
only to find it pissing down with rain on the day, I was inspired
(well... ok I got bored sometime later on) to write a small computer
program to check up on the BBC. Are their five day weather
predictions useful, or are they no good at all? Is the super computer in
the MET office accurately modelling the world's climate, or is it resting its
brain and picking out sun and rain symbols at random? We will find the answer!
So how do we tell if they're right?
Well, rather than scurry around the whole of the UK in order to check their
weather divinations, I've chosen to take the easy route and use the BBC's own
web pages. The five day weather predictions also include today's weather.
I'm making the assumption that predicting today's weather is dead simple,
so the BBC couldn't possibly get this wrong. So every day, a computer script
reads the latest five day forecast, and compares today's weather with what the
BBC said it would be in their previous forecasts.
Over time, as this program collects more and more of the weather predictions,
we'll be able to see how often the weather forecasts were right.
What are the results?
The tables below show the accuracy of the BBC's weather forecasts, listed
by time. E.g. as I write this, the table below shows that the weather
forecast for Cambridge one day ahead was 53% accurate. In other words, the
BBC's guess about tomorrow's weather in Cambridge was right roughly half of
It's not so useful to just show the numbers for the BBC's weather accuracy on
their own. You need something to compare them to. So, the tables also show
the accuracy of another really simple weather predictor - guessing that
tomorrow's weather will be the same as today's. Hopefully the BBC weather
will be better than this?
|BBC accuracy||58 %||50 %||46 %||43 %|
|BBC (yearly)||90 %||86 %||83 %||82 %|
|My accuracy||37 %||33 %||31 %||30 %||
|BBC accuracy||57 %||48 %||44 %||40 %|
|BBC (yearly)||89 %||85 %||83 %||82 %|
|My accuracy||37 %||32 %||31 %||30 %|
|BBC accuracy||59 %||51 %||46 %||41 %|
|BBC (yearly)||90 %||86 %||85 %||80 %|
|My accuracy||38 %||33 %||31 %||30 %||
|BBC accuracy||54 %||45 %||41 %||38 %|
|BBC (yearly)||85 %||82 %||80 %||80 %|
|My accuracy||37 %||34 %||32 %||30 %|
|BBC accuracy||58 %||49 %||45 %||41 %|
|BBC (yearly)||88 %||86 %||86 %||82 %|
|My accuracy||38 %||32 %||31 %||31 %||
|BBC accuracy||56 %||48 %||45 %||39 %|
|BBC (yearly)||86 %||83 %||82 %||79 %|
|My accuracy||36 %||31 %||29 %||30 %|
|BBC accuracy||59 %||50 %||47 %||42 %|
|BBC (yearly)||89 %||84 %||82 %||80 %|
|My accuracy||37 %||32 %||32 %||31 %||
|BBC accuracy||58 %||48 %||44 %||40 %|
|BBC (yearly)||88 %||86 %||86 %||81 %|
|My accuracy||36 %||33 %||32 %||29 %|
|BBC accuracy||60 %||51 %||47 %||43 %|
|BBC (yearly)||90 %||85 %||83 %||81 %|
|My accuracy||38 %||32 %||32 %||31 %||
|BBC accuracy||56 %||47 %||43 %||39 %|
|BBC (yearly)||87 %||83 %||82 %||81 %|
|My accuracy||36 %||33 %||32 %||30 %|
How do I measure 'accuracy'?
Very simply! I take the BBC's weather icons and compare them, using a bit of
leeway. So if the prediction is 'Partly Cloudly', then 'Sunny Intervals' is
also considered equivalent. Likewise, 'Light Showers', 'Light Rain' and
'Drizzle' are all considered close enough to be an accurate forecast.
The 'yearly' accuracy is based on the last 365 days of results, to
show if the BBC is getting better or worse over time.
I'm working on adding a better scoring system, since this accuracy
measurement is a bit rubbish really...
So? Are weather reports prophetic or pathetic?
So what have we learned? Can we fault the prescience of the BBC's five day
forecasts? Yes. The weather predicted by the BBC for four days time was just
30-40% accurate - i.e. they get it wrong around two times out of three.
My alternate weather prediction scheme isn't stunning either - it gets a
fairly consistent 25-35% accuracy, for all lengths of time. This is surprising
- it means that the weather in four days time is just as likely as
tomorrow's to be identical to today's. (If you can follow that and figure out a
better way to phrase it, please let me know!) It is reassuring that the BBC
do better than this simple prediction technique!
The moral of this is don't use the forecast to book time off work! :)
Links to more weather sites