Rugby World Cup – Round 2 Review and Round 3 Picks

Just a quick update this time, as I am travelling. To recap round 2:

RWC Pool Round 2 Results

Again, not too bad. The model really only got one result badly wrong – Japan vs. Scotland, where it had picked a Japan win (albeit narrow). This highlights one of the downsides of this type of analysis – we do not really have that much data to go on for some teams, and so single results can have a big impact. Remember we are only using results from the past 2 seasons – going further back in time gives us more data to work with, but runs the risk of said data being out of date and not reflective of the teams’ rating now. We also picked the England-Wales game incorrectly, although that was always going to be close. The key difference (or so the model thought) should have been the home advantage to England – as it turned out, it may have been a disadvantage on the day…

Round 3 ratings and picks

The model has updated the ratings based on the results from round 2 (remember, RWC games count double):

Ratings 29sep15

The big changes are largely with the minnows shifting up or down depending on whether they lost by more or less than the model predicted in round 2. Scotland have improved by 4 points, although the model does not think this will be enough to save them against South Africa (see below). Japan drops by 9 – again, this is due to the limited data (number of games) to base the ratings on, which adds a certain amount of volatility. Interestingly, Argentina is continuing to improve, and is now rated just shy of Australia, England and Wales.

So, for the upcoming Round 3 games, the model’s picks are:

RWC Pool Round 3 Predictions

So, according to the model, the key games to watch are Samoa vs. Japan, and England vs. Australia. Again, the points difference for England is coming from the home advantage rating, suggesting a very close match is in store. Wales vs. Fiji may also be closer than the model suggests – it of course does not take account of injuries affecting a team’s rating.

Let’s see next week how things have gone!

RWC – Round 2 Predictions

After a reasonable amount of success predicting round 1 (apart from that game), it’s now time for round 2 before things get underway. Firstly, I have re-run the ratings model to include the results from the RWC round 1 games. Just like World Rugby’s official rankings, I have given all RWC games double weighting. The updated ratings are:

Ratings 22sep15

There have been a few movers & shakers following the Round 1 fixtures. Japan and Georgia are the big winners, moving up 8 ratings points each. The losers, unsurprisingly, are the two teams that lost against them – South Africa and Tonga. Interestingly, Namibia and Romania moved down in the ratings, despite not playing – this is due to the change in ratings of teams they have played recently, meaning the model has recalibrated its ratings for these two teams. Ireland and Argentina have also moved up, Argentina now sitting above France, and Ireland sitting clear at #2 (excluding home advantage).

One interesting result is the instability of the home advantage score (this is one reason why it is in multiples of 5). Our home advantage estimate of several of the teams that performed better than expected (Argentina, Georgia, Ireland, Japan) has dropped, as their good away-from-home performance forced the model to recalibrate. Similarly, South Africa’s home advantage estimate has increased to 10 – it is much harder to beat the Boks on the veldt than at the English seaside.

So, what does this mean for our match predictions? Here they are:

RWC Pool Round 2 Predictions

First, note that I have only included home advantage for England (and Wales, when they play in Wales). This is more correct than my half-way solution from the previous round – it is certainly more in-line with the data inputs into the model.

So, the interesting predictions: Japan to beat Scotland in a very close game, South Africa to easily account for Samoa despite their loss to Japan, Argentina to show Northern Hemisphere scribes that they are significantly better than a team knocking on the door of the 6 nations, and NZ to have the biggest win of the round. Let’s see how we go!

RWC – Round 1 Review

If you recall, last week we published the predictions for round 1 of the RWC from our (fairly basic) rugby model. Let’s look how things went…

RWC Pool Round 1 Results

I’ve decided to group the predictions into 3 classes: green for those where the model picks the right winner and the margin prediction was within 7 points (i.e. a converted try) of the actual margin; blue for those where the right winner is picked, but the predicted margin is wrong by more than 7 points; and red for those where the model predicts the wrong winner (irrespective of margin). There is no great science behind this, but it provides us with an OK indicator of model accuracy without getting bogged down in too many numbers. Given the model itself is fairly approximate, it’s about right that the measurement of its effectiveness is fairly approximate too.

Apart from one glaring match, the model did not do too badly. Half of the games were rated green, and we even got one game bang on (USA v. Samoa). The model did not make the same mistake as many journalist pundits did (and continue to do) of under-rating Argentina – the model got that one pretty much right (and it would have been even better had the ABs not bombed so many tries…).

Of the two rated blue, the Ireland Canada game was difficult because of the large gulf between the teams’ ratings and the final score – any model will struggle to get a blow-out to within 7 points. And the model was actually out by only 9 – pretty close to green actually.

The two games the model did not predict accurately were the two surprises of the round – Tonga v. Georgia and the biggie, Japan v. South Africa. The model was not alone in not correctly predicting the Japan game – and I include the South African management in that.

So, a pretty good start. Round 2 starts tonight – I will update the model to take account of the round 1 games, and make some more predictions!

The Rugby World Cup – its all in the numbers

Like most Kiwis, right now my mind is on the Rugby World Cup (and little else). Unfortunately in Norway no one knows what rugby is, so I’ve been spending my time educating them. A common response, apart from “why do the big guys keep making the turtle thing” – they mean the scrum (I think) – has been “you’re a numbers dude, why don’t you make some match predictions.

So I have.

Using results from all matches between the 20 teams involved in this year’s RWC, I put together a pretty simple model that calculates team ratings. The difference between the ratings for any two teams is the model’s prediction for the expected score difference should they play each other. The idea behind the model that it will try to minimise the prediction error (well, the square of the error) overall. Also, I’ve added also in a “home advantage” rating – realising that playing at home of course generally (but interestingly, not always…) makes you perform better.

So, all in all not a super sophisticated model, but neither should it be really. We only have past data to go on here – each of the teams playing is potentially quite different to previous incarnations of themselves – different players, coaches, tactics, hair gel, the lot. So the past is only a so-so guide. Any model is only as good as the information used to build it, and it is a waste of time building a sophisticated model on a house of sand.

The ratings

Anyway, so what are the ratings? I used two data sets – the first only including all matches since 2014, and the second all matches since 2012. The thought was that this would give us some indication of ratings stability and also recent form. So, here they are:

Ratings 18sep15

With the ratings, you can see things like the value of Twickenham for England (5 points), that Wales performs as well away as at home, and that Ireland and Australia most definitely outperform on home soil. And of course the ABs are best. As if there was any doubt.

Note that the Home Advantage ratings are in multiples of 5 only – there is a limit to accuracy here and it’s no good pretending otherwise. This also means of course that two teams with 1, 2 or 3 points between them in ratings are essentially equal – or at least the model has difficulty in separating them anyway.

Rugby World Cup – Round 1 match predictions

So, let’s use these to predict the results of the first round of pool play, that starts tonight (yes!). Note as well as giving England full home advantage, I’ve given the other “home” nations half their Home Advantage ratings, reflecting that they are – nearly – playing in front of home crowds. Anyway, the predictions:

RWC Pool Round 1 Predictions

No great surprises here – the closest games then look like being Samoa v. USA, Tonga v. Georgia and NZ v. Argentina. We’ll come back next week and see how the model has done.

Go ABs!

(Note: I inadvertently transposed Australia and Argentina in my original post, so have fixed this – sorry for any confusion this may have caused!)