Football rankings
July 2009 (index)
Many sports have an official ranking of their participants. Football (soccer) does not (as fair as I am aware). So this is an attempt to rank English football teams.
We'll use results data for the 2008-09 season from http://www.betexplorer.com and a Bayesian ranking system. This based on a description of a ranking system for croquet with some changes. These changes are partly complications to allow for differences between the games (unlike croquet, football matches can be drawn, and the home team tends to win more games than the away team) and partly simplifications (for now we'll assume ability is constant over the season).
The idea is that each team has a numerical skill level, and the difference in skill level between two teams tells us how likely each team is to win. We can't observe the skill levels directly but we can infer information about them based on the results of matches using Bayes theorem.
Bayes theorem
Bayes theorem is all about conditional probabilities; the probability of X being true given that we know that Y is true.
To take a slightly whimsical example. Suppose we are picking a person at random from a list of everyone in the UK. We'll use "M" for the event that the person is male; and "N" for the event that the person is called Norman.
We'll take a wild guess at the numbers. Let's say there are 60,000,000 people in the UK; 30,000,000 males; 300,000 men called Norman; and small number X of women called Norman.
So P(M)=30,000,000/60,000,000=50% and P(N)=300,000/60,000,000=0.5%
The conditional probability P(N|M) is the chance that someone is called Norman given that we know the person is male. P(N|M)=300,000/30,000,000=1%.
The conditional probability P(M|N) is the chance that someone is male given that we know the person is called Norman. P(M|N)=300,000/(300,000+X). I'm guessing that X is less than 300 so P(M|N) is greater than 99.9%.
Note that P(N|M) and P(M|N) are wildly different in this case.
Bayes theorem gives us a formula that relates these probabilities:
P(N|M)P(M)=P(M|N)P(N)
In our case we can check that it works 1% x 50% = 99.9..% x 0.5%
In this example we could work with the actual numbers of people (the "natural frequencies"). Most people find this easier when it's possible, but Bayes' theorem also works when we only know the probabilities, not the frequencies.
Our ranking system
Our simple model (before worrying about draws and home advantage) is that a team that is ranked x points higher than its opponent has a probability P(x)=logistic(x)=1/(1+exp(-x)) of winning.
This has the necessary properties that P(x)+P(-x)=1 (so the chance of each team winning adds up to 1) and P(x) is always between 0 and 1 so we always get a valid probability.
Rankings and Bayes theorem
In our case we know the results of matches but we don't know the skill levels.
Bayes tells us that:
P(Skills|Win)P(Win)=P(Win|Skills)P(Skills) and
P(Skills|Lose)P(Lose)=P(Lose|Skills)P(Skills)
Note we write "Skills" plural, since the probabilities depend on the skills of both teams.
We'll start with an initial view of the distribution of skill of both teams (with a lots of uncertainty), and then this formula will let us update these skills based on the result of the match. Gradually the uncertainty will decrease.
An example:
Suppose teams A and B play and we think that each has a 50% chance of being ranked 11 or 10. It turns out that A wins. What are the new ranks?
Initially we think there's a 25% chance of the combinations of skill levels for A and B of (11,10), (11,11), (10,11), (10,10).
If the skills are equal then they each have a 50% chance of winning.
If the skills are different then the team with an extra skill point has a 73% chance of winning.
Suppose the outcome is that A wins. We can calculate as follows (writing S for Skills for brevity).
Skills P(S) P(A wins|S) P(S)P(A wins|S) P(S|A wins)
(11,11) 25% 50% 12.5% 25%
(11,10) 25% 73% 18.25% 37.5%
(10,11) 25% 27% 6.75% 13.5%
(10,10) 25% 50% 12.5% 25%
After the game we think there's a 62.5% chance that A has a skill of 11 and a 37.5% chance that A has a skill of 10. Because A won it's more likely that A has a high skill, and we're slightly less uncertain about the skill level.
Implementation
In practice we'll model the skill of each team with a normal distribution and store the mean and standard deviation. When we update for a match we'll model this as 11 possible skill levels for each team (at -5,-4,...,+4,+5 standard deviations from the mean), giving 121 pairs of skill levels with appropriate probabilities. We then update the probabilities at these 121 points based on the result of the match and then reduce these to a mean and standard deviation for each team.
Draws and home advantage
We'll modify our simple formula so that if the home and away teams have skills H and A then:
P(Home win) = logistic((H+B)-A-D)
P(Away win) = logistic(A-(H+B)-D)
P(Draw) = 1-P(Home win)-P(Away win)
logistic(x)=1/(1+exp(-k*x))
B and D are constants that we'll estimate
k is a constant that we'll choose
Effectively B is a bias for the home team (it's as if a team's skill is this many points higher when playing at home) and D is a drag term that reduces the chance of each team winning, hence giving a non-zero probability of a draw. We could try more complex approaches (perhaps the home bias is linked to the skill-level of a team) but this will do for now.
Since only the difference in skill levels matters we could choose for an average team to have any value. We'll use 100 as the average and choose a value for k=1/50 so that (empirically) skill values for a single league are typically in the range 0-200; rather than using k=1 and having them all in the range 98-102.
Testing this
Our skill levels will predict the probability of a result. We want to be able to test how good the results are (to compare to random or simple systems) and also to calibrate to the values of B and D that give the best predictive ability.
We'll do this using the score "statistic 1" described in ***LINK GOES HERE***
But first, to give a baseline for our results, we'll look at the score of some crude predictions, ignoring the identity of the teams and not using the rankings.
100% probability of home win: Score 317.0
50:50 chance of home/away win (no draw): Score 190.0
1/3 chance for each of home/draw/away: Score 178.8
(46%,28%,26%) chance of home/draw/away: Score 172.6
Now to fit to the actual results. We'll set the home bias B to zero for now and try different values for D for draws.
We initialise each team with a normally distributed skill of N(100,1002) and run through the 2008-2009 season ten times to get good convergence.
D Score
5 152.5
10 149.2
15 146.6
20 144.8
25 143.6
30 143.1
35 143.1
40 143.5
45 144.3
50 145.4
Now taking D=32.5 and trying different values for home bias H.
H Score
0 143.0
5 140.7
10 138.9
15 137.7
20 137.1
25 137.1
30 137.5
35 138.5
40 139.9
45 141.6
50 143.8
So D=32.5, H=22.5 look like reasonable values.
This gives a score of 137.0 and rankings of:
TEAM MEAN STDEV
Manchester Utd 185.9 6.0
Liverpool 171.8 5.7
Chelsea 164.7 5.6
Arsenal 144.8 5.2
Everton 126.6 5.1
Aston Villa 118.4 5.1
Fulham 105.0 5.1
Tottenham 98.6 5.2
West Ham 98.2 5.1
Manchester City 89.6 5.3
Stoke 82.6 5.1
Wigan 81.8 5.1
Portsmouth 74.6 5.0
Blackburn 71.6 5.0
Bolton 68.8 5.2
Newcastle 65.0 5.0
Sunderland 63.6 5.2
Hull 62.1 5.2
Middlesbrough 54.0 5.2
West Brom 52.0 5.3
Note that the rankings aren't evenly distributed. The average ranking is exactly 100 and we have a lot of teams a bit below average and a few exceptionally good teams. The gap in skill between Chelsea and Fulham is bigger than that between Fulham and West Brom.
We can also calculate the probability of results of certain games based on these rankings. Looking at matches between the top and bottom teams (Manchester United and West Brom) the rankings say that:
Man U's chance of a win is 92% at home and 83% away.
West Brom's chance is 5% at home and 2% away.
Visualising this
We can also look at visualising rankings for different years.
Thoughts for later
Some other things we could do with this:
(i) Compare the rankings to the league table
(ii) Compare rankings across divisions/countries
(iii) Make a random league table based on rankings - eg given the rankings how likely were team X to be relegated?
ibbly.com contact