89 replies to this topic

[MP422]

It's nicely presented but it's flawed IMO.

[PlatenGlass]

Things like this are always interesting, but also always flawed. The problem is that they can only really compare drivers against their team-mates, and one person isn't enough because they can get better or worse! In the all-time list, De Angelis is higher than Mansell because he did better when they were team-mates. But I think most people would agree that Mansell improved as a driver. Also, Frentzen features highly in the all-time list because he was supposedly better than all his team-mates other than Villeneuve. So the argument is that he was a world-class driver who just happened to perform badly the one time he had a good car. Except I'd argue that both Trulli and Heidfeld outperformed him as well.

[lbennie]

What is webber ranked if Rosberg is 6th on all-time list?

[emmanuelrubi]

Maldonado above Kevin Magnussen & Sergio Perez bc who cares about logic anyways.

[hittheapex]

What is webber ranked if Rosberg is 6th on all-time list?

Indeed, and I just remembered that Jacques Villenueve is ranked well below Frentzen too which I can't work out. Like MP422 wrote, "nicely presented, but flawed."

[OO7]

Straight question - should 2007 mean that Hamilton is always considered the equal or better of Alonso? 

No, I'm just looking at how the system is flawed.  If you consider the all-time rankings, Hamilton in 12th has beaten Alonso ranked 3rd, twice beaten Rosberg ranked 7th and also twice beaten Button (and lost to him once) who is ranked 20th.

 

Alonso has beaten Raikkonen ranked 14th, Massa ranked 29th, Trulli ranked 30th, Fisichella ranked 32nd.

[Rurouni]

Maybe if the calculation could be started and ended at different years (meaning only took the data from a certain years) we could have more accurate data. Like only count for 10 years period.

[RunningMan]

Maybe if the calculation could be started and ended at different years (meaning only took the data from a certain years) we could have more accurate data. Like only count for 10 years period.

 

But wouldn't that introduce bias? Actively selecting which year to choose would mean you would ignore "off" years and as a result, wouldn't be representative. Also some drivers peak at different times. Take Button for example. Button has had a better second half to his career than first, but by choosing which half of his career to input, you would be introducing bias.

[MotorsportFerrari]

Even as a man who works in the area of mathematics and as a Ferrari-fan, I doubt the modell.

The sober, clear head dictates Kimi had a very bad season. You can't take into consideration what he did in previous years, because we talk about this year.

You can attack me, but you should know I have six posters of Kimi on my walls, so if anyone can be blamed for hating him, it is clearly not me, he completed his worst year. You can blame the tyres, Alonso, Ferrari's newest pizza or whatever, he lost the sense of looking after the tyres (it was in Hungary and Brazil very good tough), massively lacked of speed in qualy and also in races he was usually stuck behind others. In some races he had such a delta between his stints, in one stint he was on the pace of Fernando, in the next he was around 6-8 tenths slower then him.

All in all there are many drivers who performed much better then him this year. So the metric is just weird.

But there are other examples. Kvyat vs Vergne, that's also weird.

 

It is very tough to decide who was the best: I would say it is between Ricciardo, Alonso and Hamilton, but I would say it was Ricciardo. Ham has made too many mistakes in qualies despite having impressive speed compared to his team-mate, Alonso lost his exceptional sense of pulling out crual overtaking manoeuvres, but you shouldn't forget that this Ferrari PU lacked any real top-speed in races because of the ERS.

[hittheapex]

But wouldn't that introduce bias? Actively selecting which year to choose would mean you would ignore "off" years and as a result, wouldn't be representative. Also some drivers peak at different times. Take Button for example. Button has had a better second half to his career than first, but by choosing which half of his career to input, you would be introducing bias.

I think it comes down to fundamentally different approaches: listing drivers comparing only peak seasons, or comparing their careers as a whole.

[MortenF1]

Can ratings from previous seasons be found? This guy has clearly been watching F1 for a few years.

[Jimisgod]

I always hate how everyone says Alonso was thumped by Hamilton in 2007, but every other driver to get beaten (HAM 2011, VET 2014, RAI 2008 and 2014) suddenly has these magical issues with the car which put those seasons in the "not true form" category. Only Alonso's 2007 can never, ever be explained as him having a poor year and he is always "beaten" despite finishing equal on points.

[aditya-now]

That same model ranks Frentzen and Watson higher than Ayrton Senna.

 

Interesting model, then. Completely inexplicable.

[Collombin]

Interesting model, then. Completely inexplicable.

I would have said the opposite. Uninteresting because it produces nonsense, but "explicable" once you understand the metrics used.

Wattie nearly outscored Prost at McLaren, he outqualified Peterson 100% at Lotus, comfortably outscored Lauda at McLaren etc. If you tell a computer things like that without making any allowances for career paths, changes in form etc then of course some absurd conclusions can be drawn.

[aditya-now]

I don't understand how critical people are of these things, be it rankings about drivers or any other type of sportsmen/teams (for example the widely mocked FIFA Rankings), it's well clear this is an attempt to estimate performance through a mathematical model, he himself admits its flaws, it's not like it's a madman shouting "WE HAVE SCIENTIFICAL PROOF FRENTZEN WAS BETTER THAN SENNA". No method will ever be 100% reliable, but it can help trim out cognitive bias.

 

I think it's highly interesting, very well thought of and very well explained, my only criticism is that indeed age/experience is a missing factor in his equations. I think it would be very possible to introduce this factor and it would give out more accurate overall results, although it might introduce some other biases (for example overestimating highly prepared rookies like Hamilton, who didn't improve much on their rookie seasons), but smaller than the bias we have at the moment due to the lack of this factor. And not only would it help the results, a by-product of introducing this factor is that maybe we could also analyse how important is the level of experience, what is the "ideal" age for performing in F1, and how sharp is the decline at a veteran age and/or the handicap of going in at an early age.

 

How then would you factor in experience (rising from year 1 till career end - say from 70% to 100% capability?) and age (coming in at what age and descending by what factor? - seems to depend on the driver - one driver descends massively (Coulthard 2008), others graciously (Lauda 1984, 1985))?

[aditya-now]

I would have said the opposite. Uninteresting because it produces nonsense, but "explicable" once you understand the metrics used.

Wattie nearly outscored Prost at McLaren, he outqualified Peterson 100% at Lotus, comfortably outscored Lauda at McLaren etc. If you tell a computer things like that without making any allowances for career paths, changes in form etc then of course some absurd conclusions can be drawn.

 

Well, there you have it. Like the model, also our language fails when conveying our impressions to each other.

 

Interesting, that the model produces such nonsense, although it uses objective parameters.

Inexplicable, that the said objective parameters lead to such irrelevant results.

 

Concerning Wattie - maybe he was not so bad after all....

Edited by aditya-now, 28 November 2014 - 12:09.

[hittheapex]

Well, there you have it. Like the model, also our language fails when conveying our impressions to each other.

 

Interesting, that the model produces such nonsense, although it uses objective parameters.

Inexplicable, that the said objective parameters lead to such irrelevant results.

 

Concerning Wattie - maybe he was not so bad after all....

 

Watson had a tough collection of teammates. He had a season alongside Prost, albeit Prost was less experienced at the time. He had 3 more alongside Lauda, he had Tambay for a season as well.

Edited by hittheapex, 28 November 2014 - 12:58.

[noikeee]

I would have said the opposite. Uninteresting because it produces nonsense, but "explicable" once you understand the metrics used.

Wattie nearly outscored Prost at McLaren, he outqualified Peterson 100% at Lotus, comfortably outscored Lauda at McLaren etc. If you tell a computer things like that without making any allowances for career paths, changes in form etc then of course some absurd conclusions can be drawn.

 

Or maybe the computer tells us we have a cognitive bias against Watson as we don't assign significance to the seasons he did well, and overinflate the significance of the seasons he didn't. How would we see him had he got a couple more points in 1982 and had become champion?

 

But I definitely agree he's overestimated due to facing a rookie Prost, which the model doesn't look at.

 

How then would you factor in experience (rising from year 1 till career end - say from 70% to 100% capability?) and age (coming in at what age and descending by what factor? - seems to depend on the driver - one driver descends massively (Coulthard 2008), others graciously (Lauda 1984, 1985))?

 

Read the final part of what I wrote, that could be used as starting point. What I mean is first you'd run an analysis of how drivers perform compared to their best relative to age. That will tell you on average at which age drivers do better, and how much age affects performance. Then use that to offset the results to calculate a driver's worth handicapped per age.

 

Measuring experience is tougher as it's not a hard number like age, I'd suggest experience = the number of full seasons in F1 in which a driver has previously competed. Then you need to find a way of measure drivers that did partial seasons, my suggestion is since the number of GPs is highly variable through the history of F1, you'd divide the number of races a driver did per the number of GPs available, for example, if a guy did 1 race in a season there was 10 GPs he'd get 0,1 seasons of experience; had he done 1 race in a season of 20 GPs he'd get 0,05 seasons of experience.

 

Then you run the same analysis as you did for age to try to assess on average, how much is experience worth, I suspect you'd get a very sharp increase of performance in the scope of the first 3 seasons in F1 but then the curve gets pretty much flat (it's hard to imagine a guy that did 10 years of F1 is any better because of it, than a guy who did 7 years - but if it's 4 vs 1 it's an entirely different story...).

 

Introducing these factors wouldn't make the model bullet-proof at all, and I agree it's particularly hard to assess at which age does a driver lose motivation, as that's a very personal thing and not necessarily just tied to age. It's particularly difficult by having to fit earlier eras of F1, when drivers ran much later into their careers such as Fangio champion in his 40s. But I'm pretty confident overall it would improve the model.

Edited by noikeee, 28 November 2014 - 13:56.

[Collombin]

How would we see him had he got a couple more points in 1982 and had become champion?

We'd see him as extremely lucky. Especially if he beat Rosberg by less than 8 points.

[HP]

 

Wow

All this tells me that is that Kimi was paired with Alonso, arguably the best in F1 this year, nothing about tyres. Also there was Kimi not being paid at Lotus last year, which might need to take into consideration as well.

 

Anyhow I never am comfortable with cold, hard numbers.  There is more to life than that.

 

For a point in case, Massa's performance declined after 2008. There is an obvious reason for that. Not that Massa was the most error free driver before that year,

[AndriesvanOverbeeke]

Correct me if I'm wrong but I believe that graph includes the 2013 Austin GP where Kimi didn't drive.

[Nonesuch]

Of all the diagrams, this one struck me as the most interesting. The author describes how he 'took all race results relative to their teammates, excluding races where either driver had a mechanical DNF. Using a sliding window 9 races long, I calculated the fraction of races in which they beat their teammate. Alonso’s record is shown below.' This one supposedly shows how Rosberg 'ran into progressively more difficulties against Schumacher across 2010-2012.'

 

 

For some reason Schumacher's comeback with Mercedes never really excited me. In Räikkönen Ferrari had a new champion, and in Alonso a new hero. Schumacher at Mercedes just didn't interest me all that much. But this at least suggests that perhaps I should have paid closer attention.

 

 
Wow

 
For those who haven't read the article, this lists the races in which Räikkönen either finished in front of, or behind his teammate. From where it switches to 'harder tyres', Hungary 2013 goes to Räikkönen (who was 2nd to Grosjean's 6th), Belgium is given to neither because Räikkönen retired (while he was in front of Grosjean), Italy is given to Grosjean because he finished 8th to Räikkönen's 11th (you'll recall Räikkönen was forced to make a first lap pitstop for a new wing, putting him last after lap 1). Singapore is again given to neither because Grosjean retired (while in front of Räikkönen, who eventually ended up 3rd). South Korea is given to Räikkönen who was 2nd in front of Grosjean in 3rd. Japan is then given to Grosjean (3rd to Räikkönen's 5th), as is India (3rd to Räikkönen's 7th), and the third one is supposedly Abu Dhabi (where Räikkönen crashed out on lap 1, after starting from 22nd due to a penalty.
 
All in all then, this diagram suggests a bigger difference then I think there was. Grosjean was great late last year, that's definitely true, but races aren't binary.
 
As for 2014; Alonso...
 
 

Correct me if I'm wrong but I believe that graph includes the 2013 Austin GP where Kimi didn't drive.

 
Abu Dhabi, actually.

Edited by Nonesuch, 29 November 2014 - 00:45.

[AndriesvanOverbeeke]

You're right, I forgot about the crash rule ;) Nevertheless, a thought-provoking article: even if you disagree you still need to find out what's wrong with the model in your eyes.

[Acathla]

I always hate how everyone says Alonso was thumped by Hamilton in 2007, but every other driver to get beaten (HAM 2011, VET 2014, RAI 2008 and 2014) suddenly has these magical issues with the car which put those seasons in the "not true form" category. Only Alonso's 2007 can never, ever be explained as him having a poor year and he is always "beaten" despite finishing equal on points.

 

Exactly this. The Alonso versus Hamilton is such a "given" win for Hamilton. Those who say so, forget Hamilton got the ultimate support of the team and Ron Dennis. People who say that Mclaren doesn't have a #1 one driver/pref. /policy, read the autobiography of DC. But hey, Alonso got "beaten" in 2007. At least it wasn't for some bull that the front of the car wasn't to his liking or no "clean"FP, Q or races. Indeed, Alonso got beaten and that's the truth, he's so ousted at McLaren that they eagerly want him back. Difficult to work with and all. All of that is total nonsense. 

[El_Rápido]

Exactly this. The Alonso versus Hamilton is such a "given" win for Hamilton. Those who say so, forget Hamilton got the ultimate support of the team and Ron Dennis. People who say that Mclaren doesn't have a #1 one driver/pref. /policy, read the autobiography of DC. But hey, Alonso got "beaten" in 2007. At least it wasn't for some bull that the front of the car wasn't to his liking or no "clean"FP, Q or races. Indeed, Alonso got beaten and that's the truth, he's so ousted at McLaren that they eagerly want him back. Difficult to work with and all. All of that is total nonsense. 

 

If I'm not mistaken... Whitmarsh himself insinuated that McLaren had made a mistake letting Fernando go so easily around 2009.

[Atreiu]

Straight question - should 2007 mean that Hamilton is always considered the equal or better of Alonso? 

 

It's up to debate. That's all we know with certainty. Or we can say a rattled Alonso is more or less the equal of Hamilton with his customary speed and inexplicable blunder weekends.

 

Anyhow, the site is interesting but whatever metric they use is clearly aimed at projecting the exact results they want.

[Kenstate]

It's up to debate. That's all we know with certainty. Or we can say a rattled Alonso is more or less the equal of Hamilton with his customary speed and inexplicable blunder weekends.

 

Anyhow, the site is interesting but whatever metric they use is clearly aimed at projecting the exact results they want.

 

say what you want about the accuracy/ validity of the model, but it's pretty clearly stated in the abstract that the results are purely the outcome of the mathematical model without bias, as even the author disagrees with some of the outcomes. So I don't see how this is a case of "using whatever metric to project the exact results [they] want", as you can't even begin to speculate where such biases lie.

[ensign14]

Especially in cases like Schumacher who probably lost huge amount of points for his Mercedes years - while Rosberg gained those.

 

Whereas surely Schumacher gains a huge amount of points when there was next to no competition.  Even his team-mate was shackled.

 

The talent depth in the early 2000s was a paddling pool.  Compare the deep-sea dive of the mid-60s with Clark, Hill, Surtees, Brabham, Stewart and Gurney - and that's just the start of the Anglophone world.  Coulthard would have been lucky to score points in 1965...

[Atreiu]

Read the top 60 and give it a few moments/minutes of thoughts and you'll find the bias.
The math has no bias only as far as how it treats input, but not as to how the model was built (and subsequently the results it produces).

Edited by Atreiu, 29 November 2014 - 03:15.

[Kenstate]

Get serious, read the top 60 and give it a few moments/minutes of thoughts and you'll find the bias.

The math has no bias only as far as how it treats input, but not as to how the model was built (and subsequently the results it produces).

 

Which part of the particular parameters he defined in the model do you find biased? The only real problem I see with the model is that it assumes the level of performance of a driver is consistent across his entire career, irrespective of age and experience, but that tends to vary for all drivers.

[Melbourne Park]

A good example of the flaws in the system is Jack Brabham, who is ranked in the late 50's. The modeller acknowledges that Jack (who built many of the cars he raced and is the only maker to win a WDC himself) had much greater reliability than his teammates, which hurts Jack's ranking as a faster driver. 

 

Jack in fact could be very quick but mostly only when the need arose. And he didn't mind being beaten by a team mate either. Afterall his teammates were driving his own cars - it was in Jack's interest that if they were good, then they'd beat him. He also survived in an era where many died.

 

So, Sir Jack doesn't rate highly in the model. A very stupid result because the model acknowledges many things, but not causation which is the number one factor of all.

 

I liked the idea though behind the model, but if you don't factor all the issues, then its going to be greatly effected by the Rubbish In - Rubbish Out characteristic, which has some bazaar effects -  for instance with Black Jack.

Edited by Melbourne Park, 29 November 2014 - 05:55.

[Kenstate]

A good example of the flaws in the system is Jack Brabham, who is ranked in the late 50's. The modeller acknowledges that Jack (who built many of the cars he raced and is the only maker to win a WDC himself) had much greater reliability than his teammates, which hurts Jack's ranking as a faster driver. 

 

Jack in fact could be very quick but mostly only when the need arose. And he didn't mind being beaten by a team mate either. Afterall his teammates were driving his own cars - it was in Jack's interest that if they were good, then they'd beat him. He also survived in an era where many died.

 

So, Sir Jack doesn't rate highly in the model. A very stupid result because the model acknowledges many things, but not causation which is the number one factor of all.

 

I liked the idea though behind the model, but if you don't factor all the issues, then its going to be greatly effected by the Rubbish In - Rubbish Out characteristic, which has some bazaar effects -  for instance with Black Jack.

 

You're right, i don't know how you can model for that because it was such an uncommon situation unlikely to ever be seen again. But I never claimed that the model wasn't flawed, I was just responding to a previous poster who claimed that the author built the mathematical model with a purpose to slant bias. Which you kinda disproved yourself with that example of brabham, because the model only takes raw results into account, while the author acknowledges how brabham's greatness as a car owner isn't factored in and thanks for that little bit about letting his teammates win.

 

Once you start accounting for anectotal stuff like that, you're injecting a certain human subjectiveness into the discussion. and THAT'S bias. I kinda just appreciate it for the pure, mathematical analysis of it all, and people will rationalize why X driver wasn't higher for themselves (the author already kinda does this in all of the explanations). Jack was obviously a special case, as I don't think (not to my knowledge anyway) any of the other drivers on that list were car owners/drivers.

 

It's kinda the same exercise I go through (because I'm a baseball fan), when I read an analysis that compiles the top WAR, across a five year period for every player since the 1950's and see what the math spits out as the greatest baseball player of that time frame. Obviously, there are bound to be a few surprises, but the interesting part is delving in and going "Well, WHY did that mathematical model pick X player as the best player and not Y?" Same thing here, it's beauty of having no preconception/or having your preconceptions shattered that makes these kinds of analysis so interesting.

[velgajski1]

Which part of the particular parameters he defined in the model do you find biased? The only real problem I see with the model is that it assumes the level of performance of a driver is consistent across his entire career, irrespective of age and experience, but that tends to vary for all drivers.

 

That is a big flaw in the model. Having a big flaw like that in the model makes model pointless. It's like having a meteorology model that will occasionaly produce heavy snow and rain even if data show no clouds.

 

Some other flaws mentioned by the author:

- discounting mechanical DNF's caused by aggressive driving/setup

- not accounting for team orders

- not attributing blame for crashes

 

One big flaw that I see and its kinda related to the one you mentioned is the 'ability to win under pressure' / 'ability to consistently race at top level'. I don't think that F1 drivers would say that its the same thing driving in the middle of the pack and fighting for F1 titles. So many drivers (e.g. Fisico / Frentzen) were considered great but for some reason had their worst seasons when they had competitive cars.

 

This is why I think that this is just an  elaborate if A beats B and B beats C then A is better than C. As such it can serve as corrective to a model that is based exclusively on results, but in itself is quite worthless.

Edited by velgajski1, 29 November 2014 - 09:21.

[Amphicar]

Jack was obviously a special case, as I don't think (not to my knowledge anyway) any of the other drivers on that list were car owners/drivers.

 

Jack Brabham was far from unique as a driver/constructor: Bruce McLaren, Dan Gurney, Graham Hill, Chris Amon were too. Black Jack was however easily the most successful - of the others I've named only Dan Gurney (1967 Belgian GP) managed to win a Championship GP as a constructor/driver. Brabham wasn't a constructor throughout his career of course - from 1955 to 1961 he (mostly) drove for Cooper.

[Collombin]

I love mathemathical analyses and I love motor sport - but I don't think they tend to go well together. Chips and custard.

Rather than break these things down and analyze the results as if they mean anything, let's just celebrate that the sport is too complex to be able to be easily disected in this way. Far better to use your own judgement and take into account all the factors relevant to you. But all too often that gets dismissed as fanboyism or bias.

[HP]

As for 2014; Alonso...

Why don't we give Alonso all WDC's from 2005 til 2014 and be done with it?  But then I'm glad that F1 doesn't work that way.

[HP]

I love mathemathical analyses and I love motor sport - but I don't think they tend to go well together. Chips and custard.

Rather than break these things down and analyze the results as if they mean anything, let's just celebrate that the sport is too complex to be able to be easily disected in this way. Far better to use your own judgement and take into account all the factors relevant to you. But all too often that gets dismissed as fanboyism or bias.

Of course is this bias. And we're all better for it. The opposite is that we think all the exact same way like that statistic program (more precisely, we'd think exactly like that person who put the stats together). If we did, we'd have a huge problem.

[Collombin]

I think I meant bias in the sense of rating a driver higher because of liking them rather than because of how good they were. Liking Prost and Piquet far more than Senna and Mansell does not cloud my judgement in admitting who were the better drivers during their respective rivalries as team mates, for example.

[garoidb]

No, I'm just looking at how the system is flawed.  If you consider the all-time rankings, Hamilton in 12th has beaten Alonso ranked 3rd, twice beaten Rosberg ranked 7th and also twice beaten Button (and lost to him once) who is ranked 20th.

 

Alonso has beaten Raikkonen ranked 14th, Massa ranked 29th, Trulli ranked 30th, Fisichella ranked 32nd.

 

He beat Raikkonen, Fisichella and Massa by a lot. Lewis's margin over Alonso was non-existent, and over Button was slim. I don't think the model is up to date for the second half of 2014, which may disadvantage Lewis with respect to Rosberg.

 

The model also appears not to regard drivers as rubbish just because Alonso has beaten them, which is logical and refreshing to see. 

[anneomoly]

I love mathemathical analyses and I love motor sport - but I don't think they tend to go well together. Chips and custard.

Rather than break these things down and analyze the results as if they mean anything, let's just celebrate that the sport is too complex to be able to be easily disected in this way. Far better to use your own judgement and take into account all the factors relevant to you. But all too often that gets dismissed as fanboyism or bias.

 

Yes, but "I used my own judgement" is not so secret code for "I got a nice shiny knife and stuck it in my least favourite driver. Metaphorically, but repeatedly. Then elevated my favourite driver to a god." Because it often is fanboyism or bias under a thin veneer of pretend impartiality.

 

At least the article doesn't make (too many) excuses for drivers ending up lower than where the author thought they ought to be and generally accepts the principle of the thing. It's just refreshing to read something that isn't kicking someone when they're down or raising their national driver beyond where there should be.

 

(It helps that the rankings make sense to me, mainly.)