Jump to content


Photo
- - - - -

90% predictions


  • Please log in to reply
24 replies to this topic

#1 Forghieri

Forghieri
  • Member

  • 680 posts
  • Joined: March 16

Posted 14 February 2019 - 09:59

In psychology there’s an interesting bias called "overconfidence effect", wherein people subjective confidence in their judgements is reliably greater than the objective accuracy of those judgements, especially when confidence is high. The data show that confidence systematically exceeds accuracy, meaning people are more sure that they are correct than they deserve to be. For instance when people provide a 90 percent confidence interval around their estimate of some number, the truth often falls inside their confidence intervals less than 50 percent of the time.

 

So I would like to apply this concept to our WCC predictions, seeing if we can overcome the confidence effect and gauge correctly our accuracy.

 

Can we correctly predict the WCC standings with a 90% confidence level? The goal would be to provide for each constructor a range with a 90% probability to be correct, that is to get 9 predictions right out of 10. If we get all 10 predictions right, it will mean the ranges we chose were too large; if we get less than 9 right, they are too narrow.

 

Here is my take:

 

Mercedes: 1-2

Ferrari: 1-3

Red Bull: 1-3

Renault: 3-5

Racing Point: 3-7

Alfa Romeo: 5-8

Haas: 5-8

Toro Rosso: 6-10

McLaren: 6-10

Williams: 7-10



Advertisement

#2 sgtkate

sgtkate
  • Member

  • 5,559 posts
  • Joined: May 17

Posted 14 February 2019 - 10:12

Isn't that the Dunning-Kruger effect, kinda....when you have a little bit of knowledge you massively overestimate your abilities, then you learn more and realise you don't actually know all that much, and then when you get loads of experience you start once more to believe in yourself again.

 

Interesting idea anyway :)


Edited by sgtkate, 14 February 2019 - 10:14.


#3 JG

JG
  • Member

  • 568 posts
  • Joined: March 00

Posted 14 February 2019 - 10:21

I wouldn't say that Red Bull Honda would have the same odds as Ferrari. Otherwise the team ranking seems to be right on. Though, McLaren could surprise a bit this year, having new sponsors and team boss. 



#4 Kalmake

Kalmake
  • Member

  • 4,492 posts
  • Joined: November 07

Posted 14 February 2019 - 10:40

Mercedes 1-2

Ferrari 1-2

Red bull 2-4

Renault 3-4

Racing Point 5-7

Alfa 5-8

Haas 6-8

Toro Rosso 7-10

McLaren 7-10

Williams 7-10



#5 Kalmake

Kalmake
  • Member

  • 4,492 posts
  • Joined: November 07

Posted 14 February 2019 - 10:46

I wouldn't say that Red Bull Honda would have the same odds as Ferrari.

Those are not odds.



#6 FullOppositeLock

FullOppositeLock
  • Member

  • 10,960 posts
  • Joined: September 15

Posted 14 February 2019 - 11:10

Mercedes: 1-3

Ferrari: 1-4

Red Bull: 1-4

Renault: 3-6

McLaren: 4-8

Racing Point: 4-8

Haas: 4-9

Alfa Romeo: 4-10

Williams: 4-10
Toro Rosso: 6-10


#7 Anderis

Anderis
  • Member

  • 7,345 posts
  • Joined: December 09

Posted 14 February 2019 - 12:06

Mercedes 1-9

Ferrari 1-9

Red Bull 1-9

Renault 1-9

McLaren 2-10

Racing Point 2-10

Haas 2-10

Alfa Romeo 2-10

Williams 2-10

Toro Rosso 2-10



#8 sopa

sopa
  • Member

  • 12,230 posts
  • Joined: April 07

Posted 14 February 2019 - 12:09

Mercedes 1-9

Ferrari 1-9

Red Bull 1-9

Renault 1-9

McLaren 2-10

Racing Point 2-10

Haas 2-10

Alfa Romeo 2-10

Williams 2-10

Toro Rosso 2-10

 

I'm not sure, how did you predict this with 90% confidence level, instead of 100%. But I assume there is always a chance one of the first four teams gets disqualified in WCC, hence making your overall prediction 90% correct.  :p



#9 Grayson

Grayson
  • Autosport digital product manager

  • 3,497 posts
  • Joined: July 08

Posted 15 February 2019 - 14:11

Can we correctly predict the WCC standings with a 90% confidence level? The goal would be to provide for each constructor a range with a 90% probability to be correct, that is to get 9 predictions right out of 10. If we get all 10 predictions right, it will mean the ranges we chose were too large; if we get less than 9 right, they are too narrow.

 

That's true in theory, but there are a few caveats:

 

- These predictions aren't independent of each other. Let's say that there were only two teams in the championship and I thought that there was a 90% chance of Team 1 beating Team 2. If I predicted that Team 1 would win (90% probability) and Team 2 would come 2nd (90% probability). The chance on being wrong on two independent 90% predictions if I'm properly calibrated should be 1% (0.1 x 0.1) but in this instance I should be wrong with both of my two 90% predictions if the 10% chance of Team 2 beating Team 1 comes in.

 

- Ten predictions are far too small a sample size to figure out whether your confidence level was right. If your predictions are perfectly calibrated (ie. your predictions really were exactly 90% likely), there's only a 39% chance of you getting exactly 9/10. That's almost as high as the probability of getting 10/10 at 35%.

 

- Simple scoring (the idea that we're aiming for 9/10) ignores the fact that we might be underconfident about some teams and overconfident or just plain wrong about others without this balancing out. I can get full marks by predicting that Mercedes will finish 17th while the others will all finish somewhere between 1st and 10th!

 

That said, I'll have a crack anyway:

 

Mercedes 1-2

Ferrari 1-3

Red Bull 1-4

Renault 3-7

Racing Point 4-8

Haas 4-8

Alfa Romeo 5-9

McLaren 5-9

Toro Rosso 6-10

Williams 6-10



#10 Nonesuch

Nonesuch
  • Member

  • 15,870 posts
  • Joined: October 08

Posted 15 February 2019 - 15:22

Here goes:

 

 

Mercedes: 1

Ferrari: 2-3

Red Bull: 2-4

Renault: 3-5

Alfa Romeo: 4-6

Haas: 5-7

McLaren: 5-8

Racing Point: 6-8

Toro Rosso: 7-10

Williams: 8-10



#11 Radoye

Radoye
  • Member

  • 3,365 posts
  • Joined: March 09

Posted 15 February 2019 - 15:37

Mercedes 1-2

Ferrari 1-3

Red Bull 2-5

Renault 3-5

Alfa 4-6

McLaren 4-7

Haas 5-8

RP 6-9

STR 7-10

Williams 8-10



#12 Mc_Silver

Mc_Silver
  • Member

  • 5,339 posts
  • Joined: May 09

Posted 15 February 2019 - 15:44

Mercedes 1
Ferrari 2
Red Bull 3-4
Renault 4-5
McLaren 4-6
Alfa 5-7
Haas 5-8
RP 6-9
STR 7-10
Williams 8-10

#13 efuloni

efuloni
  • Member

  • 763 posts
  • Joined: November 16

Posted 15 February 2019 - 17:30

Mercedes 1-2

Ferrari 1-3

Red bull 1-4

Renault 3-4

Racing Point 4-7

Alfa 5-7

Haas 6-8

Toro Rosso 9-10

McLaren 6-9

Williams 8-10



#14 rodlamas

rodlamas
  • Member

  • 11,318 posts
  • Joined: February 04

Posted 15 February 2019 - 17:58

Mercedes 1
Ferrari 2-3
Red Bull 2-4
Renault 3-5
McLaren 4-6
Alfa 5-7
Haas 6-8
RP 6-9
STR 7-10
Williams 9-10 



#15 thegforcemaybewithyou

thegforcemaybewithyou
  • Member

  • 4,006 posts
  • Joined: April 12

Posted 15 February 2019 - 18:57

Mercedes 1-2
Ferrari 1-3
Red Bull 1-3
Renault 4-7
McLaren 5-8
Alfa Romeo 6-9
Haas 5-8
Racing Paint 5-7
Toro Rosso 6-9
Williams 8-10



#16 GrumpyYoungMan

GrumpyYoungMan
  • Member

  • 6,994 posts
  • Joined: July 12

Posted 16 February 2019 - 07:07

I wouldn't say that Red Bull Honda would have the same odds as Ferrari. Otherwise the team ranking seems to be right on. Though, McLaren could surprise a bit this year, having new sponsors and team boss.

Think it’ll take a few seasons for the damage at McLaren to be recovered from slightly!

#17 Gary Davies

Gary Davies
  • Member

  • 6,459 posts
  • Joined: April 01

Posted 16 February 2019 - 07:25

Think it’ll take a few seasons for the damage at McLaren to be recovered from slightly!

That’s the received wisdom, and probably the best bet. But.... I wonder if in 2019 they may be something of a wildcard. I’m not saying wildcard as in beating, say, Mercedes but here’s why I’m speculating.
• Their 2018 car had one massive and dominating aerodynamic flaw, which we’re led to believe they came to fully understand during the season. So let’s assume they will not repeat that mistake.
• I believe (I’m sure the cognscenti here will put me right if necessary) that other aero and design aspects of MCL33 were pretty good.
• It’s looking increasingly likely that their engine supplier, which has been recruiting, regrouping and upgrading its resources as fast as it can over the last two years, may have made its biggest improvement since the current engine regs have applied.
I’ll concede there are a few ifs in there, but if all three factors coalesce, this team could make a bigger leap than most in 2019.

#18 w1Y

w1Y
  • Member

  • 10,606 posts
  • Joined: March 16

Posted 16 February 2019 - 09:32

Ferrari 1-2
Merc 1-3
Red Bull 2-3
Haas 4-6
renault 4-6
TR 5-8
RP 5-8
mcalren 6-9
alfa 7-9
Williams 10

#19 Forghieri

Forghieri
  • Member

  • 680 posts
  • Joined: March 16

Posted 16 February 2019 - 09:54

That's true in theory, but there are a few caveats:

 

- These predictions aren't independent of each other. Let's say that there were only two teams in the championship and I thought that there was a 90% chance of Team 1 beating Team 2. If I predicted that Team 1 would win (90% probability) and Team 2 would come 2nd (90% probability). The chance on being wrong on two independent 90% predictions if I'm properly calibrated should be 1% (0.1 x 0.1) but in this instance I should be wrong with both of my two 90% predictions if the 10% chance of Team 2 beating Team 1 comes in.

 

- Ten predictions are far too small a sample size to figure out whether your confidence level was right. If your predictions are perfectly calibrated (ie. your predictions really were exactly 90% likely), there's only a 39% chance of you getting exactly 9/10. That's almost as high as the probability of getting 10/10 at 35%.

 

- Simple scoring (the idea that we're aiming for 9/10) ignores the fact that we might be underconfident about some teams and overconfident or just plain wrong about others without this balancing out. I can get full marks by predicting that Mercedes will finish 17th while the others will all finish somewhere between 1st and 10th!

 

You're right, thank you for your comments. Concerning the third point, I meant that each single prediction should have a 90% probability of being true, not just the ten predictions together. If there are enough participants, I am curious to see what would happen if we put together all our predictions, say by taking the 90 percentile of each interval.

 

[Edited for clarity]


Edited by Forghieri, 16 February 2019 - 10:31.


Advertisement

#20 Forghieri

Forghieri
  • Member

  • 680 posts
  • Joined: March 16

Posted 16 February 2019 - 10:03

Isn't that the Dunning-Kruger effect, kinda....when you have a little bit of knowledge you massively overestimate your abilities, then you learn more and realise you don't actually know all that much, and then when you get loads of experience you start once more to believe in yourself again.

 

Interesting idea anyway :)

 

Thank you. Yes, I would say they are related, although the Dunning-Kruger effect is often misrepresented, with a curve like this:

 

tnNFgnF.png

 

whereas the actual curve is more like this:

 

Z9ZY8LD.png


Edited by Forghieri, 16 February 2019 - 10:12.


#21 Anderis

Anderis
  • Member

  • 7,345 posts
  • Joined: December 09

Posted 16 February 2019 - 11:25

I'm not sure, how did you predict this with 90% confidence level, instead of 100%. But I assume there is always a chance one of the first four teams gets disqualified in WCC, hence making your overall prediction 90% correct.  :p

There are 10 possible places where a team can finish in WCC, so to have 90% confidence I need to cover 9 out of 10 possible places. :p



#22 Tsarwash

Tsarwash
  • Member

  • 13,725 posts
  • Joined: August 10

Posted 16 February 2019 - 17:26

I don't know if anybody has already mentioned it, but isn't it impossible to get 9 out of 10 predictions correct ? You can do any other number correct, but not that one. 



#23 scheivlak

scheivlak
  • Member

  • 16,473 posts
  • Joined: August 01

Posted 16 February 2019 - 17:51

Mercedes: 1-2

Ferrari: 1-2

Red Bull: 3-4

Renault: 4-5

Racing Point: 5-9

Alfa Romeo: 5-8

Haas: 5-8

Toro Rosso: 7-10

McLaren: 6-9

Williams: 8-10



#24 thegforcemaybewithyou

thegforcemaybewithyou
  • Member

  • 4,006 posts
  • Joined: April 12

Posted 16 February 2019 - 18:02

I don't know if anybody has already mentioned it, but isn't it impossible to get 9 out of 10 predictions correct ? You can do any other number correct, but not that one.


You would be correct if every team would be connected to a single place, but for this thread you can give a range of outcomes which means overlaps are possible.

#25 apoka

apoka
  • Member

  • 5,878 posts
  • Joined: May 09

Posted 16 February 2019 - 18:02

I don't know if anybody has already mentioned it, but isn't it impossible to get 9 out of 10 predictions correct ? You can do any other number correct, but not that one.


Why?

Maybe you are only thinking of predictions where you distribute the 10 possible positions over each team, but we use ranges of positions in this thread.