Here is the literal translation into English, ready to be used:
There is something deeply touching about the ritual of electoral polls. They are like horoscopes for people who know how to use Excel. Nobody admits to believing in them, yet everyone consults them before leaving home. In any case, today we are not here to complain that polls fail. We have come to ask ourselves if they always make mistakes towards the same side of the sofa.
To understand this, we must get down into the mud of the data. The starting point is elementary statistics:
\[Error = Estimate - Actual\ Result\]
But a 3-point error doesn’t carry the same weight in every election. So, we do what anyone with too much free time would do: standardize. We convert that error into standard deviations relative to the set of comparable polls. From here, I have calculated the average of those errors for the parties in the government bloc. I have dubbed it:
PGDI: Pro-Government Deviation Index
Results (or when intuition gets slapped)
Applying this thermometer to the history of general elections in Spain, the numbers have decided to stop being discreet.
| Institution | PGDI (Average) |
|---|---|
| CIS | \(\approx 0.86\) |
| Private Pollsters | \(\approx -0.43\) |
This is no longer an anecdote, because when we contrast the data:
- The difference is significant (\(p \approx .009\)).
- With a massive effect size (\(d \approx 2.15\)).
To put it simply, the CIS doesn’t just deviate; it behaves differently from the rest and in the opposite direction. While others tend to fall short, the CIS always provides a floor for the Government.
The unit nobody asked for: The Matute
Saying that a poll has “0.86 standard deviations” is not sexy. It is not something you can toss out at a Christmas dinner to win an argument. That is why I have decided to define a new unit of measurement. Initially, the temptation to call it the “Tezanos” was almost irresistible for obvious empirical reasons. However, for the sake of hygiene and elegance, I opted for a more technical solution with the same weight:
1 Matute = 0.86 standard deviations
Now reality is manageable. We can already say:
- “This poll deviates exactly 1 Matute in favor of the Executive,” or,
- “This other one barely reaches 0.2 Matutes,” as examples.
But be very careful: none of the above proves manipulation. Polls are complex, involving variations in how to manage models, weightings, and “kitchen” decisions. But when the bias is not random and the toast always lands on the same side, it stops being a technical error and becomes a pattern. And when faced with a pattern, the most elegant response is not to raise your voice, but to take out the calipers and measure it in Matutes.
To learn about the entire process followed, access the data, and replicate (or even critically evaluate the above), you can access all documentation in the OSF repository:
- Project link: https://osf.io/y6e42/
- Article preprint: https://osf.io/preprints/socarxiv/8cfyt_v1
Source of the cover photo: Bundesarchiv, Bild 183-1990-0318-032 / Oberst, Klaus / CC-BY-SA 3.0, CC BY-SA 3.0 DE https://creativecommons.org/licenses/by-sa/3.0/de/deed.en, via Wikimedia Commons