Allen was never the problem on ITV, it was the production team and their lowest common denominator approach to push the channel's Lewhysteria rhetoric. Plus he suffered from coming straight off the back of Murray and being judged in comparison to him, rather then these days where he's compared to Croft (with very similar flaws to James) and Legard (cliched and cringeworthy)
Agreed on the Murray comparison and I was guilty of it myself in my younger days. The lead commentator gig post-Murray was pretty much the definition of poisoned chalice, anyone was going to fail no matter who they were.
I still feel he didn't "fit" but I still honestly don't feel I can be objective on that because I was always subconsciously comparing him to Murray. I did sometimes sense he was trying a bit to hard with the race starts and "xxxx WINS!!!" whereas with Murray it came across natural. It might be just he was (is) a genuinely highly knowledgeable and informative journalist (going by his written work) who wasn't suited to voice commentary, no shame in that really.
In a different way I felt Brundle didn't fit in 2011 (for the races I actually bothered to watch.. it was the first Pirelli/DRS year so went out of my way to miss a few), good as an informative analyst/co-commentator, but as lead commentator it didn't feel right.
Croft I just find very meh all round.
TL;DR - Murray is irreplaceable.