Maybe we are thinking in compare tests in a different way and with a different objective in mind... I do them and I like them to decide what is going to be my production render engine, where to spend my money if I have to, I need to compare, and the thing is that I need to compare for the end result I'm going to get, is not about this render is better than this other, the Furryball is a great example of a biased comparison, for a start you can take tht test just for interior viz, and one not too complex, and for the end, they didn't even tried to get the same result and with the best time with every render, this is a biased test used commercially to convince people that Furryball is better, IMHO this test is easily beatable, in fact here is where eyeball comparison can tell you something is wrong with this test, different refraction bounces, different noise level, different AA, those times and render results are completely random.
But if a good test is done, like some tests that are out there comparing vray with mental ray, they can help decide you if that render engine is good for your project or not.
Now this is a completely subjective matter, I mean, I can get my conclussion, I can think Corona is better than vRay based on my experience, I can think Corona is better than iRay based on my tests and feature set, but is subjective, but a good comparison is always to be subjective, so you can try to do the best test you can, being as fair as you can with each compared render engine, and maybe you will get some helpful results for your type of job.
Now, what I'm not trying to say is that a comparison will tell us wich render is the best render, what I'm trying to say is that it's completely valid to compare vray with it's biased methods against Corona completely unbiased if you want, because the idea is what you've said in the quoted phrase:
Regarding end-result for client, that's absolutely different matter. Almost all renderers can give you identical result if you try.
"Almost all renderers can give you identical result if you try." YES, but in what time? that's the test I meant :)
Cheers.
EDIT: "“To put it into perspective, when Pixar was developing the movie ‘Cars 2,’ its average rendering time per frame was 11.5 hours,” says Bassett. “We’re rendering at seven seconds a frame using Octane. It changes everything. There are now Hollywood movies being made using this technology.” " this is marketing crap, it's a non-sense :P