Ahhhhh, I've found the cause...... The camera is moving too slow.
I've tested speeding up the camera and I'm getting a more colourful velocity pass.

We're doing a timelapse kind a animation for a client.
Here is the first test I did and I got good results. I was happy :)
I didn't expect this headache today.
In my scene, the camera is moving slow but the Anima characters are sped up 2000% (same as the one above)
Any ideas how I can do this for this shot. I need the people to be whizzing through my scene (with the same trails as the video above).
I'll try the world setting in the velocity pass. The fact that it's set to camera should have sounded warning bells in my head.
The fact that Anima characters aren't actually animated in the scene, might also be an issue. They seem to be getting their pass colouring from the camera. I animated a cube and that's giving a brighter colour