Simple render test with Grant Warwick' Material Ball.
http://www.grantwarwick.net/mastering-vraynew-page/The material setup uses the same bitmap with separate composite mixes for Diffuse, Reflection, Glossiness, and Bump.
First Image is with Pyramidal Filtering, Blur Values of 1.0, 0.5, and 0.01
Second Image is with Summed Area, Blur Values of 1.0, 0.5, and 0.01
Third Image is with None, Blur Values of 1.0, 0.5, and 0.01
No filtering and 0.01 in the bump slot causes material problems. Which Keymaster does say Bump maps need some sort of filtering.
With these simple renders, No Filtering with any amount of blur produces the same render times, which are faster then any of the filters.