Yes, in some scenes the benefit of distributed rendering is really small, or worse the render takes longer than it does on one machine. For the most part I've learned to optimize for my particular network setup to avoid these issues, but they do occasionally creep up.
But the primary reason to do this is that sometimes getting four 6-hour images at once is better than running them one at a timed having each take 2 hours. A frequent scenario I run into is a queue of let's say 30 images that I run overnight. I check the server after being away from it for 12+ hours to find that all of my slaves except one or two hung, destroying any DR gains and keeping other images from being done.
Long story short, I don't think it should be all or nothing. I've often been saved by V-Ray's checkbox that allows me to disable DR.