Ahhh this has been/still is a battle for me every single day and has been for years.
So what do I do during 'production downtime' other than beat myself up for doing nothing :)
It depends - I've slowly learnt to not be so hard on myself. I often work long hours, work late, work weekends, so if I end up doing nothing while waiting for renders etc then I'm cool with that. I just see it as reclaiming some time back from those long hours. And that's where my 'balance' comes in.
Other tasks I do, you've already mentioned, but sorting out asset libraries, cleaning my desk/office - clear workspace clear mind and all that - stuff that I like to call 'productive procrastination'.
As James said - I'll take a walk with a podcast on. (I have a dog). Podcasts I listen to are Diary of a CEO by Steven Bartlett, and The Collective by Ash Thorp.
I'll take a nap. Why not. I'm often tired late morning/early afternoon. We drink when we're thirsty, we eat when we're hungry, so I sleep when I'm tired. It helps that I work for myself from home in that regard though. Although the reason I work for myself is because 'office hours' never suited me. I'd end up doing very little during my tired hours and then staying at the office late to make up for those hours.
I'll play the Xbox - again it helps I work from home and the Xbox is next to me, but I get personal pleasure out of this and depending on the game, I sometimes find it inspires work. Currently admiring the cinematography/lighting/landscapes of Red Dead Redemption 2.
I subscribe to a tonne of Youtube channels that I'll catch up on - some are work related some are general tech related and some are completely unrelated to what we do.
I'll watch the latest Patreon video from whoever I subscribe to - Fabien Escudero is a current favourite - his Tyflow work is exceptional.
I'll use the time for project specific learning - learn a new technique/software or improve on an old technique/scene - currently trying to get my head round Substance to see how it could fit into or improve my current workflows.
I'll use the time for project specific 'creative' - testing quick sketches on the iPad, or simply gathering photographic reference into Pureref for different CGI's, or maybe flicking through my latest coffee table book with a cup of tea and adding sticky notes/bookmarks to certain pages that I might be able to refer back to for lighting or composition reference.
Read a book, magazine, article, making of, tutorial (i have tonnes of articles bookmarked) - both work related and none work related.
I'll come on these forums and ask my own questions or help others answer theirs.
I'll create todo lists or make notes - I'm always thinking and coming up with ideas - so I write them down so I don't forget. Whether I revisit them or not is a different story, but they're there if I want them at a later date and it keeps my head clear.
Having said all that I do find it difficult, there are plenty of times where the options of 'what to do' feel so vast, that you end up doing none of them. It's like sitting down to watch something on Netflix, you spend hours deciding what to watch only to realise it's now too late to watch anything at all :)
But the main point from all of the above is not to be too hard on yourself regardless of what you choose to do. Life's short. Enjoy it. As long as I can pay my bills and feed and cloth my kids, then I'm happy with all of my other choices.