Author Topic: Render times increasing every frame?  (Read 8497 times)

2019-11-19, 23:32:39

rojharris

  • Active Users
  • **
  • Posts: 99
    • View Profile
Hello,

I have a scene with a reasonably complex product with a few lights and a few not too big textures, nothing out of the ordinary. I'm doing a spin round render over team render on a mac and PC. The Mac is i9 and the PC threadripper. When the spin started each frame was taking 2mins each on each machine, now, 25 frames in they are taking 10 minutes per frame!! This is nuts.. Both machines have 64Gb Ram and Corona is set for UHD cache and 15 pass limit. Why on earth would it be getting longer and longer?
I know this used to be a problem back in the early days but surely it must have been solved by now? We're on V4 and nearly 5 for goodness sake.

I cannot send the scene as its very NDA and I daren't stop it as it must be finished in the morning.

Any ideas what could be causing such an issue? Is it just team render perhaps?

Note the view hasn't changed much in 25 frames so it should not be taking longer.

Thanks

2019-11-19, 23:39:37
Reply #1

Jak

  • Active Users
  • **
  • Posts: 63
    • View Profile
I had a similar problem, check out the following link:

https://forum.corona-renderer.com/index.php?topic=25751.0;topicseen

Are you using libraries from external hard drive or through a network?

In the end I did not manage to solve it, had to finish rendering the animation with increasing render times, and unfortunately had to extend the deadline.

2019-11-20, 09:17:41
Reply #2

rojharris

  • Active Users
  • **
  • Posts: 99
    • View Profile
Thanks Jak,

Nope, no external libraries. It's quite a simple scene. I just can't understand how this can happen on a so called production ready renderer. Don't get me wrong I love Corona. I prefer it to Redshift and Octane, both of which I have, but I always try to use Corona. This is the first time I've used it for animation though and it's quite a shock.

Cheers,
R

« Last Edit: 2019-11-25, 11:14:57 by rojharris »

2019-11-20, 15:05:34
Reply #3

rojharris

  • Active Users
  • **
  • Posts: 99
    • View Profile
So here's a bit of insanity:

The render didn't finish overnight and the first batch has just finished (Team render broke it into two batches of 150 frames each). The render times stayed high for all the frames but as the last few were nearing completion, they started to go down again!! What??? The last few frames of the spin are pretty much exactly the same as the product hardly moves, and yet the times drop from 7 mins to 2 mins. What on Earth is the logic behind that happening......

2019-11-25, 09:52:34
Reply #4

rojharris

  • Active Users
  • **
  • Posts: 99
    • View Profile
This has happened again on a 4k render and even happens on Rebus, which means we're paying too much for our farmed out renders. It looks like render times increase on a bell curve, getting longer in the middle and faster at the start and end. I even tried on one machine... same results, fist few frames 2-3mins/frame, increasing to 30mins/frame, then back down to 2mins for the last few.. So the bulk of the animation is taking way too long.

I am going to do some experiments with a simple scene to see if I can show this happening and will file a bug report as this should definitely not be happening..

I've just rendered out a couple of 11000 px stills that rendered in 10mins! This is faster that the 3480px frames by 300% . !! Nuts!


« Last Edit: 2019-11-25, 11:14:32 by rojharris »

2019-11-25, 15:58:47
Reply #5

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5434
    • View Profile
Sorry for the delay in writing back - we've been looking into this (using another scene that gets slower with repeated renders - but of course we can't confirm it's the same issue as you have reported, since unfortunately we can't access your scene). For our reference, (Internal ID=427033657)

Some questions for you, if you don't mind
- Which version of Cinema 4D, which version of Corona, which OS?
- What are the Corona Team Render settings? Are they default, or did you modify them at all?
- What speed is your internal network, 1GB or 10GB?
- Does the scene have high geometry, e.g. something like grass that isn't using instancing at all, or similar?


We understand you may not have time to do any testing, but here are some tests that would provide useful information for us

- Does the same scene show this slowdown if it is rendered locally, and not using Team Render? Or does the slowdown only occur using Team Render?

- If it is Team Render related, it could be that data is "backing up" on the network, which might explain the bell curve (first renders are fine, as the backlog increases it takes longer to have a frame marked as complete, and at the end that backlog begins to catch up again). Since these are large resolutions, that's not impossible. You could try swapping to "Abritrated (experimental)" which is intended to help with any network congestion. If that alone doesn't help, you could try leaving it set to Arbitrated and increasing both the client update interval and the Maximum size of packet (so that larger packets of data are sent, but less often) - for larger resolutions, packet sizes of 256 or 512 may be useful, and update intervals of 20 or 30s may be useful

- If you have any information on what part takes longer, that would be great, e.g. is the parsing time for the scene getting longer with each render? Or is that staying constant, but the rendering stage is taking longer? This could be seen in the VFB stats tab.

Also, the VFB render stamp (which can be added as a multipass render element in the latest versions of Corona 5, and saved separately by enabling saving of all passes in the C4D save set up) can be used to show parsing time, render time, memory use, number of primitives being rendered, and so on. In case you are able to look into that, here's the render stamp I use in my tests that includes the relevant info, you can copy and paste:
Corona Renderer | Corona Mem %mcp | Corona + C4D Mem %mpp | Parsing Time %ptp | Noise %pe | Time: %pt | Passes: %pp | Primitives: %si | Rays/s: %pr

------------------

If you are able to replicate the issue in a scene that is safe to share with us privately, that would of course be awesome. We appreciate your patience, and any additional information you can give (and if you can do any specific tests, very much appreciated!)
Tom Grimes | chaos-corona.com
Product Manager | contact us

2019-11-25, 16:19:37
Reply #6

rojharris

  • Active Users
  • **
  • Posts: 99
    • View Profile
Hi Tom,

I mentioned a bit more further down the post but to answer what I can right now:

Cinema 4D R20, Corona 5RC2, Mojave and Win 10... Same problem on both machines. Mac is intel i9, Win is Threadripper. both 64GB RAM

Same thing happens with and without team render... but only on animation... stills are fine.
Same happens on Rebus farm too!

Scene is a fairly simple product model.. Not polygon heavy and only a few instances. I can share it once the product is launched but that may not be for a while.


I've started using Arbitrated for TR but as I say, it makes no difference whether I use TR or not. Just rendering on local machine (either mac or PC) gives same problem.

At the moment I don't know which part of the render is taking the time. I'll do some tests as you suggest later in the week if I get time.

Many thanks,
Roger

2019-11-25, 16:30:00
Reply #7

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5434
    • View Profile
Thank you Roger for that information, very useful! Especially knowing it happens whether TR is used or not - that increases the odds that it's the same issue as we are seeing in this other scene that also gets slower the more frames that are rendered (that scene, it's the parsing time that gets slower - memory use and primitives stay the same). We do understand that testing is often impossible, as you have all the production work to take care of, so we truly appreciate any that you are able to do. Meantime, this information is very useful, and as I say, it sounds more like the same issue we have a test scene for, so if we find a cause/solution there, we can see if it helps for your scene too.

Thanks again for your patience!
Tom Grimes | chaos-corona.com
Product Manager | contact us

2019-11-26, 14:25:12
Reply #8

rojharris

  • Active Users
  • **
  • Posts: 99
    • View Profile
Hi Tom,

Here's a little movie showing the render stamp of the first 20 frames of the animation (cropped from 4k). As you can see, although the parsing does increase by a small amount, it seems to be the rays/sec calculation that's slowing down considerably. Even after 20 frames we're at triple the initial frame time. What's a bit freaky is that the final 20 frames will speed up again in the opposite to this...

Hope it helps.

2019-11-26, 14:47:40
Reply #9

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5434
    • View Profile
Hi Roger! Again, thanks for that, awesome information! I've grabbed the movie and added it to our internal tracking. We'll let you know as soon as we have any further information, and you can give us a holler if you think of anything else that might be a pointer to what is happening!
Tom Grimes | chaos-corona.com
Product Manager | contact us

2019-11-29, 12:58:15
Reply #10

rojharris

  • Active Users
  • **
  • Posts: 99
    • View Profile
Hi Tom,

I've messed around with the file to take out the really sensitive stuff and I wonder if I can send you the scene to try and work out why the times increase so much. Note this is nothing to do with team render, it happens on both my machines on their own.

Please be aware that the model is still very much NDA so I need your assurance that it will not be seen by anyone other than the developers. i.e. any visitors wandering around wouldn't get to see it etc. :-)


Cheers,
Roger

2019-11-29, 14:19:56
Reply #11

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5434
    • View Profile
Hi! Thank you, that would be awesome, we appreciate you taking the time to do that. The uploader is at https://corona-renderer.com/upload

As specified on the page, the scenes are never used for anything other than testing; no images, renders, animations, and no part or whole of the scene is ever shared outside of us; and you can specify whether we can keep the scene for testing only after we resolve your particular bug (it can be handy in the future for checking the same thing didn't break again when changes are made, for instance), or whether we should just delete it once testing for this particular bug is done. So, definitely safe to share with us :)
Tom Grimes | chaos-corona.com
Product Manager | contact us

2019-11-29, 14:26:48
Reply #12

rojharris

  • Active Users
  • **
  • Posts: 99
    • View Profile
Thanks Tom, That's great. I'm uploading it to the 10GB Dropbox location as the zip is 300Mb.

I don't know how quick you guys get to these things but I think I have to render this again fairly soon and of course with frame time issue it's costing me a lot of render points over on Rebus. If someone opens the file and see's that I've checked the wrong check-box and that's causing all my problems then please let me know :-)

I doubt it will be that simple though.

The file is there now.

Cheers
R

2021-04-13, 17:27:10
Reply #13

thomaslivings

  • Active Users
  • **
  • Posts: 28
    • View Profile
Hi, did the problem here get resolved? I have the same increasing frame time problem.
thanks in advance,
Tom.

2021-04-14, 09:40:41
Reply #14

mmarcotic

  • Former Corona Team Member
  • Active Users
  • **
  • Posts: 544
  • Jan - C4D QA
    • View Profile
Hello,

we have made several improvements in this, so we would be glad to know some details of your issue.
If you could, is it possible to send us your scene?
Also, have you ever noticed at which point the slowdown is visible? (e.g. geometry parsing, scene prep, rendering time -- visible in VFB if you catch the frame, not in Picture Viewer)
And as usual, please can you tell us your corona version, c4d version, pc specs?

Thanks,
Jan
Learn how to report bugs for Corona in C4D here.