Author Topic: DR internal functions dissected/explained  (Read 1169 times)

2021-08-28, 12:38:47

kingmax

  • Active Users
  • **
  • Posts: 22
    • View Profile
Hi.

Recently working on a heavy scene, I've noticed that my render nodes are taking quite some time to pick up the job and actually starting to render. Talking about 5-10 minutes on a 2.1 GB scene and 2.24 GB assets across 444 files.

So, everything is pulled out from a pure SSD NAS running over 10Gbe. The render nodes uses "standard" SSD local disks (app. 500/500 R/W performance)

All the assets in Max have the appropriate UNC path set, and all the nodes uses the same account name and can see/read those files.

What I've noticed is that in the VFB the nodes spend quite some time on "downloading assets"?

Can anyone shed some light on how exactly does DR internally work?

1. Prior being sent to the nodes, the working .max file gets saved locally or in the original opened location? I'm asking this because even if the 10Gbe link get's saturated between the workstation and the NAS in normal network traffic, it's still faster to save the scene on a local NVMe drive running on PCIe.  and copy that large file to the NAS.

2. What does "PARSING SCENE" mean? I'm guessing that's the time required to actually open the file and load all the assets on the slave machines?

3. "DOWNLOADING ASSETS" - does this refer to actually copying the assets on the target machine, or it's the action where the node reads the assets from a shared network location?

Here's an extract from tone of the render nodes logs:

2021-08-28 11:35:04   Ready for new connection
2021-08-28 11:42:39   Accepted remote connection from 192.168.2.30
2021-08-28 11:42:39   Started connection #3
2021-08-28 11:42:39   Handshake: <root><DataType>0</DataType><Handshake>init</Handshake><FrameName>0</FrameName><HostVersion>Max2021</HostVersion><CameraName>CoronaCamera001</CameraName><Region>0 0 0 0</Region><ImageSize>1800 1350</ImageSize><MD5>47b6082ca67f9a723bd7e7dd7d2bd227</MD5><CoreVersion>7 (Hotfix 1)</CoreVersion><IpAddress>192.168.2.30</IpAddress></root>
2021-08-28 11:42:39   Saving incoming scene to C:/Users/backburner/AppData/Local/CoronaRenderer/DrData/received_scene_1841907443.max
2021-08-28 11:42:49   Downloaded scene: 2.161 M bytes in 4 s
2021-08-28 11:42:56   Received sampling focus mask (region 0 0 1800 1350)
2021-08-28 11:42:56   Render for connection #3 started
2021-08-28 11:45:37   Received a list of missing assets
2021-08-28 11:45:37   Got a message from remote: all_assets_sent
2021-08-28 11:45:38   Sending command to slave: <root><Command>all_assets_sent</Command></root>
2021-08-28 11:45:38   Received a list of missing assets
2021-08-28 11:45:39   Got a message from remote: all_assets_sent
2021-08-28 11:45:39   Sending command to slave: <root><Command>all_assets_sent</Command></root>
2021-08-28 11:45:40   Received a list of missing assets
2021-08-28 11:45:40   Got a message from remote: all_assets_sent
2021-08-28 11:45:40   Sending command to slave: <root><Command>all_assets_sent</Command></root>
2021-08-28 11:45:41   Received a list of missing assets
2021-08-28 11:45:41   Got a message from remote: all_assets_sent
2021-08-28 11:45:42   Sending command to slave: <root><Command>all_assets_sent</Command></root>
2021-08-28 11:45:42   Received a list of missing assets
2021-08-28 11:45:42   Got a message from remote: all_assets_sent
2021-08-28 11:45:43   Sending command to slave: <root><Command>all_assets_sent</Command></root>
2021-08-28 11:46:21   Received a list of missing assets
2021-08-28 11:46:22   Got a message from remote: all_assets_sent
2021-08-28 11:46:22   Sending command to slave: <root><Command>all_assets_sent</Command></root>
2021-08-28 11:47:09   Received a list of missing assets
2021-08-28 11:47:09   Got a message from remote: all_assets_sent
2021-08-28 11:47:10   Sending command to slave: <root><Command>all_assets_sent</Command></root>
2021-08-28 11:50:11   Started EXR dump (402.082 bytes, filename: C:/Users/backburner/AppData/Local/CoronaRenderer/DrData/dump4306.exr)
2021-08-28 11:50:11   Finished EXR dump after 0 s
2021-08-28 11:50:12   Received sampling focus mask (region 0 0 1800 1350)
2021-08-28 11:51:13   Started EXR dump (202 M bytes, filename: C:/Users/backburner/AppData/Local/CoronaRenderer/DrData/dump4307.exr)
2021-08-28 11:51:14   Finished EXR dump after 1 s
2021-08-28 11:51:14   Received sampling focus mask (region 0 0 1800 1350)
2021-08-28 11:52:16   Started EXR dump (204 M bytes, filename: C:/Users/backburner/AppData/Local/CoronaRenderer/DrData/dump4308.exr)
2021-08-28 11:52:16   Finished EXR dump after 0 s
2021-08-28 11:52:16   Received sampling focus mask (region 0 0 1800 1350)
2021-08-28 11:52:39   Got a message from remote: terminate_hard
2021-08-28 11:52:40   Sending command to slave: <root><DataType>0</DataType><Command>terminate_hard</Command></root>
2021-08-28 11:52:41   Slave 3ds Max ended render
2021-08-28 11:52:41   DR is finished
2021-08-28 11:52:42   Ready for new connection

I can see the the scene was copied in 4 seconds which is as expected, but can't understand why it took almost 5 minutes to actually start working on the job?
And why are there 7 entries for "Received a list of missing assets"?

Upon further examination of the DrData folder on one of the remote machines, the only files there corresponding to that job are the .max file , 2 config files, and a couple of small files from Forest Pack dist maps and custom CUBE files from the frame buffer.

Since I've switched from Vray a couple of years ago, I can't do a comparison test, but I recall scenes would load much faster on the render nodes using Vray DR back at the time.

2021-09-24, 10:23:24
Reply #1

hrvojezg00

  • Active Users
  • **
  • Posts: 270
    • View Profile
    • www.as-soba.com
I`d like to know the answer as well. We have tried both mapped and UNC paths and with both we get the "downloading assets" line which can take some time, even though all nodes have direct access to all assets. Any news on this, it`s been going on for some time now.

Thanks,
H

2021-09-25, 23:42:58
Reply #2

Avi

  • Corona Team
  • Active Users
  • ****
  • Posts: 476
    • View Profile
Hi,

In reply to your questions.

1. Before sending to nodes, the working max file must be on a shared drive network which every node can access after that Scene file is sent to the temporary folder in DR nodes which is defined here: https://bit.ly/3m0PZCB
You can also change its location if you like.

2. In Short "Parsing scene" basically means preparing scene to start rendering and this basically involves all the steps from opening the file, locating its textures, checking all proxies location, calculating any displacement present in the scene, etc.

3. Downloading asset means copying assets to the nodes.

So, if you see, there could be multiple factors why it takes a bit of time to start the rendering. Even if you have the world's best hardware but software and its factor also plays a huge role. Such as 3dsmax itself being single-threaded and will perform badly with the complex and heavy scene, other factors could be how many 3rd party plugins are used to create the scene. Also, if you have a million trees with high displacement on the scene will parse for a long time as it will try to calculate displacement on every single tree.

In short, there could be multiple factors that can cause slowdowns ranging from hardware to how scenes were created using the software.

If you have a scene in which you are facing any issue, you can always send it to us and we will try to troubleshoot why the slowdown is happening and we would be able to suggest the workaround/solution accordingly.

I hope this ans some of your questions.

Regards,
Avi
Arpit Pandey | chaos-corona.com
3D Support Specialist - Corona | contact us