Author Topic: Team Render Job failed  (Read 6610 times)

2019-01-25, 07:59:20

friedli3d

  • Active Users
  • **
  • Posts: 33
    • View Profile
Hello,

I have purchased a corona license for c4d and im trying to make to work with team render but im having an issue.

The error says (Error) Render-Job failed: Rendering stopped because of an out-of-memory or unknown error

I have tried with a different file, with simple materials, and the team render worked fine, but with this file, which has a lot of objects, refs, etc, it gets always this error.

I send the message from console, with a render that worked fine and the one who doesn't.

Thank you for your help


    > 3designs-iMac-Pro1.local. Download-Speed 14.73 MiB\s (1x)
Peer-to-Peer Statistics End
Grabbed Render-Job 'pv' from machine '3designs-iMac-Pro1.local.'
Created Asset 'Untitled 1.c4d'
Downloaded Asset(s) in 0.009 seconds
Start Rendering for Machine 3designs-iMac-Pro1.local.
Rendering frame 0 of job 'pv'
2019/01/25 07:32:46  [Corona4D] Parsing scene
2019/01/25 07:32:46  [Corona4D] ====RENDER STARTED====
2019/01/25 07:32:46  [Corona4D] Core build timestamp: Jan 18 2019 18:35:55
2019/01/25 07:32:46  [Corona4D]
Sending HELLO to server
2019/01/25 07:32:46  [Corona4D]
Connection established
2019/01/25 07:32:46  [Corona4D] REPORT: unchanged: 0, Prim: 400, Inst: 0, Mtl: 0, Env-tentative: 1
2019/01/25 07:32:46  [Corona4D] Core::renderFrame before onFrame
2019/01/25 07:32:46  [Corona4D] Parsing scene took 0.166 seconds
2019/01/25 07:32:46  [Corona4D] Calculating displacement
2019/01/25 07:32:46  [Corona4D] Preparing geometry took 0 seconds
2019/01/25 07:32:46  [Corona4D] Preparing geometry
2019/01/25 07:32:46  [Corona4D] Embree: memory usage: 21.2 MB. Total embree build time is 65ms, top level commit took: 59ms. There are 400 embree geometry groups, 0 instances, 400/0/0/0 tri/animTri/hair/custom primitives
2019/01/25 07:32:46  [Corona4D] Core::renderFrame after onFrame
2019/01/25 07:32:46  [Corona4D] CoronaCore::renderFrame before unique materials enum
2019/01/25 07:32:46  [Corona4D] CoronaCore::renderFrame after unique materials enum
2019/01/25 07:32:46  [Corona4D] CoronaCore::renderFrame after unique materials update
2019/01/25 07:32:46  [Corona4D] Preparing geometry took 0.155 seconds
2019/01/25 07:32:46  [Corona4D] Preparing lights
2019/01/25 07:32:46  [Corona4D] TreeAdapt built: nodes 0, time 0 ms, memory 0.088 kB
2019/01/25 07:32:46  [Corona4D] CutCache initialized: records 299499, time 4 ms, memory 11.9802 MB
2019/01/25 07:32:46  [Corona4D] CoronaCore:RenderFrame: after directLight
2019/01/25 07:32:46  [Corona4D] Preparing lights took 0.005 seconds
2019/01/25 07:32:46  [Corona4D] Computing sec. GI
2019/01/25 07:32:46  [Corona4D] CoronaCore::renderFrame: after GI precompute
2019/01/25 07:32:46  [Corona4D] Computing primary GI took 0.052 seconds
2019/01/25 07:32:46  [Corona4D] CoronaCore::renderFrame: before render
2019/01/25 07:32:46  [Corona4D] Rendering initial pass
2019/01/25 07:32:46  [Corona4D] Rendering pass 3
2019/01/25 07:32:47  [Corona4D] Rendering pass 6
2019/01/25 07:32:47  [Corona4D]
Sending chunk 0/1 to the server
2019/01/25 07:32:47  [Corona4D] Rendering initial pass
2019/01/25 07:32:47  [Corona4D] Rendering pass 2
2019/01/25 07:32:48  [Corona4D]
Sending chunk 0/1 to the server
2019/01/25 07:32:48  [Corona4D] Rendering initial pass
2019/01/25 07:32:49  [Corona4D] Rendering pass 2
2019/01/25 07:32:49  [Corona4D] Rendering pass 3
2019/01/25 07:32:50  [Corona4D] Rendering pass 4
2019/01/25 07:32:50  [Corona4D] Rendering pass 2
2019/01/25 07:32:50  [Corona4D]
Sending chunk 0/1 to the server
2019/01/25 07:32:51  [Corona4D] Rendering pass 4
2019/01/25 07:32:51  [Corona4D] Rendering pass 9
2019/01/25 07:32:51  [Corona4D] Rendering pass 11
2019/01/25 07:32:52  [Corona4D] Rendering pass 14
2019/01/25 07:32:52  [Corona4D] Rendering pass 15
2019/01/25 07:32:52  [Corona4D] Rendering pass 16
2019/01/25 07:32:53  [Corona4D]
Sending chunk 0/1 to the server
2019/01/25 07:32:53  [Corona4D] Rendering pass 2
2019/01/25 07:32:53  [Corona4D] Rendering pass 3
2019/01/25 07:32:53  [Corona4D] CoronaCore::renderFrame: after render
2019/01/25 07:32:53  [Corona4D] Terminating DR slaves
2019/01/25 07:32:53  [Corona4D]  - terminating slave handlers
2019/01/25 07:32:53  [Corona4D]  - waiting for broadcast thread to finish
2019/01/25 07:32:53  [Corona4D]  - clearing slave handlers
2019/01/25 07:32:53  [Corona4D] Terminating DR slaves ended
2019/01/25 07:32:53  [Corona4D] Rendering took 7.303 seconds
2019/01/25 07:32:53  [Corona4D] Cleaning up
2019/01/25 07:32:53  [Corona4D] Terminating DR slaves
2019/01/25 07:32:53  [Corona4D]  - terminating slave handlers
2019/01/25 07:32:53  [Corona4D]  - waiting for broadcast thread to finish
2019/01/25 07:32:53  [Corona4D]  - clearing slave handlers
2019/01/25 07:32:53  [Corona4D] Terminating DR slaves ended
2019/01/25 07:32:53  [Corona4D] CutCache memory: 12.1504 MB
2019/01/25 07:32:53  [Corona4D] Unique Primitives: 297432
2019/01/25 07:32:53  [Corona4D] Primitives with instancing: 297432
2019/01/25 07:32:53  [Corona4D] Area lights: 2
2019/01/25 07:32:53  [Corona4D] Geometry groups: 400
2019/01/25 07:32:53  [Corona4D] Materials: 1
2019/01/25 07:32:53  [Corona4D] Instances: 400
2019/01/25 07:32:53  [Corona4D] Portals: 0
2019/01/25 07:32:53  [Corona4D] Area lights: 2
2019/01/25 07:32:53  [Corona4D] Avg samples per pixel: 1.23473
2019/01/25 07:32:53  [Corona4D] Avg rays per sample: 20.4952
2019/01/25 07:32:53  [Corona4D] Rays/s: 3.17091e+06
2019/01/25 07:32:53  [Corona4D] Samples/s: 155816
2019/01/25 07:32:53  [Corona4D] UHDCache records: 805
2019/01/25 07:32:53  [Corona4D] UHDCache records added during viz: 0
2019/01/25 07:32:53  [Corona4D] UHDCache rejected %: 71.8781
2019/01/25 07:32:53  [Corona4D] Saving + Cleaning up took 0.002 seconds
2019/01/25 07:32:53  [Corona4D] CoronaCore::exiting renderFrame
2019/01/25 07:32:53  [Corona4D] Rendered 3/0 passes
2019/01/25 07:32:54  [Corona4D]
Sending chunk 0/1 to the server
2019/01/25 07:32:54  [Corona4D]
Finished chunks: 1/1
Peer-to-Peer Statistics:
    > 3designs-iMac-Pro1.local. Download-Speed 14.12 MiB\s (1x)
Peer-to-Peer Statistics End
Grabbed Render-Job 'pv' from machine '3designs-iMac-Pro1.local.'
Created Asset 'hanni_areal_3dmodel.c4d'
Created Asset 'Corona C4D.c4d'
Created Asset 'EVOQUE.c4d'
Created Asset 'vm_v6_016_corona.c4d'
Created Asset 'VP_Free_MazdaMX5_corona.c4d'
Downloaded Asset(s) in 0.901 seconds
Start Rendering for Machine 3designs-iMac-Pro1.local.
(Error) Render-Job failed: Rendering stopped because of an out-of-memory or unknown error
Peer-to-Peer Statistics:
    > 3designs-iMac-Pro1.local. Download-Speed 42.43 MiB\s (8x)
Peer-to-Peer Statistics End

2019-01-25, 14:30:52
Reply #1

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5457
    • View Profile
Be sure to either start the Corona License Server first before the TR Clients, or best yet, update to the latest daily which resolves the need to do that (see https://forum.corona-renderer.com/index.php?topic=23346). Let us know if either of those resolve the issue. It could be of interest to have the scene that causes problems too, since it doesn't happen in every scene - you can use the private uploader. See https://help.c4d.corona-renderer.com/support/solutions/articles/12000033461-how-to-report-issues- for how to report issues and send in things via the uploader (like TR logs, scene, etc.)

Thanks!
Tom Grimes | chaos-corona.com
Product Manager | contact us

2019-01-28, 11:52:26
Reply #2

friedli3d

  • Active Users
  • **
  • Posts: 33
    • View Profile
Thank you very much for your help!

yes the (Error) Render-Job failed: Rendering stopped because of an out-of-memory or unknown error is not showing anymore, and it seems to work now.

However some xref objects are with some transparency and the team render is actually more slow than render in only one machine , but I think that may be a problem relative to Maxon.

Once again, thank you for your help and next time I will follow better how to report the issue :)

2019-01-28, 13:59:52
Reply #3

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5457
    • View Profile
Hi!

Some quick thoughts - I've only found TR slower than on one machine when the packet size is too small (it may happen with larger resolutions), and raising that by setting the TR settings in the Corona render settings to manual and using a larger packet size corrects that. What is the resolution of the image?

Other than that, it may be the xrefs are on a network drive, not sure how that might impact things if so? It could be a C4D issue, but might be interesting to know more just in case :)
Tom Grimes | chaos-corona.com
Product Manager | contact us

2019-01-31, 11:22:42
Reply #4

friedli3d

  • Active Users
  • **
  • Posts: 33
    • View Profile
You are totally right!

The ouptut size is actually not so big ( 2560x1600 ) but when I changed the size of packet, made all the difference in time difference.

So, with client interval 5s and maximum size packet 2, the render without TR was 00:15:41 and with TR was 00:18:21, so with TR took more time.

When I changed the size of packet to 128mb the render with TR was 00:08:19, so almost half the time, that was awesome!

However I still have a problem, if I put in size packet lets say, 256mb or 512mb, the render won't finish totally.

I attach a file with the issue

This may seem stupid but is there any relation between size packet and output size?

thanks again for your help





2019-01-31, 14:12:22
Reply #5

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5457
    • View Profile
Hi!

Can you give more information about the "won't finish totally" - was it rendering to noise, passes, or time? How many passes did it say it had done, compared to how many it does on the local machine only? Did it finish receiving all data from the clients (ie was it really finished)? Does it just stop normally, or with any sort of error (either directly in C4D, or shown in the logs in the clients)? What does the image look like from just the local machine?

The scene might be useful in this case so that we can test here, since that sounds a bit unusual, can always use the private uploader to send it along, info for that is on the https://help.c4d.corona-renderer.com/support/solutions/articles/12000033461-how-to-report-issues- page (along with other useful info :) )
Tom Grimes | chaos-corona.com
Product Manager | contact us

2019-01-31, 15:18:42
Reply #6

friedli3d

  • Active Users
  • **
  • Posts: 33
    • View Profile
Hi!

Sure, so I have max 30 passes with denoising and im also using LightMix. The render appears as finished, so it completes 30/30 passes without any warning or stop, but the final looks like the image I sent.

I will send another two images, one with and without TR with size packet at 256mb. Both with denoising and 30 passes completed

I will also upload the file throw private uploader.

Thank you once again




2019-01-31, 15:40:24
Reply #7

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5457
    • View Profile
Definitely looks like it hasn't received all the data from the Clients, which is why the top half looks perfect (all data received), and the bottom is just an early pass. With LightMix layers being included, I wonder if it wasn't still receiving data and hadn't finished that step (but the render would be done, so the VFB would look like it was done)?
Tom Grimes | chaos-corona.com
Product Manager | contact us

2019-01-31, 16:58:42
Reply #8

houska

  • Former Corona Team Member
  • Active Users
  • **
  • Posts: 1512
  • Cestmir Houska
    • View Profile
Hi, did you by any chance hit the stop button twice?

2019-02-01, 09:39:44
Reply #9

friedli3d

  • Active Users
  • **
  • Posts: 33
    • View Profile
Hi all,

I assume the render was finished because it appears the green light in picture viewer when the render is finish.

I tested without the LightMix and the render is ok, so I assume is something relative to LightMix.

@houska , no I haven't hit the stop button twice, not even once, I let the render finish :)

I already sent the file throw private uploader

Thank you all


2019-02-01, 10:40:02
Reply #10

Beanzvision

  • Corona Team
  • Active Users
  • ****
  • Posts: 3855
  • Bengamin
    • View Profile
    • Cormats
Hi all,

I assume the render was finished because it appears the green light in picture viewer when the render is finish.

I tested without the LightMix and the render is ok, so I assume is something relative to LightMix.

@houska , no I haven't hit the stop button twice, not even once, I let the render finish :)

I already sent the file throw private uploader

Thank you all

Hi, do you recall what the file is called that you uploaded, was it to dropbox or the regular uploader?
Bengamin Jerrems l chaos-corona.com
3D Support Specialist - Corona l contact us
Corona Uploader l Upload
Portfolio l Click me!

2019-02-01, 11:37:09
Reply #11

friedli3d

  • Active Users
  • **
  • Posts: 33
    • View Profile
hi,
the file is hanni areal 3dmodel.zip

I may send it twice, because took a while to send and I was not sure if it send it. the file is not to heavy, 173.5mb

thank you

2019-02-01, 13:54:45
Reply #12

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5457
    • View Profile
Not sure if the green light simply means "All rendering calculations are finished" - it may not mean "and all data has been sent over the network from the slaves and combined with the original". There could be a lot of data to move with LightMix elements. What happens if you let it sit there for 5 minutes after the green light?

PS also, for the devs, do you know when the light goes green here, on calculations being finished only, or on "all data received and combined and displayed"? :)
Tom Grimes | chaos-corona.com
Product Manager | contact us

2019-02-01, 15:40:44
Reply #13

friedli3d

  • Active Users
  • **
  • Posts: 33
    • View Profile
Well, I don't know why but now it works correctly, the render with and without Team Render.
I just deleted the xref that a I had,and then insert them again, because thoose xrefs were showing some transparency, and now it works fine.

But when the final render with TR was showing the problem, if I waited 5/10m the problem maintains.

Not sure if I understood the other question. I assume that when the light circle in picture viewer and also on TR Client is green all data received and combined and displayed are completed.

Do you already have opportunity to test it with the file I sent?

Thank you!

2019-02-01, 16:01:46
Reply #14

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5457
    • View Profile
I can't test on a Mac, I am Windows only :) So that would be for the developers to have a look into.
Tom Grimes | chaos-corona.com
Product Manager | contact us