Author Topic: Tutorial: Stereoscopic panorama with Corona + Cardboard  (Read 136930 times)

2015-12-17, 13:21:56

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12712
  • Marcin
    • View Profile
TO AVOID ANY CONFUSION: This thread was started on 17.12.2015. Since then, native support for VR (stereo spherical camera) was added to Corona, and now probably different approach is required. However, maybe there is still some helpful info in this thread.

UPDATE: see the new "better" guide here: https://forum.corona-renderer.com/index.php/topic,10592.msg67811.html#msg67811



NOTE: Corona alread... soon will have built-in sterescopic pano controls. I am posting this guide mainly as a curiosity. Maybe it will serve as a base for further experimenting for users with better knowledge of the topic.
NOTE #2: example files are below.

So, here we go:

What we will need:
-3ds Max - it was chosen as the host application, we will be using Max Script so it won't work with any other host, unless you can create a similar script/tool
-Corona - any version with spherical camera support should work
-krpano - make sure you download the 1.19 version!
-VR-ready devices (I used my good old Moto G + basic version of Google Cardboard which cost about 2€), you can also view your pano on the monitor, but it's not so much fun ;)
-software to view the vr pano - I am using Firefox for Android
-some file browser for your Android (I am using Amaze, but anything which can go to a location and save it's path to clipboard should be ok)


1.  Configure cameras - basically it is a head+eyes setup:
My setup was based on Rambambulli's one. Basically I have 3 cameras. Camera-center is used for viewport/IR preview only.
cameraL is moved 3,5cm to the left from Camera-center. - it is ESSENTIAL to use the name cameraL for the object unless you edit the script
cameraR is moved 3,5cm to the right from Camera-center.  -it is ESSENTIAL to use the name cameraR for the object unless you edit the script
All cameras' targets are placed in one point, which is centered (so that it aligns with Camera-center), and moved away from the cameras (about 1-2m should be fine, I did not experiment with this, basically this is the exact point you are looking at).
The cameras and their targets are linked to a circle (rotation-helper). The circle is animated, it does a full 360 degree rotation through the whole animation range.
The cameras are aligned to the very center of the circle helper, and then are moved a little (7 cm) towards front. This is because your eyes are away from the spine - in this case the pivot of the circle helper.
All of the cameras have Corona Camera modifier applied with spherical projection type enabled (you can disable it for the center camera for previews if you wish).
The whole rig should be at some realistic height, so I guess 160cm+ - remember your eyes are a little lower than your total height. 


2. Animate the helper:
It needs to do a full 360-degree spin which takes the same amount of frames as the whole animation. Remember that 0-100 frames is actually 101 frames in total.


3. Model the environment:
You can place whatever you want around the cameras. there will be probably some problems if something is extremely close to the camera. I'm not sure.


4. Render setup and animation length:
This is very important! The way the script is constructed requires the following:
-The total number of frames should be the same as width of the render output. So if you are rendering 2400x1200px image - use 2400 frames of animation. If you are rendering a 600x300px image - 600 frames
-The circle helper must do the 360-degree spin in that number of frames

Render setup:
-set pass limit
-disable "lock sampling pattern" in performance tab
-you can set VFB to none
-remember about the correct 2:1 ratio in output size (2400x1200, 600x300 etc)


5. Run the script:
Note: the script was made by Rambambulli. All the credit goes to him.

Make sure you have the correct viewport selected.

Go to MAXScript > MAXScript Editor...
Paste the script:

Code: [Select]

--set render mode to crop
setRenderType #crop

-- change so the output width of a image divided by the stripwidth makes a whole number
stripwidth = 1
 
-- the left eye image
PanLeft = bitmap renderwidth renderheight
-- the right eye image
PanRight = bitmap renderwidth renderheight
 
--set a region to render a crop
EditRenderRegion.EditRegion()
EditRenderRegion.UpdateRegion()
viewport.setRegionRect viewport.activeViewport (Box2 (renderwidth/2) 0 1 renderheight)
EditRenderRegion.UpdateRegion()
EditRenderRegion.EditRegion()
 
parts = renderwidth/stripwidth
-- for testing parts = 100  or whatever you like

-- for the left eye

for t = 0 to parts do (
display panLeft
--make sure the left eye camera is cold cameraL. Or change the name here
viewport.setCamera $cameraL
strip = render rendertype:#crop vfb:false frame:t 
    pastebitmap strip panLeft [0,0] [(t*stripwidth),0]
)
 
-- and for the right eye
 
for t = 0 to parts do (
display panRight
--make sure the right eye camera is cold cameraR. Or change the name here
viewport.setCamera $cameraR
strip = render rendertype:#crop vfb:false frame:t 
pastebitmap strip panRight [0,0] [(t*stripwidth),0]


Then left-click in the pink text field in the lower-left side of the 3ds Max ui. Once the cursor appears in it, press Esc key - you should then see **interrupted** text in the lower, white text field. If something goes wrong, you should do this again before running the script again.

Get back to the script editor window, click Tools > Evaluate All.

If everything goes fine, you should see standard 3ds Max VFB pop up and a vertical line going from left to right with the rendered content appearing. You will need to wait for the two panoramas to render out this way.


6. Save the outputs:
After the rendering finishes, save the two images as pano_l.jpg and pano_r.jpg (IT IS REQUIRED TO SAVE THE FILES UNDER THE NAMES SHOWN HERE! the first one which renders out is the left one, the second one is right).


7. Create the stereoscopic VR tour:
Open your krpano installation folder. Select the two images (pano_l.jpg and pano_r.jpg) in explorer, grab them, and drag-and-drop them onto "MAKE VTOUR (MULTIRES) droplet.bat". You should see a command window that will do its job and then tell you to press any key.
Press any key.

This should convert your images into tiles, and save them in a folder structure in the place where they were originally stored. Go to that location.

In that location, go to "vtour" folder and open the file tour.xml in notepad (or any other text editor). We will need to do the following:

-remove the whole second <scene> section:  http://s29.postimg.org/fbjo3u1qv/remove.jpg
-add stereo="true" stereolabels="l|r" to the <image> tag: http://s17.postimg.org/5jk61wm4v/image_add.jpg
-in the <cube url> change 'pano_l.tiles' to 'pano_%t.tiles'
-save the file (just Ctrl+S it - resave under the same name)

After that is done, you can run the tour.html in your web browser on your PC to check how it works. You should see the panorama running, with a bottom menu where you can pick the Google Cardboard icon.


8. Send it to your phone:
I just connected my phone via USB and uploaded the whole folder with the original images and the pano directory to internal storage. For some reason it did not want to work when I uploaded it to the SD card. Maybe it's a useful hint, maybe I made some mistake.

Then, on my phone I used Amaze file browser (but I guess you can use anything), tracked the folder with the tour.html file, copied the path to clipboard (in Amaze you long-press on the path).

Switch to Firefox. Click on address bar. Paste the path, then add "tour.html" at the very end of it. Go!

Then just put your phone inside the Carboard/other viewer and enjoy the random teapots around you.

If you have done everything right, it should just work. Let me know if something goes wrong, there is high chance I made a mistake while typing all of this. :)

---

Known issues:
-Once you run the script, it cannot be paused/canceled (at least I do not know a way) so you will probably need to kill 3dsmax process. Incremental saves are advised. ;)
-The demo version of krpano leaves ugly watermarks all over the panorama - this can be probably fixed by using other software or buying the full version

Thanks for everyone who helped me do this, especially the guys over krpano forum who had to read my posts where I bitched how hard it is to find basic info, while it was my mistakes. ;)

« Last Edit: 2016-01-13, 17:41:21 by maru »
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2015-12-20, 12:38:18
Reply #1

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12712
  • Marcin
    • View Profile
I'm surprised there is no interest in this. :(
I have just found a faster and less painful way with no watermarks. And with daily builds' VR support you do not have to use any scripts. I will post everything soon.
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2015-12-20, 13:08:29
Reply #2

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be
Don't worry, I am interested. But at the moment trying to catch a deadline, .....rendering an animation.
 

2015-12-20, 13:51:26
Reply #3

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
I'd really love to see panos in 3D, but i haven't device in which i could do so.
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2015-12-20, 15:02:16
Reply #4

amitshneor

  • Active Users
  • **
  • Posts: 53
    • View Profile
of course im interested in a clean easy 360 stereo pano solution for corona !!
so far i could only do it in vray or octane with their rendertime solution....

i will test your way asap !  ....and share my outcome...

thanks a lot man!

2015-12-20, 20:46:06
Reply #5

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12712
  • Marcin
    • View Profile
I'd really love to see panos in 3D, but i haven't device in which i could do so.
It depends whether you want to get some really nice experience, or just see how it works. If you just want to see it, then I would recommend getting a cheap Cardboard viewer (AFAIR I bought mine for about 2€ just for fun) and a budget Android phone. I am using a moto G which I bought at the beginning of 2015 for about 150€ and I am still hapy about using it - no freezing or lags, checked with some 3d games and they run pretty smooth too. It could be a good excuse for getting yourself a new phone. Scientific purposes. ;)
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2015-12-20, 21:52:21
Reply #6

amitshneor

  • Active Users
  • **
  • Posts: 53
    • View Profile
well,here is a good place to start : https://labs.chaosgroup.com/index.php/portfolio/guide-to-virtual-reality-2/

its from chaos group but the pdf guide is very informative. and the images they rendered in vray cube vr are amazing in 360 stereo vr.(18k wide !!!)

a client of mine gave me a samsung gear vr and a samsung galaxy s6 edge to go with it, so im telling you the experience is superior over any cardboard or some other
cheap plastics. the FOV is very large and you are immersed in the 360 world.

this is what pushed me to learn more about vr and making and rendering content for it.
i will try maru's script and workflow, hoping to get something from the dev team as an addon to corona.

cheers mates !

2015-12-20, 22:22:43
Reply #7

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
It depends whether you want to get some really nice experience, or just see how it works. If you just want to see it, then I would recommend getting a cheap Cardboard viewer (AFAIR I bought mine for about 2€ just for fun) and a budget Android phone. I am using a moto G which I bought at the beginning of 2015 for about 150€ and I am still hapy about using it - no freezing or lags, checked with some 3d games and they run pretty smooth too. It could be a good excuse for getting yourself a new phone. Scientific purposes. ;)

To see how it works would be enough. Thing is, i already have android device, but it hasn't gyro, so it's not very useful for VR experience. Or am i wrong?
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2015-12-21, 08:20:45
Reply #8

88qba88

  • Active Users
  • **
  • Posts: 80
    • View Profile
I'm surprised there is no interest in this. :(
I have just found a faster and less painful way with no watermarks. And with daily builds' VR support you do not have to use any scripts. I will post everything soon.

I;m waiting for your tutorial Maru!

2015-12-21, 17:06:09
Reply #9

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12712
  • Marcin
    • View Profile
New guide!
Android + any VR viewer such as Cardboard again. This time it's much more simple and has no watermarks.


You will need:

Either: the script mentioned in the 1st post - this will give you 2 images which you can just use
Or: new version of Corona with the built-in VR support - this will give you one image, which you will need to split into two
This app on your phone: PSViewer

Requirements:
-Each of the images needs to be wider than 2000px
-Width needs to be 2x height (2200x1100 etc)
-left eye image needs to end with _l.jpg
-right - surprise...  _r.jpg

Howto:
1. Render the image(s)
2. Save as xxx_l.jpg and xxx_r.jpg - each one's width has to be above 2000px!
3. Copy the files to your phone, into one folder
4. Start PSVIewer, pick "load photospheres", find the directory where you copied the files, pick one of the images (for some reason it displays only the _l.jpg image for me - it is fine), pick "add photospheres to gallery"
5. Pick "view gallery"
6. It should just work. To switch to next panorama, you need to tilt your head (either left or right, depends on the sensors in your phone)

The End


Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2015-12-22, 10:43:37
Reply #10

88qba88

  • Active Users
  • **
  • Posts: 80
    • View Profile
Thanks a lot Maru! I've just tried it and it works like a charm!

2015-12-22, 10:44:39
Reply #11

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12712
  • Marcin
    • View Profile
Great! Did you use the script or Corona's DB?
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2015-12-22, 12:05:52
Reply #12

88qba88

  • Active Users
  • **
  • Posts: 80
    • View Profile
I used Corona Cam modifier with VR on, resolution 4000x4000, then I separated files in external program. Using PSViewer with correct phone (The program doesn't care about screen size, so when it's too big you have to have big exotropia to see it correctly!) it looks fantastic. Now it's time to test some interiors.

2015-12-22, 15:03:31
Reply #13

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12712
  • Marcin
    • View Profile
soap bubbles and godrays, somehow the 3d effects are not that apparent on my device
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2015-12-22, 15:21:29
Reply #14

88qba88

  • Active Users
  • **
  • Posts: 80
    • View Profile
I've done this one just for another quick test, 3d effects are clear and strong. I use
Eye separation =6,3cm
Eye front offset =8cm
Converge eyes = unticked

Another problem in PSViewer is that on some phones it won't keep the horizon in horizontal position, so you can't tilt your head too much.