Structurising a texture library

[mase]

So I have been collecting various textures for quite some time now and was never sure what is the best system to keep everything organized and quick to use. My current setup is that I have one folder called '_previews' where I have all the texture categories (brick, metal, wood, grunge, etc.) with their sub-categories and I only keep my diffuse textures here. Then I have the same category-tree in a folder called '_maps' where I put all the other maps (like gloss, bump, displacement, etc.) so if I find the diffuse I like, I change '_previews' to '_maps' in the folder path and get into the same folder except instead of diffuse I see other maps of those textures.

It takes so much time to sort every new collection to their actual folders and to separate each diffuse from other maps. On the other hand, when I'm browsing for textures, I don't need to click 4-5 times just to see another diffuse texture so it really helps.

As of right now, my texture library weights about 300gb and I'm already struggling a bit. But I believe professional freelancers or big offices must have much more and was wondering what system do they use to keep it nice and simple. I know Windows explorer might not be the best way to organize a file library but so far I haven't been able to find a better solution to this problem so I'm sticking to whatever works best.

[Juraj]

I am in to hear the answers :- ).

On top of stuff you mention I have issue with keeping 'legacy' (whathever random crap I needed at some point) and 'production' (Tiled, PBR-ready, with sub-maps) together in same folder because only I know which one sucks and which one doesn't.
The rest of my team is utterly lost in which textures they're supposed to use.
I occasionally send 'legacy' into 'legacy' sub-folder to die (and re-link former scenes if I need to use them in future), but I keep adding new crappy ones alongside. I can't stop myself from differentiating between temporary and pristine organized.

One (but partial only) solution is that I try to build basic materials for Connector library instead of relying on textures. If I or someone from my team needs 'White Oak' they will grab ready-to-use material with all sub-textures inside it and they can continue to combine it in some way.


Age old mantra: Waste time organising stuff forever ? Or waste time trying to find stuff each time. It just might be possible that in the end the net result is the same.

[mase]

On top of stuff you mention I have issue with keeping 'legacy' (whathever random crap I needed at some point) and 'production' (Tiled, PBR-ready, with sub-maps) together in same folder

This. I believe your way of putting production textures into materials and then keeping all the other ones aside might be a good option. Although it takes even more time than putting everything into a system of some sort, it might be worth it in the end.

Age old mantra: Waste time organising stuff forever ? Or waste time trying to find stuff each time. It just might be possible that in the end the net result is the same.

I think this depends on a person/office structure. In my opinion, having a clear file structure means having a clear mind. On the other hand, others prefer a more chaotic creative workflow so I guess there's no one answer to this question.

[romullus]

If i would be in need to organize my assets, i would probably go the tagging way. Takes much time to properly tag every asset, but then makes your work with them like a breeze.

[mase]

I also heard much about tagging but isn't it done in some 3rd party software? If so, which one?

[romullus]

If only for tags, i like Adobe Lightroom the most, but it's not suitable to work with CG assets, so i chose Connecter. It's free, it's actively maintained and it gets better with each update.

[Juraj]

I am currently contemplating on tagging my models as 3rd tier of organising. To me it's more of a search feature than division.

In Connector it would look followingly:

1st tier: 'Folder' structure (taken from file explorer): Beds/Chairs/Bookcases/Decorations/etc..
2nd tier: 'Categories' (items from single folder in multiple categories): Kitchen armchairs, Midcentury archchairs, Scandinavian Decoration, Kitchen Decoration,etc..
3rd tier: Tags: Chair (Armchair, wooden, scandinavian, midcentury, old, PBR,...)

Now perhaps this is how I should approach the textures the same way. The good thing about Categories and Tags is that they are virtual feature outside of physical disc hierarchy so they are flexible to crate and change, and don't affect filepaths.

I think that might be it for me. I once contemplated tagging inside Adobe Bridge but I couldn't get used to having both Bridge and File explorer open at same time. But now with 3rd monitor, Connector will be always ON.

[mase]

I never really looked into Connecter, thanks for pointing that out. It really seems to do a lot of work, will give it some time over the weekend.

Juraj, that seems to be the right way of putting things in order. You need proper folder structure but you also need tags. Can Connecter be used for textures as well as models? That would be great.

[Juraj]

Connecter can be used for pretty much anything it really revolutionized our workflow.

I use it currently for Models, IES, HDRi and building material library. But I know Daniel Reutersward and others use it for textures too I think.

The most incredible tool I use from it is auto-generating the assets. I simply unpack .zip file from 3dSky/DesignConnected/etc..., then drag&drop it into 3dsMax viewport and with menu pop-up it will put it into folder structure (generating "/maps" subfolder and putting all textures there, auto-relinking them), into categories you place, description you write for it, generate a thumbnail file from studio scene I prepared. I also set it up to convert it to Corona :- ).

I was part of private dev chat for more than a year I think now and the devs are very motivated and user-request listening. Some of the stuff Swedish boys asked for is almost ridiculous and everything gets implemented next week :- ).

[lupaz]

Thanks! I didn’t know about comnecter either

[mase]

That is great stuff, can't wait for the weekend to really mess with this thing.

On a related note, do you also use this for your material library? I've had great success with CMPP and was wondering if I should transfer it into Connecter.

[romullus]

The most incredible tool I use from it is auto-generating the assets. I simply unpack .zip file from 3dSky/DesignConnected/etc..., then drag&drop it into 3dsMax viewport and with menu pop-up it will put it into folder structure (generating "/maps" subfolder and putting all textures there, auto-relinking them), into categories you place, description you write for it, generate a thumbnail file from studio scene I prepared. I also set it up to convert it to Corona :- ).

Wow, i didn't even know that all of this is possible with Connecter. Definitely worth to look deeper.

[johan belmans]

The most incredible tool I use from it is auto-generating the assets. I simply unpack .zip file from 3dSky/DesignConnected/etc..., then drag&drop it into 3dsMax viewport and with menu pop-up it will put it into folder structure (generating "/maps" subfolder and putting all textures there, auto-relinking them), into categories you place, description you write for it, generate a thumbnail file from studio scene I prepared. I also set it up to convert it to Corona :- ).

Wow, i didn't even know that all of this is possible with Connecter. Definitely worth to look deeper.

Yep,we will dive deeper in connector too. It changed our workflow tremendously.
Hopefully they will develop integration for Cinema 4D. As this is will be the DCC package at the moment for us to replace MAX in the future.

[Nejc Kilar]

Yep,we will dive deeper in connector too. It changed our workflow tremendously.
Hopefully they will develop integration for Cinema 4D. As this is will be the DCC package at the moment for us to replace MAX in the future.

Cinema 4D has a fairly good content browser that allows for saving of assets - it saves them all in a lib4d file which can get super huge but its very useful. I hope you've explored that option. It is however obviously not as advanced as the connector :)

[johan belmans]

Yep,we will dive deeper in connector too. It changed our workflow tremendously.
Hopefully they will develop integration for Cinema 4D. As this is will be the DCC package at the moment for us to replace MAX in the future.

Cinema 4D has a fairly good content browser that allows for saving of assets - it saves them all in a lib4d file which can get super huge but its very useful. I hope you've explored that option. It is however obviously not as advanced as the connector :)

Hi Niklar, thanks for the tip!

[JohnNinos]

Connecter can be used for pretty much anything it really revolutionized our workflow.

I use it currently for Models, IES, HDRi and building material library. But I know Daniel Reutersward and others use it for textures too I think.

The most incredible tool I use from it is auto-generating the assets. I simply unpack .zip file from 3dSky/DesignConnected/etc..., then drag&drop it into 3dsMax viewport and with menu pop-up it will put it into folder structure (generating "/maps" subfolder and putting all textures there, auto-relinking them), into categories you place, description you write for it, generate a thumbnail file from studio scene I prepared. I also set it up to convert it to Corona :- ).

I was part of private dev chat for more than a year I think now and the devs are very motivated and user-request listening. Some of the stuff Swedish boys asked for is almost ridiculous and everything gets implemented next week :- ).

Hello fellas, i hope you are running on a great week!

I am trying to be on the new organization era because my library is extremely messy and i think that i am gonna give Connecter a try, is there any video or thread that describes a little more in depth the workflow and some tips & tricks regarding that? (Your trick with the tags is really smart, are the tags remaining thought if you reinstall Connecter?) Additionally, i have heard that having the folders directly in the HDD ( D:/Decorative sets/Candle Holders ) is a lot faster than having fro example D:/Library/3dsky/Decorative sets/Candle holders does this worth having your HDD with a tone of folders? And finally, would an SSD dedicated for the library would make things better? Really thanks guys, Cheers :)

[Benny]

I use it currently for Models, IES, HDRi and building material library. But I know Daniel Reutersward and others use it for textures too I think.

Could you develop a little on how you are using it for materials? Afaik 3D Connected can't display a .mat file. Are you creating a max file for every material? That would seam very tedious but perhaps there is a smart workflow for this?

[Benny]

No one is using 3D Connecter for materials?

[Benny]

I guess that makes sense as it can’t open a .mat file, and the only way I’ve found myself would be to save a max scene with a material on a suitable object. Problem is that even if one can use Connecter to categorize them, there is no way to conveniently load them or apply them in a scene so pretty useless really.

Oh well, the search goes on.

[romullus]

You definitely can load materials from Connecter to a max scene (assign them to the objects or place in material library). Also you can batch render previews in your own studio setup. I don't think it's useless at all.

[romullus]

By the way, with the latest update, you can manage external files for materials - view, rename, relink bitmaps. Couldn't be better.

[Benny]

Sounds like a well kept secret, I haven’t been able to figure out half of that stuff. How do you assign a material from the library to an object in your scene?

And perhaps equally crucial, can I grab a material in my scene and add it to the library?

[romullus]

Actually, i only start discovering Connecter for materials myself. So far i like what i see, so much that i started to create my own material library for the very first time :]

Materials goes from Connecter to a max scene, in the same way as the models do, you just drag and drop it to a viewport and from contextual menu, choose an action - assign to selection, add to mat editor, generate previews, etc. As for adding stuff to connecter, i do it through quad menus, don't know if there is other way. Just right click in a viewport, then add (or update) asset to Connecter and choose what you want to add, model or material.

[Benny]

Well, I've played around with it and starting to like it more and more. It is not very well documented though, is it? I can for example not figure out how to et up my own render studio in order to generate preview thumbnails of materials. Did you get that to work?

It also seems as if I can generate a rendered thumbnail while grabbing from a scene, but not for materials already in the connecter library?

[romullus]

Documentation isn't that bad, although there is room for improvement for sure. Also they're releasing short how-to videos from time to time, so it's not that hard to figure things out :]

Making your own materials' render studio is easy. In 3ds max create sphere of 1m size for reference, place it at scene origin, create and adjust camera, lights, backdrop. Make adjustments to render settings and tonemapping (dubcat's guide to material calibration can be very good guide here) and add render limit. Finally delete the sphere (shader ball will be loaded from separate file) and save the scene. Now drag any material from Connecter to 3ds max viewport and from contextual menu choose generate preview. In process files window, go to options, choose render studio option for asset preview, then configure that studio by clicking on arrow to the right. In the next window go to render studio -> settings and add new studio - navigate to the file you saved before. Go back to the previous window and choose what cameras and shader balls to use. Also configure the way you want to preview images to be saved. Once you've done, you can close all configuration windows and press process button - the Connecter will do the rest. Don't forget that you can add multiple assets to generate preview window, before processing them. The whole process looks a bit complicated at first, but once you get hang on it, you will realise, that it is very flexible and powerful tool. You can even use your own shader ball.

[Benny]

Yes, obviously really easy and straightforward. ;)

Kidding aside, big thanks for the help Romullus! I will mess with it and see if I can figure it out.

[romullus]

I'm sure you will love it. Before Connecter, i wouldn't imagine that assets management actually can be a fun thing :]

[Benny]

You were right, I do love it.

I'm having a bit of problems with the test render studio and the general scale of it. How did you know the shader ball should be around 1m? I'm using a scene I've been using before, but the shader ball becomes enormous when generating previews. If I just load it normally and render everything looks fine. Is Connecter doing something to the camera (since I specify a camera)?

The other thing that bugs me is that once I've generated a preview or assigned an existing image to a file, I would like to hide those but it doesn't seem as if I can hide images per folder? I can do it manually, but quickly becomes tedious. What is your workflow?

[romullus]

Shaderball not necessary needs to be 1m size, it's just i find that size most convenient and i think Connecter's default sphere is 1m size. Nevertheless, you can use whatever size you want, also you can have many different sized shaderballs, you just have to have multiple cameras for different spheres. If shaderball appears bigger or smaller than it's intendent to be, then check if you forgot to lock its transforms, or maybe it's a camera that has to be locked, can't remember exactly - better lock them both :]

Regarding preview hiding, after generating previews, Connecter should hide them by default, if not, search for such option somwhere in preview generating window. I don't know if there's an option to automatically hide already existing previews, but you can use filters to hide by file types.

[Benny]

I can not get that to work, locking transforms? You mean doing reset Xform? If I load a file with a shader ball in a scene and just hit render, it would seem to me that the result should be the same if I do it from within Connecter? Unless Connecter do something.

The connecter studio render also ignores the pivot point of my shader ball. Even if I align the pivot to the bottom of my shader ball object, the shader object still becomes centered around 0,0,0.

Of course none of these things are neither documented or in a video.  :/

[Benny]

Sorry, had a brain fart, didn't read properly. Locking transforms and scale takes care of the problem.

Still, in my view an unusual approach that would save people a lot of time if they gave some hints in the documentation.

[romullus]

Don't tell me, tell it to Connecter guys :] I agree with you that some parts of Connecter seems unnecessary overcomplicated, especially preview generating with its countless windows. I wasn't able to figure the case of shifting shaderball by myself, untill i was told about locking trick in their trello board. I guess it's not an easy task to achieve good balance between functionality ant simplicity of UI, but as we can see from Corona example, it isn't impossible task :]

[Benny]

Well what do you know, a new update this morning auto hides jpgs used for previews.  :D

[Marcus]

Quick question for dummies:

What is the best way to add several materials at once with its corresponding previews?

[Benny]

Do you mean from a scene or an existing library?

[romullus]

Well what do you know, a new update this morning auto hides jpgs used for previews.  :D

What a timing! Didn't you forget to buy lottery ticket today? :]
Those guys simply rocks! Lots of new cool features in this update.

Quick question for dummies:

What is the best way to add several materials at once with its corresponding previews?

From 3ds max to Connecter: same procedure as with single material - right click in viewport, add assets to connecter and choose the options that are best suits to your situation.

From Connecter to 3ds max: select multiple materials and drag them to 3ds max viewport.

[Benny]

Romullus, for your render studio, do you have different setups for different scales?

I must say that given one has to lock transformations and other non-obvious subtleties it would have been great if a render studio was built in instead of everyone having to invent their own wheel.

[Marcus]

Do you mean from a scene or an existing library?

For a scene.  I have the problem that if I add multiple materials to Connecter it always creates one Mat-File with several preview renderings.

Would be cool if it would be possible to automate this more. I had a chat with on of the Connecter Guys a few days ago and mentioned this "issue". They plan to do this in one of the next updates. So just a little bit of waiting. :)

[Benny]

If you talk to these guys ask for a built in material studio as well!  :)

[Marcus]

If you talk to these guys ask for a built in material studio as well!  :)

You mean something like the Material-Studio in Cmpp?

[romullus]

Romullus, for your render studio, do you have different setups for different scales?

No, i haven't, majority of my materials fits well enough onto 1m sized sphere. But i think it wouldn't be hard to setup studio for materials of different scales - just add few more cameras to a studio file and few more shaderballs to according file.

[Benny]

If you talk to these guys ask for a built in material studio as well!  :)

You mean something like the Material-Studio in Cmpp?

I haven't looked at those for a while, had the Vray version and there was no cross-upgrade, but that sounds like it. Is Siger still using it's own plugins with their materials?

[lupaz]

Hi all.
Anyone knows a tutorial that shows the full workflow of connecter?