Gaussian Material Synthesis

In the search of higher level AI utilization in computer graphic field, GMS(Gaussian Material Synthesis)  teaches AI the concept of various materials and the newly synthesized materials can be visualized in real-time via neural rendering. The website below provides the video, thesis and trained AI data, which are all with permissive license to use. I believe this will be a good topic for the couse in this term.

For overview, this tech is applied as an addon for Blender and the workflow shows like:
1. The user is presented with a gallery and asked to assign scores to the shown materials.
2. After choosing a threshold value to control the variability of the output, a set of recommendations
are computed and visualized via neural rendering.

I'm going to make it work on my computer first, but I'm not sure if this could be the full content for this term. Or should I build up something new upon it? For the longer purpose, I wonder if it's doable in unity as well.

https://users.cg.tuwien.ac.at/zsolnai/gfx/gaussian-material-synthesis/


Update on 4/12/2020
I started to install all the sources that are needed for the project, while I met many problems on pareparation even it's at the very beginning. According to the instruction, Python envrionment and a few extra package are required, but not all these package are friendly compatible to windows system. I searched some resolutions and they are based on very skillful python coding ability, I have to back to the start point and learn the basic using approach first. The issues remained are:

 1. Package climi and GPy installation for python environment 
https://github.com/SheffieldML/GPy/issues/649 (resolution for Gpy )
???? (climin)

 2. https://blender.stackexchange.com/a/51800 (link the anaconda environment to Blender)

 3. issues may happen 
Error when importing numpy, see here: https://stackoverflow.com/a/56009839
Error when importing keras/GPy, see here: https://stackoverflow.com/a/19105436
  

Comments

Post a Comment

Popular posts from this blog

Thesis Progress - Noise Map

Image
 In another around of research, I found a toolkit and try to use it for my project.  Resource:  https://github.com/mtwoodard/TextureGenerator It's a 3D texture generator designed for raymarching clouds. As we know, the process of my project includes the steps of asset shape then blurred by noise map. Both of them are based on 3D texture. The initial idea was to apply perlin and worely noise, cuz that's the method introduced by many paper. But now, I got this tool which makes it easier.  From last blog, I already had a framework that can drive the 3D texture for some extent. And the noise map makes it pretty close to the final result. Here's what it looks like and it makes me HAPPY. There's an example in the toolkit , I applied it with my 3D texture script. It's not that clear now, then I juggled with some parameters on Blur Material.  Transparency: 0.4 OpacityCorrection: 2.4 Then the outcome shows some really nice work. And each of those plane is attached to tex...
Read more

Thesis Progress - Schedule Check

Image
According to the timetable, the goal of this quarter is to finish the first two parts which are the function experiment for thesis project. So far,  the first method I came up with is going through 40% around, still need some more learning on coding to realize modification. At the same time, I'm still trying to find the other ways that might be better or easier, which could replace the current method. The way I make it should be covered by my capability as much as possible.  Other than that, I'm also considering whether I should have a comparsion between my method and traditional way of making 3D texture. It's mostly from the thesis perspective that I need to show the advantage or benefit of my own method. That requires more specific arrangement on timeline. 
Read more

Thesis Progress - Procedural 3D texture

Image
 According to tutorials, there's some outcome. The function realized plane creating, which would be saved later for my further using.  The problem so far is that the sphere generating function is way too complicate to modify myself, and it closely related to the planes, so it might cost more time on learning.  There's another easier example, what I can understand is the switch from object to worldspace. Once the cube is out of range, the texture could not sample on the object accurately. But not sure if I would use this later, but good to know what's going on in codes.  Resources archive:  https://www.youtube.com/watch?v=FUpsRzricPA&t=3s            The second example is from unity documentation that introduces the usage of 3D texture, this part could be as the framework of raymarching that allows to modify the shape possibily. This part will be put behind currently.  Resources archive:  https://docs.unity3d.com/Manual/...
Read more