Reference-DLSS from NVIDA

As I thought, there's not many examples for AI working with graphics technologies. But there's a big name, NVIDA. NVIDA published 20 series graphics card. One of their new technologies is DLSS, deep learning super sampling. Basically, it is for correcting the missing frames from raytracing, because the AI system has been trained to fill up the missing part in the images from a pointed game.


More presentation:
https://www.youtube.com/watch?v=KJRZTkttgLw#action=share
https://www.nvidia.cn/geforce/gaming-laptops/20-series/


I mean, I can't really know how they realize this with every line of codes, but it provides me with the basic logic of what AI can do with graphics. After another around of research, I have a chart.


Like I planned in the last blog, for now, I will get into some simple AI system, try to find out its sampling process and learning target.


Except the hardware, another example Prof Paul showed us in class is 
https://www.nvidia.com/research/inpainting/  NVIDA gaugan. I put it there as recorded reference, I'll be working on this later.




Comments

Post a Comment

Popular posts from this blog

Thesis Progress - Noise Map

Image
 In another around of research, I found a toolkit and try to use it for my project.  Resource:  https://github.com/mtwoodard/TextureGenerator It's a 3D texture generator designed for raymarching clouds. As we know, the process of my project includes the steps of asset shape then blurred by noise map. Both of them are based on 3D texture. The initial idea was to apply perlin and worely noise, cuz that's the method introduced by many paper. But now, I got this tool which makes it easier.  From last blog, I already had a framework that can drive the 3D texture for some extent. And the noise map makes it pretty close to the final result. Here's what it looks like and it makes me HAPPY. There's an example in the toolkit , I applied it with my 3D texture script. It's not that clear now, then I juggled with some parameters on Blur Material.  Transparency: 0.4 OpacityCorrection: 2.4 Then the outcome shows some really nice work. And each of those plane is attached to tex...
Read more

Thesis Progress - Schedule Check

Image
According to the timetable, the goal of this quarter is to finish the first two parts which are the function experiment for thesis project. So far,  the first method I came up with is going through 40% around, still need some more learning on coding to realize modification. At the same time, I'm still trying to find the other ways that might be better or easier, which could replace the current method. The way I make it should be covered by my capability as much as possible.  Other than that, I'm also considering whether I should have a comparsion between my method and traditional way of making 3D texture. It's mostly from the thesis perspective that I need to show the advantage or benefit of my own method. That requires more specific arrangement on timeline. 
Read more

Thesis Progress - Procedural 3D texture

Image
 According to tutorials, there's some outcome. The function realized plane creating, which would be saved later for my further using.  The problem so far is that the sphere generating function is way too complicate to modify myself, and it closely related to the planes, so it might cost more time on learning.  There's another easier example, what I can understand is the switch from object to worldspace. Once the cube is out of range, the texture could not sample on the object accurately. But not sure if I would use this later, but good to know what's going on in codes.  Resources archive:  https://www.youtube.com/watch?v=FUpsRzricPA&t=3s            The second example is from unity documentation that introduces the usage of 3D texture, this part could be as the framework of raymarching that allows to modify the shape possibily. This part will be put behind currently.  Resources archive:  https://docs.unity3d.com/Manual/...
Read more