DIGM540 Record

Outline: Address Problem: The comparison of efficiency between traditional pipeline workflow and compute shader generated asset. efficiency could be time saving for production process, rendering optimization, storage saving, etc. In the project I did in 540, it mostly shows on Who's facing it:   •  Traditional Artist (Aesthetics and Modelling) •  Technical Artist (Visualization Improvement) •   Game developer (Using massive assets with light work) How could they have better coordination and faster workflow? Who would save whose work of what? Solution/ Related work:  1. List some existing approaches that have already been implemented for solving this kind of problems 2. What tools they used? How does they work? How's the result?  3. Based on those solutions so far, what's left? what issues still remain. Select one of the issues or tools and have in-depth research on it, including history, development, current situation...

Project Progress: In the search of overcoming the incompability of RaymarchingToolkit among different renderers, I eventually found one that could be used on latest RP. This new Toolkit does have the melting effect as the previous one, what it's able to do is to separate a single cube into mutiple hexgons, the amount of which only depends on the scale of cube. I thinks it's pretty fit for the candle body. Also, I'm also trying to give my scene a table cloth as physical-realistic as possible. Fortunately, I found something really interesting. https://github.com/Ninjajie/Fusion#cloth-plugin-user-guide The plugin realized physical object simulation in unity based on Physical-Based-Dynamic, which is not perfectly accurate as physical stuff but fast enough for real-time rendering. Furthermore, authors had a comparsion of the same althorigm between CPU and GPU. They used compute shaders with HLSL for caculations on GPU. I tried to do the simulation and the differenc...

Paper1: Renato Farias,Marcelo Kallmann.  Improved Shortest Path Maps with GPU Shaders The paper designs two approaches with three types of models to realize rendering for points cloud. What inspired me is the potentials of application to WebGPU which currently counldn't have a fairly good performance as shader does in other fields. I don't really know the workflow of Web rendering for the time being, and also confused on what kind of restricition should I specifically jump into. Maybe a good start is search the terms like 'limitation of XXX', 'limitation of shaders',etc. Paper2:   KERRY A. SEITZ, JR,TIM FOLEY,SERBAN D. PORUMBESCU,JOHN D. OWENS.  Staged Metaprogramming for Shader System Development This is even a more advanced paper for me cuz it mentioned about metaprogramming and its super long with pages of codes. However, except metaprogramming, I feel it's readable for the rest of content,  the main purpose is to improve the shader...

Compute Shader Intro: https://docs.unity3d.com/Manual/class-ComputeShader.html http://kylehalladay.com/blog/tutorial/2014/06/27/Compute-Shaders-Are-Nifty.html https://www.youtube.com/watch?v=9RHGLZLUuwc&t=193s Trying to realize the basic function and then apply it into other scene Paper1: Francisco Sans and Rhadam ́es Carmona [2017] A Comparison between GPU-based Volume RayCasting Implementations:  Fragment Shader, ComputeShader, OpenCL, and CUDA In the beginning part of writing a cirtical paper, potential issues need to be listed first. Also, It is very clear that the author gives an analysis of why GPU-based raycasting intersection worths to be considered as a problem. When he's analyzing, he firstly introduces the working method for this technology and then describes the limitation for it, which gives readers a basic frame of what is this paper about and would it be useful for different individuals. Periodical conclusion is important, becau...

I tried many ways to fix the issues, but none of them worked, not even on github. One of the packages 'climin' has very rare information online and is barely used. Also, the compatibility between python version and blender is very uncertain, I'm not sure which one would work, so I gave up this AI stuff and tried to look into something new. https://www.youtube.com/watch?v=JVlfSjkvefE There is a image to material tool, materialize, which is able to edit a picture into material with diffuse,normal,metallic maps and etc. After you created the project, it's also easy to import to unity. Maybe I could build up something upon that.

I went to the second try for linking to blender, and here are the resolutions of some practical issues. 1. download package from github with anaconda I figured out how to use github link with python language pip last time, but it turned out that it still needs to use anaconda words to make sure every package can be managed with it. They are actually different environment setting, probably because they are different paths, but only connected.  To install this package with conda run: conda install -c anaconda git clone https://github.com/BRML/climin.git 2. Virtual Environment re-setup According to last week's attempt, package keras-gpu is not compatiable with python3.8, so I built another 3.7 version and it worked well this time. 3. Link to blender Author had mentioned two potiential issues when linking to blender, not surprisingly, it happened for sure on me. And here's the resolution from author: Even though it's very specific explanatio...

According to the record last week, I'tried to finish the tasks below by order. Since I havent used python before, those works were more difficult than they seem to be. 1. Package climi and GPy installation for python environment  https://github.com/SheffieldML/GPy/issues/649 (resolution for Gpy ) ???? (climin) 2. https://blender.stackexchange.com/a/51800 (link the anaconda environment to Blender) 3. Issues may happen  Error when importing numpy, see here: https://stackoverflow.com/a/56009839 Error when importing keras/GPy, see here: https://stackoverflow.com/a/19105436 • Climin installation This package cannot be found in anaconda package manager, it is currently only available via git from github. I took the research on like ' how to install packages on python', ' how to download packages from github'. All those operations are based on some very simple codes, it still took me a while though. The right approach is:  (1) install gitbash and set it...

In the search of higher level AI utilization in computer graphic field, GMS(Gaussian Material Synthesis)  teaches AI the concept of various materials and the newly synthesized materials can be visualized in real-time via neural rendering. The website below provides the video, thesis and trained AI data, which are all with permissive license to use. I believe this will be a good topic for the couse in this term. For overview, this tech is applied as an addon for Blender and the workflow shows like: 1. The user is presented with a gallery and asked to assign scores to the shown materials. 2. After choosing a threshold value to control the variability of the output, a set of recommendations are computed and visualized via neural rendering. I'm going to make it work on my computer first, but I'm not sure if this could be the full content for this term. Or should I build up something new upon it? For the longer purpose, I wonder if it's doable in unity as well. https://...

Section1 After a few examples of practice, Im trying to put them togethoer and see if multipuly shaders could work together. The first attempt is that disappear effect and the forcefield. What I want to realize is to make the disappear effect totally cover the forcefild ball inchluding the texture when it's concrete. But just like that previous example, the newly added shader can't affact on the original one on the object. So it's the same old problem, is it doable to let the added shader have an impact on other shaders or textures on the object ?  I probably can't fix this for now,  but I still recorded my attempts.    The experience told me it's not gonna work before I mixed them together, it looks not that bad though. There's one cheating way I came up with is to open postprocessing to make it blurry and super shining, so it basically realized my target as long as I can lie to myself. I don't know how to modify passes yet, but I ...

This time I created a very simple disappearing effect on a cube first. I want to put this effect on some models to go further, so I downloaded a ugly man, and tried to give the material to him. But when I put this model in the scene, there's already a material with texture and shader on it. I didn't know if there would be conflict when two shaders are on one object. Finally, it turned out that it would definitely have conflicts. When I played it, that disappear couldn't make the texture work together. Since the previous shader on this ugly guy was based OpenGUI codes, I don't really know how to change it, so the better way is to add texture in visual scripts. First thing was to delete its old shader material and give it a color on PBR component. Then, give the disappear shader a texture node. Cuz the texture2D is a public property, I need to add the texture outside. Finally, it worked. The only problem left is I c...

https://pathmind.com/wiki/generative-adversarial-network-gan Notes: Discriminative models learn the boundary between classes Generative models model the distribution of individual classes i.e. the discriminator decides whether each instance of data that it reviews belongs to the actual training dataset or not. Here are the steps a GAN takes: The generator takes in random numbers and returns an image. This generated image is fed into the discriminator alongside a stream of images taken from the actual, ground-truth dataset. The discriminator takes in both real and fake images and returns probabilities, a number between 0 and 1, with 1 representing a prediction of authenticity and 0 representing fake. https://ai.googleblog.com/2017/12/introducing-new-foveation-pipeline-for.html Notes: However, current VR/MR technologies present a fundamental challenge: to present images at the extremely high resolution required for immer...

http://galigogo.com/OtherStuff/NewMediaProject/

Finally, I found a simple package of a dumb AI. It's about car chasing, the truck will simulate the red car's direction which is controlled by me.  Now, I can try to check the scripts, to be continued...

As I thought, there's not many examples for AI working with graphics technologies. But there's a big name, NVIDA. NVIDA published 20 series graphics card. One of their new technologies is DLSS, deep learning super sampling. Basically, it is for correcting the missing frames from raytracing, because the AI system has been trained to fill up the missing part in the images from a pointed game. More presentation: https://www.youtube.com/watch?v=KJRZTkttgLw#action=share https://www.nvidia.cn/geforce/gaming-laptops/20-series/ I mean, I can't really know how they realize this with every line of codes, but it provides me with the basic logic of what AI can do with graphics. After another around of research, I have a chart. Like I planned in the last blog, for now, I will get into some simple AI system, try to find out its sampling process and learning target . Except the hardware, another example Prof Paul showed us in class is  https://www.nvidi...