Topic Development / Developer's Corner / Completing lights branch - integration into object rendering
![[de]](/mwf/flags/de.png "Germany")
PeterW told me that finding solutions for the bugs that I have found in the lights rendering so far will be manageable. However, the other thing that is blocking the release of this branch is the integration into object rendering. This, he told me, would be much more of a challenge since the code for rendering were all over the place in the project, plus involves the work that had been done by others (recently) - like Clonk-Karl's mesh rendering.Currently, the raycasting algorithm outputs a 2d lighting texture for the viewport which is then processed in the shader by putting it together with a 2d normal map texture for the landscape. This lighting texture is currently not applied to neither sprite object graphics nor meshes. This is the part that still needs doing.
The data is there - the same 2d lighting texture could be applied to mesh and object rendering (that are not GUI/HUD elements I suppose). Now, Clonk-Karl asked me to write something more specific about what must be done here to complete this task. Basically, two things:
1. Apply lighting texture to the rendering of sprites (that are not GUI/HUD elements)
2. Apply lighting texture to the rendering of meshes (that are not GUI/HUD elements)
How this will be done is open for discussion. The most simple solution would be to shade each object with the light strength at the center pixel of the that object. If a more elaborate solution can't be done for the lack of manpower or know-how, this is the minimum solution we can go with. However, it's not really that cool. First, the direction of light which had been calculated by the algorithm is not used, so all the objects are still lighted as if the light comes from the camera - this kinda breaks the 3d illusion. Secondly, as objects would be shaded equally by the light strength at their center point, for bigger objects which are at the edge of the view range, it'd look bad cause one side of the object appears to dark, the other side too bright.
That is why PeterW imagined it to be done with a pixel shader as well, this is the optimum solution:
1. Use a pixel shader to shade every pixel of a sprite according to the light strength at that position
2. Use a pixel shader to shade every pixel of a mesh according to the light strength and light direction at that position. Both information are there in the lighting texture. So the actual light calculation would be done in a shader, the lighting via GL_LIGHTs would be discarded.
Especially for mesh rendering, this is super cool and will enhance the 3d illusion somewhat. Also, Peter's raycasting algorithm had been designed from the start to handle any number of dynamic light sources. In the future, we could even add colored light etc.
Note that I think we already use shaders for sprites - for the PlayerByOwner areas - and as well for the mesh rendering - also for the PlayerByOwner areas plus recently for normal map support. So the light calculation would have to be weaved into these shaders. Peter had doubts about this being implemented in this lights branch revival project because I lack the know-how for shader programming and the internals of the Clonk rendering system. But, as it so happens, I know (at least) two people who I think are quite adept at those things, Clonk-Karl and Zapper. So, my hopes is that you guys will help here to complete this project.
Any questions or statements?
![[us]](/mwf/flags/us.png "United States")
> Note that I think we already use shaders for sprites - for the PlayerByOwner areas - and as well for the mesh rendering - also for the PlayerByOwner areas plus recently for normal map support.
I have recently introduced pixel shaders for sprite rendering -- not for ColorByOwner but as part of some refactoring for batch-rendering the PXS. ColorByOwner at the moment is realized by two rendering passes, but I think it could (and should) easily be replaced by a single pass and some additional shader code.
Overall, without having looked into the details yet, I don't think it would be much effort to add the light calculations into the shaders. I could likely take care of that.
Right,... I forgot that PXS and particles are even something different which have to be shaded.
I just did some more refactoring and cleanup so that after merging the master branch into the lights branch, the lighting code would need to be added in 3 places, I think: mesh rendering (CStdGL::PerformMesh), sprite rendering (CStdGL::SetupMultiBlit), and particles rendering (C4ParticleList::Draw). The classic landscape renderer is now using the default sprite rendering, and I'm not sure whether it will need some custom handling as well, but it's probably not used very wildly anymore anyway.
> The classic landscape renderer is now using the default sprite rendering
What do you mean with classic landscape renderer?
I guess the one you get when you disable the zoom shader for performance reasons (for integrated Intel GPUs in laptops, etc.).
Question is what its place would be if we want to shader-ify everything anyway. Note that theoretically there actually is an option here to do everything without shaders - we'd simply skip all the normal stuff, and instead alpha-blit the frame buffer over the whole screen. This means we could get, say, blurry landscape with lights (w/o normal mapping). Not sure that's worth the trouble though.
An option to tone down effects for slow GPUs is probably good. But maybe it's sufficient to just provide two sets of shaders for that? I think that would be easier to maintain than a "no shader" variant.
Yes, sounds good. A simple "blocky" landscape shader for Intel GPUs would be worth a try, for example.
The current landscape scaling shader works nicely on my notebook with Intel HD 4400 graphics, by the way (one of the ULV-Haswells). Haven't tried the lights branch yet, though...
At 1920x1200 it can look kind of muddy.Actually it's more of a texture problem...
Attachment: Screenshot014.png (2262k)
That depends very much on the screen size. E.g. there's some notebooks with 14" display and crazy high resolution.
Why do we allow zooming beyond that then?
Because unlike Clonk Rage, zoom limits are imposed by how far you can see. Otherwise, people with higher resolution would see more of the landscape, have smaller clonks, etc. I think the current approach is much better.If we want to restrict zooming, we would have to limit the maximum resolution.
That's actually the landscape textures that isn't high-resolution enough. The shader is okay until roughly 16x zoom. If you want to check, try setting "MaterialZoom" in the [Landscape] section of Game.txt higher.
Edit: Also for future reference - if for some reason we need higher zoom than 16x, we'd just have to scale Scaler.bmp up in some image processing program. I simply used GIMP.
Edit: Also for future reference - if for some reason we need higher zoom than 16x, we'd just have to scale Scaler.bmp up in some image processing program. I simply used GIMP.
Wow, this makes the maze look sooo much better. Maybe the engine could set the material zoom intelligentely by taking the screen resolution and the maximal zoom level set by the scenario into account? So that the landscape textures are exactly displayed 1:1 when fully zoomed in, no matter what resolution you are playing on. (we are getting kinda off topic here, sorry)
How would you define "fully zoomed in"? I believe the zoom limits can be adjusted dynamically... As far as I'm concerned, we could think about pushing the material zoom to 8 eventually. Might need bigger textures though so it doesn't become repetitive when zoomed out...
1920x1080. Lights branch doesn't seem to make it much worse, either.
Light calculation isn't complicated - you'd bind the light texture of the viewpoint's light region, calculate the texture coordinates (not quite screen coordinates - it's actually landscape coordinates plus an offset, see
Region), then get intensity out of the R channel and direction out of the GB channels.
How would this work when there is more than one light source? Is there a light texture for each light source?
No, the texture contains the light map of all lights at the same time. Means we can't illuminate an object from two sides at once, yes. Hopefully this won't look too strange.
On the bright side, this means that graphics performance GPU won't be impacted much by the number of lights. The line-of-sight checks will likely be our bottleneck then.
On the bright side, this means that graphics performance GPU won't be impacted much by the number of lights. The line-of-sight checks will likely be our bottleneck then.
I have done that now for sprite drawing, and it mostly works. For sprite graphics, I am simply multiplying the sprite color with
It seems to me there must be something else which I'm missing... maybe the Z component of the light vector going into the dot product?
lightPix.r * vec3(1.0, 1.0, 1.0). This works for making things darker. However, for the landscape, the region immediately around the Clonk is actually lighted up, and this is not the case for the sprites. How is that done? I see that the landscape shader actually calculates 2.0 * shadeBright * dot(...), howevery if I multiply everything by a factor of 2 for the sprites, everything just becomes white.It seems to me there must be something else which I'm missing... maybe the Z component of the light vector going into the dot product?
Note the somewhat darkened firestone below the Clonk, and the darkening being applied to the sky background. Meshes are not darkened yet...
Btw, the twig objects are still rotated by 180 degrees in this map? :(
I guess so... didn't change them just for this screenshot ;)
> maybe the Z component of the light vector going into the dot product?
Yeah, basically that was it. I have now changed it such that the normal vector of sprites is always facing the camera (0,0,1), and the light direction is normalize(x,y,0.3), where x and y are the g and b components of the lights texture. (*-3 + 1). I have also implemented the lighting for meshes, and committed the current state in the lights branch.
* Note that the only light source in the screenshot is the clonk, there is no light coming from the sky.
* The clonk is now also lightening up the sky right behind it, like it would light up tunnel. Maybe that should not happen.
* Sky far away from the clonk is getting darker. This is basically coupled to the previous point, again I'm not sure this is what we want. We could just draw all the sky like in my previous screenshot, which would correspond to a light direction of (0,0,1) instead of (x,y,0.3). But then we would have different light calculations for the sky than for the rest of the world...
* The artifacts seen in the sky darkening are probably the same ones as in water, just that they are more pronounced with the sky being brighter.
* Sprite objects at the edge of tunnel look funny, because the sprite normals point towards the screen, while the edge of the material normals are mostly pointing into the x-y plane. This makes the sprite objects very dark compared to the tunnel edge. Could maybe be fixed by taking the normals for sprites from the landscape instead of hard-coding as (0,0,1)?
* When the clonk is standing underneath a tree, the tree is mostly dark. This is because the tree is not lit from above, but the normals of the tree are mostly pointing upwards. Maybe for faces that can be seen from both sides (i.e. when backface culling is disabled), the absolute value of the scalar product should be taken, instead of clamping negative values to 0.
* The light still needs to be applied to particles. This should be easy.
Otherwise I think this whole thing has a lot of potential! :)
> When the clonk is standing underneath a tree, the tree is mostly dark. This is because the tree is not lit from above, but the normals of the tree are mostly pointing upwards. Maybe for faces that can be seen from both sides (i.e. when backface culling is disabled), the absolute value of the scalar product should be taken, instead of clamping negative values to 0.
Alternatively, or in addition to that, we could add more emissive components to mesh materials, so they are partly self-illuminated. This is already possible, and I have turned it on slightly for the Clonk, so it is not covered in total black in some parts...
Absolutely fabulous!> * Note that the only light source in the screenshot is the clonk, there is no light coming from the sky.
(...and related problems)
This would be solved by the feature I talked about implementing earlier. Mix the light calculation out of the X% of the light texture and (1-X%) of a global ambient lighting for which the light is coming from the "default" direction (the lighting we have now). X is then the fade value in the vertical section of the landscape specified in the scenario txt. You follow me?
>* Sprite objects at the edge of tunnel look funny, because the sprite normals point towards the screen, while the edge of the material normals are mostly pointing into the x-y plane. This makes the sprite objects very dark compared to the tunnel edge. Could maybe be fixed by taking the normals for sprites from the landscape instead of hard-coding as (0,0,1)?
I think this will look odd - as if the sprite is just a sticker on top of the landscape. For the case of that firestone, the left part that is already on the tunnel would be glued flat on the tunnel. I would remove the logic with the normals for sprites. So the light for sprites is always coming from the front, only the "strength" channel of the light texture is used. How about that?
> I think this will look odd - as if the sprite is just a sticker on top of the landscape. For the case of that firestone, the left part that is already on the tunnel would be glued flat on the tunnel. I would remove the logic with the normals for sprites. So the light for sprites is always coming from the front, only the "strength" channel of the light texture is used. How about that?
If you want to see how this looks, in C4DrawGL.cpp, you can change these lines in the shader code (l. 618) from
lightDir = normalize(vec3(vec2(1.0, 1.0) - lightPx.gb * 3.0, 0.3));
lightIntensity = 2.0 * lightPx.r;to
lightDir = vec3(0.0, 0.0, 1.0);
lightIntensity = lightPx.r;
Also, it is good that the shadowing is currently very extreme because it is that extreme in the landscape. I think there is one central screw we can turn to make the light direction flatten a bit in the light texture generation code so that the light comes more from the camera then from the 2D landscape. (So it doesn't have to be done in each and every shader)
The light texture contains only the x and y component of the light direction; the Z component (how far it is coming from the camera), is at the moment specified in the shader. Maybe the light texture could be extended to contain it in the alpha channel, or in the normalization of the x-y vector. Or, more natural, use rgb as xyz, and the normalization specifies the intensity.
If you "let it come more from the camera" I expect the shadows will still be hard for objects that are far away from the light source, but softer for objects closer to it.
If you "let it come more from the camera" I expect the shadows will still be hard for objects that are far away from the light source, but softer for objects closer to it.
Right, yeah. That was the "central screw" I turned in my tests so far. Not too central if we have an own shader for every type of thing on the screen.> Maybe the light texture could be extended to contain it in the alpha channel
Why would you want to have the Z-part of the light be variable for every landscape pixel? A global constant seems enough to me.
What about just increasing the ambient lighting component a bit for that?
The ambient lighting component could be a scenario option as well, so that one would have no ambient light in cave scenarios and more in overground scenarios. Or make the ambient light depend on the world y coordinate, in a similar way like Newton suggested. Or we go all the way and compute some sort of ambient light map by how far a certain pixel is from any sky ;)
>Or we go all the way and compute some sort of ambient light map by how far a certain pixel is from any sky
Hm, maybe it even looks nice if the ambient lighting would also be day-time dependant, with the maximum at midday and the minimum at midnight?
So shadows would stand out more at morning and evening, and less at midday.
> Or we go all the way and compute some sort of ambient light map by how far a certain pixel is from any sky ;)
In that case, the map should rather encode if stuff had been explored already so it could make stuff that hasn't been explored yet black, stuff that had been explored but isn't visible dark and the visible stuff bright.
Maybe even make it like a strategy game fog of war. Make enemy and enemy equipment invisible and only update map changes when illuminated again...
And then implement dynamic loading/unloading of map regions so that there can be much larger worlds and whooo.
That sounds like something that's different from what I was talking about. I was basically just wondering how to determine the ambient light on a certain point on the map. Whether we can use the light feature to differentiate explored from unexplored areas is a completely different question in my view.
>The ambient lighting component could be a scenario option as well
I don't like having too many options (like in Clonk Endevaour: Scale and Hangle Rule, Collect Animals Rule.. all that), this always demands something from the player. It would be nicer if things would adjust automatically - even if small compromises would have to be made.
Would it be possible to define a normal map for sprites? Most sprites are resources (firestones, etc.). For the player, it would be good if they popped out of the landscape a bit.
With the heavy resources you mean? I haven't seen much work on that lately, so I don't think it's such a bad idea to get into a decent state what we have right now (i.e. lighting the sprites).
Topic Development / Developer's Corner / Completing lights branch - integration into object rendering
Powered by mwForum 2.29.7 © 1999-2015 Markus Wichitill
