1) a tutorial on the same topic by Red Blob Games: http://www.redblobgames.com/articles/visibility/
2) a public domain Javascript visibility polygon library that runs in O(n log n), which is actually used in the Nothing to Hide game: https://code.google.com/p/visibility-polygon-js/
3) a blog post by author of said library discussing the algorithm: http://byronknoll.blogspot.ca/2013/05/on-log-n.html
Shooting a ray to every vertex and then naively computing intersections takes O(n^2) since there are n rays with up to O(n) intersections each. It is interesting to read and think about O(n log n) algorithms for computing the visibility polygon.
If you only care about reading code, open source or not, I think the first Deus Ex's in game scripting code was visible in the game's editor. (And the rest was the first Unreal engine, whose might also be available for reading nowadays. Or you just read the source of the various Quakes.)
First the scene must be put into a 2d bsp (that requires slicing some edge). After that, the bsp must be traversed front-to-back and that gives the edges in the right order. Just keep track of which angles are occluded then while traversing. During the process, when an edge is proccessed, cast/create the shadow accordingly. The algorithm stop when the field of vision is fully occluded - and not farther.
you can certainly walk 'outwards' from a node but then you are missing the real benefit to the bsp for this problem imo... which is that you just carved the world into convex volumes.
convex polygons are easier to work with. they allow applying many more algorithms and make others trivial enough to re-invent on the fly as needed... instead of walking a structure at all you could keep a list of which edges are visible at each leaf node - if you have 'loads' of memory (e.g. > 1MB) then you might be able to store the edges themselves rather than indexes and mitigate memory lookup expenses as well (which are so far away from anything in a broswer... but yeah).
I had fun implementing this effect a long time, in a pre-graphics-shader era [1].
(Note: I have no connection to this developer. Just thought it was worth mentioning.)
I prefer using the technique shown in this classic article on 2D soft shadows: http://archive.gamedev.net/archive/reference/programming/fea...
The basic idea is to project the geometry, rather than ray cast. I've implemented this sort of technique in the past and it works well.
https://github.com/sq/Illuminant
It's hardware-accelerated and has a few other useful features; we're using it in Escape Goat 2 and it works on Windows/Linux/Mac on pretty much any D3D9-spec hardware. You can see some of the levels using it here: https://www.youtube.com/watch?v=-9s1PyotS18#t=1m20s
For EG2 we have a pair of separate lighting environments; one for the foreground plane and one for the background plane. Most light sources exist on both planes but have different parameters in order to create a depth effect; some light sources are only on the background plane. We apply a customized ramp texture to the light sources in order to replace the smooth attenuation with quantized attenuation, and we don't use soft shadows (though we do render lighting at a lower resolution and upscale it, because that softens things a little.) We also do a simple HDR camera model by sampling lighting intensity across the scene's foreground/background planes and using that to adjust the range of the final lightmap so that we can automatically tune exposure/white point etc to highlight certain parts of a level.
I also have a similar pure-software-rendering lighting library from around ~8 years ago available here, in C++:
https://github.com/kg/Fury2/blob/master/libraries/SoftFX/mod...
That one's garbage, but may still be of some use.
I'll note that the author's approach (needing extra intersection points at +- 0.00001 radian) is a little perplexing. I'm not sure why that is needed; I've never needed it.
A better approach for 'fuzzy' shadows is to model the penumbra; you can render it as a single triangle that borders on the actual shadow umbra and then make it smooth by using a simple lookup texture. This technique is described here:
http://archive.gamedev.net/archive/reference/programming/fea...
Another great option for 2D lighting is to associate normal maps with your sprites and do per-pixel lighting against each light source using the normal map. This can produce some really great results. I think that SpriteLamp, http://snakehillgames.com/spritelamp/ does this; the Fury2 library I linked above also did per-pixel sprite lighting with normal maps. The upcoming game Full Bore (http://www.wholehog-games.com/fullbore/) is using per-pixel lighting on its sprites and environment art, and I believe Spelunky for PC/XBLA uses it on environment art.
No criticism is intended towards OP's game; the "un-stealth game" is a different and interesting idea. I'm just throwing it out there for those dreaming up yet another top-down cone-of-vision stealth shooter to chew on.
Try out Mark of the Ninja and tell me they were 'just trying to make an FPS'.
Picking an appropriate perspective for a game involves a lot of tradeoffs. While certain mechanics are most appropriate for certain perspectives (top-down 2d, isometric, first-person 3d...) a general approach to 'visibility' is not one of them. Visibility (or an approximation of it) applies to AI in almost every action game type, regardless of perspective. Whether they can see you or not is an essential consideration for creating convincing enemies, and AI allies need to know whether they can see foes.
You can also extensively use vision cone mechanics without having anything to do with an FPS. Classic games of the Rogue variety (i.e. nethack, angband...) all leverage vision cones and line-of-sight despite the fact that they predated 3D games of any kind.
Having some dynamic lighting does not mean that you must make it part of your game's mechanics. It's perfectly fine to do that kind of thing just for polish.
just blindly tracing to all of the verts 'feels wrong' its the sort of thing that is intuitively unreasonable with large data sets and is quite obviously going to be constrained by available horsepower at some size.
precomputed vis data is the obvious optimisation and removes the O(n) behaviour. not only do you not have to test everypoint but your raycast itself need not test every edge. i wrote something ages ago about generating PVS in 2d as an example of how to approach the problem in 3D...
(http://jheriko-rtw.blogspot.co.uk/2011/05/visibility-determi...)
the other thing that bugs me here is the jitter. it is the classic 'lazy' way to soften shadows... a better effect can be gotten for less - if you offset the light position perpendicular to the direction toward the test vert, in both directions by the same amount then you can generate two ray casts. triangulating the resulting mesh might be a little painful (unless you cut the world into convex chunks - then it is nearly trivial) but you will get the outline for both edges of the shadow and can then do something like vertex interpolated colour to give you the same visual result as infinite raycasts for the price of two...
This is a nice little post that reasonably describes a technique that has been used by a fair number of professional, commercial products. That anyone would try to downplay it to make themselves feel superior is silly.
To provide some actual value, most of the the images in the post are actually interactive. It's pretty cool. I somehow read the whole thing and didn't notice.
Why would you think that was my goal? Did i say the author was stupid or didn't pay attention? I simply pointed out a limitation that might not be obvious to others at first sight and which wasn't discussed in the article itself, thus also providing for others an easy starting point to improve on the algorithm as presented.
Heck, in the circles* i run the first part of my comment is even praise. Figuring out ways to approximate "real" results so they can be done in realtime is not easy.
*Have an example of EXTREMELY competent faking from those circles: https://www.youtube.com/watch?v=MJfceF0syK8
"Today, I will show you how to make something like this: (move your mouse around in the box below)"
"The demo below just draws a bunch of line segments and tracks your mouse position."
"Here's what all that math looks like: (move your mouse over the box)"
http://i.imgur.com/G5U43ZU.png
Calculate the edge points of the circle, draw a line between them, and then your shadow is simply a rectangle. Overlay the circle over the shadow to hide the line.
This will work with circles and ovals, but not more complex curves.
