Motivation
It seems that I'm working a lot on spheres recently... or that someone close to me does so ;) Well, thing is I recently got some new equations for a neat trick that you can add to your toolbox too. Say you have a planet, or in other words, a sphere (where you most probably grow some procedural mountains or whatever). Now, say you have objects both at planetary surface level as well as in the sky or space. Given a camera position (both in surface or space), you want to know what objects are for sure NOT visible because of planet occlussion. Objects in the other side of the planet will not be visible of course, but many others will also not be because of the curvature of the planet, and that can count for a BIG amount of geometry when you are at surface level ("objects" includes "mountains" also of course).
The trick presented here can be seen at a kind of horizon mapping, and it gives analytical micropixel perfect occlussion results, for a spherical occluder (the planet) and spherical objects. Now, you can use spheres around every node of your octree to quickly discard complete subtrees of geometry on your planet :)
The maths

where 
Let's call and put it all together to get:
This is much nicer already than the angle nightmare. However we can do better and get rid of the square root. Note that we want to flag objects as visible when the inequality holds. If the left part of the inequality is negative, the object will be this visible for sure. If it's positive, we have to compute the complete thing. Before going on, let's introduce
and see that we can check for the sign of the left part by
Now, the complete equation can be simplyfied by taking squares in both sides. The resulting inequality is:
what is quite cool (simple and fast to compute, ie, elegant!) You can reinterprete it as
what looks like the a kind of cosine law btw :)
ImplementationThe implementation is straightforward, it only requires two inverse squareroots (invsqrtf), or only one per camera position if you precompute the all data relative to oc: int visible( const vec3 & o, const float R, const vec3 & op, const float Rp, const vec3 & c ) { const vec3 ac = o  c; const vec3 bc = op  c; const float ia = 1.0/length( ac ); const float ib = 1.0/length( bc ); const float k0 = dot( ac, bc ) * ia*ib; const float k1 = R * ia; const float k2 = Rp * ib; if( k0*k0 + k1*k1 + k2*k2 + 2.0f*k0*k1*k2  1.0f < 0.0f ) return 1; if( k0*k0 + k1*k1 + k2*k2  2.0f*k0*k1*k2  1.0f < 0.0f ) return 2; return 3; } Note that this method is a kind of 2D overlap test, it will report occlusion when the object is the "shadow" of the planet but still in front of it. The case can be easily fixed by adding a plane test. The plane is formed by the circular cap where the cone formed by the planet the the camera (being the later the apex) intersects the sphere (a disk). The center of the disk can be computed as in this article, and then a dot product will tell you if your object is between that plane and the camera. A realtime interactive implementation of the code above can be found here (click in the title to naviate to the source code, or simply move the mouse along the image to change the position of the spheres. 