Well, where to begin.
I use Geometry shaders in some QC plugins (for work), and you can only do it in a Plugin due to some specifics of how you have to init, specify number of output vertices, and specify in and out geometry primitives for the shader, and of course because the built in GLSL patch does not support it*. But its 100% doable in QC in a plugin if you are sure to reset and not leak OpenGL State (something very very important in 3rd party plugins). Another important note is that although some Geometry Shaders are supported under 10.6, most run in software fallback for vertex/geom processing (not fragment).
As far as OpenCL, I have limited experience with only because support is so all over the place and its so finicky right now. But my understanding is you can do arbitrary tessellation and output of new primitives because you have full control over the pipeline. It basically is a programmable pipeline where the concept of ‘stages’ is fully removed (OpenGL has various fixed function, and programmable stages for how it handles lighting, rasterization etc).
What you are talking about is tessellation, and its 100% doable in GLSL/GL_GEOMETRY_SHADER_4 and OpenCL, however you have to be incredibly careful in the ordering of your output primitives and the indices you output that describe the render order. OpenGL is very sensitive to these things, and getting it right makes all the difference.
Another issue for how this works in QC, esp with regard to meshes, is the order and primitive you are sending to the plugin (or the OpenCL patch). You don’t always have a lot of say in that area, which could throw some wrenches in a plan to make it globally usable (I think).
*on the topic of the built in GLSL patch – I avoid it like the plague. I’ve had too many odd issues with it, plus texture matrices / coordinates are not always what you want, when you ask for anything in a unit higher than 1. Also, In one shader, you don’t have to have a vertex program to have a fragment program, and vice versa (although you do need a vertex program if you want a geometry program). Apples GLSL patch forces some unnecessary requirements on you, coupled with oddities in their handling of image texture matrices and coordinates, makes me *always* use a custom plugin. Plus, near as I can tell, its faster.