float textures, cpu buffer support

Home Forums Syphon Syphon Development – Developer float textures, cpu buffer support

Viewing 11 posts - 1 through 11 (of 11 total)
  • Author
    Posts
  • #5183
    Brian Chasalow
    Participant

    float32 texture support, and cpu-side per-pixel access of a texture.
    These are two features i could really use for some application-specific stuff. I mentioned these awhile back, but now that some time has passed I was curious if you guys had any thoughts on the viability of such features, and whether they makes sense. (Or, are they already implemented in some hackerly fashion I was not aware of? )
    Brian

    #5184
    bangnoise
    Keymaster

    Float texture support is partially implemented in trunk on svn – pass in

    [NSDictionary dictionaryWithObject:SyphonImageFormatRGBA32 forKey:SyphonServerOptionImageFormat]

    as the options to SyphonServer (see SyphonServer.h).

    What isn’t finished is the fallback to something that works if a client or server is in a GL context which doesn’t support float textures – otherwise it should work, and I’d be interested to hear how you get on with it.

    We’ve discussed CPU-side pixel access a lot. The main concern is that direct access may open up an opportunity for a client app to block a remote server app indefinitely. We’d prefer people do buffered read-back in OpenGL – draw into FBO, download to PBO(s) – which is what we do in Syphon Recorder.

    Cheers, Tom

    #5185
    Brian Chasalow
    Participant

    oo, interesting! Will poke at float textures in a bit. do you have an example of FBO -> PBO readback that I could take a peek at / hack apart? can go to email if necessary. thanks!
    B

    #5187
    bangnoise
    Keymaster

    The exact way you do it depends on usage, and how much state you can preserve between calls, but here are the basics.

    Setup

    // Set-up: usually do this once and re-use these resources, however you may
    // have to recreate them if the dimensions change
    
    GLuint tex, fbo, pbo;
    GLint prevFBO, prevReadFBO, prevDrawFBO;
    
    // Store previous state
    glGetIntegerv(GL_FRAMEBUFFER_BINDING_EXT, &prevFBO);
    glGetIntegerv(GL_READ_FRAMEBUFFER_BINDING_EXT, &prevReadFBO);
    glGetIntegerv(GL_DRAW_FRAMEBUFFER_BINDING_EXT, &prevDrawFBO);
    
    // Push attribs
    glPushAttrib(GL_ALL_ATTRIB_BITS);
    
    glEnable(GL_TEXTURE_RECTANGLE_ARB);
    
    // Create the texture we draw into
    glGenTextures(1, &tex);
    glBindTexture(GL_TEXTURE_RECTANGLE_ARB, tex);
    glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_RGBA8, width, height, 0, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, NULL);		
    
    // Create the FBO
    glGenFramebuffersEXT(1, &fbo);
    glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fbo);
    
    // Test that binding works
    glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_RECTANGLE_ARB, tex, 0);
    GLenum status = glCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT);
    if (status != GL_FRAMEBUFFER_COMPLETE_EXT)
    {
    		// Deal with this error - you won't be able to draw into the FBO
    }
    
    // Restore state we're done with thus-far
    glBindTexture(GL_TEXTURE_RECTANGLE_ARB, 0);
    glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, prevFBO);
    glBindFramebufferEXT(GL_READ_FRAMEBUFFER_EXT, prevReadFBO);
    glBindFramebufferEXT(GL_DRAW_FRAMEBUFFER_EXT, prevDrawFBO);
    
    // Save PBO state
    GLint prevPBO;
    glGetIntegerv(GL_PIXEL_PACK_BUFFER_BINDING, &prevPBO);
    
    // Create our PBO and request storage for it
    glGenBuffers(1, &pbo);
    glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo);
    glBufferData(GL_PIXEL_PACK_BUFFER, width * height * 4, NULL, GL_DYNAMIC_READ);
    if (glGetError() != GL_NO_ERROR)
    {
    		// Storage for the PBO couldn't be allocated, deal with it here
    }
    
    // Restore state
    glBindBuffer(GL_PIXEL_PACK_BUFFER, prevPBO);
    glPopAttrib();

    Per-frame

    // Save state as above, skipped for brevity
    // ...
    
    // The first thing we do is read data from the previous render-cycle
    // This means the GPU has had a full frame's time to perform the download to PBO
    // Skip this the first frame
    if (thisIsNotTheFirstFrame)
    {
    	glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo);
    	void *pixelData = glMapBuffer(GL_PIXEL_PACK_BUFFER, GL_READ_ONLY);
    	// Do something with the pixel data
    }
    
    // Now start the current frame downloading
    
    // We have a SyphonImage from somewhere, e.g.
    SyphonImage *image = [myClient newFrameImageForContext:cgl_ctx];
    
    // Attach the FBO
    glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fbo);
    glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_RECTANGLE_ARB, tex, 0);
    
    // Set up required state
    glViewport(0, 0,  width, height);
    glMatrixMode(GL_PROJECTION);
    glPushMatrix();
    glLoadIdentity();
    
    glOrtho(0.0, width,  0.0,  height, -1, 1);		
    
    glMatrixMode(GL_MODELVIEW);
    glPushMatrix();
    glLoadIdentity();
    
    // Clear
    glClearColor(0.0, 0.0, 0.0, 0.0);
    glClear(GL_COLOR_BUFFER_BIT);
    
    // Bind the texture
    glEnable(GL_TEXTURE_RECTANGLE_ARB);
    glActiveTexture(GL_TEXTURE0);
    glBindTexture(GL_TEXTURE_RECTANGLE_EXT, image.textureName);
    
    // Configure texturing as we want it
    glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_WRAP_S, GL_CLAMP);
    glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_WRAP_T, GL_CLAMP);
    glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
    glEnable(GL_BLEND);
    glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
    glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);
    
    glColor4f(1.0, 1.0, 1.0, 1.0);
    
    // Draw it
    // These coords flip the texture vertically because often you'll want to do that
    GLfloat texCoords[] =
    {
    	0.0, height,
    	width, height,
    	width, 0.0,
    	0.0, 0.0
    };
    
    GLfloat verts[] =
    {
    	0.0, 0.0,
    	width, 0.0,
    	width, height,
    	0.0, height
    };
    
    glEnableClientState( GL_TEXTURE_COORD_ARRAY );
    glTexCoordPointer(2, GL_FLOAT, 0, texCoords );
    glEnableClientState(GL_VERTEX_ARRAY);
    glVertexPointer(2, GL_FLOAT, 0, verts);
    glDrawArrays( GL_TRIANGLE_FAN, 0, 4 );
    
    // Now perform the download into the PBO
    
    glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo);
    glBindTexture(GL_TEXTURE_RECTANGLE_ARB, tex);
    
    // This is a minimal setup of pixel storage - if anything else might have touched it
    // be more explicit
    glPixelStorei(GL_PACK_ROW_LENGTH, width);
    
    // Start the download to PBO
    glGetTexImage(GL_TEXTURE_RECTANGLE_ARB, 0, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, (GLvoid *)0);
    
    // Restore state, skipped for brevity, see set-up section
    // ...

    #5188
    bangnoise
    Keymaster
    #5189
    Brian Chasalow
    Participant

    awesome. thanks tom. will take a look at it in a bit. B

    #5190
    Andrea Cremaschi
    Participant

    @bangnoise: thanks for the code!

    Actually, it does something different in respect of the one i posted: you are drawing in a -single- PBO, while the CVOpenGLBufferPool code manages -a pool- of frame buffers, using Core Video framework.
    This helps when a new frame may be available in Syphon server before the old one has been entirely downloaded in CPU memory (i.e. in case of heavy GPU image manipulation retarding the frame download). This is also a bit more time consuming, since a new buffer must be attached to the openGL context for every new available Syphon image (ie the “CVOpenGLBufferAttach” method call). But with a slightly more complex code than the one I posted (in a multithread app), it is possible to avoid this delay, attaching the buffer as soon as the new Syphon image has been copied to the opengl buffer.

    So, always speaking to the ones who want to copy syphon images to CPU memory: who just wants speed, doesn’t want to mess with multithreading and doesn’t need to do great image manipulation on the new Syphon image can go with the “opengl PBO method”. Who needs to do heavy GPU image manipulation, or need to process ALL the frames should go with the “CVOpenGLBufferPool method”..

    ciao 🙂

    ac

    #5191
    bangnoise
    Keymaster

    The above is intended as a solid start rather than a finished implementation.

    You probably would want to use more than one PBO, although the way you do it (a low fixed number, or a pool) would depend on what you were doing with the buffers subsequently.

    CoreVideo is likely doing something very similar to the above behind the scenes – its advantage is simply lightness of code.

    #33034
    rsodre
    Participant

    Hello!

    Any news on the float texture support?

    I saw some 1-year old comment on the old google code roadmap, I will need it pretty soon, just want to know if it’s already implemented or I need to hack my way.

    Thanks!

    #33035
    bangnoise
    Keymaster

    Yo!

    The GitHub framework still has the float branch, but it is considerably behind master and will be some work to merge. It still needs work to cope with a server and client running in contexts with different levels of support for float textures, and I can’t remember how it interacts with clients using the current framework – may need work there too.

    If you’re interested/keen, we’d gladly consider any pull requests to the float branch – if you’re making sweeping changes, probably best to discuss them here or on GitHub before starting.

    #33076
    rsodre
    Participant

    I took a look in the float branch, not sure if I can help to properly finish it.
    For my project, I can have server and client using the same version, so I’ll try to compile and use this branch, I’ll report what happens when I do it.
    Thanks!

Viewing 11 posts - 1 through 11 (of 11 total)
  • You must be logged in to reply to this topic.