NSImage to Syhon

Home Forums Syphon Syphon Development – Developer NSImage to Syhon

This topic contains 7 replies, has 2 voices, and was last updated by  Rick Burnett 10 months, 2 weeks ago.

Viewing 8 posts - 1 through 8 (of 8 total)
  • Author
    Posts
  • #59226

    Rick Burnett
    Participant

    I’m not sure what I am doing wrong, I’ve spent about 4 days trying different things but nothing seems to work.

    I’m using some code that creates an NSImage from a view (this is from an NSOpenGLView). I’ve tested that an actual image does get created as the writePNGToURL works great (commented out in example below).

    The code following that I found in a working example (I downloaded and tested) that puts a PNG image into an NSImage and then display through Syphon. Yet for whatever reason, that same exact code does not work for me.

    This code recreates a texture over and over which I’d not do in the end, I’d just create a texture and replace the contents, but I tried that as well and same results, black screen in testing code (in this case VDMX).

    Any idea what I am doing wrong?

          NSImage *im = [self imageFromView];
          //[im writePNGToURL: [NSURL fileURLWithPath:@"./test.png"]  outputSizeInPixels: NSMakeSize(640,480) error:nil];
    
        const size_t bitsPerComponent = 8;
        const int componentsPerPixel = 4;
        const size_t bytesPerRow=((bitsPerComponent * [im size].width) / 8) * componentsPerPixel;
        CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
    
        NSBitmapImageRep* imageRep=[[NSBitmapImageRep alloc] initWithData:[im TIFFRepresentation]];
        CGImageRef pixelData = [imageRep CGImage];
    
        CGContextRef gtx = CGBitmapContextCreate(NULL, [im size].width, [im size].height, bitsPerComponent, bytesPerRow, colorSpace, kCGImageAlphaPremultipliedLast);
        
        CGContextDrawImage(gtx, CGRectMake(0, 0, 1280, 960), pixelData);
        CGContextFlush(gtx);
        NSLog(@"GL Error = %u", glGetError());
        GLKTextureInfo* tex = [GLKTextureLoader textureWithCGImage:pixelData options:NULL error:NULL];
        NSLog(@"GL Error = %u", glGetError());
        NSLog(@"texture: %i %ix%i    %i %i", tex.name, tex.width, tex.height , tex.target , tex.alphaState);
        [myServer publishFrameTexture:tex.name
                        textureTarget:GL_TEXTURE_2D
                          imageRegion:NSMakeRect(0, 0, tex.width, tex.width)
                    textureDimensions:NSMakeSize(tex.width, tex.height)
                              flipped:YES];
    
    #59227

    vade
    Keymaster

    Is your OpenGL Context active and current?

    Is this the same OpenGL Context you’ve used to make a Syphon Server?

    Is this context legacy or core profile? (use legacy)

    You are positive texture target is GL_TEXTURE_2D?

    You are positive your CGContext contains an image?

    No logs? No errors? The code above is not enough to really get a context for what could be going wrong.

    https://www.mikeash.com/getting_answers.html

    #59228

    Rick Burnett
    Participant

    I’m using:

    
    CGLSetCurrentContext( [context CGLContextObj] );

    to set the current context, this is the context from my NSOpenGLVuew, I don’t see anything that would make it not current. This is also the context I am passing into the Syphon when I start the server.

    I think what I have is Core context correct?

    Yes, I am positive on the texture target. I used apitrace on the running app to just make sure textures are being constructed properly and I can view them at any frame. They look correct in apitrace.

    I don’t see any errors being reported, including in API trace.

    Just for the heck of it, I created a garbage texture basically just with some fake data to see if it would pass it along. I did get it to send something, but it only draws 1/2 of the display diagonal.

    glTexImage2D(GL_TEXTURE_2D,0,GL_RGBA8,2,2,0,GL_RGBA,GL_UNSIGNED_BYTE,@"312313532433");

    Context is being generated in my main like so:

    StretchView *view = [[StretchView alloc] initWithFrame: NSMakeRect (0, 0, WIDTH, HEIGHT) ];

    StretchView is derived from NSOpenGLView. (This was sample code I found for Cairo_nsgl usage)

    And the initialization of all the Cairo stuff is (and syphon)

    context = [super openGLContext];
    
        [context makeCurrentContext];
        glEnable(GL_TEXTURE_2D);
        glGenTextures(1,&texture);
    	
       
    
        cairo_device_t *device = cairo_nsgl_device_create (context);
    		 
        surface = cairo_gl_surface_create_for_view (device, self, WIDTH, HEIGHT);
    
        cr = cairo_create (surface);
    
        
        renderTimer = [NSTimer timerWithTimeInterval:0.001   //a 1ms time interval
                                    target:self
                                    selector:@selector(timerFired:)
                                    userInfo:nil
                                    repeats:YES];
     
        [[NSRunLoop currentRunLoop] addTimer:renderTimer
                                    forMode:NSDefaultRunLoopMode];
        [[NSRunLoop currentRunLoop] addTimer:renderTimer
                                    forMode:NSEventTrackingRunLoopMode]; //Ensure timer fires during resize
    
      
       //start syphon
       myServer = [[SyphonServer alloc] initWithName:@"Cairo Output" context:[context CGLContextObj]  options:nil];

    I initially didn’t want to post in a million things I tried until I had some sort of direction to start. Most of my GL work has been on the Raspberry Pi platform lately so still learning a lot about how it’s put together in OSX.

    Thanks for taking a look.

    • This reply was modified 10 months, 2 weeks ago by  Rick Burnett. Reason: fixing code tags
    #59230

    Rick Burnett
    Participant

    I’ve cleaned out a lot of unnecessary code, but here is a copy of what I am basically running at the moment:

    #import <Cocoa/Cocoa.h>
    #import <OpenGL/gl.h>
    #import <cairo.h>
    #import <cairo-gl.h>
    #import <CoreMIDI/CoreMIDI.h>
    #import <Syphon/Syphon.h>
    #import <GLKit/GLKit.h>
    
    #define WIDTH 640
    #define HEIGHT 480
    
    #define SLOTS 12
    int drawTable[SLOTS][4];
    int dtPtr;
    
    @interface NSImage (SSWPNGAdditions)
    
    - (BOOL)writePNGToURL:(NSURL*)URL outputSizeInPixels:(NSSize)outputSizePx error:(NSError*__autoreleasing*)error;
    
    @end
    
    @implementation NSImage (SSWPNGAdditions)
    
    - (BOOL)writePNGToURL:(NSURL*)URL outputSizeInPixels:(NSSize)outputSizePx error:(NSError*__autoreleasing*)error
    {
        BOOL result = YES;
        NSImage* scalingImage = [NSImage imageWithSize:[self size] flipped:NO drawingHandler:^BOOL(NSRect dstRect) {
            [self drawAtPoint:NSMakePoint(0.0, 0.0) fromRect:dstRect operation:NSCompositeSourceOver fraction:1.0];
            return YES;
        }];
        NSRect proposedRect = NSMakeRect(0.0, 0.0, outputSizePx.width, outputSizePx.height);
        CGColorSpaceRef colorSpace = CGColorSpaceCreateWithName(kCGColorSpaceGenericRGB);
        CGContextRef cgContext = CGBitmapContextCreate(NULL, proposedRect.size.width, proposedRect.size.height, 8, 4*proposedRect.size.width, colorSpace, kCGBitmapByteOrderDefault|kCGImageAlphaPremultipliedLast);
        CGColorSpaceRelease(colorSpace);
        NSGraphicsContext* context = [NSGraphicsContext graphicsContextWithGraphicsPort:cgContext flipped:NO];
        CGContextRelease(cgContext);
        CGImageRef cgImage = [scalingImage CGImageForProposedRect:&proposedRect context:context hints:nil];
        CGImageDestinationRef destination = CGImageDestinationCreateWithURL((__bridge CFURLRef)(URL), kUTTypePNG, 1, NULL);
        CGImageDestinationAddImage(destination, cgImage, nil);
        if(!CGImageDestinationFinalize(destination))
        {
            NSDictionary* details = @{NSLocalizedDescriptionKey:@"Error writing PNG image"};
            [details setValue:@"ran out of money" forKey:NSLocalizedDescriptionKey];
            *error = [NSError errorWithDomain:@"SSWPNGAdditionsErrorDomain" code:10 userInfo:details];
            result = NO;
        }
        CFRelease(destination);
        return result;
    }
    
    @end
    
    @interface StretchView : NSOpenGLView {
    @private
        NSTimer *renderTimer;
        cairo_surface_t *surface;
        cairo_t *cr;
        NSTimeInterval now;
        MIDIClientRef midiClient;
        MIDIPortRef inputPort;
        MIDIEndpointRef virtDest;
        SyphonServer *myServer;
        GLuint texture; 
        NSOpenGLContext *context;
    }
    - (void) draw;
    
    @end
    
    @implementation StretchView
    
    	static void memxor(unsigned char *dst, unsigned char *src, unsigned int bytes)
    	{
    		while (bytes--) *dst++ ^= *src++;
    	}
    	
    	static void memswap(unsigned char *a, unsigned char *b, unsigned int bytes)
    	{
    		memxor(a, b, bytes);
    		memxor(b, a, bytes);
    		memxor(a, b, bytes);
    	}
    
    - (NSImage *) imageFromView
    {
    	NSRect bounds;
    	int height, width, row, bytesPerRow;
    	NSBitmapImageRep *imageRep;
    	unsigned char *bitmapData;
    	NSImage *image;
    	
    	bounds = [self bounds];
    	
    	height = bounds.size.height;
    	width = bounds.size.width;
    	
    	imageRep = [[NSBitmapImageRep alloc] initWithBitmapDataPlanes: NULL
    				pixelsWide: width
    				pixelsHigh: height
    				bitsPerSample: 8
    				samplesPerPixel: 4
    				hasAlpha: YES
    				isPlanar: NO
    				colorSpaceName: NSCalibratedRGBColorSpace
    				bytesPerRow: 0				// indicates no empty bytes at row end
    				bitsPerPixel: 0];
    				
    	[[self openGLContext] makeCurrentContext];
    				
    	bitmapData = [imageRep bitmapData];
    
    	bytesPerRow = [imageRep bytesPerRow];
    	
    	glPixelStorei(GL_PACK_ROW_LENGTH, 8*bytesPerRow/[imageRep bitsPerPixel]);
    
    	glReadPixels(0, 0, width, height, GL_RGBA, GL_UNSIGNED_BYTE, bitmapData);
    
    	// Flip the bitmap vertically to account for OpenGL coordinate system difference
    	// from NSImage coordinate system.
    	
    	for (row = 0; row < height/2; row++)
    	{
    		unsigned char *a, *b;
    		
    		a = bitmapData + row * bytesPerRow;
    		b = bitmapData + (height - 1 - row) * bytesPerRow;
    		
    		memswap(a, b, bytesPerRow);
    	}
    
    	// Create the NSImage from the bitmap
    
    	image = [[[NSImage alloc] initWithSize: NSMakeSize(width, height)] autorelease];
    	[image addRepresentation: imageRep];
    	
    	// Release memory
    	
    	[imageRep release];
    
    	// Previously we did not flip the bitmap, and instead did [image setFlipped:YES];
    	// This does not work properly (i.e., the image remained inverted) when pasting 
    	// the image to AppleWorks or GraphicConvertor.
    
    	return image;
    }
    + (NSOpenGLPixelFormat*)defaultPixelFormat
    {	
    
         NSOpenGLPixelFormatAttribute attrs[] =
        {
    		NSOpenGLPFADoubleBuffer,
    		NSOpenGLPFAAccelerated,
    		NSOpenGLPFADepthSize, 24,
    		NSOpenGLPFAMultisample,
    		NSOpenGLPFASampleBuffers, 1,
    		NSOpenGLPFASamples, 4,
    		(NSOpenGLPixelFormatAttribute) 0
        };
    	
        NSOpenGLPixelFormat *classPixelFormat = [[[NSOpenGLPixelFormat alloc] initWithAttributes:attrs] autorelease];
        if (!classPixelFormat)
    	    exit (-1);
    
        return classPixelFormat;
    }
    
    - (void)prepareOpenGL
    {  
        context = [super openGLContext];
    
        [context makeCurrentContext];
        glEnable(GL_TEXTURE_2D);
        glGenTextures(1,&texture);
    	
        //GLint swapInt = 1;
        //[context setValues:&swapInt forParameter:NSOpenGLCPSwapInterval]; // set to vbl sync
    
        cairo_device_t *device = cairo_nsgl_device_create (context);
    		 
        surface = cairo_gl_surface_create_for_view (device, self, WIDTH, HEIGHT);
    
        cr = cairo_create (surface);
    
        
        renderTimer = [NSTimer timerWithTimeInterval:0.001   //a 1ms time interval
                                    target:self
                                    selector:@selector(timerFired:)
                                    userInfo:nil
                                    repeats:YES];
     
        [[NSRunLoop currentRunLoop] addTimer:renderTimer
                                    forMode:NSDefaultRunLoopMode];
        [[NSRunLoop currentRunLoop] addTimer:renderTimer
                                    forMode:NSEventTrackingRunLoopMode]; //Ensure timer fires during resize
    
       // Initialize the draw table
       for(int i=0;i<SLOTS;i++){
          for(int param=0;param<4;param++) {
             drawTable[i][param]=0;
          }
       }
       dtPtr=0;
    
       //start syphon
       myServer = [[SyphonServer alloc] initWithName:@"Cairo Output" context:[context CGLContextObj]  options:nil];
    
    }
    // Timer callback method
    - (void)timerFired:(id)sender
    {
        // It is good practice in a Cocoa application to allow the system to send the -drawRect:
        // message when it needs to draw, and not to invoke it directly from the timer.
        // All we do here is tell the display it needs a refresh
        //[self setNeedsDisplay:YES];
        [self draw];
    }
    
    int first = 1;
    
    - (void)drawRect:(NSRect)dirtyRect 
    {   //CGLSetCurrentContext( myServer.context );
        cairo_save (cr);
        cairo_rectangle (cr, 0, 0, WIDTH, HEIGHT);
        if(first) {
          cairo_set_source_rgb (cr, 0, 0, 0);
          first = 0;
        } else {
          cairo_set_source_rgba (cr, 0, 0, 0, 0.01);
        }
        cairo_fill (cr);
    
        for(int i=0; i<SLOTS;i++){
          if( drawTable[i][2] > 0 ) {
             int x = drawTable[i][0];
             int y = drawTable[i][1];
             float lum = drawTable[i][2]/256.0f;
    
             cairo_set_source_rgba (cr, lum,lum,lum, 0.8);
             cairo_arc(cr, x,y, drawTable[i][2], 0, 2* M_PI);
             cairo_stroke(cr);
             //cairo_set_font_size (cr, 18);
             //cairo_show_text (cr, "ONE NIGHT IN CAIRO!");
             drawTable[i][1]+=drawTable[i][3]/10;
             drawTable[i][2]-=8;
          } else {
             drawTable[i][2] = 256;
             drawTable[i][0] = i*30;
          }
       }
    
       glFinish();
       glFlushRenderAPPLE();
        
       NSImage *im = [self imageFromView];
       //[im writePNGToURL: [NSURL fileURLWithPath:@"./test.png"]  outputSizeInPixels: NSMakeSize(640,480) error:nil];
       CGLSetCurrentContext( [context CGLContextObj] );
       glEnable(GL_TEXTURE_2D);
    
       glBindTexture(GL_TEXTURE_2D, texture);
       NSBitmapImageRep *bitmap = [[NSBitmapImageRep alloc] initWithData:[im TIFFRepresentation]];
       glPixelStorei(GL_UNPACK_ROW_LENGTH, (GLint)[bitmap pixelsWide]);
    
       glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
    
       glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
       glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
       glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
       glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
    
       NSInteger samplesPerPixel = [bitmap samplesPerPixel];
       
        if(![bitmap isPlanar] && (samplesPerPixel == 3 || samplesPerPixel == 4)) {
    
            glTexImage2D(GL_TEXTURE_2D, 0,
                         samplesPerPixel == 4 ? GL_RGBA8 : GL_RGB8,
                         (GLint)[bitmap pixelsWide],
                         (GLint)[bitmap pixelsHigh],
                         0,
                         samplesPerPixel == 4 ? GL_RGBA : GL_RGB,
                         GL_UNSIGNED_BYTE,
                         [bitmap bitmapData]);
        }else{
            NSLog(@"Problem");
            [[NSException exceptionWithName:@"ImageFormat" reason:@"Unsupported image format" userInfo:nil] raise];
        }
        //exprimental test
        //glTexImage2D(GL_TEXTURE_2D,0,GL_RGBA8,2,2,0,GL_RGBA,GL_UNSIGNED_BYTE,@"312313532433");
        [myServer publishFrameTexture:texture textureTarget:GL_TEXTURE_2D imageRegion:NSMakeRect(0, 0, 640,480 ) textureDimensions:NSMakeSize(640, 480) flipped:NO];
      
       
    
         cairo_gl_surface_swapbuffers (surface);
         cairo_restore (cr);
    
        //[im unlockFocus];
    }
    
    - (void) draw
    {
        [self setNeedsDisplay: YES];
    }
    	
    @end
    
    int main (int argc, char **argv)
    {
        NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
        [NSApplication sharedApplication];
        [NSApp setActivationPolicy:NSApplicationActivationPolicyRegular];
        
        int style = NSClosableWindowMask |
        		NSTexturedBackgroundWindowMask | NSTitledWindowMask |
        		NSMiniaturizableWindowMask;
    
        NSScreen *screen = [NSScreen mainScreen];
        NSRect frame = [screen visibleFrame];
        int frame_height = frame.size.height;
    
        NSWindow *win = [[ NSWindow alloc] initWithContentRect: NSMakeRect (0, frame_height - HEIGHT, WIDTH, HEIGHT)
    					styleMask: style
    					backing: NSBackingStoreBuffered
    					defer: NO];
    
        StretchView *view = [[StretchView alloc] initWithFrame: NSMakeRect (0, 0, WIDTH, HEIGHT) ];
        [win setContentView: view];
        [win makeKeyAndOrderFront: win];
        [NSApp run];
    
        [win release];
        [view release];
    
        [pool release];
    
        return 0;
    }
    

    And the compile command I am using is:

    gcc -Wall -framework GLKit -framework Syphon -framework CoreMidi -framework Cocoa -framework OpenGL vade.m -I/usr/local/Cellar/cairo/1.14.8/include/cairo -lcairo -L/usr/local/lib -o vade.app

    For this stripped down version. It pops up a window showing the output (a bunch of circles drawing on top of themselves, normally it has midi driven, but I pulled all that out and just hacked that part together).

    Simple Client connects to this and just shows black.

    #59231

    Rick Burnett
    Participant

    Since it won’t let me edit above, correction, I believe my context is Legacy. I check the GL version and it reports 2.1. It appears I have to tell it I want Core I am guessing through the attributes? Reading some conflicting information on that.

    #59232

    Rick Burnett
    Participant

    Pertaining to being in the right context, I also wrote this to check which does not print out any messages to indicate the contexts got destroyed and recreated, seems to be the same as what I started syphon with:

    if( CGLGetCurrentContext() != [myServer context] ) {
          NSLog(@"Contexts don't match yo!");
        }
    
    #59233

    Rick Burnett
    Participant

    So to continue, I disabled all the Cairo_gl stuff, just commented it out and now my fake texture works properly. So Cairo is doing something to the context and I’m not sure what that is.

    I’ll have to dig into the Cairo code and find out what those functions are doing.

    #59234

    Rick Burnett
    Participant

    Alright, so anyone thinking about using the Cairo_nsgl fork, digging into their code, they are doing things with the textures of the context. No matter what I tried to do, I could not get it to stop altering the texture I was creating for Syphon.

    So instead, given I was going through the process of creating a texture anyways, I changed to using Cairo with an image surface. It writes directly to a buffer that I convert directly in to a GL_TEXTURE_RECTANGLE_EXT and SyphonServer can use that directly.

    Makes the code so much smaller. The maintainer of Cairo_nsgl hasn’t touched it in 4 years so I am not sure it’s even worth investing in at this point.

    So problem solved.

Viewing 8 posts - 1 through 8 (of 8 total)

You must be logged in to reply to this topic.