Handling interlaced video

[nisar.med@gmail.com]

In my App, I am sending frames captured from Blackmagic DeckLink card at 25fps to Quartz Composer using Syphon Framework, since the frame is interlaced, I would like to split a frame into 2 fields and send it from my App using SyphonServer so I am expecting QC to display those fields at 50fps.

As soon as a frame is received inside my App, I split it into 2 fields and send these fields in a loop but I think that QC is displaying only one field or dropping the other field.

My question is, is it ok to send frames like this using SyponServer and receive without dropping any fames inside QC using SyphonQC plugin?

if ([syServer hasClients]) {

for(int i=0;i<2;i++) {
CVOpenGLTextureRef texture = NULL;
CVPixelBufferRef field = [self getFieldFromFrame:frame offset:i];
NSInteger width = CVPixelBufferGetWidth(field);
NSInteger height = CVPixelBufferGetHeight(field);

CVOpenGLTextureCacheCreateTextureFromImage(NULL, textureCache,
field, 0, &texture);

[syServer publishFrameTexture:CVOpenGLTextureGetName(texture)
textureTarget:CVOpenGLTextureGetTarget(texture)
imageRegion:NSMakeRect(0, 0, width, height)
textureDimensions:NSMakeSize(width, height)
flipped:YES];

CVBufferRelease(texture);

}
}

[bangnoise]

A Syphon server presents a single live surface, and you shouldn’t think of it as a series of frames. If, as you do here, you draw twice into the server then the second draw is likely to replace the first one before a client has seen it.

If you have to split the fields inside your app, then you’d be best drawing them in to a Syphon server side by side and splitting them again in QC – or if you can draw the entire interlaced frame then do that and perform the entire split inside QC.

That make sense?

[vade]

Alternatively, you have to put in a delay to simulate the next field drawing.

Interlaced video the fields are drawn at framerate * 2.0 intervals – so if your framerate is 25hz, you need to wait 1/50th of a second before you send the second field to Syphon.

What Tom said is true, Syphon isn’t a series of frames, but a single image that is shared. You could update that image 10000nd times and if the other application is waiting, or lazily drawing, you’ll only see the last frame. We don’t keep a ‘queue’ of frames for many reason (performance and memory costs being one), and so theres only one surface.

Try the delay, it might be more natural, but you might also miss a frame or two depending on phasing of QC’s and your applications frame rates.

• This reply was modified 6 years, 7 months ago by vade.

[nisar.med@gmail.com]

Thanks I have understood the limitation here.

vade, I was thinking the same an it has fixed the issue to some extent but like you said it would be impossible to sync QC and the Field producing App, the time will overlap and field order will keep on changing.

I would be great if Syphon could attach some info with the published frame so that the receiver can perform some action based on it.

Thanks again

[bangnoise]

Per-frame metadata would not be in sync with the surface (if you query metadata it may be out of date by the time you come to use the surface). I appreciate that it makes some things more complicated, but it also makes Syphon very fast and means apps can’t screw each other up by holding on to (or worse, crashing in) shared locks.

[vade]

Why not just de-interlace the video?

[nisar.med@gmail.com]

I tried Deinterlace macro patch from Kineme. http://kineme.net/composition/smokris/Deinterlace

The results are not as good as splitting, scaling and merging, specially when you have a video stream which displays a text crawling from left of the screen to right.

[nisar.med@gmail.com]

I forgot to tell that in another application the QCRender is feeding frames to another DeckLink card which is set to interlaced output so if my output was a computer monitor then deinterlacing would have worked well, but in my case its a broadcast vision mixer or video monitor.

[Author]

Posts