Frame rate issues in a client with Unity Syphon server

Home Forums Syphon Syphon Implementations – User Frame rate issues in a client with Unity Syphon server

Viewing 6 posts - 21 through 26 (of 26 total)
  • Author
    Posts
  • #4925
    WeivCo
    Participant

    Sorry to add another message here. Above I say that I attach OpenGL Profiler to “our app” but that is misleading. I meant to say that I attach it to the Simple Client and our own custom client which also runs as a separate process.

    #4926
    vade
    Keymaster

    glFlushRendererApple != CGLFlushRenderer. Its not just for single buffered outputs, but when FBOs and other buffers are rendered to, or when *secondary OpenGL Threads update the contents of buffers*, flushing needs to happen so resources can be synced. IOSurface is one such instance where an additional flush is needed to synchronize a shared resource.

    Look at “Using glFlush Effectively” section

    The CGLFlushDrawable issue applies only during a single application, on a single window you own. You can totally call CGLFlushDrawable on multiple windows at multiple times, more than once, during a period between refreshes.

    You also have to understand that GL profiler “hides” much complexity of the programmable pipeline and “collapses” GLSL shader programs, ARB programs, etc into some calls it lists in the states. So when you see draw call usage for CGLFlushDrawable, or GLFlush go “up”, its most likely due to either fill rate issues, or due to using the programmable pipeline. Notice there is no higher GPU usage or rates for shader calls, even though unity and many apps use them. Much stuff gets “rolled” into other calls, making identifying some of the performance issues more nuanced than relying solely on GL Profiler and trusting its output.

    Secondly, IOSurfaceLock/Unlock is never used in Syphon because we never hit the CPU. Those calls are, as you guessed, only for when a surface is changed in main memory, and needs to be synchronized, and re-flushed to the GPU. The locking is so that any GPU based app cannot read into the texture while it is being updated from another app, potentially causing issues, incomplete reads, etc.
    
    I think you are over-thinking this issue to be honest. This sounds suspiciously like a fill rate issue, in that you simply can’t draw that much on screen. What size frames are you sending back and forth? I think its 2400x 600? (triple head 800×600 size no?)

    Understand that, because you are using Unity’s render buffer, and syphons “publish frame image” api, and drawing to the screen in unity, and then drawing in another app, you end up drawing 2400×600 a few times.

    1) Render Unity Scene to a texture
    2) Render resulting unity texture to Syphons IOSurface via publishFrame Image
    3) Unity renders the resulting texture in the Unity app.
    4) App using Syphon renders the Syphon texture.

    Due to the lack of CPU spikes, VRam usage, etc, this seriously looks like a fill rate limitation to me.

    You might find that you get better performance if you simple do a glCopyTexSubImage2D on the front buffer and the end of the unity rendering, getting *everything* rendered at that point into a texture. You can also cheat this way, and copy that directly into SyphonServers texture. This eliminates 3 render passes (Rendering the Unity Scene to its own internal texture, Unity Rendering that Texture, and rendering that texture to Syphons internal texture), and adds a gl copy / blit pass, and of course Unity still renders the scene once, rather than rendering it to texture, then rendering that texture (both which use up available fill rate).

    This is what we do in the yet to be finalized/fixed for 10.7 Screen Capture app for getting the entire scene, look at :

    SyphonScreenCaptureAppDelegate.m line 404 (thats amusing…)

    #4927
    WeivCo
    Participant

    Thanks for the info vade. You are truly an expert in this area. đŸ™‚

    With regards to the high res, my latest round of testing for the last couple days has been with 1024×768, but I’ve noticed this issue the entire time in 640×480 resolution as well. And like I also mentioned before, it is rather odd, but the effect pretty much goes away when we set Unity’s timeScale to less than 1.

    FWIW, I just built a custom version of Simple Client with NSOpenGLCPSwapInterval off and it is noticeably better displaying this Weiv scene. I still get occasional jitters, but qualitatively speaking it is definitely less noticeable. Now time spent in CGLFlushDrawable averages 200-300 microseconds much more consistently as opposed to regularly jumping up to 3000 or much higher like it did before during the “grinding.”

    Lastly, as a control, I enlarged the Simple Server to ~800×600 or so and looked at the vsync-ed Simple Client. I can notice some occasional jitter from that, too, although it’s subtle. It looks more like our scene through the non-sync-ed Simple Client.

    And thanks for the final suggestion. I’ll look into it and talk to my partner John (you would know him by AIResearcher here). Maybe I can get that going and see what kind of effect it has on our performance. We do need to display the texture in our “live view” in Unity, but the more duplication we can cut out, the better.

    #4928
    Brian Chasalow
    Participant

    got a msg from Aras @ Unity that may help in future dev so i thought I’d share the relevant bits:

    “Each “window” in the editor has it’s own GL context. At Update() time, probably the current context is the one that got repainted last. It could be scene view, game view, inspector, etc. depending on layout & stuff. If however you’d hook your plugin into something like OnPreRender() of a camera, then the current context should always be game view.”

    Not sure if this is useful to Unity Syphon yet.

    #4929
    vade
    Keymaster

    What does timescale control in unity?

    Are you ensuring vertical sync for your unity rendering? Does unity render at 60hz ? What may be happening is that the additional overhead of rendering the texture to syphon is pushing it over a 1/60th refresh, causing apparent dropped frames.

    You may want to throw a timer in there and look at how long a frame takes to render with and without a client attached, and adjust the timescale programmatically to compensate?

    #4930
    vade
    Keymaster

    Also, both Simple Server and Client render on the main thread with, so you can’t really rely on that for looking at hiccups and determining the cause. Main thread / runloop issues will interrupt rendering for both (just hold a menu down). It.. Is “simple” for a reason – just a fairly straight forward implementation.

    Try loading a composition up with QCPlayer.app (Developer example that uses CVDisplayLink to drive rendering), you should see much fewer hiccups.

Viewing 6 posts - 21 through 26 (of 26 total)
  • You must be logged in to reply to this topic.