Unity3d and post effects

[AIResearcher]

We are loving the use of Syphon with Unity3d.

In some of our game levels, we perform post camera effects, or effects after the rendering point at which Syphon (server) grabs the texture.

Is there a good handle or opportunity to grab the texture later in the rendering cycle to include these effects? Or perhaps is there a way to re-render a second cycle in which Syphon can grab the frame?

[vade]

Nice! glad you are finding Unity + Syphon useable and useful 🙂

Brian Chasalow has committed some newer Unity 3D scripts and plugin to SVN. Are you using the public beta 2, or SVN versions of Syphon?

I mention because the new scripts Brian worked on have a slightly different rendering method, and I believe he has fixed some bugs with the Unity implementation.

In theory though, you ought to be able to get a texture reference from your post process pass, and send that to Syphon. I think that ought to be possible with Brians SVN version.

[Brian Chasalow]

this question should actually have an easy solution. unity appears to render in order of script placement on a gameobject. make sure the syphon script is the last (lowest) script attached to the gameobject, and any bloom/glow/etc are above/before it.

[AIResearcher]

Thanks for the insights. I have been using the SVN version and yes it is much much better.

I’m going to try to insert a yield WaitForEndOfFrame() and then create a new Texture2D with Screen.Width and Height, ReadPixels from the screen into the texture, then provide the native texture id to syphonServerPublishTexture.

I’ll let you know how it goes. I’m not sure how long ReadPixels will take, but it looks like it should work.

[AIResearcher]

Well, it works well, grabs after OnPostRender and OnGUI, etc… but unfortunately (as was expected) ReadPixels is wicked slow.

Now just have to figure out how to grab a NativeTextureID somewhere further down the pipeline or figure out how to do second rendering (maybe a Composting Camera…) I’ll post back with further investigations.

[vade]

If you make a new texture, you should not be using glReadPixels, use glCopyTexSubImage2D, which is hardware accelerated and grabs from the current specified glReadBuffer, and copies the screen (or, well, whatever buffer you specify) into the currently bound texture.

That should be plenty fast.

[Brian Chasalow]

if all you’re trying to do is capture the GUI, what if you send the unity screen texture to syphon in OnGUI() and do something like:
if(Event.current.type == EventType.Repaint)syphonServerPublishTexture(blah);
no idea if it’ll work, or even if this relates to your issue.

Sounds like you have some effects you’re using OnPostRender with that are conflicting with syphon rendering order?
I’d take a look at a small unity project in a .zip showing the issue if you want to post one.
Brian

Anton: he’s using Unity’s wrapped Texture2D.ReadPixels, which ain’t quite the same as the direct glReadPixels command. if he wanted to write a plugin he might be able to use glCopyTexSubImage2D and such, but that’s a whole nother can of worms.

[vade]

Ah, that makes sense, I guess. Thanks Brian. I thought you could, maybe, I dunno, do direct GL calls or something. Alas!

[AIResearcher]

Wow, thanks for all the help!

It would be great to have the texture grab in the OnGUI function, just grab everything before and/or after the GUI (depending on the need). Unfortunately, the only thing I am missing is what is in the “blah” statement or “unity screen texture”.

I’m not sure if you guys know what it is, but I’ll be investigating how to get the right NativeTextureID to Syphon and post back here if/when I figure it out.

[Brian Chasalow]

that’s a good question, regarding the GUI pre/post switchable situation. I will think about it as well and see if I can come up with anything.

edit: there’s no easy answer for capturing the OnGUI layer I’m afraid. I will keep at it, and do let me know if you have any luck.

Brian

[AIResearcher]

There is no hook for OnGUI that will give anything like a TextureID. It may be possible to use something to grab it out of the FBO… However!

The next release of Unity (3.4) should include the ability to prioritize scripts. So that if we have one script with a OnImageRender( ) that we want to capture the results from, we can now make sure that it is called before the second script, (The syphon script that includes it’s OnImageRender( )).

I don’t have access to the beta so I haven’t confirmed this, but am eagerly anticipating it’s release to verify that if our Syphon script calls the very last OnImageRender, that we should be able to include everything except for OnGUI stuff. (OnPostRender) doesn’t provide a TextureID and I’m not sure what is traditional done in this function, but I’m hoping it’s just script updating stuff like positions and AI, and not rendering.

[Author]

Posts