The answer is really “when you see problems” – do you?
The complexity of operations you perform will cause problems before the number of Syphon steps does. Try running each stage in isolation and note resource usage. Computer vision in particular is resource-hungry.
For what it’s worth, Syphon’s main cost is a chunk of video memory for each server – but your problem probably isn’t Syphon, but what you’re doing at each step.