Seth Hunter's PHD thesis "WaaZam! : supporting creative play at a distance in customized video environments" goal was "to design an environment that facilitates more engaging shared experiences for geographically separated adults and children placing them together in fantastic worlds".
Seth contacted me in 2014 to implement an openframeworks based software in the form of an addon and examples that could form the basis for WaaZam!
ofxGstRTP used openframeworks along with gStreamer to allow sending video, audio, 3d data from kinect or similar depth cameras and additional data like osc in sync.
The video on the right shows a short conversation between the two of us between Barcelona and Boston where video, depth and audio were sent in sync, creating a shared 3d space between geographically separated locations.
Additionally some of the examples I developed based on Seth's ideas sent osc so the two parties could for example draw in a shared space.
Sending osc in sync with video, audio and depth allowed for many other interactive possibilities between users who are geographically separated following Seth's thesis.
The depth stream not only allowed for point cloud style visualizations but also for the straightforward implementation of effects like green screening and similar which gave even more possibilities for the implementation of games and other interactive experiences across geographically separated locations.
As part of the software I developed specific compression algorithms for the depth and osc streams.
Using external libraries the addon allowed noise, echo cancellation and NAT hole punching allowing two parties to start a conversation without any special configuration on the network or router just by using an account on a service that supported the XMPP protocol like gmail hangouts or opensource alternatives.