Using the front camera and tensorflow.js, the smartphone becomes a “window” into the real world. Video and image content appear as if they were seen through this window. To do this, the viewer’s position is determined using a neural network. The viewed content is then moved according to the viewer’s position. This makes it seem like the content is physically behind the smartphone and is viewed through the smartphone’s screen. This effect is especially useful for content captured using an ultra-wide lens.
