Facebook is one of the leading proponents of 360-degree video. The format has been highly featured on the social networking site and according to Digital Trends, has over 250,000 360-degree videos. Now it has been reported that Facebook is working on an image stabilization software for the format. Since the videos aren’t as widespread as normal content yet, Facebook’s timing is great to get an early start on it. 360-degree video cameras have started to ship out in large numbers like the Nokia Ozo and the bublcam and the Ricoh Theta S and are being aggressively promoted for video content.
Most 360-degree video is filmed poorly. The technology simply hasn’t been experimented with well enough and given it’s nature, shaky cam is particularly abhorrent on it. There are already so many angles through which to look in 360-degree video; adding shaky cam to it just ruins the experience. Facebook’s idea for image stabilization isn’t new but it’s innovative in the sense that it is localized. Companies like Video Stitch and Mettle are already working on this technology but anyone can use their software for their personal videos. Facebook’s software only works if you upload your video on Facebook.
How Does It Work?
It’s fairly simple. The video is sliced up in to bits and pieces so that it can be analysed little by little. Objects are recognized and focused on so that the environment around them can be stabilized. For instance if the central object in the video is a flower or a chair or a building, Facebook centers in on its position in a frame. Then it follows the object throughout the video. Whenever the object changes position due to shaky cam, frames are rotated so that it maintains its position. Other object are positioned accordingly as to not disturb the video as a whole.
The software can differentiate between rotational and translational motion and uses both 2D and 3D motion models. This way, spikes are detected in the motion of objects around centered objects and readjusted.
Facebook has invested a lot of money on data centers and servers to ensure a smooth video watching experience. So far they haven’t been able to catch up to the seamless video playback on YouTube but the latter wasn’t always the greatest at it either.
The technology isn’t out yet and is being tested for a lot of bugs which will no doubt be present even after the release. Video processing is an oft unforgiving chore which gives unsavoury results even with the best of software. You can even find technical faults in feature length Hollywood films like a changing frame rate or a blurry outline. However, Facebook has made sure that processing at least doesn’t eat up too much time. It processes video at 22 milliseconds per frame and there’s only a 10-20% reduction in bit rate.
Stabilizing 360 degree video also makes it easier to store and upload, hence less of a burden on Facebook’s data centers.
A hyperlapse tool is also in the works. Timelapses are pretty popular among photographers and vloggers so an algorithm to speed up video is not a bad idea. But there will have to be a check on video acceleration since it can cause motion sickness for VR headset users.