Google has released an open source tool, Autoflip, that can make bad cropping a thing of the past by intelligently reformatting videos to properly adjust alternative aspect ratios.
In a blog post by Google The AI team wrote that footage recorded for television and desktop computers is usually in 1
Autoflip aims to fix this with a framework that uses video stabilizer-like techniques to focus the camera on what is important in the footage. The team from Google wrote that it can use "ML-enabled object recognition and tracking technologies for intelligent understanding of video content", which are based on the MediaPipe framework, to adjust the frame of a video on the fly.
In addition, auto flip is performed automatically. Adapts between scenes by "identifying changes in composition that mean scene changes to isolate scenes for processing," the company said . Finally, each scene is analyzed to determine if a static frame or tracking mode should be used.
This is fairly neat and offers obvious advantages over static cropping of videos, although it's probably better for things like news material and Snapchat videos than movies and TV shows (although it is more important to watch an entire recording).
For a more technical explanation of how it all works, the Google AI team explained the various technologies involved in their blog post. The project's source code can also be viewed on Github along with instructions on how to take it for a spin.