Recently Lush’s video production team was asked to create a promotional video for a soon-to-be-released smartphone application.

The video was to demonstrate the app’s various features and show how to use it, but the problem was that the app was still in development and wouldn’t be ready before we needed to film the video.

That meant we had to film our talent using a smartphone and then insert the app onto the phone’s screen later on, in post-production.

So, how did we do it?

If you’re adding things in post-production, planning is key

To create realistic interaction with a smartphone application that isn’t actually there requires some advance planning. Before filming, we made a plan of which functions of the application we would be demonstrating, what gestures would be needed to interact with the application, and where the various buttons would be on the screen. Although the application wasn’t yet finished, the developers did have some mock-ups of how each function worked, which helped a lot.

Since we knew we would need to replace the contents of the phone’s screen in post-production, we were able to place some tracking markers onto the screen of the phone before filming. Having tracking markers makes the process of tracking the screen much easier and more accurate.

During filming, we shot multiple takes of the actors performing each interaction with the application, having them perform the interaction in different ways so that we could choose the one which best matched the application’s animation.

Tracking the phones and inserting the image

In post-production, we used Mocha Pro by BorisFX to track the movement of the phone in each scene. Mocha Pro is a planar tracker, which doesn’t just track the position of the tracked item, but also its changes in rotation, scale, shear, and perspective. Once tracking was complete, we used the corner-pin effect in Adobe After Effects to insert the application into the correct place on the screen.

Then we used Mocha Pro again for rotoscoping the actor’s fingers, so their fingers remained on top of the new phone screen rather than behind it. This was by far the most time-consuming process.

Creating the application

The developers of the application weren’t able to send us a working version of the application to insert into the video, but they were able to send us a few screenshots. Our solution was to use those screenshots to create a close copy of the application from scratch inside of Adobe After Effects, which we could animate and then use that to insert into the phone’s screen.

 Using this method, we were able to exactly match the animation of the application’s interface with the gestures made by the actor, to help sell the effect of them interacting with the app.

If you need tricky special effects and post-production for your next video, get in touch with the team at Lush. We have a very experienced and talented team of video editors who go the extra mile to achieve fantastic results.