Giving Photographs a New Life 📷, Controlling Video Games with Your Face 🎮, and Automatic Lip Syncing your Next YouTube Hit 💋
This Week in Creative Technology
Using First Order Motion to Turn a Roman Bust into a Moving Mouth
Here’s an example of the First Order Motion Model (not the one from Star Wars), a model that takes a source image and a video and applies the face movements from the video onto the source image. There are really cool results you can get by using source images that don’t look entirely like faces, for example this half a head Roman bust.
Creating 3D Photography by using Photo Inpainting
This machine learning algorithm has been catching the AI world by storm recently. It takes a source image and runs a depth algorithm on it, extracting depth information from the image. Our eyes do this naturally, so it’s cool to see computers getting this ability as well. Watch out Harry Potter Portraits, here we come!
First Order Motion Mona Lisa
Another phenomenal example of the First Motion Model, Emily Xie created a web app that has a Mona Lisa that tracks your face!
Realtime Face Movement Tracking
This repo allows you to convert head movements into keyboard commands using only 100 lines of Python! Really great potential applications for multiplayer games where there’s only one keyboard between multiple players.
GPT-3 Explorer
GPT-3 examples have been making their rounds around Twitter lately, but many who have used it have had trouble with saving their responses and having an easy way to share the prompts they create. The GPT-3 Explorer addresses these issues beautifully! Here is an example of a shared run. You can see the prompt typed, the completion returned, and the parameters used.
If you have a GPT-3 API key, give the Explorer a shot!
Automatic Lip Syncing
This past week, I was looking for a way to automatically lip sync a 3D character from a text file and WAV audio file. It turns out, carykh had the same idea! His implementation is fascinating, and it goes to show how automation applied correctly can save a ton of time on tedious tasks like animation.
That’s All Folks!
This is a new format I’m trying for the newsletter, so I’d love to hear your thoughts about this vs. my regular essay format. You can email me directly at: bram@sharingfiction.com.
Thanks for reading!