[Originally written on November 2023. ]
Are you looking to create a captivating video for your brand, but don't have a whole lot of budget?
With the rise of AI tools like Runway ML, generating stunning cinematic videos is easier than ever.
In this AI tutorial, I'll walk you through the full process of using Runway ML and other AI tools to make an AI-powered trailer from start to finish. You'll learn how to:
This is a process anyone can replicate and adapt to their own brand, to create eye catching trailers right from their living room.
Let's get started!
The first step is coming up with a compelling trailer story and structure. For this, we can use the new Bing search engine.
Powered by GPT-4 (just like ChatGPT), Bing allows you to have a back and forth conversation to refine your ideas. It can also browse the web to find up to date information on what your competitors are creating, or what's trending.
Simply share the basic premise, characters, and any other creative direction.
For example, we prompted Bing to help conceive a hero story related to our example brand, HubSpot. Here's the initial prompt we used:
Bing will provide various story and character suggestions. Refine them to create something unique to your brand.
These are some of the first generic concepts I got:
Then I utilized other prompts to refine the story like: "I'd like it to be inspired by the 1984 Apple Ad". And I got a more specific response.
This AI-assisted brainstorming ensures you have an exciting narrative structure before moving to the visuals.
With the story fleshed out, the next step is breaking it down shot-by-shot. This helps visualize the trailer before spending time on generating images and videos.
I prompted Bing to provide a detailed shot list for the story, including scene descriptions and actions. It grouped the shot list into a chart, which was key for organization.
Here's an example shot list Bing generated for our HubSpot trailer:
Now we can start bringing the storyboard to life with AI-generated images.
Midjourney is an excellent image generation tool for this.
The best part of creating a shot list? All you have to do is feed your shot descriptions from the storyboard into Midjourney prompts. For example:
"A dystopian city skyline establishing wide shot"
Midjourney will create initial concept art for each scene. You may need to iteratively refine prompts to get the right mood and style.
Here's some tips for consistent, high-quality images:
The seed is like the starting point for an image. If you use the same seed in your prompts, the images will end up looking very similar, which is key to creating consistent videos.
To extract the seed from an image, react to the generation you want using the envelope image in discord.
This will prompt the bot to give you a seed number (like in the image below). Then copy that seed number and use it in your next prompt with --seed.
Another way to ensure consistency in images is to create variants from a source image and utilizing the remix feature in Midjourney.
For this, you need to enable remix mode by prompting /prefer and selecting "Remix" mode.
From now on, every time you click on the Variation buttons for a generation, you can add an additional prompt to guide the newer generation.
For example, below I had a different shots of engraved pens, and by creating the "V2" button, we were able to add additional details to the background.
As I generated images from the shot list, I downloaded them into my computer.
Then, using an editing software like Premiere Pro, I imported the images and arranged in the timeline. It was helpful to have the shot list as a reference, so I could sequence the images properly.
Then I added transitions between the images, music and voice over, and started to see it come together.
Generating video takes way longer than generating images. So it was important to ensure the images worked well together before moving into animation.
But before we get there, let's talk about voice-overs.
To make the story come to life, I wanted an epic voiceover narration.
Now my voice is not nearly as epic as the story demanded, so I had to look for a different solution. The text-to-speech capability in Eleven Labs was my choice.
Using Eleven Labs is super easy. I listened through the different voice models that they offer, and picked the one I liked the most, which was optimized for deep, narrative audios.
I then played around with some of the fine tuning parameters, and copied the script that Bing had put together for me.
I clicked "Generate" and done! I got a voice over that was really good quality.
I added it to the Premiere timeline and used the Essential Sound panel to make the voice feel warmer and less robotic. I also cut the audio to adjust the pacing and make it flow better with the music.
And that's how we create the voice over and have the full boardomatic!
Now let's make the images come to life using Runway ML. Their Gen-2 AI platform generates cinematic videos based on source images or text.
For this example, I used their Image to Video option. I uploaded the Midjourney images into Gen-2, and played with some of the motion parameters they have available.
If you have a specific camera motion in mind, you can specify it with the camera controls. Otherwise, just move the general motion slider to test out different outputs.
I also added simple description prompts to give the tool more context about the image and what I wanted to see in the video.
One caveat: getting a winning shot right off the bat is not easy. I had to test multiple versions of each shot and save the best takes. I tweaked the amount of motion and the text descriptions until I was happy with the result.
This step is the most time-intensive, so be selective about the shots to animate.
One way to save time was to batch generate the content. This means I had multiple shots or images being generated at the same time, which could help cut down the waiting time between generations.
Finally, I brought all of the generated videos from Runway into the Premiere Pro timeline, replacing the previous images from Midjourney. I also refined the transitions between scenes to achieve a more polished look.
And that's it! We've got a complete AI-generated movie trailer!
By combining the creative capabilities of tools like Bing, Midjourney, Runway ML, and Eleven Labs, anyone can produce studio-quality trailers. This gives you the power to showcase your brand story like never before.
Follow this tutorial, and you'll have the AI assistant, creative assets, animation, and voiceover to craft an epic vision. Let us know if you have any other questions! We hope this guide inspired you to start your AI movie trailer journey today.
We hope these tips help you get started making AI trailers! Let us know if you have any other questions.
Feel free to share your epic trailers with us once they're ready!