Learn how to write effective text prompts to generate video clips that align with your vision.
With Generate video (beta), you can turn basic text and image prompts into visually compelling videos that express creative intent, fill gaps in your timeline, or add elements to an existing shot. Here are some suggestions to get the most from Generate video (beta) available on the Adobe Firefly website.
Be clear and descriptive
When writing your text prompt in Generate video (beta), use as many words as possible to be specific about lighting, cinematography, color grade, mood, and aesthetic style. A well-structured, concise prompt helps Firefly understand what kind of video you want to create.
Whether you’re describing a scene, action, or setting, avoid ambiguity and ensure your language is simple enough for Firefly to understand your intent. The structure of your text prompt should follow – Shot Type Description + Character + Action + Location + Aesthetic.
- Shot Type Description: What is the camera perspective? How is it moving? For example - "a close-up shot with a slow zoom-in."
- Character Description: Who is the character? What do they look like? What are they wearing? What’s their emotion? For example - "a large polar bear with bright white fur looking pensive."
- Action: What is the character doing in the scene? For example - "the polar bear is walking softly but confidently toward a hole it has previously opened in the ice to hunt beneath the surface of the ice."
- Location: Where is the character? What’s the weather? What’s the terrain? For example - "the location is barren and snowy; gray clouds are moving slowly in the distance."
- Aesthetic: What type of shot is it? What’s the ambience? What’s the depth of field? For example - "cinematic, 35mm film, highly detailed, shallow depth of field, bokeh."
More than four subjects can often confuse Firefly, so it's recommended that you limit the number of subjects when building your prompt.
Specify the visual style
It’s important to be specific on the visual tone and style of the video, such as the type of look you want to generate. For example, do you want a realistic, cinematic, animated, or surreal? The more specific you can be about the aesthetic, the closer Firefly can get to your desired result.
An example prompt– "Create an animated, high-contrast video of a snow mobile racing through mountains at dusk."
Some visual style ideas you can try:
- Cinematic: Think dramatic lighting, lens flares, high contrast.
- Realistic: Aim for natural lighting, clear details, and lifelike characters.
- Animated: Use terms like cartoonish, 3D animation, or 2D animated.
- Artistic: Use adjectives like impressionistic, minimalist, or abstract to guide the art direction.
Define actions clearly
If you want your video to involve movement or action with one or more of your characters, describe these actions with specific verbs and adverbs. This helps Firefly understand the pacing, rhythm, and flow of the action you want. It's recommended you use dynamic verbs such as running, flying, swimming or dancing, and include pacing such as slowly, quickly, or gradually.
An example prompt– "A dog sprints gleefully across the beach and catches a ball in the air."
Use descriptive adjectives
Getting the correct atmosphere of the video is crucial when writing an effective prompt. Be specific about what you want the overall atmosphere to be. For example, do you want the video to feel calming, mysterious, or energetic? If you use very descriptive adjectives that evoke the feeling you want your video to convey, Firefly can generate the most accurate output.
An example prompt– "A peaceful, misty morning on the beach, with soft sunlight filtering through a beach chair."
Provide context or background story
For more complex video projects, you may want Firefly to incorporate specific plot elements or characters. Providing background or context helps Firefly generate a coherent and engaging sequence. For example, you can pick still frames from clips you are using in Premiere Pro and use those stills to direct new action from existing characters or actors. Using uploaded still frames or images can create character continuity between generations to help you tell a longer, more detailed narrative.
Use camera angles and movements
Firefly can often simulate real-world camera work, so help it out by being specific to what angles or movements you want the camera to make for a very personal touch.
Here are some common camera directions:
To highlight specific details or expressions, a close-up shot is often used to allow the audience to see subtle expressions or convey emotion. When a close-up is used on animals, Firefly will showcase animal fur in great detail.
For example– "Cinematic closeup and detailed portrait of a golden retriever dog in a field of sunflowers at golden hour. The lighting is cinematic, gorgeous, and soft, with beautiful, strong backlight and lens flare. The color grade is warm, sun-drenched, and sunlit. The dog is extremely realistic with detailed fur texture. The movement is subtle and soft. The camera doesn't move. There is heavy film grain and textures."
To establish location or scale, a long or establishing shot gives viewers a clear understanding of the location, time, and context in which the action is taking place to help orient the audience.
To give a sense of motion or focus to a subject, Firefly can create some unique camera movements if you tell it to.
Aeriel shots are a great way to give a broader context of location, helping viewers understand the scale of an area, such as a city or landscape.
A low-angle shot is often used to create specific effects or convey a particular emotion. For example, a low-angle shot makes the subject appear larger and more dominant, which implies power. This type of shot is often used in heroic stories.
Include temporal elements (Time of day, weather)
Lighting conditions change, as well as weather throughout the day. To affect the mood of your video, be specific about these temporal elements if they are important to your story. Foggy, cloudy, thunderstorms or bright sunny days are good examples.
Use iterative prompting
Firefly works best through trial and error. Start with a basic prompt and refine it with more details in each iteration.
For example, try the following prompts:
Prompt - "Full scene, eye level, wide shot, a giant mech in the street, high quality, high details."
Prompt - "Full scene, eye level, wide shot, a giant mech, yellow and orange armor, in desolate streets, wires, LEDs, cybernetic parts, high quality, high details."
Prompt - "Full scene, eye level, wide shot, a giant black and yellow mech, accents of orange neon LEDs, marching, firing lasers, scanning, in the street of a destroyed city, rubble, fires, decayed buildings, desolate, ominous, high quality, great details."
Prompt - "Cinematic action scene, a group of giant mechs is invading the city, they are menacing, giant black and yellow mechs, yellow and orange matte armor, a dystopian future, in the street of a destroyed city, rubble, fires, decayed buildings, desolate, ominous, high quality, high details, volumetric lighting."
Add elements to existing videos
Firefly can also generate several interesting editorial use cases that can help enhance your creativity when editing.
Here are some of the examples:
- Create transition masks that can be used in Premiere Pro to transition between two different clips. These masks control which parts of the screen show the outgoing clip and which show the incoming one. This can lead to unique and creative effects beyond the standard transitions like cross-dissolves or wipes.
- Create elements, such as animations, graphics, effects, or objects, on green or black backgrounds to composite in Premiere Pro or After Effects.
- Create overlays, like light flares, particle effects, or fire, on a black background that can be blended into footage in Premiere Pro.
Additional prompt considerations
Prompt length
There aren't hard limits on how long or short a prompt should be, but Firefly has a maximum word limit of 175 words. Long prompts don't always yield better results.
Language style
Vivid details and concrete language produce more predictable results. Using poetic or abstract wording can produce surprising results so it’s often a good idea to mix and match depending on your objectives. As you become more experienced, you can play with the structure, length, and style of your video prompts.
Additional descriptors
Consider adding additional descriptors to your prompts for even greater control over generated video content:
- Cinematography terms such as shallow depth of field, shot on film, and cinematic.
- Composition terms such as wide, close up, shot from above, and shot from below.
- Lighting terms such as backlight, soft light, and hard light.
- Color grading terms such as saturated colors, high contrast, and warm or cool tones.
- Mood or tone terms such as somber, tense, euphoric, and mysterious.
- Camera movement terms such as pan right or left, tilt up or down, and dolly in or out.