skill-video

name: skill-video description: Complete end-to-end video creation, editing, and series generation using ByteDance Seedance models. Use this skill when asked to create a video, a video series, generate storyboards, or stitch together clips with audio (voiceovers, narrations, background music).

Video Creator and Editor Skill

This skill provides the procedural knowledge required to create high-quality videos and video series autonomously. The process involves storyboard creation, prompt engineering, video generation using ByteDance Seedance models, audio generation (TTS and music), and final compositing using ffmpeg.

Workflow Overview

When the user asks to create a video or a series (e.g., "create a series of condor hero"):

1. Storyboard Creation & Structuring

   • Break the overarching narrative into a detailed storyboard.
   • Divide the storyboard into individual scenes.
   • Important Constraint: Each scene MUST have a maximum duration of 12 seconds.
   • For each scene, write out the exact visual description, action, camera movement, and any required voiceover/narration text.

2. Prompt Engineering for Seedance

   • Translate the visual description of each scene into optimized, descriptive prompts suitable for the ByteDance Seedance API.
   • Improve prompts by adding stylistic keywords (e.g., "cinematic lighting", "4k resolution", "hyper-realistic", "dynamic motion").

3. Video Generation

   • Use the scripts/generate_video.py helper to send the optimized prompts to the Seedance endpoint.
   • Wait for the generation to complete and retrieve the individual .mp4 scene files.

4. Audio Generation

   • If narration or voiceover is required, use the scripts/generate_audio.py helper to convert text into speech.
   • If background music is needed, generate or procure the appropriate audio track.

5. Compositing and Assembly

   • Once all visual and audio assets are ready, use the scripts/video_editor.py helper to composite the final video.
   • The editor script utilizes ffmpeg to concatenate the scenes sequentially.
   • It will overlay the narration tracks at the correct timestamps and mix in the background music (with appropriate audio ducking or volume adjustment).

Bundled Resources

This skill includes several helper scripts in the scripts/ directory to facilitate execution. You must use these scripts to interact with APIs and process media.

scripts/generate_video.py

Handles communication with the ByteDance Seedance endpoint to generate video clips from text prompts.

Usage:

python scripts/generate_video.py --prompt "Cinematic shot of a hero standing on a mountain peak, epic lighting, 4k" --duration 12 --output scene_1.mp4

scripts/generate_audio.py

Handles generating text-to-speech (TTS) voiceovers and background music.

Usage:

# Generate voiceover
python scripts/generate_audio.py --type tts --text "In a world of martial arts..." --output narration_1.mp3

# Generate background music
python scripts/generate_audio.py --type music --prompt "Epic orchestral battle music" --output bgm.mp3

scripts/video_editor.py

Handles joining video clips and mixing in audio tracks using ffmpeg.

Usage:

# Join multiple clips
python scripts/video_editor.py --action join --clips scene_1.mp4,scene_2.mp4,scene_3.mp4 --output final_visuals.mp4

# Add audio to video
python scripts/video_editor.py --action add_audio --video final_visuals.mp4 --audio narration_1.mp3 --bgm bgm.mp3 --output final_master.mp4

Setup & Environment Variables

Before using the helper scripts, ensure that the environment variables are correctly loaded. The required keys are detailed in .env.example:

• SEEDANCE_API_KEY
• SEEDANCE_ENDPOINT
• (Optional) ELEVENLABS_API_KEY for TTS
• (Optional) SUNO_API_KEY for music generation

Ensure you remind the user to populate their .env file if API calls fail due to unauthorized errors.