Building an AI Avatar Video Pipeline with HeyGen and ElevenLabs

From Static Image to AI Avatar Video

The process starts with a static image of a character. HeyGen converts the image into a photo avatar and generates a video where the character can move and gesture naturally while delivering the provided text. This makes the result fundamentally different from a simple talking-head animation. The generated video is built around the full character, allowing body movement and gestures to become part of the presentation. The application exposes these capabilities through a REST API instead of requiring each external service to be called manually.

Voice Cloning and Text-to-Speech

ElevenLabs handles the voice side of the pipeline. A voice can be cloned from an uploaded recording and then used to synthesize speech from arbitrary text. Long audio recordings can exceed upload limits, so FFmpeg is used to split the source recording into smaller chunks before sending them to ElevenLabs. The application validates the input and then stores the resulting voice identifier for use during video generation. This keeps voice generation independent from the avatar generation process and makes it possible to control the character's voice separately from the visual appearance.

Avatar and Motion Pipeline

The video generation flow is divided into several API operations. First, the source image is uploaded to HeyGen and converted into an asset. A photo avatar group is then created from the uploaded image. The avatar can optionally receive a motion prompt that influences how the character behaves during the generated video. Once the avatar and voice are ready, ElevenLabs generates the speech audio. That audio is uploaded to HeyGen and used together with the avatar to create the final video. The application then checks the generation status until HeyGen provides the completed video URL. This separation keeps each stage focused on a single responsibility while allowing the complete process to be controlled through the Node.js API.

Multi-Photo Video Generation

The pipeline can also be extended to videos containing multiple photos or scenes. Each image can be processed as a separate avatar video segment with its own text. After the individual videos are generated, FFmpeg can combine the segments into a single video file. This makes it possible to build a sequence of different scenes while keeping the avatar generation process independent for each one.

Instagram Publishing

Once the video has been generated, the API can publish it directly to Instagram as a Reel through the Instagram Graph API. The publishing endpoint receives the publicly accessible video URL and caption, sends the content to Instagram, and returns the resulting post identifier. This turns the workflow into a complete content pipeline: starting with a static image and text and ending with a published social media video.

API Architecture

The application is built with Node.js and Express. HTTP handling, validation, external API clients, and media processing are separated into dedicated layers. HeyGen and ElevenLabs integrations are encapsulated in their own services, while the provider layer isolates video-generation logic from the rest of the application. FFmpeg is used for local media processing, particularly for audio chunking and combining multiple generated video segments. This structure keeps the integrations isolated and makes the pipeline easier to extend as new capabilities are added.

Conclusion

The interesting part of this system is not a single AI API. It is the orchestration between several specialized services. HeyGen handles the visual avatar and video generation, ElevenLabs provides voice cloning and speech synthesis, FFmpeg handles media processing, and the Instagram API handles publishing. Together, these components turn a static character image and a text script into a complete AI-generated video that can be published automatically.

The source code is available on GitHub:

ai-avatar-video-publisher