如何生成视频音频文案

Lipsync可以为视频配音配口型，生成视频最长可达30s，目前只支持对PixVerse生成的视频进行口型适配。Lipsync的优点是什么？支持多种语言（英语、汉语、法语、日语等等皆可适配）最长可以生成30s多样化音频适配，演讲、音乐、歌剧等等都允许[heading3]使用指南[heading4]上传图片[content]选择一张带有人脸的图片上传，写好提示词，点击生成视频。为保证最佳生成效果，建议使用单人图片。[heading4]口型同步[content]点击生成的视频，在生成的视频下方找到“Lipsync”并点击。之后，您可以输入文案，从右边的预设声音中选择合适的声音，或者点击“Upload Audio”上传一段音频，最后点击“create”生成视频。注意：生成视频的长度取决于您文案或音频的长度，最长为30s。例如，5s视频+3s音频=3s语音视频，5s视频+30s音频=30s语音视频。[heading4]口型同步实例[content]注：以下示例有声音，请打开声音后观看。文案：Ladies and gentlemen,fellow Americans,Thank you for entrusting me once again with the incredible honor of serving as your President.God bless you,and God bless the United States of America.声音：Chloe[pixverse-preview%2Fmp4%2Fmedia%2Fweb%2F86478b9d-ac02-4e3a-8f0d-0250f05aafc1_seed1823532749.mp4](https://bytedance.feishu.cn/space/api/box/stream/download/all/JNrTbr4NCoMFfhxDaT8cSO53nFh?allow_redirect=1)

Our research stands out from existing video-to-audio solutions because it can understand raw pixels and adding a text prompt is optional.与现有的视频音频解决方案相比，我们的研究与众不同，因为它可以理解原始像素，而且可以选择添加文字提示。Also,the system doesn't need manual alignment of the generated sound with the video,which involves tediously adjusting different elements of sounds,visuals and timings.此外，该系统无需手动调整生成的声音和视频，因为手动调整需要对声音、视觉效果和时间等不同元素进行繁琐的调整。[V2A Guitar.mp4](https://bytedance.feishu.cn/space/api/box/stream/download/all/EwA0bUu8qo2iL6xAk4Tc2dqUnvf?allow_redirect=1)[V2A Bike.mp4](https://bytedance.feishu.cn/space/api/box/stream/download/all/QjPCbCFBloIt5lxtFC8cWkKynKh?allow_redirect=1)Still,there are a number of other limitations we’re trying to address and further research is underway.不过，我们还在努力解决其他一些限制因素，进一步的研究正在进行中。Since the quality of the audio output is dependent on the quality of the video input,artifacts or distortions in the video,which are outside the model’s training distribution,can lead to a noticeable drop in audio quality.由于音频输出的质量取决于视频输入的质量，视频中超出模型训练分布范围的假象或失真会导致音频质量明显下降。We’re also improving lip synchronization for videos that involve speech.V2A attempts to generate speech from the input transcripts and synchronize it with characters' lip movements.But the paired video generation model may not be conditioned on transcripts.This creates a mismatch,often resulting in uncanny lip-syncing,as the video model doesn’t generate mouth movements that match the transcript.

Our research stands out from existing video-to-audio solutions because it can understand raw pixels and adding a text prompt is optional.与现有的视频音频解决方案相比，我们的研究与众不同，因为它可以理解原始像素，而且可以选择添加文字提示。Also,the system doesn't need manual alignment of the generated sound with the video,which involves tediously adjusting different elements of sounds,visuals and timings.此外，该系统无需手动调整生成的声音和视频，因为手动调整需要对声音、视觉效果和时间等不同元素进行繁琐的调整。[V2A Guitar.mp4](https://bytedance.feishu.cn/space/api/box/stream/download/all/EwA0bUu8qo2iL6xAk4Tc2dqUnvf?allow_redirect=1)[V2A Bike.mp4](https://bytedance.feishu.cn/space/api/box/stream/download/all/QjPCbCFBloIt5lxtFC8cWkKynKh?allow_redirect=1)Still,there are a number of other limitations we’re trying to address and further research is underway.不过，我们还在努力解决其他一些限制因素，进一步的研究正在进行中。Since the quality of the audio output is dependent on the quality of the video input,artifacts or distortions in the video,which are outside the model’s training distribution,can lead to a noticeable drop in audio quality.由于音频输出的质量取决于视频输入的质量，视频中超出模型训练分布范围的假象或失真会导致音频质量明显下降。We’re also improving lip synchronization for videos that involve speech.V2A attempts to generate speech from the input transcripts and synchronize it with characters' lip movements.But the paired video generation model may not be conditioned on transcripts.This creates a mismatch,often resulting in uncanny lip-syncing,as the video model doesn’t generate mouth movements that match the transcript.