生成短视屏

Much prior work has studied generative modeling of video data using a variety of methods,including recurrent networks,generative adversarial networks,autoregressive transformers,and diffusion models.These works often focus on a narrow category of visual data,on shorter videos,or on videos of a fixed size.Sora is a generalist model of visual data—it can generate videos and images spanning diverse durations,aspect ratios and resolutions,up to a full minute of high definition video.以往的许多研究通过使用各种方法对视频数据进行生成建模，包括循环网络、生成对抗网络、自回归变换器和扩散模型。这些工作通常关注于视觉数据的狭窄类别、较短视频或固定大小的视频。Sora是一个视觉数据的通用模型——它可以生成持续时间、纵横比和分辨率各异的视频和图像，长达一分钟的高清视频。[heading2]Turning visual data into patches

If you want your video to involve movement or action with one or more of your characters,describe these actions with specific verbs and adverbs.This helps Firefly understand the pacing,rhythm,and flow of the action you want.It's recommended you use dynamic verbs such as running,flying,swimming or dancing,and include pacing such as slowly,quickly,or gradually.如果您希望视频涉及一个或多个角色的动作或动作，请使用特定的动词和副词来描述这些动作。这有助于Firefly了解所需操作的节奏、节奏和流程。建议您使用动态动词，例如跑步、飞行、游泳或跳舞，并包括节奏，例如慢、快或逐渐。An example prompt–"A dog sprints gleefully across the beach and catches a ball in the air."“一只狗兴高采烈地冲过海滩，在空中接住了一个球。[heading2]Use descriptive adjectives使用描述性形容词[content]Getting the correct atmosphere of the video is crucial when writing an effective prompt.Be specific about what you want the overall atmosphere to be.For example,do you want the video to feel calming,mysterious,or energetic?If you use very descriptive adjectives that evoke the feeling you want your video to convey,Firefly can generate the most accurate output.在编写有效的提示时，获得正确的视频氛围至关重要。具体说明您希望整体氛围是什么。例如，您希望视频感觉平静、神秘还是充满活力？如果您使用描述性很强的形容词来唤起您希望视频传达的感觉，Firefly可以生成最准确的输出。An example prompt–"A peaceful,misty morning on the beach,with soft sunlight filtering through a beach chair."“海滩上一个宁静、薄雾缭绕的早晨，柔和的阳光透过沙滩椅洒进来。

Training text-to-video generation systems requires a large amount of videos with corresponding text captions.We apply the re-captioning technique introduced in DALL·E 3 to videos.We first train a highly descriptive captioner model and then use it to produce text captions for all videos in our training set.We find that training on highly descriptive video captions improves text fidelity as well as the overall quality of videos.训练文本到视频生成系统需要大量带有相应文本标题的视频。我们应用在DALL·E 3中介绍的重新标题技术到视频。我们首先训练一个高度描述性的标题生成模型，然后用它为我们训练集中的所有视频生成文本标题。我们发现，在高度描述性的视频标题上训练改善了文本的忠实度以及视频的整体质量。Similar to DALL·E 3,we also leverage GPT to turn short user prompts into longer detailed captions that are sent to the video model.This enables Sora to generate high quality videos that accurately follow user prompts.类似于DALL·E 3，我们还利用GPT将简短的用户提示转换为更长的详细标题，然后发送给视频模型。这使得Sora能够生成高质量的视频，准确地遵循用户提示。