[論文レビュー] SketchDynamics: Exploring Free-Form Sketches for Dynamic Intent Expression in Animation Generation

Boyu Li, Lin-Ping Yuan+2 more|arXiv (Cornell University)|Jan 28, 2026

Human Motion and Animation被引用数 0

ひとこと要約

tldr: SketchDynamics introduces free-form sketch storyboards as dynamic intent prompts for AI-driven animation, with an adaptive clarification interface and iterative editing, validated through a three-stage user study.

ABSTRACT

Sketching provides an intuitive way to convey dynamic intent in animation authoring (i.e., how elements change over time and space), making it a natural medium for automatic content creation. Yet existing approaches often constrain sketches to fixed command tokens or predefined visual forms, overlooking their freeform nature and the central role of humans in shaping intention. To address this, we introduce an interaction paradigm where users convey dynamic intent to a vision-language model via free-form sketching, instantiated here in a sketch storyboard to motion graphics workflow. We implement an interface and improve it through a three-stage study with 24 participants. The study shows how sketches convey motion with minimal input, how their inherent ambiguity requires users to be involved for clarification, and how sketches can visually guide video refinement. Our findings reveal the potential of sketch and AI interaction to bridge the gap between intention and outcome, and demonstrate its applicability to 3D animation and video generation.

研究の動機と目的

• objective: ["Investigate how free-form sketches can convey dynamic intent for animation beyond fixed commands.", "Develop SketchDynamics to interpret sketches via a vision–language model and render vector animations from storyboards.", "Evaluate the workflow through a three-stage user study to assess interpretability, ambiguity handling, and refinement.", "Demonstrate applicability to explainer-style motion graphics and potential extension to 3D animation and video generation."]

提案手法

• method: ["Implement a unified web interface with sketch input, storyboard sequencing, and a video rendering view.", "Use prompts that pair sketch descriptions with executable code to guide generation.", "Render output using executable Manim (Python) code to produce scalable vector animations.", "Introduce an adaptive clarification cue that categorizes sketch ambiguity into four levels and enables user intervention.", "In Stage 3, develop a frame-based interactive refinement approach combining keyframe extraction and annotation for precise control.", "Conduct a three-stage study with 24 total attempts to assess expressiveness, interpretation, and refinement."]],
• research_questions
• key_findings
• table_headers
• table_rows

実験結果

リサーチクエスチョン

• RQ1 research_questions: ["How effectively can free-form sketches convey dynamic animation intent to a vision–language model?", "What types of ambiguity arise when interpreting free-form sketches, and how can clarification interventions help?", "Does an adaptive clarification guide improve alignment between user intent and generated animations?", "Can users meaningfully refine outputs through on-canvas or frame-based editing to close gaps between intent and result."]

主な発見

• key_findings: ["Free-form sketches can express animation intent with minimal input, though interpretation is often underspecified due to ambiguity.", "Clarification cues help users engage in the interpretation process and reduce misinterpretation through adaptive prompts.", "Some intents remain under-specified and require viewing output and refinement to become concrete.", "An iterative visual editing approach, including frame-based refinement, enables efficient, low-effort user guidance toward the desired result.", "The study demonstrates the potential of sketch–AI interaction to bridge intent and outcome and indicates applicability to 3D animation and video generation.", "Across 24 attempts, 5 outputs were judged as failures, highlighting current limitations in sketch-based intent expression."]