Visual Storytelling: A Generative Adversarial Networks (GANs) and Graph Embedding Framework

Abstract

Visual storytelling is a powerful educational tool, using image sequences to convey complex ideas and establish emotional connections with the audience. A study at the Chinese University of Hong Kong found that 92.7% of students prefer visual storytelling through animation over text alone [21]. Our approach integrates dual coding and propositional theory to generate visual representations of text, such as graphs and images, thereby enhancing students' memory retention and visualization skills. We use Generative Adversarial Networks (GANs) with graph data to generate images while preserving semantic consistency across objects, encompassing their attributes and relationships. By incorporating graph embedding, which includes node and relation embedding, we further enhance the semantic consistency of the generated high-quality images, improving the effectiveness of visual storytelling in education.