| 引用本文： |
-
梁骁,黄文明,姚俊,温雅媛,邓珍荣.结合多注意力和条件变分自编码器的宋词生成模型[J].广西科学,2022,29(2):308-315. [点击复制]
- LIANG Xiao,HUANG Wenming,YAO Jun,WEN Yayuan,DENG Zhenrong.Song Ci Generation Model Based on Multi-attention and Conditional Variational Auto-encoder[J].Guangxi Sciences,2022,29(2):308-315. [点击复制]
|
|
| 本文已被：浏览 476次 下载 604次 |
码上扫一扫！ |
| 结合多注意力和条件变分自编码器的宋词生成模型 |
|
梁骁1, 黄文明1,2, 姚俊3, 温雅媛4, 邓珍荣1,2
|
|
|
| (1.桂林电子科技大学计算机与信息安全学院, 广西桂林 541004;2.广西图形图像与智能处理重点实验室, 广西桂林 541004;3.广西壮族自治区高级人民法院, 广西南宁 530000;4.广西师范大学电子工程学院, 广西桂林 541004) |
|
| 摘要: |
| 现有的诗词生成方法忽略了风格的重要性。另外，由于宋词大部分词牌词句较多，逐句生成宋词的过程中容易产生上下文缺乏连贯性的现象，在上下文连贯性方面仍存在提升空间。针对这两个问题，在编码解码的文本生成框架基础上，引入自注意力机制的句子表示算法计算多注意力权重矩阵，用于提取词句的多种重要语义特征，让模型更多地关注上文的显著信息来提高上下文连贯性。引入条件变分自编码器(CVAE)将每条宋词数据转化为隐空间中不同风格特征的高维高斯分布，从各自的分布中采样隐变量来控制宋词的风格。由于自构建的宋词语料库缺少完整风格分类标签，使用具有风格标签的宋词微调BERT模型，将其作为风格分类器标注全部的宋词数据。在上述关键技术的基础上实现了宋词生成模型，生成上下文连贯的婉约词以及豪放词。通过与其他4种基准方法进行对比实验，结果表明引入自注意力机制的句子表示算法和条件变分自编码器，在上下文连贯性和风格控制方面有一定的提升。 |
| 关键词: 条件变分自编码器 宋词风格 宋词生成 Bi-GRU 自注意力机制 |
| DOI：10.13656/j.cnki.gxkx.20220526.011 |
| 投稿时间：2021-04-16 |
| 基金项目:广西科技计划项目(桂科AB20238013)，广西图像图形与智能处理重点实验室培育基地(桂林电子科技大学)开放基金项目(GIIP2011)和广西高校中青年教师科研基础能力提升项目(2019KY0238)资助。 |
|
| Song Ci Generation Model Based on Multi-attention and Conditional Variational Auto-encoder |
|
LIANG Xiao1, HUANG Wenming1,2, YAO Jun3, WEN Yayuan4, DENG Zhenrong1,2
|
| (1.College of Computer Science&Information Security, Guilin University of Electronic Technology, Guilin, Guangxi, 541004, China;2.Guangxi Key Laboratory of Image and Graphic, Intelligent Processing, Guilin, Guangxi, 541004, China;3.Guangxi Zhuang Autonomous Region Higher People Court, Nanning, Guangxi, 530000, China;4.College of Electronic Engineering, Guangxi Normal University, Guilin, Guangxi, 541004, China) |
| Abstract: |
| The existing poetry generation methods ignore the importance of style.In addition,since there are the large number of Cipai and phrases in most of Song Ci,it is easy to produce the phenomenon that the context lacks coherence in the process of generating Song Ci sentence by sentence,and there is still space for improvement in context coherence.To solve these two problems,based on the encoding and decoding text generation framework,a sentence representation algorithm based on self-attention mechanism is introduced to calculate multi-attention weight matrix,which is used to extract a variety of important semantic features of words and sentences,so that the model can pay more attention to the significant information above to improve context coherence.The conditional variational self-encoder is introduced to transform each Song Ci data into a high-dimensional Gaussian distribution with different style features in the hidden space,and the hidden variables are sampled from the respective distributions to control the style of Song Ci.Since the self-constructed corpus of Song Ci lacks a complete style classification label,a fine-tuning BERT model of Song Ci with style label is used as a style classifier to label all the data of Song Ci.Based on the above key technologies,the generation model of Song Ci is realized,which generates euphemistic poetry and bold poetry with coherent context.Compared with the other four benchmark methods,the results show that the sentence representation algorithm with self-attention mechanism and the conditional variation self-encoder have certain improvements in context coherence and style control. |
| Key words: conditional variational auto-encoder Song Ci style generation of Song Ci Bi-GRU self-attention mechanism |
