Generating Poetry Title Based on Semantic Relevance with Convolutional Neural Network

Several approaches have been proposed to automatically generate Chinese classical poetry (CCP) in the past few years, but automatically generating the title of CCP is still a difficult problem. The difficulties are mainly reflected in two aspects. First, the words used in CCP are very different from modern Chinese words and there are no valid word segmentation tools. Second, the semantic relevance of characters in CCP not only exists in one sentence but also exists between the same positions of adjacent sentences, which is hard to grasp by the traditional text summarization models. In this paper, we propose an encoder-decoder model for generating the title of CCP. Our model encoder is a convolutional neural network (CNN) with two kinds of filters. To capture the commonly used words in one sentence, one kind of filters covers two characters horizontally at each step. The other covers two characters vertically at each step and can grasp the semantic relevance of characters between adjacent sentences. Experimental results show that our model is better than several other related models and can capture the semantic relevance of CCP more accurately.


Introduction
CCP is a great treasure of Chinese culture. According to the structural, rhythmical and tonal patterns, CCP can be divided into many genres. Among all the genres, the quatrain perhaps is the most popular one. There are four lines in the quatrain and each line has five or seven characters. An example of a 5char quatrain written by Li Bai, a famous poet in Tang Dynasty of China, is shown in figure 1. In recent years, there are several studies using deep learning to write CCP, for examples, the RNNPG model [1] with three neural networks and the encoder-decoder model [2] with a recurrent neural network (RNN) as the encoder and another RNN as the decoder. Given some keywords, those models can generate poems line by line meeting the structural, rhythmical, tonal constraints. However, it is still difficult to automatically generate the title of CCP. The difficulties mainly come from the differences between the CCP and the modern Chinese articles, which will be described in detail in section 2.
In this paper, we focus on making machines grasp the semantic meaning of the CCP and automatically generate a semantic relevance title. We treat this as an automatic text summarization problem. The research of automatic text summarization is very active in recent years and the encoderdecoder model is widely used recently. Rush, Chopra and Weston [3] first introduce the encoderdecoder model into the automatic text summarization problem and they name their model as ABS. [4] uses a RNN for encoding and attention mechanism is used in the RNN encoder. Many techniques have been proposed based on the encoder-decoder model to improve the summarization result, such as feature-rich encoder [5], minimum risk training [6] and copy mechanism [7]. Our work is based on the encoder-decoder model and we improve the structure of the encoder, which can capture the semantic relevance in poems and is more suitable for the title generation of CCP.
The paper is organized as follows: In section 2, we describe the problem of poem title generation. The details of poem title generation (PTG) model we proposed are introduced in section 3. At last, the evaluation result and conclusion are shown in section 4 and 5 respectively.

Problem description
In this paper, we focus on the task of automatically generating the title of CCP. In this task, each input is a quatrain, which is a famous genre of CCP. We first train word2vec [8] Given the content of a poem X , the model needs to generate the title This task can be taken as an automatic text summarization problem. However, there are several differences between our problem and the summarization problem of modern articles. First, the words in CCP are very different from the words in the modern Chinese and there are no valid word segmentation tools we can use. Just using characters is not enough and we need to make use of the information of words. Second, there are structural, rhythmical, tonal constraints in CCP, so it's better for our model to take advantage of that information. Third, the title of CCP is extremely short (usually less than five characters), so the popular evaluation metrics such as ROUGE [9] are not suitable for this problem. All the differences bring new challenges to the poem title generation problem. For the first and second difficulties, we proposed our poem title generation (PTG) model. We also introduce a new kind of evaluation metric which can reflect the semantic differences between titles to solve the third difficulty. In this section, we first describe the semantic features of CCP, and then the framework for poem title generation is presented. After that, we show the implementation details of our model and how the model can capture the semantic features.

The semantic features of CCP
The coherence is an important feature of CCP. When writing a CCP, the poet pays attention to not only the semantic relevance of characters in one sentence, but also the semantic relevance of characters between adjacent sentences. So the semantic features of CCP are mainly reflected in two aspects, the intra-sentence semantic relevance and the inter-sentence semantic relevance.
The intra-sentence semantic relevance of characters means some characters usually appears together in one sentence and form words. In CCP, the words usually consist of two characters. In figure 1, "moon" and "light" in the first sentence of the quatrain are an example of the intra-sentence semantic relevance, which form the word "moonlight" together.
The inter-sentence semantic relevance of characters is the antithesis used between two adjacent sentences in the poem. If two characters appear in the same position of two adjacent sentences and the meanings of the two characters are related to each other, then there is inter-sentence semantic relevance between the two characters. In the quatrain in figure 1, the first character "raise" of the third sentence and the first character "stoop" of the fourth sentence are an example of the inter-sentence semantic relevance.
There are difficulties for a model to capture the two kinds of semantic features. The words used in the CCP have many differences with the model Chinese words and there are no valid segmentation tools. So our model should learn the information of words directly from the characters. In addition, the inter-sentence semantic relevance is reflected from the characters in the same position of adjacent sentences, so the input form of our model should not simply be a sequence of characters, which usually is the input form of traditional encoder-decoder models. We will show how our model overcomes those difficulties and captures both the two kinds of semantic features of CCP in the below sections. The poem title generation problem is similar to the automatic text summarization problem. Given an input, which is the quatrain in our problem, the PTG model needs to generate an output which represents the main ideas of the input and is the title of the quatrain here. The entire framework for poem title generation is shown in Figure 2. The title of a CCP can be generated through three steps in our PTG model. Step 1, by looking up the word embedding matrix trained by word2vec model, each character in the poem is transformed into a -k dimensional vector.

The framework for poem title generation
Step 2, the model encoder which is a CNN encodes the poem into a fixed length vector, which represents the semantic meaning of the original poem.
Step 3, the model decoder which is a RNN generates the title character by character as the above vector being the context. Our model is similar to the encoder-decoder model raised in [10], but the encoder is tailored CNN for our task. We will describe the details of the CNN encoder in the next section. The encoder is to encode the input into a fixed length vector. Here, each input is a quatrain X shown in equation . The quatrain has four sentences and each sentence consists of five or seven characters, so here m is 4 and n is 5 or 7. Because of the special structure of quatrains, we can take quatrains as squared pictures.  The h-filter covers two adjoining characters in one sentence. After training on the dataset, the hfilter can learn the commonly used words of two characters without word segmentation, so the intrasentence semantic relevance is captured by our model. The v-filter covers two characters in the same position of two adjacent sentences, so it can capture the inter-sentence semantic relevance. When applying the h-filter to row i of X , we can get an intra-sentence feature list denoted by  , which is the representation of the whole quatrain. Our model can capture both the two kinds of semantic features mentioned above, so the vector c can represent the semantic meaning of the corresponding quatrain.

Experiments and results
The training dataset we use is 40047 quatrains with titles. We compare our model with two baseline systems, which are both encoder-decoder models. The first is RNN encoder-decoder model raised in [11]. The second one [4] is similar with the first one, but uses the attention mechanism [12] in the encoder.

Evaluation metrics
The traditional evaluation metrics for automatic text summarization task such as ROUGE can only reflect the surface similarity between the standard title and the generated title. This is not suited for our poem title generation task. As we mentioned before, the title of CCP usually has less than five characters and directly comparing the surface similarity is ineffective. The semantically similar titles may have low scores under the traditional evaluation metric. In order to reflect the semantic similarity between the standard title and the generated title, we proposed two kinds of evaluation metrics. Let  s can reflect the semantic differences between different titles. However, with the fact that the title of CCP is usually very short, directly comparing the title is not very effective, so we propose the second evaluation metric. Instead of comparing the standard titles and generated titles directly, the second evaluation metric compares the titles with the corresponding poems. The title represents the central thought of the poem, so a good title is usually closely related to the poem. Inspired by those ideas, we propose the 2 s calculated as below:     figure 5). This indicates that the proposed PTG model is better in generating semantic relevance titles.

Case study
To demonstrate the effectiveness of our model more specifically, we analyze the different titles generated by different models corresponding to the same quatrain. The quatrain with the standard title is shown in Figure 6. It is a poem that praises the beauty of willow.  The titles generated by different models is shown in table 2. From table 2, we can observe that the title the RNN encoder-decoder model generated is "The spring" and the title the RNN encoderdecoder model with attention mechanism generated is "The early spring". They both focus on the topic "spring". Although the season described in the poem is the spring, but that is not the key idea this poem focuses on. In contrast, the PTG model titled this poem "The willow". Our model gets the main idea of this poem although there is no word "willow" in the content of the poem.

Conclusions
In this paper, we propose an encoder-decoder model for automatically generating the title of CCP. The encoder is a CNN and it is to encode the poem into a fixed length vector. The decoder is a RNN that generates the title character by character as the above vector being the input. We also propose an evaluation metric that is more suitable for evaluating our task. Experimental results show that our model is outperformed than the other related models. The future work of this paper may be trying to combine the attention mechanism with our CNN encoder to make further improvement.