Ad Headline Generation using Self-Critical Masked Language Model
July 07, 2026
Abstract
For any E-commerce website it is a nontrivial problem to build enduring advertisements that attract shoppers. It is hard to pass the creative quality bar of the website, especially at a large scale. We thus propose a programmatic solution to generate product advertising headlines using retail content. We propose a state of the art application of Reinforcement Learning (RL) Policy gradient methods on Transformer [1] based Masked Language Models [2]. Our method creates the advertising headline by jointly conditioning on multiple products that a seller wishes to advertise. We demonstrate that our method outperforms existing Transformer and LSTM + RL methods in overlap metrics and quality audits. We also show that our model-generated headlines outperform human submitted headlines in terms of both grammar and creative quality as determined by audits.
1 Introduction↩︎
There are a various types of ads. A set of example ads that showcase products selected by sellers along with headlines that advertise them are shown in Figure 1. Sellers create multiple ad campaigns for multiple products, bid in an auction to advertise and pay for clicks on the ad.
Figure 1: Examples of different product ads from multiple websites across the internet. A variety of ad headlines accompany the products in these ads..
An E-Commerce product catalog may have millions of products which can be advertised. To ease the ad headline writing process, humans resort to programmatically padding keywords, or repasting the retail catalog content in the advertisement.
Templated creatives such as “Save Now on ..." or”Buy more (product) of (brand)" save the creative effort but fail to create any excitement or brand identity in the minds of shoppers. High quality headlines are more attractive to shoppers and offer better value proposition. In this paper, we describe how we built a Natural Language Generation (NLG) system to generate instantaneous, attractive and brand identity building headlines for advertisements that intend to promote a wide range of products offered by a brand.
The content associated with a retail product has challenging characteristics. Some product titles have poor structure, grammatical issues, or partial phrases. The product titles also include varying number of product features such as “Hyper Tough 18V Cordless Drill, 3/8 inch Chuck, Variable Speed, with 1.2Ah Nickel Cadmium Battery, Charger, Bit Holder LED Light" along with titles such as “ZIPIT Grillz Backpack, Camo Grey".
The generated headlines need to capture the information present in the retail attributes and at the same time be different and uniquely attractive. Advertisers select multiple related products that are advertised as part of a single ad campaign. The ad campaign headline is then shared across all of these related products. Thus, the headline also needs to generalize the shared characteristics of the products and cannot be specific to a single product within the campaign.
The key contributions of our work are:
We use Masked Language Model (MLM) for the generation of advertisement headlines using multiple products at the same time. Extensive test-set metrics, quality and grammar audits show that the proposed model outperforms all the baselines and the human-submitted headlines in terms of quality and grammar.
The novel usage of RL for the training of MLM allows us to directly optimize the MLM for improved headline quality metrics without changing inference setup or latency. Our method can also be applied to any other NLG task such as summarization, translation etc.
Our model reduces the extensive effort and time that is required to manually create headlines and has low latency.
2 Related Work↩︎
Natural Language Understanding (NLU) using Language Models (LM) has observed great leaps in recent years. LMs have evolved from using word level models [3] to to a variety of extensions to the Transformer [1]. The BERT [2] employs Transformer in a pre-training setting and introduced the MLM training objective.
[4] first demonstrated textual generation by using auto-regressive prediction in a seq2seq architecture. Transformer based auto-regressive methods such as GPT2 [5] and BART [6] which predict one word at a time have also shown good results. [7] concatenated BERT representations with the Encoder and Decoder layers of another LM to incorporate pre-trained LM. Another model [8] combines BERT-based Transformer Encoder with attention masking from the Transformer decoder. [9] combined pre-trained BERT Encoder with GPT decoder for NLG.
[10] framed NLG as an RL problem and the generation quality as a reward. The Self-Critical Sequence Training (SCST) approach [11] replaces the learned baseline from other approaches [12] with the model’s own inference time algorithm to normalize the rewards.
For advertising, recent works [13], [14] have combined LSTM based pointer network [15] with RL methods to generate advertisement headlines. While these methods improve the results, they fail to utilize extensive pre-training of Transformer based models and their various well-demonstrated advantages.
Our method extends BERT based generation [8] by using Self-Critical policy gradient method [11] and jointly conditioning the generated sentence on multiple products at the same time. This allows us to use pre-trained BERT based LMs that can be trained to optimize various inference time metrics that are typically non-differentiable such as BLEU, Rouge, Readability etc.
3 Self-Critical Masked Language Model↩︎
3.1 Masked Language Model↩︎
The BERT model takes an unlabeled input sequence \(x = (x_1,x_2,...,x_{|x|})\) and randomly masks some positions \(M_x\) by replacing them with a special mask token \(\textrm{[MASK]}\), to produce a sequence like \((x_1,\textrm{[MASK]},...,x_{|x|})\). All the tokens are embedded and added to special positional embeddings. It then uses \(N\) identical Transformer layers to generate contextualized representation, with each layer employing self-attention by taking in the output of the previous layer. To compute self-attention, the output of the previous layer is projected into triplets of vectors named Query, Key and Value (\(Q, K, V\)) of dimensions \(d\). The attention \(A\) is then given as:
\[\label{eq:mlm95attention} A=\textrm{softmax}(\frac{QK^T}{\sqrt{d}})V\tag{1}\]
After the final Transformer layer the model uses a feed forward layer followed by a softmax over the vocabulary to predict the masked tokens. The MLM loss for the sequence \(x\) is then calculated as:
\[\label{eq:mlm95loglikelihood} \mathcal{L}_{MLM} = - \log \prod_{m \in M_x} p(x_m| (x_{m'} \in x \setminus M_x))\tag{2}\]
where \((x_{m'} \in x \setminus M_x)\) represents all the tokens in \(x\) that are not masked and \(m \in M_x\) are all the masked positions.
3.2 Encoding multiple products and common headline for Proposed MLM↩︎
Figure 2: The sub-tokens from the product titles and headline are embedded and added with other embeddings that encode the positional and segment information. We also optionally add an embedding that represents the category of the product. During training, the masked tokens are predicted using Transformer layers and the cross-entropy (Eq. 2 ) loss and Self-Critical (Eq. 9 ) gradient is used to optimize the model. During inference, we predict one word at a time (left-to-right) in an auto-regressive manner using Beam Search..
During training, for a given advertising campaign, our model takes as input it’s headline \(x^h = (x^h_1,...,x^h_{|x^h|})\) and a set \(P\) of one or more products. Each product \(p\) is represented by its title \(x^p = (x^p_1,...,x^p_{|x^p|})\). The titles and the headline are tokenized to sub-word tokens.
To encode using the model that only accepts a single product, we simply append ‘[EOS]’ \(\in \mathbb{V}\) to both the title and the headline and concatenate their tokens. The entire concatenated sequence is
prepended with ‘[SOS]’ \(\in \mathbb{V}\).
We encode multiple products by concatenating the tokens from different products using a special token ‘[P‘_SEP]’ \(\in \mathbb{V}\). We replace a token ‘[UNUSED_0]’ \(\in \mathbb{V}\) that remains unused during pre-training, with this special token during multi-product fine-tuning. This makes a distinction between different titles as well as the source and target sub-sequences. It also
yields individual embeddings for each product for other tasks.
Only the tokens from the headline \(x^h\) are randomly masked with token ‘[MASK]’ \(\in \mathbb{V}\). We discuss results for the model that additionally also masks the source
tokens in section 5.1.
The complete process for an example such that all products in the ad have two tokens and the headline has 4 tokens is illustrated in Figure 2.
We also experimented with adding of category based embeddings. The category labels for each product such as “Cell Phones and Accessories" are tokenized to subword units, encoded using the same embedding matrix as that of the title tokens, averaged and added to the title token embeddings.
3.3 Generation using Self-Critical Masked Language Model↩︎
The BERT MLM framework with multi-directional attention discussed in Section 3.1 cannot be used for auto-regressive generation directly. This is because, during training, the masked headline words may condition on the future words which are not available during auto-regressive inference. For MLM auto-regressive generation, we employ masked attention [8] that modifies the attention from equation 1 as below:
\[\label{eq:mlm95masked95attention} A_{\textit{masked}}=\textrm{softmax}(\frac{QK^T}{\sqrt{d}} + \Phi_{ij})V\tag{3}\]
where \(\Phi_{ij}\) represents the attention mask between the positions \(i\) and \(j\). The elements are set to 0 if attention is allowed and \(- \infty\) if it is not allowed. Figure 3 illustrates the attention mask for headline generation using multiple input products.
Figure 3: Masked attention partially restricts attention for some token pairs. It prevents attention to headline tokens that would not be accessible during each step of generation during inference..
The BERT MLM uses log-likelihood (Equation 2 ) of masked words during training to optimize the model parameters. The likelihood is predicted using other ground-truth words during training and other predicted words during inference. This causes exposure bias [10], [11] and accumulates error during inference. Moreover, the training is optimized for log-likelihood, while we actually care about other more evolved measures of headline quality such as overlap metrics BLEU [16] and ROUGE [17].
To overcome these issues and improve the quality of the generated headlines, we frame the MLM as an RL problem. The model is an ‘agent’ that takes the ‘action’ of predicting masked words and updates the ‘state’ such as the self-attention weights. The MLM follows a policy \(\pi_\theta\) defined by the parameters \(\theta\) of the model. It receives a reward that is proportional to the quality of the generated headline. This quality may either be the overlap with ground truth headlines that have been approved by internal subject-matter-experts or be predicted by another model. Our goal is to maximize the reward corresponding to a generated headline \(\hat{x}^h\) during training, with the tokens at some masked positions \(M_{x^h}\) sampled from the model.
We thus minimize the negative expected reward defined by any reward function \(r(\cdot)\) for headline quality \(r(\hat{x}^h)\) as:
\[\label{eq:rl95reward95loss} \mathcal{L}_{RL} = - \mathbb{E}_{\hat{x}^h \sim \pi_\theta} [r(\hat{x}^h)]\tag{4}\]
We can compute the gradient \(\nabla_\theta \mathcal{L}_{RL}\) using the REINFORCE algorithm [18]. It is defined as:
\[\label{eq:rl95reward95reinforce95loss} \nabla_\theta \mathcal{L}_{RL} = - \mathbb{E}_{\hat{x}^h \sim \pi_\theta} [r(\hat{x}^h) \nabla_\theta P]\tag{5}\]
where, \[\label{eq:rl95probability} P = \sum_{m \in M_{\hat{x}^h}} \log p_\theta(\hat{x}^h_m| (\hat{x}^h_{m'} \in \hat{x}^h \setminus M_{\hat{x}^h})\tag{6}\]
such that \(M_{\hat{x}^h}\) are the masked positions and \(\hat{x}^h \setminus M_{\hat{x}^h}\) are all the unmasked tokens.
To reduce the variance without changing the expected gradient, the algorithm proposes to use a baseline \(b\) that does not depend on the generated headline \(\hat{x}^h\). \(b\) is used to normalize the reward along with \(P\) from equation 6 as:
\[\label{eq:rl95reward95reinforce95baseline95loss} \nabla_\theta \mathcal{L}_{RL} = - \mathbb{E}_{r(\hat{x}^h) \sim \pi_\theta} [(r(\hat{x}^h)-b) \nabla_\theta P]\tag{7}\]
A single Monte-Carlo sample for each set of products and headline can be used to approximate the gradient. Using the definition of \(P\) from equation 6 , we have the approximate gradient:
\[\label{eq:rl95reward95reinforce95baseline95loss95approx} \nabla_\theta \mathcal{L}_{RL} \approx - (r(\hat{x}^h)-b) \nabla_\theta P\tag{8}\]
Instead of using other models to estimate the expected baseline reward [10], [12], we employ Self-Critical training [11] that involves generating two headlines using the same underlying MLM. The first headline \(\hat{x}^h\) is generated by sampling from the vocabulary distributions generated by the model for the masked tokens. The second headline \(\hat{z}^h\) is generated using the inference time strategy, which uses the token with the maximum probability at each step rather than sampling. The difference in the reward achieved by these two headlines is used to compute the gradient:
\[\label{eq:rl95reward95scst95loss95approx} \nabla_\theta \mathcal{L}_{SC\_MLM} \approx - (r(\hat{x}^h)- r(\hat{z}^h)) \nabla_\theta P\tag{9}\]
where \(P\) is defined by equation 6 .
Thus, this method maximizes both the reward of the headlines generated by MLM and the likelihood of correct words by incorporating both the likelihood and the reward in the loss function.
3.4 Inference↩︎
During inference, we generate the headline auto-regressively using beam search until we reach the predetermined max length or each beam generates the end token. We have employed a modified version of Length Normalization [19] to better adapt to our headline lengths and training setup. This is necessary as the default beam search setup uses the log probability of each word to select the best headline. However, this biases the results as longer headlines would have lower probability of generation. We thus use the following normalized scores for each word to select the best headline:
\[\label{equation:length95normalization} \textrm{score}(\hat{x}^h_i) = \textrm{log-likelihood}(\hat{x}^h_i) * \frac{(2 + 1)^\alpha}{(2 + i)^\alpha}\tag{10}\]
where \(\alpha\) is the length normalization coefficient and \(\hat{x}^h_i\) is the \(i^{th}\) word of the generated headline in each beam. We also include additional Regular Expression based post-processing to remove extra spaces around various symbols such as ‘-,+()’ etc.
4 Experiments↩︎
4.1 Training and Inference↩︎
We used over 500,000 ad campaigns that were created on Amazon by sellers who have signed-up for advertising. Each campaign contains a set of related products along with an ad headline. We only selected the campaigns that contained English headlines and products with English titles. They were also de-duplicated to only have unique products-headline pairs. The mean product title length is 19.6 words and the mean headline length is 6.16 words. The entire dataset was divided into train (85%), validation (5%) and test (10%) sets. For training, we only selected the campaigns that comply with ad [20] as verified by internal experts.
We use HuggingFace [21] implementation of Transformer BERT ‘Large’ models as the base for our experiments. The models are pre-trained on WikiPedia and BookCorpus [2], [8]. We first fine-tune the pre-trained model for up-to 15 epochs with early stopping using \(\mathcal{L}_{MLM}\) and Adam [22]. We then further fine-tune the model for another 15 epochs with early stopping using Adam with \(\nabla \mathcal{L}_{SC\_MLM}\) (Equation 9 ). We use the Rouge L F1 [17] overlap with the approved headlines as the headline quality reward. For a fair comparison, the MLM-only model is fine-tuned for upto 30 epochs.
The model training is very time expensive with a single fine-tuning sub-experiment of 30 epochs taking over 20 days on an Nvidia v100. We thus only performed the essential experiments that help to determine the contribution of different sub-experiments and proposals. We estimated post-experiment that a single fine-tuning sub-experiment of 30 epochs would consume approximately 150 kWh of energy based on the GPU’s power draw.
4.2 Baseline↩︎
We used a Pointer Network [15] based bi-LSTM with intra-decoder and temporal attention. We also used Self-Critical training with the bi-LSTM, similar to other ad headline generation methods [13], [14] methods for a fair comparison to Self-Critical MLM.
4.3 Ablations↩︎
We trained a model with the same architecture, number of parameters and input as the proposed models but without MLM pre-training and separately without Self-Critical loss to study the impact of the proposals.
We also trained a model with MLM pre-training but fine-tuning only using the primary first product from each campaign instead of using all the products. This is interesting since some of the campaigns are cohesive to a degree with similar products and using only one product improves training time and inference latency.
We also report overlap metrics for model that does not use length normalization and post-processing discussed in equation 10 . We also include results for model that uses BERT Base as the base model instead of BERT Large.
5 Results↩︎
5.1 Overlap with Approved Headlines↩︎
The first evaluation criterion we adopt is overlap [23] of model headlines with subject-matter-experts approved human-submitted headlines from the test set (Table ¿tbl:table:main95results?).
Masking the source product title words reduces the performance as the titles and headlines do not follow the same sentence structure and distribution. Adding product category embedding reduces performance and our hypothesis is that this is because the base model cannot be pre-trained with these embeddings. Only using one title achieves lesser but respectable performance, highlighting the efficacy of multi-product conditioning.
“No pre-training of MLM" highlights the advantage of using non-pretrained Transformer based architecture over bi-LSTM. ‘Proposed MLM’ shows the advantage of using pre-training, BERT Large and only masking the headline. ‘Proposed Self-Critical MLM’ achieves the best scores across all the metrics and highlights the applicability of our proposed approach.
| Model | Rouge-L | CIDEr | BLEU-4 | METEOR | Avg. Cos. Sim. |
|---|---|---|---|---|---|
| Baseline bi-LSTM Pointer Network model | |||||
| bi-LSTM | - | - | - | - | - |
| Self Critical bi-LSTM | 0.62 | 0.01 | 1.06 | 0.42 | -4.31 |
| MLM Baselines and Ablations (Single Product and No Self Critical Training) | |||||
| First Product Only | 2.14 | 0.19 | 5.03 | 3.55 | 0.36 |
| First Product and Category embedding | 1.52 | 0.13 | 4.18 | 2.938 | 0.15 |
| Proposed MLM and Ablations (Multiple Products and No Self Critical Training) | |||||
| Using BERT Base instead of BERT Large | 2.85 | 0.22 | 4.96 | 3.58 | 1.53 |
| No pre-training of MLM (Training from scratch) | 3.38 | 0.27 | 5.72 | 3.79 | -0.04 |
| Additional Source Titles Masking | 4.13 | 0.29 | 4.42 | 5.41 | -2.09 |
| Proposed MLM | 5.08 | 0.42 | 7.49 | 5.46 | 1.31 |
| Proposed Self-Critical MLM (SC-MLM) and Ablation | |||||
| No beam search normalization and post-processing | 5.37 | 0.43 | 7.81 | 5.61 | 1.96 |
| Proposed Self-Critical MLM | 6.33 | 0.55 | 9.11 | 6.14 | 3.75 |
5.2 Quality and Grammar Audits↩︎
| SC-biLSTM | MLM - Single Product | Proposed MLM | Proposed SC-MLM | |
|---|---|---|---|---|
| % improvement in mean rating over human-submitted headlines | ||||
| -9.87% | 0.40% | 1.15% | 2.07% | |
| % improvement in number of headlines | ||||
| Rated \(\geq 2\) out of 3 | -4.99% | 2.75% | 2.42% | 2.37% |
| Rated 3 out of 3 | -42.96% | -0.06% | 1.22% | 6.53% |
4pt
| One of the source product’s title | Human Submitted Headline | Proposed MLM | Proposed SC-MLM |
|---|---|---|---|
| BEST Natural Hair Growth Oil for GUARANTEED Hair Strength, Thickening, Hair Gro... | All Natural Hair care products | Natural Hair Growth & Beard Care Products | Protect Your Hair and Beard With All Natural Oils |
| Royal 310DX Thermal Print Electronic Cash Register | Affordable Reliable Cash Management from Royal | Royal Cash Registers - Retail & Event Supplies | Secure your cash with Royal Cash Registers |
| Blue Copper 5 Anti-Aging Body Lift, Pregnancy Stretch Marks Prevention and Removal Cream 5 Oz | Blue Copper 5 Anti-Aging Products | Discover Osmotics Best Selling Products | Say Goodbye to Stretch Marks |
| Cosy House Collection Twin Size Bed Sheets - Cream Bedding Set - Deep Pocket - Extra Soft Luxury... | Soft & Hypoallergenic Twin Sheets | Luxury Twin Sheets - These Will Change Your Life. | Soft Luxury Sheets - These Will Change Your Life. |
| Carson Dellosa | Valentine’s Day Heart Stickers | 1-inch x 1-inch, 216ct | Share the Love with this Classroom Decor | Valentine’s Day Celebrations | Show your Valentine some love this Valentine’s Day |
| Canon GI-20 PGBK Ink Bottle, Compatible to PIXMA G6020 and G5020 MegaTank Printers | Print more for less with SuperTank printers. Canon | All-in-one solution for professional grade prints. | All-in-one solution for professional grade prints. |
| LABILUS iPhone Xs MAX case, (Rugged Armor Series) TPU Soft Stripe Designed Protective Cover Case... | ° Protection Heavy Duty for iPhone Xs Max | Rugged Protective Case for iPhone Xs MAX | Rugged Armor Protective Case for iPhone Xs Max |
| Biscotti Cookie Gift Basket, Gourmet Gift Basket, Delicious Biscotti Artfully Decorated 18 Count... | Valentines Gifts | Gourmet Chocolate Gift Baskets | Gourmet Holiday Gift Baskets |
| Le Angelique Tapered Curling Iron Wand with Glove And 2 Clips - 3/4 to 1 Inch (18-25mm) Conical ... | Tapered Curling Wands with Glove and 2 Clips | Le Angelique Tapered Curling Iron Wand | Le Angelique Tapered Curling Iron Wand |
| JUNK Brands London Fog-BBL London Fog Big Bang Lite Headband | Headbands for Every Adventure | Headbands for Every Adventure | BBL Headbands for Adventure |
| Jump&Go Portable Car Battery Jump Starter set -16,000mAh, 600A Peak, Mini Automotive Power Boost... | Portable Jump and go Jumpstarter | Jump and Go Portable Car Jump Starter | Jump and Go Portable Jump Starters |
| decanit Silver Metal Thin Edge 5x7 Picture Frames, Silver Thin Profile Photo Frames 5 by 7 Inch,... | Life-Style-Living | Metal Thin Edge Picture Frames | Thin Edge Picture Frames |
| CARSYS Coating Thickness Gauge DPM-816 Extended Range Precision Probe Fe/NFe Paint Meter for Car... | Coating Thickness Gauges | Coating Thickness Gauge - Range Precision | Coating Thickness Gauge |
| Rich & Creamy Buttermilk Syrup Original Flavor by Uncle Bob’s Butter Country 16 fl oz/1 Pack | Fresh and Premium Buttermilk Syrup | Rich and Creamy Buttermilk Syrup | Rich and Creamy Buttermilk Syrup - Taste Great |
| Sesame Street Ernie Face Tee Funny Humor Pun Adult Mens Graphic T-Shirt Apparel (Small), Orange | Sesame Street Tees for Adults | Sesame Street Men’s Shirts | Sesame Street Men’s Shirts |
| Agvee Unbreakable End Tip [3 Pack 6ft] 4A Heavy Duty USB Phone Charger Cable, Durable Charging for iPhone 11 Pro Max X XS XR, i-Phone 10x 10xs ...... | AGVEE Fast Lightning Charging Cable for iPhone | AGVEE Fast iPhone 11 X 10s 10s XR Cable | AGVEE Heavy Duty iPhone 11 Xs XS XR Cable |
We also conducted large scale crowd-sourced evaluation studies of the headlines with over 150,000 judgments. All headlines are shuffled and each headline is rated by 3 random and double-blind crowd-sourced auditors. The quality is judged on a 3-point scale of [1. Incorrect or Irrelevant, 2. Correct, 3. Correct and Attractive] and we use the mode of the 3 judgments.
In this double-blind audit, the auditors were not aware of the source of the headlines and we were not aware of the identity or demographics of any auditor. More details about the workforce may be found in the platform documentation [24]. In order to determine the compensation for the crowd-sourced workers, we used the guideline provided by the crowd-sourcing platform to “choose a price consistent with the approximate time it takes to complete a task" (Visible in the Console while creating the [25] job). We thus first conducted an internal audit by volunteers across our organization to determine the time required to complete the task (average 21.59s) and then used the remuneration recommended for the corresponding time range ($0.12 for 20s - 22s).
Table ¿tbl:table:quality95scores? summarizes the quality audits. The SC-biLSTM model performed worse compared to human-submitted headlines. The proposed SC-MLM model achieves the highest average rating and the most number of perfectly rated headlines. Using just a single product does produce correct headlines with 8% faster inference latency but fails to produce attractive headlines due to lack of input from multiple products.
We also conducted Grammar specific audits (N \(\approx\) 10000) in which the grammar of the headlines is judged independently. 98.13% of SC-MLM and 98.12% of MLM generated headlines were judged to have correct grammar against 93.14% of human submitted headlines.
Table ¿tbl:table:sample95headlines? shows a sample of headlines for campaigns in the blind test-set. Excessive keyword stuffing in source product titles does hamper headline quality at times and post-filtering using beam search score helps to filter them out. We do observe cases where both the models generate the same headline. This is an artifact of the fact that both the models share the first 15 epochs. The SC-MLM model generates more descriptive headlines and both models are able to abstract the product qualities.
6 Conclusion↩︎
Ad headline generation is a difficult problem owing to the varying nature of retail product attributes. A lot of historical methods focus on template based creation of ad headlines that are not very expressive.
We demonstrated a new NLG based method to generate headlines for multiple products. Our method achieves highest score in overlap metrics, quality audits and grammar audits compared to the baselines and human-submitted headlines.
Masked Language Models were relatively unexplored for ad headline generation and we were able to demonstrate their utility. We further extended the performance of the model by using Reinforcement Learning. The method only changes the training procedure without impacting inference latency. Thus, our work contributes to both SOTA and practical business applications.
The approach can also be used for any other NLG task.