Taiwan

Lifelong language learning seeks to have models continuously learn multiple tasks in a sequential order without suffering from catastrophic forgetting. State-of-the-art approaches rely on sparse experience replay as the primary approach to prevent forgetting. Experience replay usually adopts sampling methods for the memory population; however, the effect of the chosen sampling strategy on model performance has not yet been studied. In this paper, we investigate how relevant the selective memory population is in the lifelong learning process of text classification and question-answering tasks. We found that methods that randomly store a uniform number of samples from the entire data stream lead to high performances, especially for low memory size, which is consistent with computer vision studies.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.20/)

IJCNLP-AACL 2022

How Relevant is Selective Memory Population in Lifelong Language Learning?

lifelong language learning

text classification

continual learning

question answering

stages 

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING 2020

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

poster

The AACL, Asia-Pacific Chapter of the Association for Computational Linguistics, and the AFNLP, Asian Federation of Natural Language Processing, invite you to participate in the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL) and the 10th International Joint Conference on Natural Language Processing (IJCNLP). 
<br>
<br>

A registration is required to access this event. Access to the event site will be granted to all registered attendees on **November 16th**. If you have not already done so, please register using the link below:
<br>
<br>https://conference.iis.sinica.edu.tw/servlet/Register?ConferenceID=411

Registration Is Required

The AACL, Asia-Pacific Chapter of the Association for Computational Linguistics, and the AFNLP, Asian Federation of Natural Language Processing, invite you to participate in the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL) and the 10th International Joint Conference on Natural Language Processing (IJCNLP). 

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.31/)

Performance-Efficiency Trade-Offs in Adapting Language Models to Text Classification Tasks

Existing subword segmenters are either 1) frequency-based without semantics information or 2) neural-based but trained on parallel corpora. To address this, we present BERTSeg, an unsupervised neural subword segmenter for neural machine translation, which utilizes the contextualized semantic embeddings of words from characterBERT and maximizes the generation probability of subword segmentations. Furthermore, we propose a generation probability-based regularization method that enables BERTSeg to produce multiple segmentations for one word to improve the robustness of neural machine translation. Experimental results show that BERTSeg with regularization achieves up to 8 BLEU points improvement in 9 translation directions on ALT, IWSLT15 Vi->En, WMT16 Ro->En, and WMT15 Fi->En datasets compared with BPE. In addition, BERTSeg is efficient, needing up to 5 minutes for training.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.12/)

BERTSeg: BERT Based Unsupervised Subword Segmentation for Neural Machine Translation

Machine translation traditionally refers to translating from a single source language into a single target language. In recent years, the field has moved towards large neural models translating from or into many languages. As the input and output languages vary, the model must be correctly cued to translate into the correct target language. This is typically done by prefixing language tokens onto the source or target sequence. A single token’s content can denote the source language, target language, or language pair. The location and content of the prefix varies and many approaches exist without much justification towards one method or another. As guidance to researchers and directions for future work, we present a series of comprehensive experiments that show how the positioning and type of a target language prefix token affects translation performance. We show that source-side prefixes consistently improve performance. Further, we find that best language token content varies dependent on the supported language set.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.19/)

The Effects of Language Token Prefixing for Multilingual Machine Translation

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.1/)

Transfer Learning for Humor Detection by Twin Masked Yellow Muppets

We build a dual-way neural dictionary to retrieve words given definitions, and produce definitions for queried words. The model learns the two tasks simultaneously and handles unknown words via embeddings. It casts a word or a definition to the same representation space through a shared layer, then generates the other form in a multi-task fashion. Our method achieves promising automatic scores on previous benchmarks without extra resources. Human annotators prefer the model's outputs in both reference-less and reference-based evaluation, indicating its practicality. Analysis suggests that multiple objectives benefit learning.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.2/)

A Unified Model for Reverse Dictionary and Definition Modelling

Previous studies on neural linguistic steganography, except Ueoka et al. (2021), overlook the fact that the sender must detokenize cover texts to avoid arousing the eavesdropper’s suspicion. In this paper, we demonstrate that segmentation
ambiguity indeed causes occasional decoding failures at the receiver’s side. With the nearubiquity of subwords, this problem now affects any language. We propose simple tricks to overcome this problem, which are even applicable to languages without explicit word boundaries.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.15/)

Addressing Segmentation Ambiguity in Neural Linguistic Steganography

The Transformer Language Model is a powerful tool that has been shown to excel at various NLP tasks and has become the de-facto standard solution thanks to its versatility. In this study, we employ pre-trained transformer document embeddings in an Active Learning task to group samples with the same labels in the embedding space on domain-specific corpora. We find that the calculated class embeddings are not close to the respective samples and consequently do not partition the embedding space in a meaningful way. In addition, using the class embeddings as an Active Learning strategy yields reduced results compared to all baselines.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.45/)

Pre-Trained Sentence-BERT does not represent Class Embeddings in Active Learning for Multi-Label Text Classification

Conversational question answering (CQA) facilitates an incremental and interactive understanding of a given context, but building a CQA system is difficult for many domains due to the problem of data scarcity. In this paper, we introduce a novel method to synthesize data for CQA with various question types, including open-ended, closed-ended, and unanswerable questions. We design a different generation flow for each question type and effectively combine them in a single, shared framework. Moreover, we devise a hierarchical answerability classification (hierarchical AC) module that improves quality of the synthetic data while acquiring unanswerable questions.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.22/)

Multi-Type Conversational Question-Answer Generation with Closed-ended and Unanswerable Questions

This paper introduces a first step towards creating the NERDz dataset. A manually annotated dataset of named entities for the Algerian vernacular dialect. The annotations are built on top of a recent extension to the Algerian NArabizi Treebank, comprizing NArabizi sentences with manual transliterations into Arabic and code-switched scripts. NERDz is therefore not only the first dataset of named entities for Algerian, but it also comprises parallel entities written in Latin, Arabic, and code-switched scripts. We present a detailed overview of our annotations, inter-annotator agreement measures, and define two preliminary baselines using a neural sequence labeling approach and an Algerian BERT model. We also make the annotation guidelines and the annotations available for future work.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.13/)

NERDz: A Preliminary Dataset of Named Entities for Algerian

For low-resourced Bangla language, works on detecting emotions on textual data suffer from size and cross-domain adaptability. In our paper, we propose a manually annotated dataset of 22,698 Bangla public comments from social
media sites covering 12 different domains such as Personal, Politics, and Health, labeled for 6 fine-grained emotion categories of the Junto
Emotion Wheel. We invest efforts in the data preparation to 1) preserve the linguistic richness and 2) challenge any classification model. Our experiments to develop a benchmark classification system show that random baselines perform better than neural networks and pre-trained language models as hand-crafted features provide superior performance.

To view the proceedings, please [click here](https://aclanthology.org/2022.aacl-short.17/)

How Relevant is Selective Memory Population in Lifelong Language Learning?

Downloads

Next from IJCNLP-AACL 2022

Performance-Efficiency Trade-Offs in Adapting Language Models to Text Classification Tasks

Similar lecture

Distill and Replay for Continual Language Learning

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES