publications

2023

1. ACL Findings

   Generate then Select: Open-ended Visual Question Answering Guided by World Knowledge

   Xingyu Fu, Sheng Zhang, Gukyeong Kwon, and 10 more authors

   In Findings of the Association for Computational Linguistics: ACL 2023, Jul 2023

   Abs

   The open-ended Visual Question Answering (VQA) task requires AI models to jointly reason over visual and natural language inputs using world knowledge. Recently, pre-trained Language Models (PLM) such as GPT-3 have been applied to the task and shown to be powerful world knowledge sources. However, these methods suffer from low knowledge coverage caused by PLM bias – the tendency to generate certain tokens over other tokens regardless of prompt changes, and high dependency on the PLM quality – only models using GPT-3 can achieve the best result. To address the aforementioned challenges, we propose RASO: a new VQA pipeline that deploys a generate-then-select strategy guided by world knowledge for the first time. Rather than following the de facto standard to train a multi-modal model that directly generates the VQA answer, pasted macro ‘MODEL’name first adopts PLM to generate all the possible answers, and then trains a lightweight answer selection model for the correct answer. As proved in our analysis, RASO expands the knowledge coverage from in-domain training data by a large margin. We provide extensive experimentation and show the effectiveness of our pipeline by advancing the state-of-the-art by 4.1% on OK-VQA, without additional computation cost.

2. ACL Findings

   Benchmarking Diverse-Modal Entity Linking with Generative Models

   Sijia Wang, Alexander Hanbo Li, Henghui Zhu, and 9 more authors

   In Findings of the Association for Computational Linguistics: ACL 2023, Jul 2023

   Abs

   Entities can be expressed in diverse formats, such as texts, images, or column names and cell values in tables. While existing entity linking (EL) models work well on per modality configuration, such as text-only EL, visual grounding or schema linking, it is more challenging to design a unified model for diverse modality configurations. To bring various modality configurations together, we constructed a benchmark for diverse-modal EL (DMEL) from existing EL datasets, covering all three modalities including text, image and table. To approach the DMEL task, we proposed a generative diverse-modal model (GDMM) following a multimodal-encoder-decoder paradigm. Pre-training GDMM with rich corpora builds a solid foundation for DMEL without storing the entire KB for inference. Fine-tuning GDMM builds a stronger DMEL baseline, outperforming state-of-the-art task-specific EL models by 8.51 F1 score on average. Additionally, extensive error analyses are conducted to highlight the challenge of DMEL, facilitating future researches on this task.

3. Arxiv

   UNITE: A Unified Benchmark for Text-to-SQL Evaluation

   Wuwei Lan, Zhiguo Wang, Anuj Chauhan, and 8 more authors

   arXiv preprint arXiv:2305.16265, Jul 2023
4. ICLR

   Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness

   Shuaichen Chang, Jun Wang, Mingwen Dong, and 13 more authors

   In The Eleventh International Conference on Learning Representations , Jul 2023
5. ACL

   Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning

   Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, and 9 more authors

   In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2023

   Abs

   In this paper, we present a novel approach for data-to-text generation that addresses the limitations of current methods that primarily focus on specific types of structured data. Our proposed method aims to improve performance in multi-task training, zero-shot and few-shot scenarios by providing a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and meaning representations. We demonstrate that our proposed approach can effectively adapt to new structured forms, and can improve performance in comparison to current methods. For example, our method resulted in a 66% improvement in zero-shot BLEU scores when transferring models trained on table inputs to a knowledge graph dataset. Our proposed method is an important step towards a more general data-to-text generation framework.

6. ACL Findings

   Importance of Synthesizing High-quality Data for Text-to-SQL Parsing

   Yiqun Hu, Yiyun Zhao, Jiarong Jiang, and 15 more authors

   In Findings of the Association for Computational Linguistics: ACL 2023, Jul 2023

   Abs

   There has been increasing interest in synthesizing data to improve downstream text-to-SQL tasks. In this paper, we examined the existing synthesized datasets and discovered that state-of-the-art text-to-SQL algorithms did not further improve on popular benchmarks when trained with augmented synthetic data. We observed three shortcomings: illogical synthetic SQL queries from independent column sampling, arbitrary table joins, and language gaps between the synthesized SQL and natural language question (NLQ) pair. To address these issues, we propose a novel synthesis framework that imposes strong typing constraints, incorporates key relationships from schema, and conducts schema-distance-weighted column sampling. We also adopt an intermediate representation (IR) for the SQL-to-text task to further improve the quality of the generated NLQ. When existing powerful text-to-SQL parsers are pretrained on our high-quality synthesized data, these models have significant accuracy boosts and achieve new state-of-the-art performance on Spider. We also demonstrate the effectiveness of our techniques with ablation studies

7. ICLR

   DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases

   Donghan Yu, Sheng Zhang, Patrick Ng, and 7 more authors

   In The Eleventh International Conference on Learning Representations , Jul 2023

2022

1. EMNLP Industry

   Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding

   Jun Wang, Patrick Ng, Alexander Hanbo Li, and 5 more authors

   In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track, Dec 2022

   Abs

   Most recent research on Text-to-SQL semantic parsing relies on either parser itself or simple heuristic based approach to understand natural language query (NLQ). When synthesizing a SQL query, there is no explicit semantic information of NLQ available to the parser which leads to undesirable generalization performance. In addition, without lexical-level fine-grained query understanding, linking between query and database can only rely on fuzzy string match which leads to suboptimal performance in real applications. In view of this, in this paper we present a general-purpose, modular neural semantic parsing framework that is based on token-level fine-grained query understanding. Our framework consists of three modules: named entity recognizer (NER), neural entity linker (NEL) and neural semantic parser (NSP). By jointly modeling query and database, NER model analyzes user intents and identifies entities in the query. NEL model links typed entities to schema and cell values in database. Parser model leverages available semantic information and linking results and synthesizes tree-structured SQL queries based on dynamically generated grammar. Experiments on SQUALL, a newly released semantic parsing dataset, show that we can achieve 56.8% execution accuracy on WikiTableQuestions (WTQ) test set, which outperforms the state-of-the-art model by 2.7%.

2. AAAI

   Generation-focused table-based intermediate pre-training for free-form question answering

   Peng Shi, Patrick Ng, Feng Nan, and 8 more authors

   In Proceedings of the AAAI Conference on Artificial Intelligence, Dec 2022

2021

1. EMNLP

   Learning to Selectively Learn for Weakly-supervised Paraphrase Generation

   Kaize Ding, Dingcheng Li, Alexander Hanbo Li, and 4 more authors

   arXiv preprint arXiv:2109.12457, Dec 2021
2. EMNLP

   Contextual Rephrase Detection for Reducing Friction in Dialogue Systems

   Zhuoyi Wang, Saurabh Gupta, Jie Hao, and 4 more authors

   In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Dec 2021
3. ACL

   Dual reader-parser on hybrid textual and tabular evidence for open domain question answering

   Alexander Hanbo Li, Patrick Ng, Peng Xu, and 3 more authors

   arXiv preprint arXiv:2108.02866, Dec 2021

2020

1. AAAI

   Learning contextual representations for semantic parsing with generation-augmented pre-training

   Peng Shi, Patrick Ng, Zhiguo Wang, and 5 more authors

   arXiv preprint arXiv:2012.10309, Dec 2020
2. Arxiv

   Decomposed adversarial learned inference

   Alexander Hanbo Li, Yaqing Wang, Changyou Chen, and 1 more author

   arXiv preprint arXiv:2004.10267, Dec 2020
3. ICASSP

   Semi-supervised learning for text classification by layer partitioning

   Alexander Hanbo Li, and Abhinav Sethy

   In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Dec 2020
4. AISTATS

   Censored quantile regression forest

   Alexander Hanbo Li, and Jelena Bradic

   In International Conference on Artificial Intelligence and Statistics, Dec 2020

2019

1. Arxiv

   Knowledge enhanced attention for robust natural language inference

   Alexander Hanbo Li, and Abhinav Sethy

   arXiv preprint arXiv:1909.00102, Dec 2019

2018

1. JASA

   Boosting in the presence of outliers: adaptive classification with nonconvex loss functions

   Alexander Hanbo Li, and Jelena Bradic

   Journal of the American Statistical Association, Dec 2018

2017

1. ICML

   Forest-type regression with general losses and robust forest

   Alexander Hanbo Li, and Andrew Martin

   In International Conference on Machine Learning, Dec 2017