From 2023.12.21

1. Retrieval-Augmentation LLMs 22

• ACL 2023: Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
• TACL 2023: In-Context Retrieval-Augmented Language Models Retrieval Stride Retrieval Query Length Reranking
• NeurIPS 2023 (poster): Retrieval-Augmented Multiple Instance Learning RAM-MIL
• EMNLP 2023 (findings): SAIL: Search-Augmented Instruction Learning
• EMNLP 2023 (main): Active Retrieval Augmented Generation
• EMNLP 2023 (main): Query Rewriting for Retrieval-Augmented Large Language Models Frozen LLM and Retriever Trainable Rewriting Model
• ICLR 2024 (oral): Self-RAG: Self-reflective Retrieval Augmented Generation Framework
• ICLR 2024 (spotlight): BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models
• ICLR 2024 (poster): Making Retrieval-Augmented Language Models Robust to Irrelevant Context Robust
• ICLR 2024 (poster): RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective Augmentation Context Compress Selective Augmentation
• ICLR 2024 (poster): RA-DIT: Retrieval-Augmented Dual Instruction Tuning
• ICLR 2024 (poster): RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
• ICLR 2024 (reject): Understanding Retrieval Augmentation for Long-Form Question Answering Attribution
• ICLR 2024 (reject): Accelerating Retrieval-augmented Language Model Serving with Speculation Acceleration Framework Iterative RALMs
• NAACL 2024: Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity Trainable Classifier to determine the complex of queries. Three strategies.
• arXiv preprint. 2023.: REPLUG: Retrieval-Augmented Black-Box Language Models
• arXiv preprint. 2023.: RETA-LLM: A Retrieval-Augmented Large Language Model Toolkit
• arXiv preprint. 2024.: RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation iteratively revise CoT with RAG
• arXiv preprint. 2024.: Corrective Retrieval Augmented Generation Trainable Retrieval Evaluator Correct: Knowledge Refinement. Incorrect: Web Search. Ambiguous: Both types of processed knowledge in Correct and Incorrect are combined.
• arXiv preprint. 2024.: RAFT: Adapting Language Model to Domain Specific RAG novel SFT data Test: RAG Top-k
• arXiv preprint. 2024.: RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedback Iterative. Self-Knowledge Module. Passage Relevance Module. Question Decomposition Module.
• arXiv preprint. 2024. RAGGED: Towards Informed Design of Retrieval Augmented Generation Systems Three aspects: 1. The number of contexts; 2. The reader model; 3. The retriever model. Experiments: varied context volumes, document quality, and question domains.

2. Agent 3

• ICLR 2022 (oral): ReAct: Synergizing Reasoning and Acting in Language Models Reasoning + Acting Reasoning helps the model induce, track, update action plans as well as handle exceptions. Acting allows the model to interface with and gather additional information from external sources.
• ACL 2023 (long): Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
• arXiv preprint. 2024.: KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents

3. Fine-Tuning 4

• ICML 2019: Parameter-Efficient Transfer Learning for NLP Designing an effective adapter module and its integration with the base model.
• ACL 2022 (long): P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
• ICLR 2022 (poster): LoRA: Low-Rank Adaptation of Large Language Models freezes the pretrained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture.
• NeurIPS 2023 (oral): QLoRA: Efficient Finetuning of Quantized LLMs

4. Named Entity Recognition 3

• ACL 2023 (long): Learning In-context Learning for Named Entity Recognition Pre-training
• arXiv preprint. 2023.: GPT-NER: Named Entity Recognition via Large Language Models Prompt Construction Few-shot Demonstrations Retrieval: Random, Sentence-Level, ** Entity-Level **
• AAAI 2024: ConsistNER: Towards Instructive NER Demonstrations for LLMs with the Consistency of Ontology and Context Pre-recognition Ontology-based Demonstration Filtering Context-based Demonstration Retrieval

5. Survey 3

• arXiv preprint. 2024.: Large Language Models for Information Retrieval: A Survey LLM4IR
• arXiv preprint. 2024.: Retrieval-Augmented Generation for Large Language Models: A Survey RAG
• arXiv preprint. 2024.: Retrieval-Augmented Generation for AI-Generated Content: A Survey RAG