Research papers on LLM
Quickly access the latest research papers on large language models from arXiv.
Papers and research related to Large Language Model
- arXiv - Artificial Intelligence·Jan 20
Survey Highlights LLMs' Role in Transforming Scholarly Paper Review Process
Description
This survey paper examines the role of large language models (LLMs) in automated scholarly paper review (ASPR), highlighting their transformative potential and the challenges they present in academia.
Key Points
- The paper discusses the integration of LLMs into the peer review process, marking a shift towards automated scholarly paper review (ASPR) in academic publishing.
- It identifies technological bottlenecks that have been addressed through LLM technology and explores new methods, datasets, and systems that enhance ASPR.
- The authors analyze the performance of LLMs in ASPR, the reactions of academia and publishers, and the challenges that remain for the effective implementation of ASPR.
linkCopy linkshare_windowsShare - arXiv - Artificial Intelligence·Jan 20
Integrating LLMs with Automated Planners for Enhanced Decision-Making in Intelligent Agents
Description
This paper presents a novel architecture that combines Large Language Models (LLMs) with classical automated planners to enhance decision-making in intelligent agents, particularly in formative simulations.
Key Points
- Traditional behavior trees struggle in simulating human-like behavior, while LLMs provide plausible responses, making them suitable for decision-making tasks.
- The proposed architecture integrates LLMs for generating human-like decisions with automated planners that create sound plans, improving agent adaptability.
- This approach allows agents to handle unforeseen situations effectively, enhancing their performance in dynamic environments and simulations.
linkCopy linkshare_windowsShare - arXiv - Artificial Intelligence·Jan 20
Survey Unifies LLM Test-Time Compute: Insights on Profiling and Search Algorithms
Description
This survey reviews LLM test-time compute via search, addressing task definitions, LLM profiling, and search algorithms to facilitate comparisons among various inference frameworks in the field of artificial intelligence.
Key Points
- The survey unifies task definitions and modular definitions of LLM profiling and search procedures, enhancing clarity in LLM inference comparisons.
- It highlights the divergence of employed search algorithms from standard implementations, emphasizing the need for thorough specification of their characteristics.
- The paper discusses the performance, efficiency, and applicability of different LLM inference methods, providing insights into their operational frameworks.
linkCopy linkshare_windowsShare - arXiv - Artificial Intelligence·Jan 20
Mind Evolution: A Breakthrough in Scaling Inference for Large Language Models
Description
The paper presents Mind Evolution, an evolutionary search strategy that enhances inference time compute in Large Language Models, demonstrating superior performance in natural language planning tasks compared to traditional methods.
Key Points
- Mind Evolution leverages a language model to generate, recombine, and refine candidate responses, optimizing inference without formalizing the underlying problem.
- In benchmarks like TravelPlanner and Natural Plan, Mind Evolution achieves over 98% success in problem instances using Gemini 1.5 Pro, outperforming Best-of-N and Sequential Revision strategies.
- The research highlights the efficiency of Mind Evolution in scaling inference costs while maintaining high performance in complex natural language tasks.
linkCopy linkshare_windowsShare - arXiv - Artificial Intelligence·Jan 20
Hierarchical Transformers Enhance Robustness and Flexibility in Language Modeling
Description
This research presents a hierarchical autoregressive transformer model that integrates character-level and word-level processing, enhancing robustness and adaptability in natural language processing tasks.
Key Points
- The model addresses limitations of traditional subword tokenizers, such as large vocabularies and sensitivity to spelling errors, by using a lightweight character-level encoder.
- Hierarchical transformers demonstrate comparable performance to subword-tokenizer models while being more robust to input variations and faster in continued pretraining on out-of-domain languages.
- This approach allows for greater flexibility and generalizability across different languages and domains, paving the way for more adaptable NLP systems.
linkCopy linkshare_windowsShare - arXiv - Artificial Intelligence·Jan 20
Dual Debiasing Algorithm Enhances Fairness in Language Modeling and Translation
Description
The paper presents a novel Dual Debiasing Algorithm through Model Adaptation (2DAMA) aimed at reducing gender stereotypes in language models while preserving factual gender information for fair language processing.
Key Points
- 2DAMA effectively mitigates gender bias in English, marking a significant advancement in debiasing techniques for language models and translation tasks.
- The method ensures that while reducing stereotypical biases, it retains factual gender cues, which are essential for various natural language processing applications.
- This research highlights the importance of balancing bias reduction with the preservation of useful information, contributing to the development of more equitable language technologies.
linkCopy linkshare_windowsShare - arXiv - Artificial Intelligence·Jan 20
Advancements in Uzbek NLP: BERT Models Achieve 91% Accuracy in POS Tagging
Description
This research paper presents advancements in NLP for the Uzbek language by evaluating BERT models for part-of-speech tagging and introducing a benchmark dataset, achieving significant accuracy improvements.
Key Points
- The study evaluates two monolingual BERT models for Uzbek, achieving an average accuracy of 91% in POS tagging, surpassing multilingual BERT and rule-based methods.
- The research introduces the first publicly available UPOS-tagged benchmark dataset for Uzbek, enhancing resources for low-resource languages in NLP.
- The fine-tuned models effectively capture intermediate POS changes through affixes and demonstrate context sensitivity, addressing limitations of existing rule-based taggers.
linkCopy linkshare_windowsShare - arXiv - Artificial Intelligence·Jan 20
TARDIS: A Breakthrough in Optimizing Large Language Models for Speed and Efficiency
Description
The paper introduces TARDIS, a novel method for optimizing large language models (LLMs) by approximating non-linear activation functions with linear ones, achieving significant parameter reduction and improved inference speed.
Key Points
- TARDIS allows for an 80% reduction in parameters in feed-forward networks while maintaining up to 65% higher accuracy compared to traditional pruning methods like Wanda and RIA.
- The method integrates an online predictor for outlier inputs, ensuring that complex non-linear activations can still be effectively optimized without sacrificing performance.
- In practical applications, TARDIS provides a 1.6x speedup in inference time for a 7B model when used with the vLLM serving system, with only a 10.9% accuracy trade-off.
linkCopy linkshare_windowsShare - arXiv - Artificial Intelligence·Jan 20
AGSER Method Enhances Zero-Shot Hallucination Detection in Large Language Models
Description
The paper presents the Attention-Guided Self-Reflection (AGSER) method for zero-shot hallucination detection in Large Language Models (LLMs), significantly improving detection efficacy while reducing computational complexity.
Key Points
- AGSER categorizes input queries into attentive and non-attentive, processing them separately to compute consistency scores for hallucination estimation.
- The method requires only three passes through the LLM and utilizes two sets of tokens, enhancing efficiency in detection tasks.
- Extensive experiments with four popular LLMs across three benchmarks show AGSER's superior performance compared to existing hallucination detection methods.
linkCopy linkshare_windowsShare - arXiv - Machine Learning·Jan 20
CliMB-DC: A Human-Guided Framework for Data-Centric LLM Co-Pilots in Healthcare
Description
The paper 'Towards Human-Guided, Data-Centric LLM Co-Pilots' presents CliMB-DC, a framework that integrates human expertise with LLMs to address data-centric challenges in machine learning, particularly in healthcare.
Key Points
- CliMB-DC combines advanced data-centric tools with LLM-driven reasoning, enabling effective handling of complex data issues like missing values and label noise.
- The framework features a multi-agent reasoning system, incorporating a strategic coordinator and a specialized worker agent to enhance data processing.
- Empirical results demonstrate CliMB-DC's superior performance in transforming uncurated datasets into ML-ready formats, empowering domain experts across various fields to leverage machine learning effectively.
linkCopy linkshare_windowsShare - arXiv - Machine Learning·Jan 20
Advancements in Explainable AI: Enhancing Interpretability in Large Language Models
Description
This paper surveys advancements in Explainable AI (XAI), focusing on methods that enhance the interpretability of machine learning models, including large language models (LLMs), to foster trust and usability in critical applications.
Key Points
- The paper discusses the importance of XAI in improving transparency in AI systems, particularly in high-stakes fields like healthcare and autonomous driving, where understanding model decisions is crucial.
- It reviews various XAI techniques, from inherently interpretable models to modern approaches that utilize LLMs and vision-language models (VLMs) for generating meaningful explanations of model behavior.
- The authors highlight the strengths and weaknesses of current XAI methods, present comparative results, and outline future research directions to address existing challenges in the field.
linkCopy linkshare_windowsShare - arXiv - Machine Learning·Jan 20
MultiPruner: A Novel Approach to Efficient Pruning in Large Pre-trained Models
Description
The paper introduces MultiPruner, a novel pruning method for large pre-trained models that enhances model compression while maintaining accuracy through a multidimensional pruning strategy.
Key Points
- MultiPruner improves upon existing training-free pruning methods by employing a fine-grained, iterative approach that balances structural integrity in block-pruned models.
- The method compresses models along three dimensions: residual blocks, MLP channels, and attention heads, leading to better zero-shot accuracy on downstream tasks.
- Extensive experiments validate MultiPruner's effectiveness across various large pre-trained models, demonstrating superior compression ratios and reduced computational requirements.
linkCopy linkshare_windowsShare
