Large language models (LLMs) have demonstrated remarkable capabilities, but their outputs can sometimes be unreliable or factually incorrect. To address this, we introduce Self Logits Evolution Decoding (SLED), a novel decoding framework that enhances the truthfulness of LLMs without relying on external knowledge bases or requiring further fine-tuning. From an optimization perspective, our SLED framework leverages the latent knowledge embedded within the LLM by contrasting the output logits from the final layer with those from early layers. It then utilizes an approximate gradient approach to enable latent knowledge to guide the self-refinement of outputs, thereby effectively improving factual accuracy. Extensive experiments have been conducted on established benchmarks across a diverse range of model families (LLaMA 2, LLaMA 3, Gemma) and scales (from 2B to 70B), including more advanced architectural configurations such as the mixture of experts (MoE). Our evaluation spans a wide variety of tasks, including multi-choice, open-generation, and adaptations to chain-of-thought reasoning tasks. The results demonstrate that SLED consistently improves factual accuracy by up to 20% compared to existing decoding methods while maintaining natural language fluency and negligible latency overhead. Furthermore, it can be flexibly combined with other decoding methods to further enhance their performance.
We introduce Self Logits Evolution Decoding (SLED), a novel factuality decoding approach that leverages the latent knowledge within LLMs by contrasting the final layer’s logits with early layers' logits. SLED tracks the logits evolution process to unearth the latent knowledge within LLMs, and enables the self-evolution of the output distribution further to align it more closely with real-world facts. Furthermore, our approach recognizes that the latent knowledge within LLMs, while valuable, may not always be perfect. Therefore, instead of simply replacing the original outputs with this latent knowledge, SLED integrates it into the original logits through an operation similar to “single-step gradient descent” over the output logits during the inference time. This operation minimizes the Kullback-Leibler (KL) divergence between the latent knowledge distribution and the output distribution, effectively balancing the two and mitigating potential drawbacks such as overfitting or biased outputs. The following figure illustrates the SLED workflow, highlighting how SLED optimizes the output logits, leading to a more factual output distribution.
As a novel layer-wise contrastive decoding approach, we first benchmark SLED against the state-of-the-art approach DoLa across a diverse range of model families (LLaMA 2, LLaMA 3, Gemma) and model scales (from 2B to 70B), including the more advanced mixture of experts (MoE) architecture, as detailed in the following tables. The results showcase notable factuality improvements across a variety of tasks, including multi-choice, open-generation, and adaptations to chain-of-thought reasoning tasks.
SLED exclusively focuses on contrasting differences between layers without altering other parts of the model. Thus, it remains compatible with other techniques that incorporate additional strategies or utilize auxiliary models. This compatibility allows SLED to be seamlessly integrated into existing methods, enhancing the factuality further without the need for modifications on SLED. We test the integration of SLED with the following approaches: Inference Time Intervention (ITI), Activation Decoding (AD), Contrastive Decoding (CD) and Induce-then-Contrast Decoding (ICD). The following table shows that SLED leads to accuracy improvements from 1% to 12% across four LLaMA-2 models.
SLED achieves better performance without the need for excessive repetition penalty.
Our method, SLED, does not incur significant latency overhead. The latencies presented in the following table demonstrate that our method, SLED, just increases the decoding time of DoLa by factors ranging from 0.1% to 10%. Notably, even with an atypical setting such as evolution scale = 100, which is seldom used, the increase remains around 10%.
We provide a new interpretable perspective for understanding layer-wise contrastive decoding methods, paving the way for further developments in factuality decoding. We provide some interesting findings. For instance, we introduce the concept of 'logits evolution' to reinterpret how large language models develop over time. Below are several images that showcase our insights.
In fact, we can observe that SLED provides a new framework for subsequent inference-time algorithms. Unlike most current inference-time computing methods, which primarily focus on heuristic modifications at the sentence level or to logits, SLED integrates more closely with classical optimization algorithms, such as gradient descent. As a result, SLED not only achieves higher optimization efficiency but also opens up many potential research directions for exploration. On the other hand, compared to inference time training methods, SLED does not involve modifications at the level of model parameters, thus resulting in lower overhead for optimization efficiency while better preserving the original performance of the model.
We would greatly appreciate it if you cite our SLED paper when you find this page helpful for your research or projects.
@article{zhang2024sled,
title={SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Models},
author={Zhang, Jianyi and Juan, Da-Cheng and Rashtchian, Cyrus and Ferng, Chun-Sung and Jiang, Heinrich and Chen, Yiran},
journal={arXiv preprint arXiv:2411.02433},
year={2024}
}