Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTS

In-context Learning (ICL) enables large language models (LLMs) to tackle downstream tasks through sophisticated prompting and high-quality demonstrations. However, this traditional ICL paradigm shows limitations when facing complex mathematical reasoning tasks, primarily due to its heavy dependence on example quality and the necessity for human intervention in challenging scenarios. To address these limitations, this paper presents HiAR-ICL, a High-level Automated Reasoning paradigm in ICL that shifts focus from specific examples to abstract thinking patterns, extending the conventional concept of context in ICL. HiAR-ICL introduces five atomic reasoning actions as fundamental components for constructing chain-structured patterns. Using Monte Carlo Tree Search, we explore reasoning paths and construct thought cards to guide subsequent inference. We then develop a cognitive complexity framework that dynamically matches problems with appropriate thought cards. Experimental results demonstrate HiAR-ICL's effectiveness, achieving state-of-the-art accuracy (79.6%) on the MATH benchmark with Qwen2.5-7B-Instruct, surpassing GPT-4o (76.6%) and Claude 3.5 (71.1%).

Host: Hey everyone, and welcome back to another episode of the podcast! Today, we're diving deep into the fascinating world of large language models and, more specifically, how we can improve their complex reasoning abilities. I've got Jinyang Wu, Mingkuan Feng, Shuai Zhang, Feihu Che, Zengqi Wen, and Jianhua Tao with me today, all key contributors to a groundbreaking paper on this very topic. So buckle up, it's going to be a wild ride!

Guest: Thanks for having us, Leo! Excited to be here.

Host: Absolutely! So, the paper focuses on a new paradigm called HiAR-ICL – High-level Automated Reasoning in In-Context Learning. Now, for our listeners who aren't familiar, can you give us a quick rundown of what In-Context Learning, or ICL, actually is?

Guest: Sure. ICL is essentially a way to get LLMs to perform tasks without explicitly training them on that specific task. You just give them a few examples of the task, showing them what the input is and what the correct output should be. Then, you give it a new input, and it tries to figure out the correct output based on the examples. Think of it like showing a kid a few addition problems before asking them to solve a new one.

Host: That makes sense, a kind of 'show, don't tell' approach. So what are the limitations of this traditional ICL approach, especially when it comes to complex tasks like mathematical reasoning?

Guest: The main issue is that traditional ICL heavily relies on the quality of those examples. If the examples are poorly chosen, or don't cover all the nuances of the problem, the LLM might not perform well, even if it's a powerful model. It's also very labor-intensive; crafting those perfect examples for complex problems often requires a lot of human expertise and time. Finally, it struggles with generalization. If the format of a new problem is slightly different, even if the underlying logic is the same, the model might fail because it hasn't seen that specific format before.

Host: So HiAR-ICL aims to solve these issues. How does it approach the problem differently?

Guest: Exactly. HiAR-ICL shifts the focus from specific examples to more abstract reasoning patterns. Instead of just showing examples, we're teaching the LLM how to think. We break down the reasoning process into five atomic actions—things like analyzing the problem, taking a step-by-step approach, breaking the problem down into smaller parts, and reflecting on the solution. These actions are the building blocks of what we call 'thought cards.'

Host: Thought cards? That's an interesting concept. Can you elaborate on that?

Guest: Sure. We use Monte Carlo Tree Search (MCTS) to explore different reasoning paths on a small set of seed examples. The successful paths are then distilled into these 'thought cards,' which are essentially templates for reasoning. These cards are like pre-programmed strategies for tackling different types of problems. When given a new problem, the system assesses its complexity and selects the most suitable thought card(s) to guide the solution process.

Host: So, instead of relying on specific examples, the LLM uses these pre-learned strategies, essentially learning to fish rather than just being given the fish. Clever! How do you determine the complexity of a problem to match it with the appropriate thought card?

Guest: We developed a cognitive complexity framework that considers three key factors: the number of subproblems, the complexity of the problem's conditions, and the semantic similarity between the new problem and the seed examples. This framework helps select the most relevant thought card(s) for a given problem, improving both accuracy and efficiency.

Host: That sounds incredibly sophisticated! And what about verification? How do you ensure the LLM gets to the right answer using this approach?

Guest: That's a crucial point. We employ several verification methods, including checking the consistency of the reasoning steps, evaluating the overall solution, and even assessing the quality of the individual reasoning steps. We found that even a simple consistency check is quite effective in improving accuracy.

Host: This all sounds incredibly promising. You mentioned some impressive results in your paper. Can you highlight some of those key findings?

Guest: Absolutely. Our experiments show that HiAR-ICL significantly outperforms traditional ICL methods across several reasoning benchmarks. It's particularly effective with smaller language models, often achieving performance comparable to, or even exceeding, much larger and closed-source models like GPT-4 and Claude 3.5. For instance, on the MATH benchmark, our 7B parameter Qwen2.5 model achieved 79.6% accuracy, surpassing GPT-4o's 76.6%.

Host: That's remarkable! It seems HiAR-ICL is a major step forward in improving the reasoning capabilities of LLMs. We've only just scratched the surface here, but...