LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs

Reasoning is a fundamental capability for solving complex multi-step problems, particularly in visual contexts where sequential step-wise understanding is essential. Existing approaches lack a comprehensive framework for evaluating visual reasoning and do not emphasize step-wise problem-solving. To this end, we propose a comprehensive framework for advancing step-by-step visual reasoning in large language models (LMMs) through three key contributions. First, we introduce a visual reasoning benchmark specifically designed to evaluate multi-step reasoning tasks. The benchmark presents a diverse set of challenges with eight different categories ranging from complex visual perception to scientific reasoning with over 4k reasoning steps in total, enabling robust evaluation of LLMs' abilities to perform accurate and interpretable visual reasoning across multiple steps. Second, we propose a novel metric that assesses visual reasoning quality at the granularity of individual steps, emphasizing both correctness and logical coherence. The proposed metric offers deeper insights into reasoning performance compared to traditional end-task accuracy metrics. Third, we present a new multimodal visual reasoning model, named LlamaV-o1, trained using a multi-step curriculum learning approach, where tasks are progressively organized to facilitate incremental skill acquisition and problem-solving. The proposed LlamaV-o1 is designed for multi-step reasoning and learns step-by-step through a structured training paradigm. Extensive experiments show that our LlamaV-o1 outperforms existing open-source models and performs favorably against close-source proprietary models. Compared to the recent Llava-CoT, our LlamaV-o1 achieves an average score of 67.3 with an absolute gain of 3.8% across six benchmarks while being 5 times faster during inference scaling. Our benchmark, model, and code are publicly available.

Host: Hey everyone, and welcome back to the podcast! I'm your host, Leo, and I'm super excited for today's episode. We're diving into some seriously cutting-edge AI research today, something that's both fascinating and incredibly relevant to where the field is heading.

Host: Yeah, we've been talking a lot lately about large language models, or LLMs, and their incredible text generation capabilities. But what happens when you throw images into the mix? That's where things get really interesting. We're talking about multimodal models now, models that can understand and reason about both text and visual data.

Host: Exactly! It's one thing for an AI to generate human-like text, but it’s a whole different ball game when they can analyze an image, understand it’s content, and then use that knowledge to solve a complex problem. It's like giving AI a pair of eyes and the ability to think, not just talk. And today, we've got a research paper that's pushing this idea forward in some pretty significant ways.

Host: It’s a pretty hefty topic, but it's about improving how these multimodal models reason, especially when it comes to breaking down complex problems into a series of understandable steps. Think about it, as humans, we don't just magically arrive at an answer – we often think through each step of the way. That's what this paper is addressing for AI.

Host: The authors of this paper—and I'll just run through their names because there are quite a few: Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, Hisham Cholakkal, Ivan Laptev, Mubarak Shah, Fahad Shahbaz Khan, and Salman Khan. It’s an impressive team from some of the leading institutions in the field, like Mohamed bin Zayed University of AI, the University of Central Florida, Linköping University, and the Australian National University. So, you know this research is serious.

Host: Alright, let’s get into the meat of it. This paper basically argues that current methods for evaluating these multimodal reasoning models aren't really capturing the full picture. They tend to focus on the final answer, and they often neglect what is happening in the intermediate steps. So, the authors are addressing that. They're saying, ‘Hey, it’s not enough to just get the answer right, we need to see how the model arrived at that answer, step by step.’ And they're also emphasizing the importance of having these models tackle problems sequentially, just like we humans do when we’re solving something complicated.

Host: or VRC-Bench, specifically designed to test these step-by-step reasoning capabilities. Second, they propose a new metric to evaluate reasoning quality at the individual step level. This is important because it looks at both correctness and logical coherence, which gets past just the final answer. And third, they present a brand new multimodal model called 'LlamaV-o1', which they trained using a progressive learning approach. It's like building up from the basics to more complex problems, step-by-step.

Host: So, let’s start with their first key contribution – this new VRC-Bench benchmark. They’ve designed it to really push these models to their limits, encompassing a wide variety of scenarios. We’re talking about eight different categories here, everything from basic visual perception and math to more sophisticated things like scientific reasoning and even understanding social and cultural contexts. This isn't your typical image recognition task; it's designed to gauge deep reasoning capabilities.

Host: Yeah, the diversity is pretty impressive. They're not just throwing a bunch of similar problems at the models; they're trying to get a good feel for how these models handle very different types of tasks. From what I read, it includes around 1000 different examples, all with over 4000 manually verified reasoning steps. That's a lot of curated data and manual verification. And it highlights their commitment to getting this evaluation framework just right. It's important to note how carefully they have curated the data.

Host: It makes you wonder, though, why is this step-by-step reasoning so crucial anyway? I think it's something that we often take for granted. We, as humans, tend to break down our thought process into a sequential path to help achieve a logical conclusion. So, if we're trying to build more capable AI, should we not try to emulate how humans solve problems?

Host: I think you hit on something important there, the cognitive aspect. When we solve complex problems, we go through a series of steps. For example, if you're looking at a complicated diagram, you don't just jump to a conclusion. You look at the different components, you figure out their relationship to each other, and you reason through the implications. That's what this step-by-step approach is trying to instill in these AI models. It allows them to show their work, so to speak, making the reasoning transparent and easier to understand.

Host: And I think the interpretability is crucial. If we just care about the final output, we have no insight about its process. It's like a black box. But, if we train models to solve problems step by step, we can track the internal reasoning process. It allows us to identify flaws, refine the training, and improve their overall robustness. It can potentially also help to remove bias or inaccuracies in the generated answers. It’s a pretty big deal for building trust in these systems, I'd say.

Host: Okay, so now we come to the second contribution: this novel metric they’ve proposed. They’re emphasizing that relying on just end-task accuracy – basically, just the final answer being correct – is not enough. So, they created this metric that evaluates the visual reasoning quality at the granularity of the individual steps. They look at both the correctness of each step, but also the logical coherence between each step. It’s like they’re checking if the model is thinking straight, not just getting lucky.

Host: And that’s important, because a model could get a final answer right by sheer luck, or by taking a wrong turn and then magically correcting itself. You wouldn’t know unless you're looking at each step. By evaluating the individual steps, we're making sure the model is actually following a logical path, and not just stumbling on the right answer. It also gives us an idea about how consistent the reasoning is. The paper highlights how much more insightful it is as compared to a standard end-to-end accuracy check.

Host: Yeah, it’s not enough to have a model that’s just a good guesser; you need one that understands why the answer is correct. In real world applications, that’s critical. Imagine using this in a medical diagnostic setting – you’d want the model to show you its reasoning, so you can evaluate if it’s sound, or if it’s made some kind of a mistake.

Host: Absolutely. So, onto their third and final contribution – their new multimodal model called ‘LlamaV-o1’. They trained this using a multi-step curriculum learning approach. The curriculum learning is a pretty neat idea, they’re basically training the model progressively, starting with very simple tasks, like summarizing the input, and then moving onto the complicated multi-step reasoning tasks. Think about it like learning in school, we don't start with calculus, we start with the basics and move up from there. It’s the same idea here.

Host: And it makes sense intuitively, right? It's about gradually building the skills required for complex reasoning. They start with foundational tasks, such as summarizing the approach, or generating captions based on the given question, and then move on to the much more complicated multi-step reasoning. This approach helps models manage the complexity of the task, ensure logical coherence, and generalize effectively to more challenging scenarios. It avoids this kind of all-or-nothing approach and builds a really solid foundation.

Host: Yeah, exactly. The model also seems to incorporate something called beam search, during inference. It basically means the model is generating multiple potential reasoning paths, rather than just picking one at random, and that it ends up choosing the most optimal path from those options. I think this is an approach that balances both efficiency and output quality. So, it’s not just about accuracy, it’s also about making sure they achieve that accuracy efficiently, and I suppose it helps to get rid of some of the computational overhead.

Host: And from what I gathered, all these design choices lead to some pretty impressive results. Their experiments show that LlamaV-o1 outperforms other open-source models and even performs really well against close-source models, like Gemini and Claude. They claim to achieve a 3.8% improvement over recent models while being about five times faster during inference. Those numbers are quite significant, especially in the context of making these models usable in the real world.

Host: The speed is particularly important, right? Because, it’s not enough to have a model that does well, but also needs to do it quickly. Especially when we look at the increasing demand for real time results. And that really just goes to show the potential of combining curriculum learning with a smarter search technique. So, they’re not just relying on brute force, but they’re actually optimizing their approach. I think this paper is a demonstration that shows us the importance of structured training in these complex models.

Host: Alright, let’s delve into some of these technical details in the paper. Let’s talk about the training data, for example, because that’s a crucial aspect of any model. To implement this curriculum learning strategy they were talking about, they have divided the entire process into two stages. The first stage is where they’re focusing on simpler tasks, summarization of the visual data, and generating captions from the input. And, they used around 18,000 samples from the PixMo dataset, and 57,000 samples from the Geo170K dataset for this.

Host: Those initial tasks are pretty fundamental, right? Summarization is about getting the overall context, while captioning is about highlighting specific details. And, they have used two large multimodal datasets for that initial step. I think the PixMo dataset is quite interesting in this context, because it includes examples that have ground truth captions based on the input question. So, the model gets the chance to learn that correlation between a question and the aspects of the image. While the Geo170K dataset, according to the authors, has questions with reasoning steps. So, the model gets to learn structured reasoning in this step as well.

Host: Right, and once it has a solid grasp of the basics, they move onto the second training stage. This is where the model trains on more complex reasoning tasks. They are using the original LLaVA-CoT dataset which is around 99,000 structured samples comprising of a variety of tasks such as general VQA and science focused VQA. These samples include a summary, caption, detailed reasoning steps, and the final answer. So now, the model has to understand the input, break down the reasoning step by step, and give the correct final answer.

Host: So, it’s all about this incremental learning, from simple to complex, and it really mirrors how we learn. It’s like laying a strong foundation before building the skyscraper. I like how they structured their methodology. This progressive approach probably makes the model much more robust, compared to just training it on the most complicated reasoning tasks right away. It sort of helps to avoid what they call “catastrophic forgetting”, which can occur if you try to train a model on too many complex tasks at the same time.

Host: That’s an excellent point. It all about balancing learning efficiency with task complexity. Now, let’s dig a bit deeper into the multi-step chain of thought reasoning methodology they are proposing. They emphasize that breaking down the problem into incremental steps is critical for complex reasoning tasks. And, it aligns quite closely with the human approach. In order to do that, they have divided the reasoning process into further stages: They start with task understanding, then they go on to task summarization, where the model generates a basic summary of the visual data. Then, they move on to a more detailed caption generation where it labels specific objects and their values. And then it goes to a more structured logical reasoning step where it breaks the tasks into a series of sub-goals, before generating the final answer.

Host: It's like a recipe for solving problems. You don't just dump all the ingredients into a pot, you follow steps to get to your final outcome. By explicitly breaking down the process this way, the authors are ensuring the model is not only getting the right answer, but it’s doing so in a way that’s transparent, consistent and understandable. And they’ve structured their entire training process to optimize the model for these specific stages. It’s a fairly smart methodology, I think.

Host: And I think that’s the key, right? It’s not enough to just have a model that can ‘guess’ the answer, you want a model that can reason logically and transparently, especially if it’s going to be applied to crucial decision making contexts. I also found their choice of the base model to be quite interesting. They chose Llama-3.2-11B-Vision-Instruct, and it goes to show that having a solid base in multimodal reasoning and instruction-following capabilities can make a huge difference.

Host: Right, and it also highlights the importance of using the right tool for the job. And when we’re talking about these large models, the inference efficiency is also very important. That’s where their usage of beam search during inference comes in. I think they have managed to improve both quality and efficiency by generating multiple reasoning paths and selecting the most optimal one. It also goes to show that simple techniques can often make a significant impact if implemented correctly.

Host: And the best part of all of that, is that it helps to bring down the complexity from O(n²) which is what LLaVa-CoT uses, to O(n). That is a huge leap in terms of reducing inference time and making the model more scalable. It actually manages to cut down the inference time almost five folds, according to their claims. That is impressive by all means.

Host: Okay, so we’ve talked about the new benchmark, the new evaluation metric, the model architecture and training methodologies, now, let’s move on to the experiments. They used this Llama-3.2-11B-Vision-Instruct model as the base and trained it using the curriculum learning strategy, combined with a supervised fine tuning approach. And then they went and evaluated this model on their new VRC-Bench benchmark, and then they also compared against other models on six other well established multimodal benchmarks.