Exploring Mistral's Pixtral Large: The Next Generation Multimodal Model
Mistral AI has unveiled its latest large language model, Pixtral Large, a 124-billion parameter multimodal powerhouse. It is the second in the series, succeeding the Mistral Large 2, and introduces a robust combination of a 123 billion parameter multimodal decoder and a 1 billion parameter vision encoder.
Pixtral Large excels in both image and text processing, showcasing frontier-level image understanding while maintaining exemplary text processing capabilities. Its proficiency spans multilingual optical character recognition (OCR), reasoning, chart understanding, and document analysis. A practical example of its capabilities includes analyzing a restaurant bill using OCR to calculate the total cost and each person's share, including the tip.
In terms of performance, Pixtral Large sets new standards by achieving a 69.4% accuracy on MathVista, outperforming previous models. It also surpasses GPT-4o and Gemini-1.5 Pro on both DocVQA and ChartQA, and remains competitive on the MM-MT-Bench, outperforming models like Claude-3.5 Sonnet, Gemini-1.5 Pro, and GPT-4o.
Technically, Pixtral Large features a context window of 128,000 tokens, making it capable of handling up to 30 high-resolution images or a 300-page book. The model weights exceed 200GB, necessitating a substantial GPU setup for operation. Users can access the model via Hugging Face, with pricing set at $2 per 1M input tokens and $6 per 1M output tokens.
Pixtral Large is available under the Mistral Research License (MRL) for academic and non-commercial use, with a separate commercial license for enterprise applications. Users can integrate the model into various applications through the pixtral-large-latest API or download it for self-hosted implementations. Integration into Mistral AI's chatbot, Le Chat, and accessibility via the Mistral API expands its utility across web search, document processing, and image generation.
Looking ahead, Mistral AI plans to enhance Pixtral Large's OCR capabilities, ensuring it remains at the forefront of multimodal processing technology.
