mistral-small

Unlocking the Power of Mistral Small on Amazon Bedrock: A Comprehensive Guide

Tal Peretz

— 2 min read
Unlocking the Power of Mistral Small on Amazon Bedrock: A Comprehensive Guide

Mistral Small is a highly efficient large language model (LLM) developed by Mistral AI and now available on Amazon Bedrock. Optimized for high-volume, low-latency language-based tasks, it’s perfect for applications like classification, customer support, and text generation.

Here are some of its standout features:

  • Retrieval-Augmented Generation (RAG) specialization: Ensures important information is retained even in long context windows, up to 32K tokens.
  • Coding proficiency: Excels in code generation, review, and commenting, supporting major coding languages.
  • Multilingual capability: Delivers top-tier performance in French, German, Spanish, Italian, and English, as well as dozens of other languages.

Mistral Small is available in the US East (N. Virginia) Region within Amazon Bedrock.

Access and Usage

To get started, you need to access the model through the Amazon Bedrock console by managing model access and selecting Mistral Small. You can interact with Mistral Small programmatically using AWS CLI and AWS SDK, leveraging Amazon Bedrock APIs.

Customize the interaction by setting parameters such as temperature, max tokens, top P, and top K. Supported tasks include text summarization, text translation, information extraction, and more, which can be integrated into pipelines using constructs like the Mistral text processor.

Getting Started

  1. Access: Ensure you have access to an AWS account within a region that supports AWS Bedrock and the Mistral Small model. Configure the necessary IAM permissions.
  2. Environment Setup: Set up your local code environment with the AWS CLI and the boto3 Python library.
  3. Model Access: Follow the instructions to unlock access to the Mistral Small model through the Amazon Bedrock console.
  4. Querying the Model: Use the Converse API to query the model, configuring authentication credentials and setting environment variables such as AWS_REGION and AWS_BEDROCK_MODEL_ID.

Example Usage

Here is an example of how to query the model using Python:

import boto3
import os

region = os.environ.get("AWS_REGION")
model_id = os.environ.get("AWS_BEDROCK_MODEL_ID")

bedrock_client = boto3.client(service_name='bedrock-runtime', region_name=region)

user_msg = "Who is the best French painter? Answer in one short sentence."
messages = [{"role": "user", "content": [{"text": user_msg}]}]

temperature = 0.0
max_tokens = 1024

params = {
    "modelId": model_id,
    "messages": messages,
    "inferenceConfig": {
        "temperature": temperature,
        "maxTokens": max_tokens
    }
}

resp = bedrock_client.converse(**params)
print(resp["output"]["message"]["content"]["text"])

This example demonstrates how to set up and query the Mistral Small model using the AWS SDK.

Read more