Getting started with the LLAMA 3.2-11B with groq

Much has been happening in the AI space for a while now with different models popping up that are capable of one task or the other. Some existing models are being retained to perform a certain task more accurately than the general model. These models are capable of different functionalities with built-in capacities and adjoined capacities.

The LLAMA 3.2 variant of Meta ai is one of the most recently released models. 

According to Meta AI, Llama 3.2 included lightweight models in 1B and 3B sizes at bfloat16 (BF16) precision. Subsequent to the release, It was updated to include quantized versions of these models. 

The vision models come in two variants: 11B and 90B. These models are designed to support image reasoning. The 11B and 90B can understand and interpret documents, charts, and graphs and perform tasks such as image captioning and visual grounding. These advanced vision capabilities were made possible by integrating pre-trained image encoders with language models using adapter weights consisting of cross-attention layers.

Compared to Claude 3 Haiku and GPT-4o mini, the Llama 3.2 vision models have excelled in image recognition and various visual understanding tasks, making them robust tools for multimodal AI applications.

Below is a quick implementation of the LLAMA 3.2-11B with Groq.

Step1 

Get a Groq account on grok site

Step2

Create an API key and save it somewhere, do not lose this key

Step 3

Go to your coding environment and install groq with pip

1pip install groq

Step 4 

Import groq and create a simple completion with groq but first set your groq api key

1GROQ_KEY = "gsk_AC2gpFV1nkpzW0YG......"

Use the following code to create the completion agent.

1from groq import Groq

1client = Groq(

1    api_key=GROQ_KEY,

1)

1# define the image

1From PIL import image

1image = image.open(‘path_to_image’)

1client = Groq()

1completion = client.chat.completions.create(

1    model="llama-3.2-11b-vision-preview",

1    messages=[{"role": "system", "content": """you are an expert image analyst, analyse the attached image and explain what it is."""},

1        {

1            "role": "user",

1            "content": [

1                {

1                    "type": "text",

1                    "Text": ”what is this image”

1                },

1                {

1                    "type": "image",

1                    "image_url": {

1                        "url": image

1                    }

1                }

1            ]

1        }

1    ],

1    temperature=1,

1    top_p=1,

1    stream=False,

1    stop=None,

1)

Step 5

Capture the response using the code snipped below

1reply= completion.choices[0].message.content

With these few steps, you have been able to create an agent that uses the llama 3.2 11B vision model that can take your image, analyze it, and tells you the details about the image.

Using groq, you do not need to use a powerful system optimized for AI modeling thereby helping you save resources. The llama 3.2 11B model is very heavy and it will take a high-performing system and a fast internet connection to work with it.