(Need Help) Running LLAMA on V3D

progressEdd · February 11, 2024, 7:12am

I’ve never worked with or heard of V3D before. Is it a gpu driver for a raspberry pi?

Also did you see that llama.CPP merged GPU support

Haven’t deployed it before, but these might be some good resources for working with llama

garyexplains/examples/blob/master/how-to-run-llama-cpp-on-raspberry-pi.md

# How to Run a ChatGPT-like AI Bot on a Raspberry Pi
Here is my step-by-step guide to running Large Language Models (LLMs) using llama.cpp on a Raspberry Pi. These instructions accompany my video [How to Run a ChatGPT-like AI on Your Raspberry Pi](https://youtu.be/idZctq7WIq4).

## Links

- [Introducing Code Llama, a state-of-the-art large language model for coding](https://ai.meta.com/blog/code-llama-large-language-model-coding/)
- [llama.cpp](https://github.com/ggerganov/llama.cpp)
- [Llama-2-7b-Chat-GGUF](https://huggingface.co/TheBloke/Llama-2-7b-Chat-GGUF) - ([Q4_K_S](https://huggingface.co/TheBloke/Llama-2-7b-Chat-GGUF/resolve/main/llama-2-7b-chat.Q4_K_S.gguf)) - ([Q2_K](https://huggingface.co/TheBloke/Llama-2-7b-Chat-GGUF/resolve/main/llama-2-7b-chat.Q2_K.gguf))
- [Llama 2 13B Chat](https://huggingface.co/TheBloke/Llama-2-13B-chat-GGUF) - ([Q4_K_S](https://huggingface.co/TheBloke/Llama-2-13B-chat-GGUF/resolve/main/llama-2-13b-chat.Q4_K_S.gguf))
- [CodeLlama-7B-Instruct-GGUF](https://huggingface.co/TheBloke/CodeLlama-7B-Instruct-GGUF) - ([Q4_K_S](https://huggingface.co/TheBloke/CodeLlama-7B-Instruct-GGUF/resolve/main/codellama-7b-instruct.Q4_K_S.gguf))
- ~~[CodeLlama 7B Instruct - GGML](https://huggingface.co/TheBloke/CodeLlama-7B-Instruct-GGML/tree/main) - ([Q4_K_S](https://huggingface.co/TheBloke/CodeLlama-7B-Instruct-GGML/blob/main/codellama-7b-instruct.ggmlv3.Q4_K_S.bin))~~

## Prepare development environment
To start you need to have the C/C++ compiler, and tools like make and git.
```
sudo apt update
sudo apt install git g++ wget build-essential
```

## Download and compile llama.cpp

This file has been truncated. show original