Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 10 additions & 12 deletions src/docs/introduction.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,28 @@
# Introduction

llama.cpp lets you run large language models locally on your laptop, desktop, or server with minimal setup and state-of-the-art performance.
Llama App is a desktop app (built on llama.cpp) that lets you download and run large language models locally and chat with them in a few clicks on Mac and Windows.

Llama app is a minimalist desktop app built on llama.cpp, allowing you to download and run models and chat with them only with a few clicks. Llama app is available for both Mac and Windows.
The app lives in your Mac's menu bar or Windows system tray. Choose from models recommended for your platform, download one, and start chatting. Click the link on the [main page](https://llama.app/) to install the app. Once downloaded the app, pick the [model of your choice on Hugging Face Hub](https://huggingface.co/models?apps=llama-app&sort=trending), or simply pick one of the model recommendations on the app.

Instead of using Llama app, if you want to chat with a model in your terminal, or spin up an OpenAI-compatible server with a built-in web interface, you can use `llama cli` or `llama serve`:
<video src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/llama.cpp/llama-lfm.mp4" aria-label="Using a model from the Hub" autoplay muted loop playsinline controls></video>

## Command-line tools

Llama App also sets up the `llama` command line, so you can chat in your terminal with `llama cli` or run an OpenAI-compatible server with a built-in web interface using `llama serve`. On Linux, or if you only want the command line, use the [one-line installer](installation#one-line-install).

```sh
# Chat in your terminal
llama cli -hf ggml-org/gemma-4-e4b-it-GGUF:Q4_0
llama cli -hf LiquidAI/LFM3-Omni-2B-GGUF

# Serve an OpenAI-compatible API + web UI
llama serve -hf ggml-org/gemma-4-e4b-it-GGUF:Q4_0
llama serve -hf LiquidAI/LFM3-Omni-2B-GGUF
```

`llama cli` is great for [experimentation and terminal use](cli), while `llama serve` exposes a [full-featured web UI](webui).

## How it works

llama.cpp runs models stored in the [GGUF](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) file format. GGUF is a single file that packages everything you need together: model weights, tokenizer, metadata. Thousands of ready-to-use GGUF models are available on [Hugging Face](https://huggingface.co/models?library=gguf&sort=trending).

On top of this core, the project ships user-facing tools: `llama cli` for the terminal, `llama serve` for serving an HTTP API and web UI, plus utilities for benchmarking, quantizing, and evaluating models.

## Next steps

- [Installation](installation) — install desktop app, prebuilt binaries, use a package manager, Docker, or build from source
- [Installation](installation) — install Llama App or the command-line tools
- [Quickstart](quickstart) — download a model and run it in minutes
- [Using the CLI](cli) — chat and experiment from the terminal with `llama cli`
- [Running a server](serve) — serve models over HTTP with `llama serve`
Expand Down
10 changes: 8 additions & 2 deletions src/docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,26 @@ This guide takes you from a fresh [installation](installation) to chatting with

## Download your first model

### Llama app
### Llama App

On the Llama app, click on the downwards arrow next to one of the models recommended for your platform.
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/llama.cpp/llama-onboarding.png" alt="Onboarding" />
Once downloaded, you can click on the model, which opens chat options. Depending on your memory and the length of the conversation you want to have, select the context window > Chat with the model.

<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/llama.cpp/ctx-windows.png" alt="Context window" />

### Hub

Hugging Face Hub hosts thousands of Llama compatible models you can use with the app. You can get started with any of them by clicking "Use this Model" → Llama. This will automatically download the model.

<video src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/llama.cpp/llama-lfm.mp4" aria-label="Using a model from the Hub" autoplay muted loop playsinline controls></video>

### Llama CLI

If you want to use the CLI to chat with models, pass a Hugging Face repository with the `-hf` flag. The model is downloaded automatically the first time you use it:

```sh
llama cli -hf unsloth/gemma-4-E4B-it-GGUF:Q4_K_M
llama cli -hf unsloth/gemma-4-E4B-it-GGUF
```

We picked a 4-bit quantized version of Gemma-4 E4B, a model that can take image, text, and audio inputs, using just ~6GB of memory with a context window of 16k tokens. There are [thousands of GGUF models on Hugging Face](https://huggingface.co/models?library=gguf&sort=trending) to choose from.
Expand Down
Loading