1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
|
Metadata-Version: 2.2
Name: llm
Version: 0.23
Summary: CLI utility and Python library for interacting with Large Language Models from organizations like OpenAI, Anthropic and Gemini plus local models installed on your own machine.
Home-page: https://github.com/simonw/llm
Author: Simon Willison
License: Apache License, Version 2.0
Project-URL: Documentation, https://llm.datasette.io/
Project-URL: Issues, https://github.com/simonw/llm/issues
Project-URL: CI, https://github.com/simonw/llm/actions
Project-URL: Changelog, https://github.com/simonw/llm/releases
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click
Requires-Dist: openai>=1.55.3
Requires-Dist: click-default-group>=1.2.3
Requires-Dist: sqlite-utils>=3.37
Requires-Dist: sqlite-migrate>=0.1a2
Requires-Dist: pydantic>=2.0.0
Requires-Dist: PyYAML
Requires-Dist: pluggy
Requires-Dist: python-ulid
Requires-Dist: setuptools
Requires-Dist: pip
Requires-Dist: pyreadline3; sys_platform == "win32"
Requires-Dist: puremagic
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: numpy; extra == "test"
Requires-Dist: pytest-httpx>=0.33.0; extra == "test"
Requires-Dist: pytest-asyncio; extra == "test"
Requires-Dist: cogapp; extra == "test"
Requires-Dist: mypy>=1.10.0; extra == "test"
Requires-Dist: black>=25.1.0; extra == "test"
Requires-Dist: ruff; extra == "test"
Requires-Dist: types-click; extra == "test"
Requires-Dist: types-PyYAML; extra == "test"
Requires-Dist: types-setuptools; extra == "test"
Dynamic: author
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: project-url
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary
# LLM
[](https://pypi.org/project/llm/)
[](https://llm.datasette.io/)
[](https://llm.datasette.io/en/stable/changelog.html)
[](https://github.com/simonw/llm/actions?query=workflow%3ATest)
[](https://github.com/simonw/llm/blob/main/LICENSE)
[](https://datasette.io/discord-llm)
[](https://formulae.brew.sh/formula/llm)
A CLI utility and Python library for interacting with Large Language Models, both via remote APIs and models that can be installed and run on your own machine.
[Run prompts from the command-line](https://llm.datasette.io/en/stable/usage.html#executing-a-prompt), [store the results in SQLite](https://llm.datasette.io/en/stable/logging.html), [generate embeddings](https://llm.datasette.io/en/stable/embeddings/index.html) and more.
Consult the **[LLM plugins directory](https://llm.datasette.io/en/stable/plugins/directory.html)** for plugins that provide access to remote and local models.
Full documentation: **[llm.datasette.io](https://llm.datasette.io/)**
Background on this project:
- [llm, ttok and strip-tags—CLI tools for working with ChatGPT and other LLMs](https://simonwillison.net/2023/May/18/cli-tools-for-llms/)
- [The LLM CLI tool now supports self-hosted language models via plugins](https://simonwillison.net/2023/Jul/12/llm/)
- [Accessing Llama 2 from the command-line with the llm-replicate plugin](https://simonwillison.net/2023/Jul/18/accessing-llama-2/)
- [Run Llama 2 on your own Mac using LLM and Homebrew](https://simonwillison.net/2023/Aug/1/llama-2-mac/)
- [Catching up on the weird world of LLMs](https://simonwillison.net/2023/Aug/3/weird-world-of-llms/)
- [LLM now provides tools for working with embeddings](https://simonwillison.net/2023/Sep/4/llm-embeddings/)
- [Build an image search engine with llm-clip, chat with models with llm chat](https://simonwillison.net/2023/Sep/12/llm-clip-and-chat/)
- [Many options for running Mistral models in your terminal using LLM](https://simonwillison.net/2023/Dec/18/mistral/)
## Installation
Install this tool using `pip`:
```bash
pip install llm
```
Or using [Homebrew](https://brew.sh/):
```bash
brew install llm
```
[Detailed installation instructions](https://llm.datasette.io/en/stable/setup.html).
## Getting started
If you have an [OpenAI API key](https://platform.openai.com/api-keys) you can get started using the OpenAI models right away.
As an alternative to OpenAI, you can [install plugins](https://llm.datasette.io/en/stable/plugins/installing-plugins.html) to access models by other providers, including models that can be installed and run on your own device.
Save your OpenAI API key like this:
```bash
llm keys set openai
```
This will prompt you for your key like so:
```
Enter key: <paste here>
```
Now that you've saved a key you can run a prompt like this:
```bash
llm "Five cute names for a pet penguin"
```
```
1. Waddles
2. Pebbles
3. Bubbles
4. Flappy
5. Chilly
```
Read the [usage instructions](https://llm.datasette.io/en/stable/usage.html) for more.
## Installing a model that runs on your own machine
[LLM plugins](https://llm.datasette.io/en/stable/plugins/index.html) can add support for alternative models, including models that run on your own machine.
To download and run Mistral 7B Instruct locally, you can install the [llm-gpt4all](https://github.com/simonw/llm-gpt4all) plugin:
```bash
llm install llm-gpt4all
```
Then run this command to see which models it makes available:
```bash
llm models
```
```
gpt4all: all-MiniLM-L6-v2-f16 - SBert, 43.76MB download, needs 1GB RAM
gpt4all: orca-mini-3b-gguf2-q4_0 - Mini Orca (Small), 1.84GB download, needs 4GB RAM
gpt4all: mistral-7b-instruct-v0 - Mistral Instruct, 3.83GB download, needs 8GB RAM
...
```
Each model file will be downloaded once the first time you use it. Try Mistral out like this:
```bash
llm -m mistral-7b-instruct-v0 'difference between a pelican and a walrus'
```
You can also start a chat session with the model using the `llm chat` command:
```bash
llm chat -m mistral-7b-instruct-v0
```
```
Chatting with mistral-7b-instruct-v0
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
>
```
## Using a system prompt
You can use the `-s/--system` option to set a system prompt, providing instructions for processing other input to the tool.
To describe how the code in a file works, try this:
```bash
cat mycode.py | llm -s "Explain this code"
```
## Help
For help, run:
llm --help
You can also use:
python -m llm --help
|
