Ollama на удалённом сервере VDS

Информация о материале
Родительская категория: Заметки
Категория: Компьютерная повседневность
Просмотров: 8
Звезда не активнаЗвезда не активнаЗвезда не активнаЗвезда не активнаЗвезда не активна
 
1. Установка (Linux):
On VDS under root:
curl -fsSL https://ollama.com/install.sh | sh

2. Использование на Windows удалённо:

BEST: SSH tunnel (safe, recommended)

On Windows PowerShell:

ssh -L 11434:127.0.0.1:11434 root@ВАЩ_REMOTE_IP
 

Then in another terminal:

Invoke-RestMethod -Uri "http://127.0.0.1:11434/api/generate" `
  -Method POST `
  -ContentType "application/json" `
  -Body '{"model":"llama3","prompt":"Tell me about Olomouc"}'
 

👉 This works without opening any firewall ports.
 

Для использования в своих HTML (CORS fix)

На сервере:
1. Run

sudo systemctl edit ollama

2. The file will be empty with some comments. Type this EXACTLY between the comment lines:

[Service]
Environment="OLLAMA_ORIGINS=*"

I### Anything between here and the comment below will become the new contents of the file


[Service]
Environment="OLLAMA_ORIGINS=*"

### Lines below this comment will be discarded

3. Save and exit — in nano: Ctrl+O, Enter, Ctrl+X. In vim: :wq

4. Reload and run services:

sudo systemctl daemon-reload
sudo systemctl restart ollama

5. Verify it took effect:

sudo systemctl show ollama | grep ORIGINS

You should see OLLAMA_ORIGINS=* in the output.

Локально:
1. PowerShell: 

ssh -L 11434:127.0.0.1:11434 root@YOUR_REMOTE_IP

2. Проверка в браузере:
Open URL: 

http://127.0.0.1:11434/api/tags