So I am wondering if anyone has had success with AI, which AI, free or paid, and maybe if it is some AI that you are hosting yourself (and, if so, is it linux based?)?
I host locally using llama.cpp on a i9 64 GB ram Dell 5280 workstation and two
RTX 3060 GPUs with 12 GB each. llama.cpp utilizes both GPUs to give me a combined 24 GB of vram. I created a new door called Quantasia that uses both cloud and local LLMs, and I like the response times of llama.cpp that Quantasi
achieves when hitting a local bot based on it.
| Sysop: | Amessyroom |
|---|---|
| Location: | Fayetteville, NC |
| Users: | 60 |
| Nodes: | 6 (1 / 5) |
| Uptime: | 491598:22:19 |
| Calls: | 823 |
| Calls today: | 1 |
| Files: | 1,291 |
| D/L today: |
9 files (8,192P bytes) |
| Messages: | 257,581 |