projectmoon@lemm.eeOPtoChatGPT@lemmy.world•What happened to GPT -4o Censorship This Weekend?
4·
3 months agoThe only problem I really have, is context size. It’s harder to get larger than 8k context size and maintain decent generation speed with 16 GB of VRAM and 16 GB of RAM. Gonna get more RAM at some point though, and hope ollama/llamacpp gets better at memory management. Hopefully the distributed running from llamaccp ends up in ollama.
Anyone have any suggestions for bulk options in the Netherlands?