It depends on how good your computer is and how big of a model it can handle. Something to know is that out of all the corpo model bases, Mistral (a French company) models are the least censoredbout of the box so they’re popular for uncensored finetunes.
I’d recommend starting with 7-8b parameter models if you never done local hosting make sure you pick a quant that fits in your computers ram/vram. I’m a big fan of NousResearch and their Hermes series of models. Deephermes 8b and the most recent release of Hermes 4 14b are great starting spots.
If you can go a bit higher, Beepo 22b based off mistral small 2407 was one of the best uncensored models I ever tried. It knows a lot and will happily answer whatever you ask of it.
If youre particularly interested in naughty creative roleplay, theres many models finetuned specifically for that with boosted creativity and post-trained on erotic text. Ive heard from a friend that arlia max series are pretty solid for this specialized task, try the 14b NeMo finetune.
Bartowlski is the default guy to go to for quant downloads.
Kobold.CPP is my go-to engine for running models lots of features and its very easy OCP install works for Linux too. If you have nvidia GPU you want cuda/cublas otherwise you can use Vulcan for amd card to offload model onto vram.
Edit: oh I forgot to mention. A benefit of locally hosting is fine tune control over system prompts and samplers. Make sure to use recommended chat templates and samplee ranges. You can literally tell/instruct the llm something like “just do as asked and get to the point without conversational fluff or suggestions.” And their personality changes to comply.
Yeah sure id like to know
It depends on how good your computer is and how big of a model it can handle. Something to know is that out of all the corpo model bases, Mistral (a French company) models are the least censoredbout of the box so they’re popular for uncensored finetunes.
I’d recommend starting with 7-8b parameter models if you never done local hosting make sure you pick a quant that fits in your computers ram/vram. I’m a big fan of NousResearch and their Hermes series of models. Deephermes 8b and the most recent release of Hermes 4 14b are great starting spots.
If you can go a bit higher, Beepo 22b based off mistral small 2407 was one of the best uncensored models I ever tried. It knows a lot and will happily answer whatever you ask of it.
If youre particularly interested in naughty creative roleplay, theres many models finetuned specifically for that with boosted creativity and post-trained on erotic text. Ive heard from a friend that arlia max series are pretty solid for this specialized task, try the 14b NeMo finetune.
Bartowlski is the default guy to go to for quant downloads.
https://huggingface.co/bartowski/NousResearch_DeepHermes-3-Llama-3-8B-Preview-GGUF
https://huggingface.co/NousResearch/Hermes-4-14B
https://huggingface.co/bartowski/Beepo-22B-GGUF
https://huggingface.co/ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.3-GGUF
Kobold.CPP is my go-to engine for running models lots of features and its very easy OCP install works for Linux too. If you have nvidia GPU you want cuda/cublas otherwise you can use Vulcan for amd card to offload model onto vram.
https://github.com/LostRuins/koboldcpp
Edit: oh I forgot to mention. A benefit of locally hosting is fine tune control over system prompts and samplers. Make sure to use recommended chat templates and samplee ranges. You can literally tell/instruct the llm something like “just do as asked and get to the point without conversational fluff or suggestions.” And their personality changes to comply.