{"id":9831,"date":"2024-04-29T09:45:49","date_gmt":"2024-04-29T07:45:49","guid":{"rendered":"https:\/\/via-internet.de\/blog\/?p=9831"},"modified":"2025-09-16T11:53:32","modified_gmt":"2025-09-16T09:53:32","slug":"working-with-local-llms","status":"publish","type":"post","link":"https:\/\/via-internet.de\/blog\/2024\/04\/29\/working-with-local-llms\/","title":{"rendered":"Working with local LLMs"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">List of Frameworks and Tools<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/gpt4all.io\">gpt4all<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/ollama.com\/\">Ollama<\/a> (<a href=\"https:\/\/github.com\/ollama\/ollama\">Github<\/a>)<\/li>\n\n\n\n<li><a href=\"https:\/\/huggingface.co\/\">Hugging Face<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/python.langchain.com\/\">LangChain<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/llm.datasette.io\/en\/stable\/\">LLM<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/github.com\/lmstudio-ai\">LM Studio<\/a><\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Ollama<\/h2>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">curl -fsSL https:\/\/ollama.com\/install.sh | sh<\/pre>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"> ollama run llama2<\/pre>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">LLM<\/h2>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">pip install llm\nllm install llm-gpt4all\nllm -m the-model-name \"Your query\"<\/pre>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">llm aliases set falcon ggml-model-gpt4all-falcon-q4_0\nllm                 -m ggml-model-gpt4all-falcon-q4_0 \"Tell me a joke about computer programming\"<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Working with Hugging Face<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">List of Models<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/huggingface.co\/models?sort=trending&amp;search=gguf\">https:\/\/huggingface.co\/models?sort=trending&amp;search=gguf<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Install models with Ollama<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/huggingface.co\/docs\/hub\/ollama\">https:\/\/huggingface.co\/docs\/hub\/ollama<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The snippet would be in format:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">ollama run hf.co\/{username}\/{repository}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Please note that you can use both&nbsp;<code>hf.co<\/code>&nbsp;and&nbsp;<code>huggingface.co<\/code>&nbsp;as the domain name.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here are some models you can try:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">ollama run hf.co\/bartowski\/Llama-3.2-1B-Instruct-GGUF\nollama run hf.co\/mlabonne\/Meta-Llama-3.1-8B-Instruct-abliterated-GGUF\nollama run hf.co\/arcee-ai\/SuperNova-Medius-GGUF\nollama run hf.co\/bartowski\/Humanish-LLama3-8B-Instruct-GGUF<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>List of Frameworks and Tools Ollama LLM Working with Hugging Face List of Models https:\/\/huggingface.co\/models?sort=trending&amp;search=gguf Install models with Ollama https:\/\/huggingface.co\/docs\/hub\/ollama The snippet would be in format: Please note that you can use both&nbsp;hf.co&nbsp;and&nbsp;huggingface.co&nbsp;as the domain name. Here are some models you can try:<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_crdt_document":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-9831","post","type-post","status-publish","format-standard","hentry","category-allgemein"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/posts\/9831","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/comments?post=9831"}],"version-history":[{"count":8,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/posts\/9831\/revisions"}],"predecessor-version":[{"id":10423,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/posts\/9831\/revisions\/10423"}],"wp:attachment":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/media?parent=9831"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/categories?post=9831"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/tags?post=9831"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}