{"id":307,"date":"2025-04-23T22:19:53","date_gmt":"2025-04-24T05:19:53","guid":{"rendered":"https:\/\/www.lucaswilliams.net\/?p=307"},"modified":"2025-04-23T22:19:56","modified_gmt":"2025-04-24T05:19:56","slug":"deploying-deepseek-r1-in-ubuntu","status":"publish","type":"post","link":"https:\/\/www.lucaswilliams.net\/index.php\/2025\/04\/23\/deploying-deepseek-r1-in-ubuntu\/","title":{"rendered":"Deploying Deepseek R1 in Ubuntu"},"content":{"rendered":"\n<p>Hello everyone! Hope you all have been well. I&#8217;ve been messing around with AI and different models for my job as we are implementing AI in our software. <\/p>\n\n\n\n<p>I wanted to learn more about this, and it just so happened that Deepseek R1 was announced and I decided to start there. I originally installed this on my Macbook Pro and I installed a smaller model, and for the hardware, it worked well. However, my son needed the laptop so that he could record music so I restored it back to MacOS and am now using my old Linux laptop that I used when I was at Canonical. This laptop is a beast. It&#8217;s a little on the older side, but here&#8217;s what it has under the hood:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Intel 7th Gen Core i7 processor, 8 core, 3.8 gHz<\/li>\n\n\n\n<li>32 GB DDR4 Memory<\/li>\n\n\n\n<li>256GB SSD<\/li>\n\n\n\n<li>1TB HDD<\/li>\n\n\n\n<li>Nvidia Geforce GTX 1050 with 4GB vRAM<\/li>\n<\/ul>\n\n\n\n<p>So, these are the steps that I did to install Deepseek R1 and Open-WebUI as a docker container on my laptop for testing.<\/p>\n\n\n\n<p>First thing I did was install Ollama, which is the LLM from Meta that works with Deepseek R1 Models.<\/p>\n\n\n\n<p>First thing, you need to download and install Ollama. To do this all you need to do is run the following command:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>curl -fsSL https:\/\/ollama.com\/install.sh | sh<\/code><\/pre>\n\n\n\n<p>After this, I had to add an Environment variable to the systemd service. The systemd service is located in <code>\/etc\/systemd\/system\/ollama.service<\/code><\/p>\n\n\n\n<p>Under the <code>[Service]<\/code> section, add the following:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Environment=\"OLLAMA_HOST=0.0.0.0\"<\/code><\/pre>\n\n\n\n<p>This will allow Ollama to listen and serve on all clients. Since I use Docker this works best. I kept running into issue getting Open-WebUI to connect to my Deepseek model without doing this.<\/p>\n\n\n\n<p>Next, you need to reload the daemon and the Ollama Service:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo systemctl daemon-reload\nsudo systemctl restart ollama.service<\/code><\/pre>\n\n\n\n<p>Now, we need to load the model. I use the 8 Billion Parameter model since my laptop can handle that fairly easily. To load this model use the following command:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ollama run deepseek-r1:8b<\/code><\/pre>\n\n\n\n<p>There are other models you can use depending on your system. The 1.5 billion parameter is the smallest, and works farily well on most systems. I ran this model on Raspberry Pi&#8217;s and on my Mac Laptop with 16GB of and no GPU, and it ran well. To see the different models, you can check out the details on Ollama&#8217;s website here:<\/p>\n\n\n\n<p><a href=\"https:\/\/ollama.com\/library\/deepseek-r1\">https:\/\/ollama.com\/library\/deepseek-r1<\/a><\/p>\n\n\n\n<p>You will be dropped in to the Ollama shell where you can interact with the model here. To exit, just type <code>\/bye<\/code> in the prompt and you will be back at the Linux shell.<\/p>\n\n\n\n<p>Next, we need to install a nice Web front end. I use Open-WebUI since it works like ChatGPT, and is super simple to setup. <\/p>\n\n\n\n<p>I use Open-WebUI as a Docker container on my laptop to keep it nice and clean. If I want to disable and stop using this, I can remove the container and my system is nice and clean. Plus updating the web front end is really easy with Docker containers.<\/p>\n\n\n\n<p>Make sure you install Docker on your machine. You can use Snaps or Apt. I followed the instructions on Docker website. It&#8217;s pretty straight forward. After you install Docker, and add yourself to the docker group. After that, log out and log back in so that the group membership gets applied.<\/p>\n\n\n\n<p>I also had to install the Nvidia Container Toolkit so that I could use the GPU in my containers. To do this run the following command to add the repo to Ubuntu and then use Apt to install:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>curl -fsSL https:\/\/nvidia.github.io\/libnvidia-container\/gpgkey | sudo gpg --dearmor -o \/usr\/share\/keyrings\/nvidia-container-toolkit-keyring.gpg \\\n  &amp;&amp; curl -s -L https:\/\/nvidia.github.io\/libnvidia-container\/stable\/deb\/nvidia-container-toolkit.list | \\\n    sed 's#deb https:\/\/#deb &#91;signed-by=\/usr\/share\/keyrings\/nvidia-container-toolkit-keyring.gpg] https:\/\/#g' | \\\n    sudo tee \/etc\/apt\/sources.list.d\/nvidia-container-toolkit.list<\/code><\/pre>\n\n\n\n<p>Next, we need to update the sources and install the toolkit:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt update\nsudo apt install -y nvidia-container-toolkit<\/code><\/pre>\n\n\n\n<p>Next, we need to restart the Docker daemon so it uses the toolkit:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo systemctl restart docker<\/code><\/pre>\n\n\n\n<p>Once that has been completed, we need to pull the container:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>docker pull ghcr.io\/open-webui\/open-webui:cuda<\/code><\/pre>\n\n\n\n<p>After this, run the following command to start the container and have it run at startup:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:\/app\/backend\/data --name open-webui --restart always --gpus all ghcr.io\/open-webui\/open-webui:cuda<\/code><\/pre>\n\n\n\n<p>Now, open your web browser, and point it to:<\/p>\n\n\n\n<p><a href=\"http:\/\/localhost:3000\">http:\/\/localhost:3000<\/a><\/p>\n\n\n\n<p>On the landing page, setup a new Admin user and you done. Select your model from the pull down in the top left corner. Ask your new chatbot a question and your done.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hello everyone! Hope you all have been well. I&rsquo;ve been messing around with AI and different models for my job as we are implementing AI in our software. I wanted to learn more about this, and it just so happened that Deepseek R1 was announced and I decided to start there. I originally installed this [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[144,146,37,147,145,6],"tags":[139,148,21,142,124,55,143,140,141,22,5],"class_list":["post-307","post","type-post","status-publish","format-standard","hentry","category-ai","category-deepseek","category-howto","category-nvidia","category-ollama","category-ubuntu","tag-ai","tag-chatbot","tag-configure","tag-deepseek-r1","tag-docker","tag-howto","tag-nvidia","tag-ollama","tag-open-webui","tag-setup","tag-ubuntu"],"_links":{"self":[{"href":"https:\/\/www.lucaswilliams.net\/index.php\/wp-json\/wp\/v2\/posts\/307","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lucaswilliams.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lucaswilliams.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lucaswilliams.net\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lucaswilliams.net\/index.php\/wp-json\/wp\/v2\/comments?post=307"}],"version-history":[{"count":1,"href":"https:\/\/www.lucaswilliams.net\/index.php\/wp-json\/wp\/v2\/posts\/307\/revisions"}],"predecessor-version":[{"id":308,"href":"https:\/\/www.lucaswilliams.net\/index.php\/wp-json\/wp\/v2\/posts\/307\/revisions\/308"}],"wp:attachment":[{"href":"https:\/\/www.lucaswilliams.net\/index.php\/wp-json\/wp\/v2\/media?parent=307"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lucaswilliams.net\/index.php\/wp-json\/wp\/v2\/categories?post=307"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lucaswilliams.net\/index.php\/wp-json\/wp\/v2\/tags?post=307"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}