Goal:

LLM

• Works with GPU
• Runs locally and uses the GPU.
• (optional) can be used for vibe coding with IntelliJ.
• is accessible via web browser.
• has a WhatsApp relay. (Another phone number recommended)

Prepare

• Disable all radio devices on the Dell in the BIOS.
• Attach a cable network
  I tried first the installation also without a network attached, but this just made my life harder.
• Download Ubuntu 22.04 Server, even though a newer version is available.
  The error I ran into was :fail curtin command block-meta dev/pve/data not an existing file of block device…
  I tried several things. From partitioning by hand over vgremove, pvremove, wipefs and some other solutions unsuccessfully.
  The older installer is just not so picky as the one from 24.04.

Installation

Install Ubuntu LTS 22.04 Server:

• Choose Ubuntu Server with the HWE Kernel at boot
• Check “Ubuntu Server (minimized)”
• Check “Search for third-party drivers”
• NO LVM, I needed to disable it.
  • I also edited the automatically made partitions by reducing the size of the main partition in favor of having a swap partition of 16G.
• Check “Install OpenSSH Server”

Install GPU drivers

• update repository
  sudo apt update
• check drivers
  sudo ubuntu-drivers devices

vendor : NVIDIA Corporation
model : GM107GLM [Quadro M1200 Mobile]
driver : nvidia-driver-535-server – distro non-free
driver : nvidia-driver-470 – distro non-free
driver : nvidia-driver-450-server – distro non-free
driver : nvidia-driver-535 – distro non-free
driver : nvidia-driver-580 – distro non-free recommended
driver : nvidia-driver-580-server – distro non-free
driver : nvidia-driver-470-server – distro non-free
driver : nvidia-driver-390 – distro non-free
driver : nvidia-driver-418-server – distro non-free
driver : nvidia-driver-545 – distro non-free
driver : nvidia-driver-570 – distro non-free
driver : nvidia-driver-570-server – distro non-free
driver : xserver-xorg-video-nouveau – distro free builtin

Install the recommended one:

• remove all old nvidia packages.
  sudo apt purge 'nvidia*'
• get sources to build the dkms
  sudo apt install build-essential linux-headers-$(uname -r)
• get new gcc to prevent later compilation problems with nvcc and llama.cpp
  sudo apt install gcc-12 g++-12
• install the driver
  sudo apt install nvidia-driver-580

reboot.

Then check the installation:
nvidia-smi -> It should show your card.

Sun Feb  1 09:05:38 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.09             Driver Version: 580.126.09     CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  Quadro M1200                   Off |   00000000:01:00.0 Off |                  N/A |
| N/A   46C    P8            N/A  /  200W |       2MiB /   4096MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |

Install llama.cpp

Prepare

• Get necessary software
  sudo apt install git
sudo apt install cmake
sudo apt install dialog
sudo apt install open-ssl

• Install a newer version of the CUDA toolkit (we need version 12, but the repo has version 11)
  wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/3bf863cc.pub
sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /"
sudo apt update
sudo apt install cuda-toolkit-12-5 This will take a while; grab a coffee or clean your room.

• Assign CUDA to the paths (put those commands eventually also into .bashrc):

export CUDA_HOME=/usr/local/cuda-12.5
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH

• Test the installed version

which nvcc

/usr/local/cuda-12.5/bin/nvcc

nvcc --version

nvcc: NVIDIA ® Cuda compiler driver
Copyright © 2005-2024 NVIDIA Corporation
Built on Thu_Jun__6_02:18:23_PDT_2024
Cuda compilation tools, release 12.5, V12.5.82
Build cuda_12.5.r12.5/compiler.34385749_0

• Clone the repository
  git clone https://github.com/ggerganov/llama.cpp
• Check the following sites for the actual CUDA installation method:
  https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md
• Build a llama
  cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_C_COMPILER=gcc-12 -DCMAKE_CXX_COMPILER=g++-12
cmake --build build --config Release -j 8 j8 is the amount of parallel jobs that will be used. It speeds the build a lot.

Go cooking or make something meaningful. This will take a long time!

Install Model

• Switch to the home folder
  cd
• Create models folder
  mkdir -p models
• Download the 7b model
  curl -L -o openhermes-2.5-mistral-7b.Q4_K_M.gguf \ https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF/resolve/main/openhermes-2.5-mistral-7b.Q4_K_M.gguf

Start LLM

My Dell Precision 7520 has:

• 32 GB RAM dual channel
• 16 GB RAM single channel

Depending on the context size, the communication will be faster or slower.
--ctx-size 4096

• On 4 GB VRAM (Quadro M1200), this was a good starting point.
  --n-gpu-layers 32

~/llama.cpp/build/bin/llama-server   -m ~/models/openhermes-2.5-mistral-7b.Q4_K_M.gguf \
   --n-gpu-layers -1 \
   --host 0.0.0.0
   --port 8080

All the settings hardly depend on the model that is used. I tried different models with more or less success. However, my approach is to have 2 services, one for a smaller model and one for a bigger model. 3B models run well on the GPU, which means that most of the layers are in VRAM.
For the bigger ones, I set --n-gpu-layers 0 to prevent them from running on the GPU. They then live in my RAM and are processed in the CPU.

Despite the one used for this article, I currently have the following models running:

• cpu_model.gguf -> glm-4.7-flash-claude-4.5-opus.q4_k_m.gguf
• gpu_model.gguf -> OwlLM2-e2b.Q8_0.gguf

I recommend putting the server into a service and using systemctl to start it.

Llama.cpp brings its own web UI with it. It is available on the specified port.

I get speeds of about 6-12 tokens per second. On both.

I also tried https://github.com/openclaw/openclaw for the WhatsApp relay, but this doesn’t work well. OpenClaw needs a context size of at least 16k, which is not manageable by this system, or at least not for the GPU model. However, depending on the models, I had to wait at least 3 minutes for just a “Hello.” As an alternative, I now use https://github.com/HKUDS/nanobot which does not need such a big context size.

At the end I achieved all open points. The part with IntelliJ is still ongoing.

Foremost, it has some strange issues. But if you want to give it a try, go ahead.
xRdp will create its own X11 instance, and as far as I know, brings its own X11 server. However, this comes with a caveat. Every program I tried could only be started either on the host or on the client, but never on both at the same time.

• Sometimes the keyboard does not work well; keys don’t do what they should do, or the whole keyboard does not work at all.
• The resolution is not changeable (my workaround is to scale it to 75%).
• I have no sound, but I also have not configured it; therefore, this could be a configuration issue.

Although cosmic and xwayland ignore the xorg configuration files, the xserver from xrdp may use them. So, I will try it and extend this article with my findings.

My current system:

Host:

• PopOS 24.04 with Cosmic
• Kernel 6.16.3-76061603-generic #202508231538~1759252525~24.04~c08ae99 SMP PREEMPT_DYNAMIC Tue S x86_64 x86_64 x86_64 GNU/
• xrdp 0.9.24

Remote System:

• Windows 11 with mRemoteNG
• PopOS 22.04 LTS with Remmina

Installation Guide

Update System (optional):

sudo apt update
sudo apt list --upgradable
sudo apt upgrade

Install xRDP
sudo apt install xrdp

Configure xRDP

echo "cosmic-session" > ~/.xsession
systemctl enable xrdp
systemctl start xrdp

Use this formula for pixel movement, they told me. It will work, they told me. Yep, no.
After all, it makes totally sense why this can not work, but later more.

fun rotateCoordinate(point: Coordinate, center: Coordinate, angleDegrees: Double): Coordinate {
    val angleRadians = toRadians(angleDegrees)

    val dx = point.x – center.x
    val dy = point.y – center.y

    val rotatedX = dx * cos(angleRadians) – dy * sin(angleRadians)
    val rotatedY = dx * sin(angleRadians) + dy * cos(angleRadians)

    return Coordinate(center.x + rotatedX, center.y + rotatedY)
}

This is probably the wrong formula for the task, but for now, I just want to write down my notes.

However, after it just looked horrible in my game, I started to make it more easily visible. What we developers do in such cases—yes, we write some more code. And this is the result.

On the left side, I took the pixel coordinate of the previously rotated point and rotated it by angle 1.0.
On the right side, I took the start coordinate and rotated it by the full range.
So, whereas the rotation on the left side is, e.g., from degree 53 to 54, on the right side it is from 0 to 54.

So, what is happening here? With every rotation, the rotated point must be aligned to the coordinate system; therefore, it loses precision. It is now located somewhere else than it should be and will land somewhere else due to the next calculation, which also loses some precision. But keeping the starting point and calculating the full angle solves the issue.

But still—it could be that I am using the wrong formula, so I will continue researching it.

A few months ago, I've got a kernel update which broke my display link driver. Therefore, my second monitor stopped showing anything. However, today it broke again.

Starting DisplayLink Driver Service...
Mär 23 09:14:49 pop-os sh[140421]: modprobe: FATAL: Module evdi not found in directory /lib/modules/6.12.10-76061203-generic
Mär 23 09:14:49 pop-os sh[140447]: Kernel preparation unnecessary for this kernel. Skipping...
Mär 23 09:14:49 pop-os sh[140447]: Building module:
Mär 23 09:14:49 pop-os sh[140447]: cleaning build area...
Mär 23 09:14:51 pop-os sh[140447]: make -j8 KERNELRELEASE=6.12.10-76061203-generic all INCLUDEDIR=/lib/modules/6.12.10-76061203-generic/build/include KVERSION=6.12.10-76061203-generic DKMS_BUILD=1...(bad exit status: 2)
Mär 23 09:14:52 pop-os sh[140905]: ERROR (dkms apport): binary package for evdi: 1.14.2 not found
Mär 23 09:14:52 pop-os sh[140963]: Error! Bad return status for module build on kernel: 6.12.10-76061203-generic (x86_64)
Mär 23 09:14:52 pop-os sh[140963]: Consult /var/lib/dkms/evdi/1.14.2/build/make.log for more information.
Mär 23 09:14:52 pop-os systemd[1]: displaylink-driver.service: Control process exited, code=exited, status=10/n/a
Mär 23 09:14:52 pop-os systemd[1]: displaylink-driver.service: Failed with result 'exit-code'.
Mär 23 09:14:52 pop-os systemd[1]: Failed to start DisplayLink Driver Service.
Mär 23 09:14:52 pop-os systemd[1]: displaylink-driver.service: Consumed 10.704s CPU time.
Mär 23 09:14:57 pop-os systemd[1]: displaylink-driver.service: Scheduled restart job, restart counter is at 229.
Mär 23 09:14:57 pop-os systemd[1]: Stopped DisplayLink Driver Service.

It turned out that the module is too old for this kernel. I remember last time I needed to do some manual work to get it working. OK, I downloaded the sources from https://github.com/DisplayLink/evdi and extracted them. Then I naively started make:

:~/evdi-1.14.8$ make

And, yes it failed.

No package 'libdrm' found
evdi_lib.c:6:10: fatal error: libdrm/drm.h: Datei oder Verzeichnis nicht gefunden
    6 | #include 
      |          ^~~~~~~~~~~~~~
compilation terminated.

I installed the developer packaged by:

:~/evdi-1.14.8$ apt install libdrm-dev

And did another make. This time it ran through without issues. After that I discovered the module folder which contained an installation script, so I started that too:

:~/evdi-1.14.8/module$ sudo ./dkms_install.sh 

journalctl -f

stopped throwing errors and my screen as also my other devices started working again.

Puh, that was a day.

I was changing things on a drawing and then saved it accidentally as another existing file. And worse, I accepted the dialog for overwriting. Yeah. Fxxk.

Ok. I did a huge mistake. But, still, maybe there is a way to get back the file. Luckily, the software I use for drawing 3d models uses an JSON formatted file and I remembered at least one word that shall be in the file(LedHalterArm).

After a bit of research, I found a way to read out what’s still on the hard disk.
First, I moved the file that I have overwritten and gave it a new name:

mv GameboyBeleuchtungV5.d3ddoc GameboyBeleuchtungV5.d3ddoc.bak

That shall remove the link to the part of the hard disk.

The next task was to find the position on hard disk, for that a unique string should be used and makes the search easier.

grep -a -b "LedHalterArm" /dev/sda3

Unfortunately, I already exported this as STL file so it gave me mostly unrelated results but some of them seemed to be possible. I got some results like

123123123: "name": "LedHalterArm"

Ok, let’s try again with the more specific search term.

grep -a -b "\"name\": \"LedHalterArm\""

Here, we are, I’ve got 3 results and took the last one as starting point.

48020388426:                "name": "LedHalterArm"
94465072989:                "name": "LedHalterArm"
94465073068:            "name": "LedHalterArm",

Let us see what is there:

dd if=/dev/sda3 count=16 skip=$(expr 94465073068 / 512)

Yey :) I’ve got at least the file contents from the point of “LedHalterArm” to the end. From here now I have to go back on the hard disk until I reach the beginning of the file. This process took the most of the time, because I did it step by step.

I reduced the expr value by 512 and increased the count by 1:

dd if=/dev/sda3 count=11 skip=$(expr 94465072556 / 512)

Later on I increased the steps by factor 10 then factor 50 and finally I got the whole text. For easier handling I added a clear command to the front, with that I didn’t need to care about where the beginning is of my result. Just scrolled up.

clear && dd if=/dev/sda3 count=336 skip=$(expr 94464908716 / 512)

So, I copied the relevant part from the console into a file and saved it. It worked.