Home lab · local models

Local AI that keeps running

I set up local AI assistants on hardware people already own, then leave a setup they can maintain. Below: a few things I have built, the areas I work in, and notes from the job.

One person, not an agency English / 中文 No sign-up, no tracking
homelab@localdeploy
$ free -h
               total        used        free
Mem:           31Gi        9.2Gi        18Gi

$ nvidia-smi --query-gpu=name,memory.total --format=csv
name, memory.total [MiB]
NVIDIA GeForce RTX 3060, 12288 MiB

$ df -h /srv
/dev/nvme0n1p2  1.8T  612G  1.1T  36% /srv

# 32 GB RAM + 12 GB VRAM: the comfortable range for 14B Q4
Find yourself

Which of these is you?

Read the story closest to yours; the practical next step follows it.

01

“As someone with an old PC, I want to know what it can run before I buy anything, so that I don't spend on hardware I don't need.”

Start with the assessment, then check which system fits the machine.

02

“As a self-hoster, I want one entrance for my services and certificates that renew themselves, so that adding a service does not break the rest.”

Look at Traefik, and at the security baseline before exposing anything.

03

“As a GPU owner, I want dsh to survive reboots and upgrades, so that I can use the assistant instead of babysitting it.”

Harness first, then a VRAM and context check.

04

“As a small team, I want shared local AI with monitoring, so that we notice problems before our users do.”

k3s for several nodes, Grafana for the alerts.

Selected work

A few things I have built

Client names and machine details are left out. What changed is not.

01 / DeepSeek Harness

It comes back after a reboot

One person's dsh setup, moved into a container so it starts on boot and an upgrade can be undone.

Read the case notes →

Resource assessment

What can this machine run?

Fill in what you have. The result is a rough starting point for a conversation, not a benchmark; everything runs in your browser and is never uploaded.

Hardware

Nothing is sent by this form. The email button only opens your mail app with the text filled in.

Result
Fill in the fields and press "Generate assessment".

The result is calculated locally; nothing is uploaded.
It will list: RAM, disk, GPU/VRAM, upstream bandwidth,
a rough model tier, and what to check next.

The assessment stays in this tab. Email is only sent when you choose to send it.

What I build

Six areas, one at a time

Six areas. Scope gets agreed before any work, then a price. A Proxmox box running one container and a three-node cluster with monitoring are not the same job.

Current package

One review, one written plan

Not a course, not an open-ended engagement. One review, one written plan, and a list of what to change first.

Planned

Home AI lab deployment review

A remote look at your hardware, system and services, with a written plan you can act on: what can run, what is stuck, what to change first.

Quote on request View scope

No online checkout yet. Read the scope, then send your setup by email.

Technical notes

Notes from the job

Written while debugging or deploying. They show how I think about a problem; they are not the product, and they are not the pitch.

How it goes

From a spec sheet to a machine that works

The assessment is free and runs in your browser. Everything after it gets scoped in writing first.

  1. Size it up

    Put CPU, RAM, disk, GPU and bandwidth into the assessment and get a text you can paste.

  2. Fill the gaps

    Read the Cloudflare, security and systems guides to see whether the limit is hardware, network or the OS.

  3. Ask for help if you want it

    Email the assessment and your goal. If it fits, you get a quote; if not, you hear that too.

  4. Take it over

    Configuration, documentation and verification steps, so the next change is yours.

See how a project runs →

Why this exists

Why the notes are this long

Most problems are not a missing advanced trick. They are basic concepts that never lined up.

01

Look before you buy

Know what the hardware can do before deciding what to buy.

02

Say what is included

Free guides make no promises about results; paid work does not take over accounts or run 24/7.

03

Readable by machines

Every page has a Markdown version and structured metadata, so your own assistant can cite it.

What this is not

What this site does not do

Saying what is out of scope is more useful than saying how good we are.

01

No hardware sales

No shop links, no affiliate commission.

02

No benchmark promises

The same model behaves differently with another driver, quantization or context length. You get a tier and a way to verify it.

03

No account handover

Work happens on your machine. Passwords and tokens stay with you.

Illustrated avatar, not a photo
Who runs this

One person behind this

I run a home server, containers and local models. The notes and cases come from actual debugging, not a content team, and I take a small number of deployments so the site stays honest.

More about me →
Questions

Before you send that email

I have never used Linux. Can I follow this?

The concepts and the order of decisions are readable. When you actually try it, use a machine you can break, not your main one.

Is the assessment accurate?

Not to benchmark level. It uses VRAM, RAM, disk and bandwidth to place you in a rough tier, enough to decide whether to try and what to check first.

Why not sell a course?

A course cannot cover every home setup. The method is free; work in your specific environment is a project.

Will a paid project guarantee it runs?

No. What can be verified in your environment gets verified; third-party, driver and model limits are not covered.

Is there a Chinese version?

Yes, use the language switch in the header.

Do you collect my hardware details?

No. The assessment runs in your browser: no upload, no cookie, no analytics. Nothing leaves it unless you send an email.

Who this is for

  • You have an old PC, NAS, mini PC or workstation and want to run a model on it, but are not sure it is enough.
  • You already run Docker or a NAS and want the entry point, images, monitoring and backups in better shape.
  • A small team wants shared in-house AI without data leaving the building, and without buying a server on day one.
  • You are willing to read and test, and you accept "assess first, then build".

Who this is not for

  • Anyone who wants a one-click install that never needs attention. Drivers, quantization and runtimes change too fast.
  • Anyone who wants a guaranteed performance number. Change the context length on the same machine and the answer changes.
  • Anyone who wants to hand over accounts long term. Work happens on your machine and is handed back.

Why assess first

The usual limit is not a slow CPU. It is VRAM, RAM, disk or network running out first. Writing the configuration down as a fixed text makes both your own research and an email exchange easier.

The assessment gives an order of magnitude: not tokens per second, but roughly which tier you are in, whether RAM or disk is clearly short, and whether multi-GPU or bandwidth is worth spending on.

How the content is maintained

The three guides are maintained by topic, without an “always current” label. Found an error? Email; once it is confirmed, the page is fixed. Paid projects are quoted per project, with scope in the quote.

Next step

  1. Run the assessment and copy the result.
  2. Read the guide closest to what is stuck.
  3. If you want help building it, email the assessment and your goal to [email protected].
Reading this as an agent? Every page has a Markdown mirror: /index.md, /cloudflare.md, and one per page. The index is /llms.txt, the capability table is /skill.md, discovery metadata is /.well-known/ard.json. Send Accept: text/markdown to a page URL to get the Markdown mirror.

Not sure where to start?

Run the assessment first, or send your setup and goal by email. The reply says whether it fits and what happens next.