AI and machine learning

Private AI with Local LLMs: Ollama, Open WebUI and Offline RAG

Run AI on your own hardware so confidential plant data never leaves the building, with Ollama, Open WebUI and offline RAG.

Private AI with Local LLMs: Ollama, Open WebUI and Offline RAG is a free, self-paced online course from EDWartens for engineers at plants, EPC contractors, consultancies, defence and oil and gas operators worldwide who need AI on confidential documents, and the IT and OT staff who run the server. It has 13 modules and takes about 18.1 hours, with written notes, a practice task per module, a practical project and a final assessment of 15 questions (pass mark 60%). Learning costs nothing and needs only a free account. The optional EDWartens Certificate of Completion, verifiable by code, is a one-off US$28.99 for you; it is not a vendor credential. Updated 1 November 2026.

  • Free
  • Intermediate
  • 18.1 hours
  • Self-paced
  • Certificate optional
  • Lab practice in your browser

No card, no trial. The course is free for good. Certificate US$28.99, optional.

Who it is for
Engineers at plants, EPC contractors, consultancies, defence and oil and gas operators worldwide who need AI on confidential documents, and the IT and OT staff who run the server
Jobs
Roles that use this skill: AI engineer, machine learning engineer, data analyst and automation developer. No course guarantees a job.
Tools
Ollama, Open WebUI (in Docker), Python with the ollama, chromadb and pydantic packages, and optionally LM Studio and promptfoo. All are free to download. Hardware: A laptop with 16 GB of RAM runs the small models used in the modules. A GPU with 8 GB or more of memory, or an Apple silicon Mac with 16 GB or more, makes them faster. The sizing module shows how to size a team server; you do not need to buy one.
Updated
Private AI with Local LLMs: Ollama, Open WebUI and Offline RAG course cover
Private AI with Local LLMs: Ollama, Open WebUI and Offline RAG course cover

After this course you can

By the end you will be able to:

  • Explain when a model must run locally
  • Size VRAM, RAM and speed for a model
  • Install Ollama and write Modelfiles
  • Run Open WebUI for a team with roles and groups
  • Build an offline RAG pipeline that cites and refuses

Who it is for, and what you need first

Who it is for

Engineers at plants, EPC contractors, consultancies, defence and oil and gas operators worldwide who need AI on confidential documents, and the IT and OT staff who run the server.

Level: Intermediate. Intermediate. For engineers in plants, EPC contractors, consultancies, defence and oil and gas who want AI on data they cannot paste into a hosted chatbot. You should be comfortable with a command line and able to run a short Python script; Generative AI and LLM Foundations is the natural course before this one.

What you need

Software: Ollama, Open WebUI (in Docker), Python with the ollama, chromadb and pydantic packages, and optionally LM Studio and promptfoo. All are free to download..

Hardware: A laptop with 16 GB of RAM runs the small models used in the modules. A GPU with 8 GB or more of memory, or an Apple silicon Mac with 16 GB or more, makes them faster. The sizing module shows how to size a team server; you do not need to buy one.. What to download, and how.

How this course works

  1. Watch the lessons

    Short video lessons, one module at a time, at your own pace.

  2. Read the notes

    Open the Reading tab for the written notes and diagrams of the module.

  3. Practise in the 3D labOptional

    Try the module in the free 3D lab, in your browser.

  4. Check yourself

    Answer the questions to see what stuck before you move on.

  5. Final assessment

    Pass the final once you have been through the modules.

  6. Certificate, if you want oneOptional

    Optional, with a page anyone can use to verify it.

What you learn

13 modules in order, at your own pace. 18.1 hours of video lessons, with written notes and a practice task in each module.

  1. 1Why run AI locally: confidentiality, export control, cost and offline sites, with lab practice: 33m
  2. 2Open-weight models and their licences: Llama, Qwen, Gemma, Mistral and gpt-oss31m
  3. 3Hardware sizing: VRAM, RAM, CPU-only and quantisation (GGUF, Q4, Q8)57m
  4. 4Installing Ollama: the CLI, environment variables and the Modelfile51m
  5. 5Open WebUI: a private ChatGPT for the team1h 6m
  6. 6The Ollama REST API and Python3h 17m
  7. 7Offline RAG over drawings, specifications and manuals, with lab practice: 54m
  8. 8Structured output: extracting data from datasheets, with lab practice: 22m
Show the other 5 modules
  1. 9Vision models locally: reading nameplates and P&ID snippets28m
  2. 10Benchmarking local against cloud on your own tasks, with lab practice: 21m
  3. 11Air-gapped deployment, updates and access control40m
  4. 12Governance: what may go in, logging and acceptable use, with lab practice: 33m
  5. 13Final assessment6m

Software you need

What to download, where from, what it costs and how to install it. Every link goes to the maker's own site, never a mirror.

Open WebUI runs in Docker from ghcr.io/open-webui/open-webui, so it needs no separate download. On a Linux server use Docker Engine rather than Docker Desktop. LM Studio and promptfoo are optional.

Show what to download (4 tools)

Required

  1. 01

    Ollama

    Ollama Inc.

    Free
    Runs on
    macOS 14 Sonoma or later, Windows 10 or later, Linux
    Account
    None needed

    Ollama is free and open source (MIT licence). Each model you download has its own licence; check it before commercial use.

  2. 02

    Python

    Python Software Foundation

    Free
    Runs on
    Windows (not Windows 7 or earlier), macOS and Linux. Windows builds for x64, 32-bit and Arm64.
    Account
    None needed

    Python is free, open source software. You can use it for learning and for commercial work at no cost.

    Alternatives

  3. 03

    Docker Desktop

    Docker, Inc.

    Free for personal use, education and small businesses
    Runs on
    Windows 10 64-bit version 22H2 (build 19045) or Windows 11 64-bit version 23H2 (build 22631) or later, with WSL 2 and hardware virtualisation on, 8 GB RAM. Also macOS and Linux.
    Account
    None needed

    Free for personal use, education, non-commercial open source projects, and businesses with fewer than 250 employees and less than 10 million US dollars in annual revenue. Larger businesses need a paid subscription. Docker Engine on Linux is open source and is not covered by these terms.

    Alternatives

    • Docker Engine (Linux): Free, open source engine for Linux. Not covered by the Docker Desktop subscription terms.

Optional

Useful, not needed to finish the course.

  1. 04

    Visual Studio Code

    Microsoft

    Free
    Runs on
    Windows 64-bit (supported Windows client versions), macOS (latest and two previous releases), Linux (Ubuntu 20.04, Debian 10, RHEL 8, Fedora 36 or later)
    Account
    None needed
    Size
    Less than 200 MB download, under 500 MB installed

    Free to download and use. Extensions from the Marketplace each have their own licence.

    Official download pagecode.visualstudio.com

Checked against each maker's own page on 27 September 2026. Trial lengths and editions change; the maker's page is the final word.

Practise in the 3D lab

Practise what makes a private AI server useful: answers that cite their source, exact structured output, and personal data removed before anything is shared.

Train and testWatch

Optional. Skip it whenever you like: it never blocks your progress or your certificate.

Course project, about 14 hours

A private team AI server that answers from project specifications with citations, sized, tested and governed

Stand up Ollama and Open WebUI as a team server. Index a project's specification PDFs, make a preset that cites the clause and says NOT FOUND, size the hardware and test it on twenty questions. Then write the acceptable use policy and risk register that let engineers use it on confidential documents.

This project at a glance
  1. 5documents in the pack
  2. 6steps to follow
  3. Laboptional: in the 3D lab
  4. 5things to hand in
  5. 14 hof work, at your pace
  • URSRequirements for the private specification assistant
  • SDSDesign and sizing note for the team AI server
  • ProcedureAcceptable use policy for AI tools
  • Test reportAcceptance test of the specification assistant on project 2607
  • Risk registerRisk register for the office AI server

Do it in the lab: 4 steps.

The project is optional practice. The certificate needs only the modules and the final assessment.

After the course, your next step

Show what you can do, then keep going in the same field.

Sample EDWartens Certificate of Completion for Private AI with Local LLMs: Ollama, Open WebUI and Offline RAG

Sample. The issued certificate carries your name, your final score, a certificate ID and a QR code that opens its verification page.

Optional

A certificate you can verify

Finish the modules and pass the final assessment (15 questions, pass mark 60%) and you can add a certificate with a public verification page. One-off, US$28.99 for you. It is not a vendor credential.

Want a free certificate? Do the matching Free internship: 4 weeks, nothing to buy, with a free certificate of completion.

Hands typing on a laptop at a wooden desk with a notebook, glasses and coffee.
Free · 4 weeks · free certificate

Generative AI and Prompt Engineering

Eight short weeks of how language models work, prompting that gets better answers, and the risks to manage: notes, diagrams, a few videos and worked exercises. Finish weeks 1 to 4 and the final, then claim a free Internship Completion Certificate with your real dates and study hours, and a QR code anyone can scan to verify it. Training, not employment.

See the internship

Roles that use this skill

  • AI engineer
  • Machine learning engineer
  • Data analyst
  • Automation developer

These are the kinds of roles where the skill is used. We make no promise of a job or an interview.

Plain notes

  • Free: the whole course, the notes, the practice tasks, the final assessment and support from practising engineers.
  • Paid, and optional: the course certificate, and the Engineer-verified internship (the Free internship costs nothing).
  • No job promise: the course teaches a skill. It does not place you in a job.
  • Lessons by: Matt Williams, IBM Technology, freeCodeCamp.org, Christian Lempa, NetworkChuck, Tech With Tim, Syntax, Real Python, Sam Witteveen, pixegami, Case Done by AI, Professor Patterns, Tony Kipkemboi, Ian Wootten, Leon van Zyl, Skill Leap AI, Jason Koo, BlueSpork, AppSecEngineer. Independent creators; the lessons stream from their own channels. EDWartens wrote the study plan, notes, practice tasks and assessments.

Matt Williams · IBM Technology · freeCodeCamp.org · Christian Lempa · NetworkChuck · Tech With Tim · Syntax · Real Python · Sam Witteveen · pixegami · Case Done by AI · Professor Patterns · Tony Kipkemboi · Ian Wootten · Leon van Zyl · Skill Leap AI · Jason Koo · BlueSpork · AppSecEngineer. If you are one of these creators and want a lesson removed or credited differently, write to info@edwartens.com.

Practise what you learn in the free 3D lab

Technical work is judged on what you can do: wiring, commissioning, finding a fault. Doing a task yourself teaches more than watching it a second time, and a lab lets you try it safely before you meet real equipment. It does not replace working on real hardware.

  • Practise safely, so a mistake costs nothing.
  • The lab checks your work like a site test, so you know when it is right.
  • Free, in your browser, with nothing to install.

Why learn with EDWartens

What you get with this course, and what comes after it.

  • Free from the first lesson to the final assessment.
  • Practice in a free 3D lab in your browser, with nothing to install.
  • Video lessons with written notes and a project.
  • An optional certificate that anyone can verify online.
  • Free internships to keep practising afterwards.

Common questions

Who is this private AI and local LLM course for?

It is for engineers who want to use AI on documents they cannot paste into a hosted chatbot. That includes process, instrument, electrical and project engineers at EPC contractors, consultancies, plants, defence and public-sector projects, and oil and gas operators worldwide. IT and OT staff who will run the server for them will find it useful too.

Is the course free to learn?

Yes. Every module, the notes, the worked problems, the project and the final assessment are free, and every piece of software used is free to download. The certificate is optional.

What do I need to know first?

You should be comfortable with a command line and able to run a short Python script. If tokens, context windows and RAG are new to you, take Generative AI and LLM Foundations first. After this course, RAG and Chatbots with LangChain goes deeper into retrieval.

What hardware do I need to run local LLMs?

A laptop with 16 GB of RAM is enough for the small models used in the modules. A GPU with 8 GB or more of memory makes them several times faster. So does an Apple silicon Mac with 16 GB or more. Module 3 teaches you to size a team server from the model, context length and number of users. Then you can write the purchase request yourself.

Which software does it use?

Ollama to run the models, Open WebUI in Docker as the team chat interface, and Python with the ollama, chromadb and pydantic packages for the API, offline RAG and extraction modules. LM Studio and promptfoo appear as optional tools. Versions move every few weeks, so the notes say what was current in October 2026 and tell you to check.

Is a local model safe to use on confidential and export-controlled data?

A local model keeps the data on hardware you control, which is what most confidentiality clauses require. It does not make the answers correct. The course teaches the data classes, the locked-down server, the logging and the acceptable use policy. Export-controlled data still needs your security officer's approval for any tool. No AI output reaches a live plant without review and test.

How long does the Private AI with Local LLMs course take?

About 18 hours of video, notes and practice at your own pace, of which about 11 hours is video. The optional team server project takes about 14 hours more.

What certificate does the Private AI with Local LLMs course give?

An EDWartens certificate of completion. It is issued when you finish the modules and pass the 15-question final at 60 percent. It has a number anyone can verify on our site. It is not a credential from Ollama, Open WebUI or any model maker. None of them is affiliated with EDWartens.

Do I have to use the lab?

No. The lab is optional practice. You can skip it whenever you like, and it never blocks your progress, the final assessment or your certificate.

Is the lab free?

Yes. Every lab level for this course opens in your browser, free, with nothing to install.

What topics does the Private AI with Local LLMs course cover, module by module?

The Private AI with Local LLMs course has 13 modules. It opens with "Why run AI locally: confidentiality, export control, cost and offline sites", goes through "Open-weight models and their licences: Llama, Qwen, Gemma, Mistral and gpt-oss", "Open WebUI: a private ChatGPT for the team", "Vision models locally: reading nameplates and P&ID snippets" and "Governance: what may go in, logging and acceptable use" and ends with "Final assessment". The full list with minutes is under "What you learn" on this page.

What can I do after the Private AI with Local LLMs course?

After the Private AI with Local LLMs course: Decide which documents may go to a hosted AI tool, which must stay on a local model and which go nowhere; Compare open-weight models and their licences: Apache 2.0, MIT and the Llama community licence; Size GPU memory and speed for a model from its parameters, quantisation, context length and users. These are the practical outcomes the course is written around, and the final assessment of 15 questions checks them.

What software and hardware do I need for the Private AI with Local LLMs course, and does it cost anything?

For the Private AI with Local LLMs course, software: Ollama, Open WebUI (in Docker), Python with the ollama, chromadb and pydantic packages, and optionally LM Studio and promptfoo. All are free to download. Hardware: A laptop with 16 GB of RAM runs the small models used in the modules. A GPU with 8 GB or more of memory, or an Apple silicon Mac with 16 GB or more, makes them faster. The sizing module shows how to size a team server; you do not need to buy one. The course itself is free.

Is the Private AI with Local LLMs course suitable for me, and what level is it?

Private AI with Local LLMs is rated intermediate. Intermediate. For engineers in plants, EPC contractors, consultancies, defence and oil and gas who want AI on data they cannot paste into a hosted chatbot. You should be comfortable with a command line and able to run a short Python script; Generative AI and LLM Foundations is the natural course before this one. It is written for engineers at plants, EPC contractors, consultancies, defence and oil and gas operators worldwide who need AI on confidential documents, and the IT and OT staff who run the server, and takes about 18.1 hours at your own pace.

Which jobs use the skills from the Private AI with Local LLMs course?

Skills from the Private AI with Local LLMs course (for example: decide which documents may go to a hosted AI tool, which must stay on a local model and which go nowhere) are used in roles such as AI engineer, machine learning engineer, data analyst and automation developer, and in AI engineer work. The course gives you practical evidence of the skill; it does not promise a job, and employers decide on experience and interview.

Start free today

Private AI with Local LLMs: Ollama, Open WebUI and Offline RAG, at your own pace. Add the certificate only if you want one.