NLWeb on a VPS: Add AI Search and an MCP Endpoint to Your Website

NLWeb is Microsoft's open-source project for adding AI search to a website: visitors and AI agents ask questions in plain language, and NLWeb answers from your own content through two endpoints, /ask for people and apps and /mcp for AI assistants. You can run NLWeb on a VPS with a local vector database and a local model, and we did exactly that on Debian 13. The install took minutes and search over our own pages worked. The answers did not: with a small local model on 2 vCPUs, every question timed out, because NLWeb asks the language model to score each search result separately. This guide shows the full setup, what we measured, and when a plain llms.txt file does the job better.

Key facts, checked on 13 October 2026:

• Microsoft announced NLWeb on 19 May 2025 as an open project, and every NLWeb instance is also an MCP server (Microsoft Source).
• The code now lives at github.com/nlweb-ai/NLWeb (the old microsoft/NLWeb address redirects). It is MIT licensed, has 6,256 stars and 696 forks, and has no tagged releases. The last code change on the main branch was on 7 April 2026; after that, until 10 June 2026, only automated dependency updates were merged.
• The setup guide asks for Python 3.10 or newer (hello-world guide). The code ships connectors for Qdrant, Milvus, Elasticsearch, OpenSearch, Postgres, Azure AI Search, Snowflake and Cloudflare AutoRAG, and for OpenAI, Azure OpenAI, Anthropic, Gemini, Hugging Face, Inception, Snowflake and Ollama models.
• A path traversal flaw that could expose files such as LLM API keys was reported in May 2025 and fixed on 1 July 2025 (MobileSyrup).
• In our lab, one question caused 98 to 99 calls to the language model. With NLWeb, the Ollama models and the index loaded, the container used about 2 GB of RAM and 3.7 GB of disk.

What NLWeb does, in plain words

Most sites already describe their pages for machines. A blog post carries schema.org markup, a block of JSON-LD that says "this is a BlogPosting, here is the headline and the description". NLWeb puts that data into a vector database, which stores the meaning of each text as a list of numbers called an embedding. Similar meanings get similar numbers, so a question finds the right page even when the words differ.

When a question arrives, NLWeb does four things:

1. It rewrites the question into search phrases with a language model.
2. It looks up the closest items in the vector database (up to 50).
3. It asks the language model to score every single item for relevance, one call per item.
4. It returns the best items as JSON, in schema.org shape. In "summarize" or "generate" mode it adds an AI-written answer on top.

The same engine sits behind two doors. /ask is a normal HTTP endpoint that a search box on your site can call. /mcp speaks the Model Context Protocol, the standard way for assistants like Claude or ChatGPT to call outside tools. Its tools are ask and list_sites (REST API docs). If you want to know more about MCP itself, our guide to running your own MCP servers for Claude and ChatGPT builds one from scratch.

What worked in our lab, and what didn't

We installed NLWeb in a Debian 13 container on our Amsterdam test server, limited to 2 CPUs and 3 GB of RAM. We used no paid API at all: Ollama ran the models locally, and Qdrant ran in its built-in file mode with no separate server. The data was the schema.org markup of rscomputers-ks.com itself.

The plumbing works; the language model is the bottleneck. Below is the exact setup we ran, then why the last step fails on a CPU.

Step-by-step: NLWeb on Debian 13 with Ollama and local Qdrant

Start from a fresh Debian 13 server with a firewall (our Debian 13 hardening guide covers SSH keys and nftables). NLWeb listens on port 8000 on all interfaces, so keep that port closed to the internet until you put a proxy in front of it.

1. System packages and a user for NLWeb

# as root
apt-get update
apt-get install -y git python3 python3-venv python3-pip curl ca-certificates zstd
adduser --disabled-password --gecos "" nlweb

Debian 13 gives you Python 3.13.5 and git 2.47.3. In our lab this step took 16 seconds.

2. Ollama and two small models

Ollama runs AI models on your own server. We need two: an embedding model that turns text into numbers, and a small chat model for rewriting questions and scoring results.

# as root
curl -fsSL https://ollama.com/install.sh | sh
ollama pull nomic-embed-text
ollama pull qwen2.5:1.5b
ollama list

Expected output: the installer ends with a systemd service and a note that no GPU was found, and ollama list shows qwen2.5:1.5b at 986 MB and nomic-embed-text at 274 MB. We got Ollama 0.40.3. Ollama listens only on 127.0.0.1:11434, which is what you want.

Our first test answer took 56 seconds, and the second one hung for over four minutes with both CPUs busy. The Ollama log showed n_threads = 8, the host's core count, though our container was limited to 2. A KVM VPS normally sees only its own vCPUs, so you will probably not hit this there (we did not test it on a VM). In a CPU-limited container, set the thread count by hand:

# as root
printf 'FROM qwen2.5:1.5b\nPARAMETER num_thread 2\n' > /tmp/Modelfile
ollama create qwen2.5-2t -f /tmp/Modelfile
printf 'FROM nomic-embed-text\nPARAMETER num_thread 2\n' > /tmp/Modelfile2
ollama create nomic-embed-2t -f /tmp/Modelfile2

After that, a one-sentence answer took 8.8 seconds including 4.6 seconds of model loading, and the model wrote about 13 tokens per second on 2 vCPUs. The rest of this guide uses these two names. If you skipped this step, use qwen2.5:1.5b and nomic-embed-text instead.

3. Download NLWeb

# as the nlweb user (su - nlweb)
git clone --depth 1 https://github.com/nlweb-ai/NLWeb.git
python3 -m venv ~/venv
. ~/venv/bin/activate
cd ~/NLWeb/AskAgent/python
pip install -r requirements.txt ollama

The application code now sits in AskAgent/python, not in code/python as many 2025 tutorials say. The install took 26 seconds and brought in qdrant-client 1.19.1, mcp 2.3.0 and aiohttp 3.14.5. We tested commit b423f15 from 10 June 2026.

4. Point NLWeb at Ollama and the local Qdrant

NLWeb reads secrets from .env and choices from YAML files. The shipped configs default to Azure OpenAI and have no Ollama section, so we add one, switch off two remote backends that are on by default (an Azure AI Search index called nlweb_west, and Shopify), and set production mode, because development mode lets anyone override settings through URL parameters.

# as the nlweb user
cd ~/NLWeb
cp .env.template .env
printf '\nOLLAMA_URL="http://127.0.0.1:11434"\n' >> .env

sed -i 's/^preferred_endpoint: .*/preferred_endpoint: ollama/' config/config_llm.yaml
cat >> config/config_llm.yaml <<'YAML'

  ollama:
    api_endpoint_env: OLLAMA_URL
    llm_type: ollama
    models:
      high: qwen2.5-2t
      low: qwen2.5-2t
YAML

sed -i 's/^preferred_provider: .*/preferred_provider: ollama/' config/config_embedding.yaml
cat >> config/config_embedding.yaml <<'YAML'

  ollama:
    api_endpoint_env: OLLAMA_URL
    model: nomic-embed-2t
YAML

sed -i '/^  nlweb_west:/,/^  [a-z_]*:$/ s/enabled: true/enabled: false/' config/config_retrieval.yaml
sed -i '/^  shopify:/,/^  [a-z_]*:$/ s/enabled: true/enabled: false/' config/config_retrieval.yaml
sed -i 's/^mode: development/mode: production/' config/config_webserver.yaml

Watch the printf '\n...' line. The template file has no newline at its end, so a plain echo >> glues your variable onto the last line, and NLWeb then reports "Ollama endpoint not found in config". That happened to us first.

Now check the setup:

# as the nlweb user
cd ~/NLWeb/AskAgent/python
python testing/check_connectivity.py

Expected output on an empty index: the LLM and embedding checks pass, and the retriever check fails with "No valid output received" because there is nothing to search yet. The summary says 2/3 connections successful.

5. Load your site's schema.org data

The official guide loads RSS feeds with python -m data_loading.db_load <RSS URL> <site-name>. Our site has no feed, but every page has schema.org markup and there is a sitemap. NLWeb's own crawler, scraping.incrementalCrawlAndLoad, loaded 40 pages in 4 minutes but stored each page's JSON-LD @graph as a list, and ranking then failed on every item with 'list' object has no attribute 'get'. Production mode hides that error, so all you see is an empty answer.

Our fix: a short standard-library script that keeps one main schema.org object per page in the "URL, tab, JSON" format db_load accepts:

# as the nlweb user: save as ~/sitemap2tsv.py
import json, re, sys, urllib.request

SKIP = {"BreadcrumbList", "WebSite", "WebPage", "Organization", "ItemList"}
UA = {"User-Agent": "NLWeb-loader/1.0"}

def get(url):
    return urllib.request.urlopen(urllib.request.Request(url, headers=UA), timeout=20).read().decode("utf-8", "replace")

sitemap = sys.argv[1]
urls = re.findall(r"<loc>(.*?)</loc>", get(sitemap))
kept = 0
for url in urls:
    html = get(url)
    for block in re.findall(r'<script type="application/ld\+json">(.*?)</script>', html, re.S):
        data = json.loads(block)
        items = data.get("@graph", [data]) if isinstance(data, dict) else data
        main = [i for i in items if isinstance(i, dict) and i.get("@type") not in SKIP]
        if main:
            obj = main[0]
            obj.setdefault("url", url)
            print(url + "\t" + json.dumps(obj, ensure_ascii=False))
            kept += 1
            break
print(f"{kept} of {len(urls)} pages had a usable schema.org object", file=sys.stderr)

Run it against your sitemap, then load the file. The local Qdrant database is a folder that only one process can open at a time, so stop the NLWeb server before you load data.

# as the nlweb user
cd ~
python3 sitemap2tsv.py https://rscomputers-ks.com/sitemap.xml > rscomputers.tsv
cd ~/NLWeb/AskAgent/python
. ~/venv/bin/activate
python -m data_loading.db_load ~/rscomputers.tsv rscomputers

Expected output: "99 of 108 pages had a usable schema.org object" (82 BlogPosting, 16 Service and 1 Blog in our case), then "Loading completed. Added 99 documents to the database." Embedding 99 pages took 4 minutes 25 seconds on 2 vCPUs, and the finished index was 1.2 MB. NLWeb only knows what your markup says. Our BlogPosting objects hold the headline and the description, not the full article text, so that is all NLWeb can search.

6. Start the server and ask a question

# as the nlweb user
cd ~/NLWeb/AskAgent/python
. ~/venv/bin/activate
setsid nohup python app-aiohttp.py > ~/nlweb.log 2>&1 < /dev/null &
sleep 8
curl -s localhost:8000/health

Expected output: {"status": "healthy", ... "version": "2.0.0", "mode": "production"}. Opening http://localhost:8000/ shows the bundled "NLWeb Chat" page. Now a real query:

# as the nlweb user
curl -s -G localhost:8000/ask --data-urlencode "query=how do I run an MCP server on a VPS" -d site=rscomputers -d streaming=false -d mode=list

Pass site= every time. Without it NLWeb treats the question as a multi-site search, asks its /who router which site fits, and with one local site it returned nothing in 0.15 seconds.

7. Talk to the MCP endpoint

# as the nlweb user
curl -s -X POST localhost:8000/mcp -H 'Content-Type: application/json' -H 'Authorization: Bearer anything' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}'
curl -s -X POST localhost:8000/mcp -H 'Content-Type: application/json' -H 'Authorization: Bearer anything' \
  -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"list_sites","arguments":{}}}'

Expected output: the first call returns "protocolVersion": "2024-11-05" and "serverInfo": {"name": "nlweb-mcp-server", "version": "1.0.0"}, the second returns {"sites": ["rscomputers"]}. A tools/list call shows two tools: ask (arguments query, site, generate_mode) and list_sites. Note the header: we really sent the word "anything". More on that below.

The wall: about 100 model calls for one question

Our first /ask calls returned nothing, and the log showed "LLM call timed out after 8s". NLWeb runs several model checks on each question before searching, and on a slow model they all compete for the CPU. You can switch them off:

# as the nlweb user
cd ~/NLWeb
for k in tool_selection memory analyze_query decontextualize required_info aggregation who_endpoint; do
  sed -i "s/^${k}_enabled: true/${k}_enabled: false/" config/config_nlweb.yaml
done
grep _enabled: config/config_nlweb.yaml

After a restart the question rewrite worked (2.7 seconds, it produced "MCP server", "VPS", "virtual private server" and two more phrases) and the vector search found 31 matches in 0.24 seconds. Then the ranking step fired one model call per match, twice, because NLWeb runs a fast track and a regular track side by side. In our lab Ollama worked through them one at a time, so they queued up and NLWeb gave up on each after 8 seconds. To count the calls, read the Ollama log from the minute before your question (we asked at 14:13):

# as root
journalctl -u ollama --no-pager --since 14:13:00 | grep -c /api/chat

That printed 98 for a single MCP question, and the /ask call before it made 99. Every ranking call ended with HTTP 500 after 7.9 seconds, and the answer came back empty after 16 to 17 seconds. The 8-second limit is the default in NLWeb's ask_llm function, and up to 50 results are ranked.

That design suits a hosted API that answers many requests in parallel. It does not suit a CPU-only model, and a bigger server alone won't fix it: even at one second per scoring call, 100 calls in a row is over a minute and a half per question. So a local setup needs a GPU, or you keep Ollama for embeddings and send chat calls to a hosted model. We could not test paid APIs, so that part comes from the docs only.

Before you open it to the internet

• The server binds to 0.0.0.0:8000 and has CORS switched on, so any website can call it from a visitor's browser. Firewall the port and publish it only through a reverse proxy with HTTPS (see our Caddy, nginx and Traefik guide).
• In production mode /mcp answers 401 without a token, but the auth middleware accepts any Bearer value that is not a valid JWT "for backward compatibility", as the code comment says. We got in with Bearer anything. If you need real protection, check the token in your proxy.
• /ask is public by design. Every request can trigger up to about 100 model calls, which is a real bill on a paid API. Rate-limit it in the proxy.
• NLWeb installs missing Python packages with pip at runtime (our crawler run fetched beautifulsoup4 on its own), so run it as its own user, as above.
• Keep the checkout updated. The 2025 path traversal bug could expose the .env file, which is where your API keys live.

NLWeb, llms.txt or your own MCP server?

For a small site the honest answer is usually "not NLWeb yet". AI assistants already read your pages. What helps them most is clean, cheap-to-read content, which is what our guide to serving Markdown to AI agents with nginx or Caddy covers, together with an llms.txt file and clear crawl rules (content signals, robots.txt and llms.txt).

NLWeb pays off with thousands of well-marked-up items and visitors who ask long, fuzzy questions such as "a vegetarian dinner under 30 minutes without nuts". For 100 blog posts, a sitemap and llms.txt already do the job.

Which server for NLWeb?

Here is what used memory and disk in our lab, with everything loaded at once: the chat model server 1,241 MB, the embedding model server 650 MB, NLWeb itself 127 MB and Ollama 57 MB. On disk, the Ollama runtime took 2.1 GB (it bundles GPU libraries even on a CPU-only server), the models 1.2 GB, the Python environment 383 MB, and the code and index 29 MB.

VPS plans share their vCPUs with neighbours, while VDS plans have dedicated vCPUs, which matters when a model keeps every core busy for minutes. All plans have NVMe storage, unmetered traffic and their own IPv4 and IPv6 addresses, and you can move to a bigger plan later from the client area with a short reboot.

Frequently asked questions

Is NLWeb free?

Yes. NLWeb is open source under the MIT licence, and the code is on GitHub at nlweb-ai/NLWeb. What can cost money is the language model: NLWeb calls it once per search result, so with a paid API every question uses many requests.

Can NLWeb run without OpenAI or any API key?

Partly. NLWeb supports Ollama for both embeddings and chat, and in our test the install, data loading, vector search and MCP endpoint all worked with no key. Answers did not: on 2 vCPUs, a 1.5B model was too slow for NLWeb's per-result scoring, and every scoring call timed out after 8 seconds.

Does NLWeb need an RSS feed?

No. It also loads JSONL, CSV and tab-separated "URL and JSON" files with schema.org objects, and has a sitemap crawler. Sites without a feed can export their JSON-LD, as we did for 99 pages in 13 seconds.

Is NLWeb still maintained in 2026?

The project moved from Microsoft's GitHub account to github.com/nlweb-ai/NLWeb. On 13 October 2026 it had no tagged releases, the last code change on the main branch was from 7 April 2026, and only dependency updates were merged after that, until 10 June 2026. Treat it as an active experiment, not a finished product.

What is the difference between NLWeb and llms.txt?

llms.txt is a static file that tells AI tools what is on your site. NLWeb is a running service that answers questions about your content and offers an MCP endpoint, and it needs a vector database and a fast language model.

How much RAM does NLWeb need?

NLWeb itself used 127 MB in our lab. With local Ollama models, a 1.5B chat model and nomic-embed-text, the whole setup used about 2 GB of RAM and 3.7 GB of disk. With hosted models, a 2 GB VPS is enough to start.

Try it on a test server first

The search and MCP parts of NLWeb are ready to use; answer quality and speed depend on the model behind it. Put it on a test server, load your own schema.org data, and see whether your visitors really need a chat box before you pay per question. Our KVM VPS and VDS plans run in Amsterdam, Dublin and Prishtina, and availability by city is on the plans page. If you want us to install NLWeb or another AI app for you, we quote that per job: message us on Telegram.

← All articles