Anyone who wants to work on customer emails, contracts or personnel files with a cloud language model faces a dilemma: the strongest models run in the cloud, and the data must not go there. The AUSTR.AI PrivacyProxy solves this on the user's own computer. It detects personal data locally and replaces it with placeholders before any text leaves the device. The PrivacyProxy was developed by Florian Bieche under the AUSTR.AI brand; this post describes how it is built.

Starting point

Anyone who enters text containing personal data into a cloud language model transfers that data to the provider: names, addresses, contract numbers. Redacting by hand takes time and is error-prone. Cloud anonymisation services only move the problem: the data still leaves the building, just to a different provider.

Then there is the language. Many detection tools are designed mainly for English texts. Austrian addresses, academic titles such as "Mag." or "DI" and the names of public authorities therefore need their own recognisers. A missed name is exactly the error that matters.

Goal

  • Detection and anonymisation run entirely locally, without a detour through a third-party server, not even through an AUSTR.AI server.
  • The user chooses the model: cloud providers with their own key, or local models.
  • The answer comes back with the real names and reads as if there were no proxy.
  • When in doubt, the proxy blocks instead of letting things through.
  • People stay in control: nothing goes out without a click on "Send".

Architecture

Diagram: On the user's own computer, a request from the chat or the command line goes to the local proxy; selected snippets come from the local knowledge base. The proxy detects personal data, replaces it with codenames and keeps the mapping in the encrypted vault. Only text with placeholders goes to a cloud model or a local model. The answer gets the real data back locally. If anonymisation fails, the proxy blocks the request.

The proxy sits between the interface and the language model. Everything except the model itself runs on the user's computer.

  • Interface and command line: a chat in the browser on the user's own computer, built with Preact, and the command line aai chat.
  • Local proxy: a Python service based on Starlette that sends every request through anonymisation.
  • Detection: GLiNER as the primary model with 24 categories, spaCy as a fallback, Microsoft Presidio and custom recognisers for Austrian titles, educational institutions, public authorities and addresses.
  • Codename engine: replaces every match with a semantic placeholder in square brackets. The model knows that a person or a place sits at this position, but not the real name.
  • Encrypted vault: keeps the mapping from placeholder to original locally, encrypted with Fernet, in four protection levels with their own expiry times.
  • Knowledge base: a local SQLite database. Documents are anonymised before indexing.
  • Language model of choice: Anthropic, OpenAI, Mistral or Google with the user's own key, or locally via Ollama or LM Studio.
  • Rehydration: puts the original data back into the answer locally.

Implementation

The most important rule is fail-closed. If anonymisation fails, the proxy blocks the request with an error message. Unprotected text never goes out as a fallback.

Anonymisation covers not just the new message but the entire conversation history before every send. Model answers already contain placeholders and are flagged as such, so codenames stay stable throughout the conversation. Personal data does not end up in any log.

One feature was deliberately removed again: automatic model selection. It was a black box and contradicted the principle that users stay in control. It was replaced by three tools that people switch on deliberately:

  • Slash commands at the start of an input.
  • Skills as Markdown files with a recommended model; the user's own choice in the chat header takes precedence.
  • A knowledge base whose text snippets the user sees and selects before sending.

Power-user features are off by default. Cloud sync and a marketplace are deliberately absent. An earlier issue has been solved: some models adapted placeholders grammatically or translated them along with the text, so rehydration no longer found them. Such altered placeholders are now matched back to the real data as well.

Infrastructure and EU compliance

AUSTR.AI has two separate parts. The website austr.ai shows in a demo what the proxy can do; only this demo runs on a server, using a Mistral model. The tool itself runs entirely on the user's side. It sends no user data to any AUSTR.AI server.

The only outbound connection goes to the model provider the user has chosen, with the user's own key and with text that is already anonymised. Anyone using a local model via Ollama or LM Studio sends nothing out at all. The exception is the cloud models that Ollama now offers as well: they run at the provider, even when accessed through Ollama. On first start, the tool downloads the detection models once; no user data is involved.

Legally, this is pseudonymisation within the meaning of Article 4(5) of the GDPR: the key to the mapping sits in the local vault, and the model provider only sees placeholders. No automatic detection catches every reference to a person. Paraphrases such as "the new head of the legal department" remain a residual risk that users need to be aware of. Fail-closed behaviour and local models for the most sensitive texts keep it small.

Outcome and value calculation

The PrivacyProxy makes cloud language models usable for texts with personal data, without that data leaving the device in plain text. For the most sensitive texts, there is still the local model route, with no outbound connection at all.

The value can be calculated in three items:

  • Time saved = texts per week that used to be redacted by hand × minutes per text × internal hourly rate.
  • Newly unlocked value = tasks per week that could not go to a cloud model without anonymisation × time the model saves per task × internal hourly rate.
  • Costs = model usage with the chosen provider + setting up and maintaining the local installation.

The second item is easy to overlook: it covers tasks with sensitive texts for which a cloud model is out of the question without anonymisation. An avoided data protection incident cannot be priced honestly. It still belongs in the decision, as a separate line without an amount.

You can find more projects in the overview of reference projects.