
Alexander is a writer, traveler, and serial digital entrepreneur. He’s the founder of the Tushi, the Gen X personal growth newsletter Further, and offers small business advice at Unemployable.

Twitter

Linkedin

Facebook

Director and Chief Evangelist

Alexander Hipp


Anil is a content-obsessed marketer who loves driving organic traffic for outstanding companies. He is also the creator of Hack the Content, co-host of The Runner, and owner of several digital commerce sites.

Managing Partner

Anil Vugels


Monika is a writer, musician, and content strategist with a passion for language. When she isn’t helping clients, you can find her writing songs, hiking in a northern forest, or furiously journaling.

Director of Operations

Ann Monika

Programmer / Founder / Student

Tim Schupp


<img width="1906" height="917" alt="OpenCode blog cover" src="/static/assets/opencode-blog-cover.png" />

[Opencode](https://opencode.ai/) is a pretty cool and neatly composable terminal based llm coding agent.
For a project at [tb-software](https://timschupp.de/) and also private personal use in general I'm especially interesting in the usage of open-code or similar tools with local llms, hosted on own hardware. See [my article on running and testing local llms on a 128gb strix halo setup here](https://blog.t1m.me/blog/local-llms-on-strix-halo-128gb-shared-ram).
This article will describe my personal setup as well as some hints how to manage it using a nix-flake - this functions more as future reference - than direct instructions. But it may help someone configure open-code for local llm usage.

### Configuring Open-Code

Just create a `opencode.json` like this:

```json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": { ... },
}
```
Then start adding your providers, fistly the local llm one, e.g.: you could use here [ollama]() though there are probaly some better and more performant options out there ( _these I still have to test again / with newer models_ ). But project like localai lammacpp, ... come to mind. Anything that host api compatible with the openai api schema really.

```json
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "ollama-local",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": { ... }
```
e.g.: to test the models from [my latest benchark](https://blog.t1m.me/blog/local-llms-on-strix-halo-128gb-shared-ram) i added. _Of course you will have to pull the models and corretly setup and start the local lmm server first ( check my other post for an example using ollama on vulkan + nix )._

```json
        "qwen3-coder-next": { "name": "qwen3-coder-next" },
        "mdq100/Qwen3-Coder-30B-A3B-Instruct:30b": { "name": "qwen-code-instuct-small"},
        "nemotron-3-nano": { "name": "nemotron-3-nano" },
        "glm-4.7-flash": { "name": "glm-4.7-flash" },
        "gpt-oss:120b": { "name": "gpt-oss:120b" }
```

<img width="1906" height="917" alt="OpenCode post starting prompt example" src="/static/assets/open-code-post-starting-prompt.png" />

### Usability / Review

As already stated in my benchmark post I was impressed by the performance and usability of the small llms, especially on small simple task like file editing, moving things around, manipulating issues, making simple commits and more. *So it is definitely somewhat hopeful that these open models do seems to perform for some use-case*; So I would say with care and good guardrails they can applied for certain tasks; especially when fully private data processing is required.

Also for the full potential of open-code and priming agents to be able to perform certain tasks in certain environments; you should definitely checkout and learn [how open-code mcps](https://opencode.ai/docs/mcp-servers/) and [agent skills](https://opencode.ai/docs/skills/) are set-up. I my experience the small local models perform best with easy; repeatable and easy to verify tasks - especially when given the right context clues and tools to use - but tasks that involve steps hard to statically automate ( e.g.: text based decision; complex semi autonomous task; browser use; search etc... ). Also it is possible to build in restrictive confirmation based tools for oversight.

### External Providers

Also if you want to use other remove non local providers to test oss models, you could add the following. _Note the api keys are taken from environment you don't need to include them in the config_.

#### 1. Deepinfra

```json
    "deepinfra": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "DeepInfra",
      "options": {
        "baseURL": "https://api.deepinfra.com/v1",
        "apiKeyEnv": "DEEPINFRA_API_KEY"
      },
      "models": { ... }
```

#### 2. groq

```json
    "groq": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Groq",
      "options": {
        "baseURL": "https://api.groq.com/openai/v1",
        "apiKeyEnv": "GROQ_API_KEY"
      },
      "models": { ... }
```

#### a complete config example

```json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "qwen3-coder-next": { "name": "qwen3-coder-next" },
        "mdq100/Qwen3-Coder-30B-A3B-Instruct:30b": { "name": "qwen-code-instuct-small"},
        "nemotron-3-nano": { "name": "nemotron-3-nano" },
        "glm-4.7-flash": { "name": "glm-4.7-flash" },
        "gpt-oss:120b": { "name": "gpt-oss:120b" }
      }
    },
    "deepinfra": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "DeepInfra",
      "options": {
        "baseURL": "https://api.deepinfra.com/v1",
        "apiKeyEnv": "DEEPINFRA_API_KEY"
      },
      "models": {
        "nvidia/Nemotron-3-Nano-30B-A3B": { "name": "nvidia/Nemotron-3-Nano-30B-A3B" },
        "moonshotai/Kimi-K2.5": { "name": "moonshotai/Kimi-K2.5" },
        "Qwen/Qwen3-235B-A22B-Instruct-2507": { "name": "Qwen/Qwen3-235B-A22B-Instruct-2507" },
        "MiniMaxAI/MiniMax-M2.5": { "name": "MiniMaxAI/MiniMax-M2.5" }
      }
    },
    "groq": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Groq",
      "options": {
        "baseURL": "https://api.groq.com/openai/v1",
        "apiKeyEnv": "GROQ_API_KEY"
      },
      "models": {
        "moonshotai/kimi-k2-instruct": { "name": "moonshotai/kimi-k2-instruct" },
        "qwen/qwen3-32b": { "name": "qwen/qwen3-32b" }
      }
    }
  }
}
```

### Managing a global open-code install via nix

<img width="1906" height="917" alt="OpenCode performing git push" src="/static/assets/open-code-performing-git-push.png" />

To mange my local open-code installation and configuration I've setup a simple nix flake, in a git sub-module that stores and pulls my open-code configuration and makes it globally available on configuration update.

```nix
{
  description = "Install opencode.json as a global OpenCode config";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  };

  outputs = { self, nixpkgs }:
    let
      opencodeJsonPath = ./opencode.json;

      commonOptions = { lib, ... }:
        {
          options.opencodeOllamaLocalSetup = {
            enable = lib.mkOption {
              type = lib.types.bool;
              default = false;
              description = "Whether to install the OpenCode config file globally.";
            };
          };
        };

      nixosModule = { config, lib, ... }:
        let
          cfg = config.opencodeOllamaLocalSetup;
          defaultUsers = lib.attrNames (lib.filterAttrs (_: user:
            (user.isNormalUser or false) || ((user.uid or null) != null && user.uid >= 1000)
          ) config.users.users);
          targetUsers = if cfg.users == null then defaultUsers else cfg.users;
          existingUsers = lib.filter (name: lib.hasAttr name config.users.users) targetUsers;
          userRules = lib.concatLists (map (name:
            let
              user = config.users.users.${name};
              homeDir = user.home or "/home/${name}";
              groupName = user.group or name;
            in [
              "d ${homeDir}/.config/opencode 0700 ${name} ${groupName} -"
              "L+ ${homeDir}/.config/opencode/opencode.json - ${name} ${groupName} - ${opencodeJsonPath}"
            ]
          ) existingUsers);
        in
        {
          options.opencodeOllamaLocalSetup.users = lib.mkOption {
            type = lib.types.nullOr (lib.types.listOf lib.types.str);
            default = null;
            description = "Users to receive ~/.config/opencode/opencode.json (null = all normal users).";
          };

          config = lib.mkIf cfg.enable {
            systemd.tmpfiles.rules = userRules;
          };
        };

      homeManagerModule = { config, lib, ... }:
        let
          cfg = config.opencodeOllamaLocalSetup;
        in
        {
          config = lib.mkIf cfg.enable {
            xdg.configFile."opencode/opencode.json".source = opencodeJsonPath;
          };
        }
    in
    {
      nixosModules.default = { ... }: {
        imports = [ commonOptions nixosModule ];
      };
      homeManagerModules.default = { ... }: {
        imports = [ commonOptions homeManagerModule ];
      };
    }
}
```

<img width="1906" height="917" alt="OpenCode blog section edit example" src="/static/assets/opencode-blog-section-edit-example.png" />


Testing and configuring opencode for usage with local llms

Opencode; Usability with Local LLMs on iGPU w 128GB vram: My Tests


## Local LLMs on Strix Halo; [AMD Ryzen™ AI MAX+ 395](https://www.amd.com/en/blogs/2025/amd-ryzen-ai-max-395-processor-breakthrough-ai-.html) with 128GB Shared Ram: My Tests

LLMs are sometimes stupid, sometimes impressive, often unpredictable and almost never private.
Bigger models are generally hosted in clouds and proprietary, sometimes you can limit access or censor data presented to model providers but almost never you can restrict access fully or be really sure how and when your data is being used.

These are just some of the reasons why you might want to run models locally. But hardware especially graphics cards that can run these models are very pricey and the vram requirement for the more 'mid-range' models just exceeds what is possible with local models.

For some time Apple had an alternative to expensive graphics cards with their m-chip lineup allowing shared VRAM with integrated graphics and CPU, but the prices for higher RAM Mac configurations are still ridiculously high. 
But now there is a 'cheaper' option available for consumers. For a [tb-software](https://timschupp.de/) project I tested the highest available 128GB RAM configuration of a mini PC with the new strix-halo chip to test it and its ability for running local LLMs.

This article reports our early findings and summarizes some configuration steps and the measurement setup I used.
To this point I've tested 5 LLMs and more tests are coming. I also investigated the use and usability of local agents with OpenCode in a companion post: [Opencode; Usability with Local LLMs on iGPU w 128GB VRAM](https://blog.t1m.me/blog/opencode-with-local-llms).

### Setting up ollama with vulkan

First I needed to allocate VRAM to the integrated graphics card, I have not played with the 'dynamic' setting here yet. ( _I have now tried extending VRAM to 94GB more below_ )
For these tests I have chosen to set the VRAM to `65536 MiB` so there is about `67 GB` RAM still available to the CPU. We can also test influence on load speed and inference of varying the available amount of VRAM in the future.

<img width="1451" height="234" alt="Image" src="/static/assets/strix-halo-vram.png" />

Tests were run on NixOS and to access all the libraries required to use the Ollama Vulkan backend I had to install some additional packages and libraries, specifically I added `vulkan-loader` to the NixOS system libraries and ran Ollama with `OLLAMA_LLM_LIBRARY=vulkan OLLAMA_VULKAN=1 ollama serve`. I used `amdgpu_top` to monitor the VRAM and graphics usage ( _It feels like there are still some performance gains to be made, also VRAM is not fully utilized, especially by the smaller models_ ).

### The Test Setup

To test the LLMs for now I just wanted to know basic results for load speed and completion speeds.
I've created a simple fork of [ollama benchmark](https://github.com/aidatatools/ollama-benchmark) ( here [my fork without telemetry](https://github.com/tbscode/tims-ollama-bench-fork) ) to measure the results based on a running Ollama backend.

If you had the exact same hardware, system and configurations, tests could be replicated via:

```bash
python3 -m venv venv
source venv/bin/activate
pip install git+https://github.com/tbscode/tims-ollama-bench-fork.git
tims_llm_benchmark run --custombenchmark=<your-model-config>.yml
```

With models file:

```yaml
file_name: "<config-name>.yml"
version: 2.0.custom
models:
  - model: "<model-name>"
```

And run them all via

```bash
tims_llm_benchmark run --custombenchmark=model-configs/qwen3-coder-next.yaml
tims_llm_benchmark run --custombenchmark=model-configs/nemotron-3-nano.yaml
tims_llm_benchmark run --custombenchmark=model-configs/glm-4.7-flash.yaml
tims_llm_benchmark run --custombenchmark=model-configs/gpt-oss-120b.yaml
tims_llm_benchmark run --custombenchmark=model-configs/qwen-3.5-122b.yaml
```

<img width="1906" height="917" alt="Image" src="/static/assets/ollama-benchmark.png" />

### Test Results

Based on this benchmark and personal prompts and agents tests I've created this table to summarize the results.
There are many more tests to be done and likely also performance improvements through improved configuration, these are just my initial findings and a preliminary review. *But I can definitely say that there are functional local models out there that can definitely complete simple tasks locally on this hardware!*

| Name | Size (GB) | Quality | Load Speed | Tokens / sec |
|--|--|--|--|--|
| **qwen3-coder-next** | 61GB | 👾👾👾👾👾 | Fast | `35.094` |
| **nemotron-3-nano** | 34GB | 👾👾👾👾 | Very Fast | `63.566` |
| **glm-4.7-flash** | 40GB | 👾👾👾👾 | Slow | `50.098` |
| **gpt-oss:120b** | 70GB | 👾👾👾👾 | Very Slow | `31.532` |
| **qwen-3.5:122b** | 81GB | 👾👾👾👾 | Very Slow | `19.158` |
| **qwen-3.5:9b** | 6.6GB | 👾👾👾👾 | Very Slow | `29.52` |

All these models are usable for smaller and simpler local agentic and coding tasks, involving basic files manipulations and tool calls e.g.: via open-code. I was especially impressed by how well `qwen3-coder-next` performed, and also `nemotron-3-nano` that was incredibly fast and smart for such a small thinking model.

### Testing new qwen-3.5 with increased 94gb vram

Now I've updated the bios to allocate 94GB VRAM to allow larger models to run entirely in VRAM.

<img width="1451" height="234" alt="BIOS VRAM settings for 94GB allocation" src="/static/assets/strix_halo_increase_94gb_vram.png" />

Here's the active usage with 94GB VRAM allocated:

<img width="1906" height="917" alt="94GB VRAM active usage showing model loaded in VRAM" src="/static/assets/strix_halo_94gb_vram_active_usage_strix_halo.png" />

With the increased VRAM, I was able to download and run qwen-3.5:95b entirely in VRAM:

<img width="1451" height="917" alt="Downloading qwen-3.5:95b model with Ollama" src="/static/assets/ollama_downloading_qwen122b.png" />


Strix Halo 128GB RAM, 100% local LLM agents, my tests

Local LLMs on Strix Halo 128GB Shared Ram: My Tests


I have been testing several "Agentic" code editors to understand their current capabilities and limitations. This article compares **Antigravity** (Google), **Cursor** (Anysphere), **Windsurf**, and the open-source combination of **VSCodium + Continue**.

## The Contenders

### Antigravity (Google)
Antigravity is Google's entry into the agentic coding space, leveraging their compute and model capabilities. It integrates well for complex, multi-step reasoning tasks.

**Notes:**
- **Pricing:** No official pricing model exists yet.
- **Rate Limits:** The current rate limits are low and can interrupt workflow during intensive sessions.

![Antigravity Screenshot](/static/assets/agentic_editors/antigravity_screenshot.png)

### Cursor
Cursor is a fork of VS Code built by Anysphere. It includes a "Cursor Tab" feature for predictive edits.

**Cloud Agents:**
Cursor's agents operate in a sandboxed cloud environment. They can execute terminal commands, edit files, and create/refine Pull Requests. This is useful for repetitive, dynamic tasks. For example, I used it to integrate new automatic emails for **Little World**, where the agent edited configuration files across multiple locations and restructured code, replacing a previously manual process.

**Pricing:**
The Pro plan costs approximately **€20/month**. However, it includes a limited number of "fast" agent edits; exceeding this limit results in throttling or requires purchasing usage tokens.

![Cursor Screenshot](/static/assets/agentic_editors/cursor_screenshot.png)

### Windsurf
Windsurf is another editor in this space, emphasizing a "Flow" state. Some of the original engineers behind Windsurf are now employed at Google.

### Codium + Continue
For a fully open-source stack, **VSCodium** (a telemetry-free VS Code build) combined with the **Continue** extension allows the use of own models (e.g., Claude 3.5 Sonnet, GPT-4o) while maintaining data control.

![Codium Screenshot](/static/assets/agentic_editors/codium_screenshot.png)

**Trade-offs:**
- **Privacy:** No telemetry or data egress without explicit configuration.
- **Usability:** Major vendors (like Microsoft) do not support non-official builds. Proprietary extensions like Pylance/Pylint are often disabled or broken, requiring workarounds.

[Read my guide on setting up Codium + Continue here](https://timschupp.de/blog/self-hosted-ai-code-editing)

## Verdict

These tools are useful for specific workflows, particularly repetitive tasks involving consistent steps (e.g., adding a database field, updating the API, and modifying the frontend).

### Reality Check

They are not perfect. In my experience, they produce a significant amount of incorrect output. My acceptance rate is approximately **20-30%** of the suggested code. Even accepted code often requires rewriting to match specific requirements. They function as assistants, not replacements.

## Python Tooling Note

The official **Pylint extension for VS Code is proprietary** (part of the Pylance/Python bundle) and is disabled on non-official IDEs like VSCodium.

If using an editor other than the official VS Code or Cursor, I recommend **[basedpyright](https://github.com/DetachHead/basedpyright)** as an open-source alternative.


A comparison of Antigravity, Cursor, Windsurf, and Codium + Continue for agentic coding tasks.

Comparing Agentic Code Editors


Here’s how I keep AI-assisted coding fully under my control: Continue.dev with Codium, pointed at APIs I trust and models I host. Everything stays local or on endpoints I pick - no mistrius data blackholes, no pricy APIs with uncontrolled token usages. Just own hardware and local storage.

### Setting up the local model

`~/.continue/config.yaml`

```yaml
name: My Setup
version: 1.0.0
schema: v1

models:
  - name: "Tims Kimi"
    provider: openai
    model: moonshotai/Kimi-K2-Thinking
    apiBase: https://api.deepinfra.com/v1/openai
    apiKey: <DEEPINFRA_API_KEY>
    roles:
      - chat
      - edit
```

You can even use Google Models outside of Antrigravity:

```yaml
  - name: Gemini 3 Native
    provider: gemini
    model: gemini-3-pro-preview
    apiKey: <GOOGLEAI_API_KEY>
```

### Using self hosted models

There are several tools you can use here; the easiest is probably Ollama with its OpenAI-compatible backend.

e.g.: start qwen coder:

```bash
ollama run qwen3-coder:30b
```

Then include it in your contine.dev configuration:

```yaml
  - name: "Ollama self hosted"
    provider: openai
    model: qwen3-coder:30b
    apiBase: http://localhost:11434/v1
    roles:
      - chat
      - edit
```

And now you can edit the code completely privately using a 100% local model.


### Fixing Python Linting

Microsoft sometimes is a little b**ch, they block the pylint plugin from working with non-official VS Codes.
So I recommend using 'basedpyright'; it works with Codium, and honestly in some aspects is even better than pylint.


A comprehensive guide to setting up fully self-hosted AI code editing with Codium and Continue.dev, keeping your code and AI interactions completely private and under your control.

100% Open-Source Self-Hostable AI Code Editing: Codium & Continue.dev


As privacy preserving communication options are vanishing rapidly, there remain only a few options for private secure communication.
While messaging services like Signal promise a good level of security and anonymity, its server is still managed by a corporate entity.

To truly control your own privacy and communication, you should consider joining a Matrix server hosted by a trusted party, or consider hosting one yourself.
This blog post will show how you can quickly setup and deploy a Matrix server on Kubernetes using [ananace's Helm charts](https://gitlab.com/ananace/charts).

### Performing the Helm Install

First, browse through the example values and the templates if required.
I managed to build a very minimal Matrix configuration via:

```bash
export KUBECONFIG=./kubeconfig.yaml 
helm repo add ananace-charts https://ananace.gitlab.io/charts
helm install matrix-synapse ananace-charts/matrix-synapse \
  --set serverName=example.com \
  --set publicServerName=matrix.example.com \
  --set wellknown.enabled=true
```

Or use a values file:

```yaml
serverName: example.com
publicServerName: matrix.example.com
wellknown:
  enabled: true
ingress:
  enabled: true
  tls:
    - secretName: matrix-synapse-crt
      hosts:
        - example.com
        - matrix.example.com
  hosts:
    - matrix.example.com
  className:
  annotations:
    kubernetes.io/ingress.class: "public"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
  includeServerName: true
```

Also **you should change the default postgresql credentials**:

```yaml
postgresql:
  enabled: true
  auth:
    password: <db-password>
    username: <db-username>
    database: <db-name>
```

Same things goes for the redis server:

```yaml
redis:
  enabled: true
  password: <new-redis-password>
```

### Configuring Upload and File Size Limits

```yaml
extraConfig:
  max_upload_size: 20M
ingress:
  enabled: true
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "20m"
    nginx.ingress.kubernetes.io/client-max-body-size: "20m"
```

### Creating First User

To be able to login to your Matrix server you'll have to actually register a first user.

##### 1. Retrieve the Main Matrix Pod

```bash
export KUBECONFIG=./kubeconfig.yaml
export NS=<matrix-namespace>
export POD_NAME=$(kubectl get pods --namespace "$NS" \
  -l "app.kubernetes.io/name=matrix-synapse,app.kubernetes.io/instance=matrix-synapse,app.kubernetes.io/component=synapse" \
  -o jsonpath="{.items[0].metadata.name}")
```

##### 2. Run the 'register_new_matrix_user' command

```bash
kubectl exec --namespace "$NS" "$POD_NAME" -- register_new_matrix_user \
  -c /synapse/config/homeserver.yaml \
  -c /synapse/config/conf.d/secrets.yaml \
  -u "<your-user-name>" \
  -p "<your-strong-admin-password>" \
  --admin http://localhost:8008
```

### Setting up Element as Matrix Web-UI

Now you have a Matrix server, [you may connect to it with any client](https://doc.matrix.tu-dresden.de/clients/).
But you might also want to be able to use it directly in the web.

Element Web is the official web client for Matrix and provides a user-friendly interface for accessing your Matrix server through a browser.

To deploy Element Web, you'll need to create a values file that configures the default server URL and ingress settings:

```bash
export KUBECONFIG=./kubeconfig.yaml
helm upgrade --install element-matrix ananace-charts/element-web \
  -f values_element_web.yaml
```

Here's an example `values_element_web.yaml` configuration:

```yaml
defaultServer:
  url: 'https://matrix.example.com'
  name: 'example.com'
ingress:
  enabled: true
  tls:
    - secretName: element-matrix-crt
      hosts:
        - element.example.com
  hosts:
    - element.example.com
  className: "public"
  annotations:
    kubernetes.io/ingress.class: "public"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
```

Make sure to replace `example.com` with your actual domain and adjust the `defaultServer.url` to point to your Matrix server's public URL.

### Backup Existing Matrix Server

```bash
export NS=default
export KUBECONFIG=./kubeconfig.yaml

# List PVCs
kubectl get pvc -n $NS

# Find PostgreSQL secret
kubectl get secret -n $NS -o name | grep -i postgres

# Get PostgreSQL pod name
export PG_POD=$(kubectl get pods -n $NS -l app.kubernetes.io/name=postgresql -o jsonpath='{.items[0].metadata.name}')

# Backup database (replace <db-password> with actual password from secret)
kubectl exec -n $NS "$PG_POD" -- sh -lc '
  export PGPASSWORD="<db-password>"
  /opt/bitnami/postgresql/bin/pg_dump -Fc -h 127.0.0.1 -U "<db-user>" "<db-name>"
' > postgres-FULL-$(date +%F).dump
```

### Backup All Matrix Media

```bash
export KUBECONFIG=./kubeconfig.yaml   # adjust path if needed
export NS=default
export RELEASE=matrix-synapse

# Get one synapse pod name
export SYNAPSE_POD=$(kubectl get pods -n $NS -l app.kubernetes.io/name=$RELEASE -o jsonpath='{.items[0].metadata.name}')

# Create tar.gz backup of media directory
kubectl exec -n $NS "$SYNAPSE_POD" -- sh -lc '
  set -e
  cd /synapse/data
  tar czf /tmp/matrix-media-backup.tar.gz media
  echo "Done."
'

# Copy backup from pod to local machine
kubectl cp $NS/$SYNAPSE_POD:/tmp/matrix-media-backup.tar.gz ./matrix-media-backup-$(date +%F).tar.gz
```

### Use Matrix bot to automate messaging

**1. Login a Matrix user**

```bash
curl -X POST "<YOUR-MATRIX-SERVER>/_matrix/client/v3/login" \
  -H "Content-Type: application/json" \
  -d "{
    \"type\": \"m.login.password\",
    \"identifier\": { \"type\": \"m.id.user\", \"user\": \"<user-name>\" },
    \"password\": \"<your-strong-admin-password>\"
  }" -o login-response.json -w "\nHTTP Status: %{http_code}\n" -s -S
```

Use the token to (2) list registered users (3) send a message to a registered user.


Secure Federated Chat: Self Host Matrix on Kubernetes


[Open-Chat](https://github.com/msgmate-io/open-chat-go) is not in it's 7th iteration and fully re-written in golang.
It now as 'federation' features using libp2p, it allows you to quickly spin-up your own network of open-chat nodes.

Open-Chat network member can freely communicate with another, they are automatically joined into a completely independent self-healing overlay-network.

### Creating a Open-Chat Network

Open-Chat Networks can be established inside private networks, across public networks *or across both*.
To be able to penetrate across public and private networks open-chat need to be able to route traffic back to itself trough a router, 
or it need to have at-least one public relay peer available to use as alternate path.

#### **1. Prepare a Bootstrap and Relay Node**
On our public node, we want to use as 'relay' and 'bootstrap' peer, download the Open-Chat binary [from the github release page](https://github.com/msgmate-io/open-chat-go/releases)
Be sure to download the one with '-federation' build in.
E.g.: 

```bash
wget -O open-chat https://github.com/msgmate-io/open-chat-go/releases/download/build-10/open-chat-federation-0.0.111-linux-amd64
chmod +x open-chat # make it executable
```

#### **2. Start first federation node**

For the 'bootstrapper' node you'll have to specify base admin credentials and network-credentials
- network credentials allow 'read-only' network operations and are required to be known for joining a network
- admin credentials are individual to each node, they are required to perform any elevated privilege action, e.g.: creating proxy routes

All can simply be specified like this:

```bash
./open-chat --default-network-credentials "<network-name>:<password>" --root-credentials "<user>:<password>"
```

You can also initalize not credentials without revealing them by specifying `--root-credentials "<username>:hashed_<pdf-hashed-password>"`.
Now that our bootstrapper node is available we can use its peer-id to bootstrap other nodes.

> If you want to persis your open-chat configuration system wide just run it with `--install`

We can extract it's peer Id by either loggin in to the UI with the admin user, or via the Open-Chat cli:

```bash
./open-chat client login                # you'll be promted to ender username and password
./open-chat client identity -b64        # revals the node id in a re-usable base64 format
eyJpZCI6ICI8TE9DQUxfUEVFUl9JRD4iLCAiY29ubmVjdF9tdWx0aWFkcmVzcyI6IFsiL2lwNC88U09NRV9JUF9BRFJFU1M+L3RjcC8wL3AycC88TE9DQUxfUEVFUl9JRD4iXX0=
```

#### **3. Adding other nodes to the network**

The only thing you need is the bootstrap peer string and the networks credentials

```bash
./open-chat -dnc "<network-name>:<password>" -rc "<different-node-specific-admin-creds>" -bs "<base64-bootstrap-peer-string>"
```

Now we just need to wait a moment untill these nodes found each other and established connections.
Lets see if the nodes are connected.

```bash
./open-chat client login          # ....
./open-chat client nodes -ls    # list all available nodes
```

Now lets *see if we can send requests to the node*:

```bash
./open-chat client get-metrics --node <remote-peer-id>
```

Hurray it works! **Now where do we go from here?**

#### **4. Establish on-demand proxy routing rules**

When we have a network, we can model its traffic in any-way we like.
Open-chat nodes can configure arbitrary routing rules on per-node basis, 
every node can only route traffic that goes trough itself.

To route traffic trough another node, routing rules on that node need to be established, 
thus the admin credentials for that node must be known.

Since we control the full network we know admin credentials of very node and can establish any routing rules we want.

**Lets use open-chat to create a ngrok-like TLS to behind NAT reverse-proxy**, to achieve this we need:
1. A single public peer in our control with an `<PUBLIC-IP>`
2. A domain name A-Record pointing that IP `<HOSTNAME>`
3. A private behind NAT node that hosts some TCP service on `<peer-port>`

> All the steps presented here can also be run from a single node, by using network-requests that are build in to Open-Chat.
> For simplicity we run command on individual nodes, but there are also workflows that allow controlling routing rule requests to ANY node just from a single network member.

##### 1. Use open-chat to request TLS certificate on the public node

```bash
./open-chat client tls --hostname "<HOSTNAME>" --key-prefix "<self-defined-certificate-prefix>"
```
This will automatically start the ACME challenge process and saves the obtained certificate in the Open-Chat nodes database.

##### 2. Establish TLS to network routing rules

First we need to tell the public node to accept TLS traffic using the just obtained certificate.
It should only accept traffic coming from `"<HOSTNAME>"` and the traffic should be routed to a specific port on the system.

```bash
./open-chat client proxy --direction "egress" --traffic-origin "<HOSTNAME>" --traffic-target "<key-prefix>:<peer-port>" --kind "tls" --key-prefix "<self-defined-certificate-prefix>"
```

This create an 'egress' proxy that accepts incoming traffic from the `"<HOSTNAME>"` and runnels it into the network.

```bash
./open-chat client proxy --direction "egress" --traffic-origin "<own-peer-id>:<peer-port>" --traffic-target "<remote-peer-id>:<peer-port>" --kind tcp
```

This establishes an 'ingress' rule from the network to the remote peer node.
Now the only thing that is left to do is the local node to tunnel network traffic from that peer to the `<peer-port>` that runs the TCP service.

##### 3. Accept Network Traffic on a local Port

Traffic is now routes from the domain to the public peer, from there inside the network and to the remote peer.
But currently the remote peer would just drop the traffic as it doesn't know what to do with it, lets route it to the port that runs our service:

```bash
./open-chat client proxy --direction "ingress" --traffic-origin "<own-peer-id>:<peer-port>" --traffic-target "<remote-peer-id>:<peer-port>" --kind tcp
```


### Conclusion and Outlook

**This Article merely gives a hint what Open-Chat Federation is able to do**, options from there go way further.
Just as food for though, the following thins are only a few lines of open-chat rules away:

1. **Decentralized Multi-Site behind NAT hosting**
Host a swarm of Server behind NAT on different locations, route them trough multiple public peers in a 'load balancing' fashion
and you have a load resilient decentralized server infrastructure that masks the actual server locations, allows for easy traffic filtering and allows for easy horizontal scaling by just extending the node pool.
2. **Persistent self hosted Ngrok**. As shown in this article open-chat can easily be used to create development routing rules as tools like ngrok can do.
3. **Self managed overlay networks for anything**. This connect extends to any application, also the ones fully privately easily use open-chat routing rules to manage traffic in your local network, *you may even use it in offline local networks!*.


How Open-Chats Federation Enables anybody to host anything anywhere

Latest Posts

Opencode; Usability with Local LLMs on iGPU w 128GB vram: My Tests

Local LLMs on Strix Halo 128GB Shared Ram: My Tests

Comparing Agentic Code Editors

100% Open-Source Self-Hostable AI Code Editing: Codium & Continue.dev

Secure Federated Chat: Self Host Matrix on Kubernetes

Breaking Trough NAT: Decentralized Self-Hosting Via Open-Chat