
Alexander is a writer, traveler, and serial digital entrepreneur. He’s the founder of the Tushi, the Gen X personal growth newsletter Further, and offers small business advice at Unemployable.

Twitter

Linkedin

Facebook

Director and Chief Evangelist

Alexander Hipp


Anil is a content-obsessed marketer who loves driving organic traffic for outstanding companies. He is also the creator of Hack the Content, co-host of The Runner, and owner of several digital commerce sites.

Managing Partner

Anil Vugels


Monika is a writer, musician, and content strategist with a passion for language. When she isn’t helping clients, you can find her writing songs, hiking in a northern forest, or furiously journaling.

Director of Operations

Ann Monika

Programmer / Founder / Student

Tim Schupp


# Solid Authentication Overview

> A presentation prepared for the
> 'Data Ecosystems Lab'

## Content

This blog article delves into several key aspects of Solid authentication and OpenID Connect, providing insights into the mechanisms and practical implementation details:

- **Introduction to Solid and OpenID Connect**: An overview of the Solid framework and how OpenID Connect (OIDC) serves as a foundational technology for secure authentication.
- **The Solid Community Server's Authentication Process**: A closer look at how the Solid Community Server implements OIDC and the intricacies involved in user verification.
- **Reverse Engineering Authentication with Python**: Step-by-step guidance on how to reverse engineer client authentication processes using the Python `requests` library.
- **Jupyter Notebook Demonstration**: A practical example showcasing the authentication flow within a Jupyter notebook, accessible for interactive learning and experimentation.
- **Flow Charts and Visual Aids**: The inclusion of original and comprehensive flow charts to visualize the authentication sequence as defined by the Solid project.

> If you want to deploy solid on a self hosted kubernetes cluster [checkout this blog post](todo)

## Presentation Slides

<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vR9IbBdC5t9iGfPpiIm6voYEIJvfLBghO4rQoI1J57_cqfim40nndh1kjCajXyoTt77bQVvMgLaLkDJ/embed?start=false&loop=true&delayms=5000" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>

## Flow Chart

- To view the original flow chart as published by the Solid project, please visit this link: [Solid OIDC Flow Chart](https://solidproject.org/TR/oidc)
- For a full-size preview of the flow chart, click here: [Full Size Flow Chart Preview](https://viewer.diagrams.net/?tags=%7B%7D&highlight=0000ff&edit=_blank&layers=1&nav=1&title=diagramm.xml#R7V3bcptIGn6WuVDV7oUozodLW46SbM3UeOKkUnuVQqIlsUFqBpDszNNvNydB8yNAAoRkchHbTQMN%2Ff3nAxNptn376Jnu5g9sIWci8tbbRHqaiKKgqgb5QUd%2BRSOywUcDa8%2B24knHgRf7HxQPJtP2toX83MQAYyew3fzgEu92aBnkxkzPw6%2F5aSvs5O%2FqmmtUGHhZmk5x9LttBZtoVFf44%2FgnZK83yZ0FPj6yNZPJ8YC%2FMS38mhmSPkykmYdxEP22fZshh7685L1E581LjqYL89AuqHOC%2BYfh%2FTpIbwdNdP%2FGb389ffn0YRpf5WA6%2B%2FiBJ6LqkOs9rjC5LHlh5jI6oP69pyt9nOHtCnuBaR6H6PaagTmlp0x9b5mbvwkCulUPdGHinE7xuTXGaweZru1zS7wlw0ufTJmvzK3tUJyA91DX9Oc3H3nJEuPrJYfi1xz8SvaOvHGX%2FrrfOr%2FbK%2BTYO%2FLXo4s8e4sCchnpyYmHn49jj68bO0AvbvTcrwTV4UNsHfKXQH4lQAtMcgo9nSd%2FWx52v5reGgXxwBI7jun69iJcBh3x0HLv%2BfYBfUF%2BBHA6ivcBvfUsBS4ddMljkxE%2F8Mg96CAy%2FeAV%2BfT5duj1g0XAmTyg9nh8PSg9cHxj9J2nx50Fzpydea0zZlLJBQ7IC2xCFiWnkgc8ICt3Lp%2Bb4OH9zirM0J4yh6L3W4R1glGyAvSWGYph%2FhFhsnXeLzIlPipLnCBFJyVcJ6bB1yMJp4S6yZCvnhCrGbONdXrxI2WRX2LiakBoIkBoDFwzIHMx2f1wCcrjRHliUEswssFrvDOdLG5BPAUhMl8oIfhFSniHYCvC6yRfrI85JQc4DQJcEW%2Ba0hXelCrG3gPHrik8Iu79SGX1yNvvjNxa5%2B0iL3K6UMncxV6Zu3ZrxEbePyU1%2FjtafLbqklyGUBLEPDj2ekfGFjgIyErpha0HqnXTMQcvf%2BbgSxCfh2oCS%2BkphWAKt5i8Uvzwp%2FDj470XP%2F1J8RskxFw2T43m0TWfRKOHHDMgVJ%2B3NwBcxac%2BU4F%2BRLFg5MWFIDHQjB4oPuuITvJuzV%2BZabGeUHqfVOgk91EYG4Fdl8rMl0%2FPT2kPvj75JVoxc3ayfLxa%2BSiYsOSXvufzKVJvpG7VgTN20Y6KBNPfpPyMHI4tVv0KQN8j5eXpBf81fVlMZ%2Fajo60%2B4uTBuwawqDIAVhkAR5R2MYAlFpBqLYAlSplYscoSMjsDhac243bkQiwP%2BGcPH2zrqI0tvGTGw57IWJF%2FQQS7wGGiBUUILZ8yanh3rOHFb0g5ycQaWFWyxuVJGFL2JEjZ41tQ9kCiNhqJltGS78%2BSP8mDa2OOlThGPVNeEFoA3M7531ravL18M5T%2FPqFv%2F3w7vO6nYjNlZkRct4grIKkuCE8gjpPyJq0hJmyvAnWJ26l1LidADstB6y4e%2BntP8ZdatfwTXu63KFQx7t7AjQ3XrIF7kpn0biCwns6WLFxJOcsCLTMQ2FWWmDEdmbIwJd6cd%2BlIiX9%2BnpFTaHB2Za%2FPNwXujDBPqZWd06WWpxexK7pkPENihSdJUhRwXTXpmL3bEOm40htAbE%2Fs5YhsIkp8%2BC9Hi0fKSAYj4g9tL0LrvMC7bxF5NSXg%2FgL5Hz98nVCH%2FNxCWzx1MRHYc9fDK5tqffOl6ZEBaYuaegsIiZqOgxy89swto%2FvmjjVwGqzsN5Q4%2BqK%2FHWeW7pW00pdouaTGb%2BDhnyhzZKErclumsGjAgi6rIYpQTLsNuwQEtHiDgmlt%2B1HkY%2BbYGc1wFEkDEklSR8EQiYnRJ%2Fc5T1UsrHIAIqahp%2BCOwh5GEdLXjO%2BxTqRUG2o7PMJoTZLcCNISfHZf4ZGEgkaNKJQxz3%2B%2BxCoRh%2B0l1YeIvBr1nwTqYomUyKV9pOGCXjQgSaoCcBPtxN7ZgenQ10yAH6snvBnG2zrRSXY49OS2yL99wtYCVuUJB%2Be2kyyNrCD%2Ba1gpHTHOeI7n4yVcmuTBM4g1utFrROY%2BMl%2BR5CE2nK8p4HPUEzKiUvEWBqA3SVBMrz9BNDQzKX9XHzu2pXA%2BYSSmMyX35Sx0CA%2BNkimiH0HnGBKBYoZ6v7JJrpRN7wrTzLMP9ZmixXIcV0pdt%2B8vsxSkWzJElbq4kFT1QqpM%2FGPCAGhQbosGz4G675q7M6EeXSjYutO9701DCTCR%2Ba1pO1wgbLktPfMr8gNBlH7LIC6643lpXvcHxPjoVJBY8SDLxei%2BmJnWCzgVyPpuxaLox1mUMzZiN1XW1uDr2xqMbVDDqBDhLb%2FQYmCywuWaqnJTg0HQNPA%2BdRX6Qk5eTcOmLXVdaZaCdztuzu7dkuzW1Q2UNve1S%2BB96kFMYSzaukHq1vzorTkmh6MGV%2BsBRokakEZquVfy7qY%2Fd%2FiVSPk5JRnbmka5HXuP4BLvbkDo9xSt1QtCH0pd7ixgC%2Bd%2BVaH6ltCql6A1ahBAKxioaURYVL7IrXmkN5%2FgXgG%2BEpwAaCp3dKscE2rUgVzQEm2xjRJHEDuV3oQ7ws4HWh1JkfM7XtvUhJp5yPJvCEHF%2FgcqYAx3CKFTjvgxjX0QaeyXibeCDSCI9RLWRa2rjHV5LMy5Y3zJIsDBIOWJLfFsD18j%2F7obfCl8AV9gi6Be8VWZTtnECUwAsfO3Nj3NibUYqqF1lEoQxvq%2Fbuxdu46SnIsvdi6Wu%2FhOdhjp37Ui6zSVIPNPzV%2BzraxGNvofOxFLE%2B2ZlhCyXjHfYOc38%2BQUnYWMAjCA%2BL9y3Vjp0BLRyGTbcsn%2FMecYQ%2F3HVJsCnFVI7%2B01jlMZZOwyeFhmStM%2BEeQ9Ex4f0Hy2bNtG0IA%2BJ3R4hl19YV%2FB%2FN6nburs3iu97v1YUwQyrtAp7W1H1nWKdSlQhkRXrMvfLIld8pNoerP51zfnO7%2F7tAe7z16miA4yjpdPZgXfhNSRUlqIQiiSwWmMJto45leqLCp8rRt2HR%2BubH55DYn4Ja2KnkXcKY6Y8XNMnc5JQRr%2FsFwi%2F7S3%2BSaEpViEAuTD0cCUrDac0CChiW1CIxUz79SorizLSnyylaWGcm%2BFWUVU1mwK19gwl%2FKGs6JVVMDrOiugGzW7g3hvza58HdnaIAEOkjd%2F3tmBHdbeWGTWV6KF0ct9QZbtUUfszfNiRWBjyooG2K2i2FV6LAiFcoenZR9qb%2FHCXP5ch290Ghs7dKftaEcjAyQz7q0X%2F4o8zXzy49%2BxMUOhklgZD6GYTnB3PF4ja3aiPEq8RF3%2BSS2ftSQ%2Fo0IuYo%2FIE%2B0xKu%2F64cUA%2B7H3bOrbPo0qEH65N%2FXebBpB4TlVqsR1NqWin84ILX1Wo2UmZ1GHzIo8nmpu6XbuFj79kWqeX8L2CEkmF%2F9hZ7mRWLgpZifoeU4H9uKHvDRt6Jxw55cbasbfBryStk%2BffX9%2FSQfYayNHAJEDJTV0lXVVzkdccKtoVHiarCzcLLIyBRZT12dIrofoflGpbO4simR%2BohJhbaEfyw2VSjtiDFAVlieHo4PEUAnoGEfVtE2opS32tkOPJmHOvedcrrClw26b6Lwoq4tFpwQ0AAL5GtuopD0xN5YYMmSRJC9kBOkPdBSjxxOG7pO2TKSvQP1NXeposWqJ4WqFXFewdFbtrGwP5Lo31NfqfOb7gpzQtOXnYQUpMW%2BoF3LvOti0ehXZLSdLaxCA%2Bs6Wruwn0ISvuB7euuFJ0U754c5lqi067142oE4hhdBKfay019mDqdPTtG48iIIgc%2FCdyr%2FfYrBnNPIhFvMuC8%2FWrwsR1orfaeXqOdhvGel6PxWpWqMkMwC17DrbizjCiIRyg89m%2BDGfT2Qxb1Fwjry%2BtTZRoDPa6Et2sDxd7yj%2BxLaF0qu%2B%2FSXI4LpOyJpT129Ms4X3MIDEULHSc3dnlisNVqRuu9sxV1VZU3UNMlclQZYUOd6WbIAi%2FNeOGctSGvjJGOiLlJ1ZsNW9xocGQTuNwGZceRMxjN5BvRBvt2PzFbQ4oDghcRS23mOZFW%2FJnc6uF2BXOgQD5C4akAwDiGq97W3eGbkARPVCILIr7dimAIPIo5Xbg5UrCwaX9K%2FpuvXS8VZnd18CVttxgq1cmd%2FQWuCn2JRmcKpLU4vgOo%2FUQmvXpC15ksokzeM8phAsp5pWDsdS6CkxSSwk3MlAv3LR6DewlUiQdtxUx898HTX0XO766Jmq9kxdw%2BPUVcazYLDRB6XKhyQ0PkMrPM2FfqcrJ0nDhAplVoy6X8GTe73enErNuNilCiJTNdAc3TVjk60hF6pGvkPTuE7xi9oJEKkNrec%2FF6wkdTatG9ECl7StYG5V14qWyy7QASDLKyvHCmmgQnoZ6mxjhfSJCmnoG0P9Vkg3UwTGplMDbzrFMG4NBFgRX%2ByH2doLFDUT1yO%2BusXXRbmQRZe1XrOpWWfV1mNLxiGh6%2BKWjAz3UsEGIgC82uiZB%2BKrWUvGgRob59sWBUOlh4Cd0tFHKYoBO6XRZykUvmKdHdsZQke%2B1cSlygdRVfo7bTXRB9ALypnBaUItEDUFuyzznFR6t9IOEUrxLKX6LKF4llx9lnjWCjWt%2Bl6VhGywPgNoJwbQyqK6rvdd%2BRZy3y8Oex40Yll3718Q9CKwIQ1K7jMkCNl%2F91Nh7JF3ZKMDzSmcqLEo%2FRGhMlNXTLYwqlvPFBjT9xZK3U1YcmXSLyvSty%2FyC0RWnf3O4g0XFLPMWoU6dHXVKwHmqa19eukuMrPDDA1Cb6YTbKam6xZzNaJcWWlOy%2BYnYRYGTcOIGnxE%2F0eQvpUEjZ4qj4XCN1JUoJur1OFHGUF2XF5HdA%2Fs%2BNjwgXBY2iEYe%2FY%2FUbuZkOM%2Bmr69TNkyGXt6xs%2FRoU%2FItMKP8tw822WrBVQd4LmSAHYs6qxgGaoXuDlHypUSmlmzsWY%2ByTkOkpJb1faQVK616zrJ%2B25os%2FRQ1O7c3S%2BcDCdzPfsQHfjPd2oi%2FUS%2F7pGRQQ1p%2BmZkJ9xw94CwKgkaCsy7F6BigY9BFXe9Q68y1aQJYp6%2FUU7BloKnfuH34AZumoV1KgSZzRo8EQnvQEEwuJJC6dadysyXfZLob5mzVpZFToCXVu6BLp5S1dZYKJ5SkeMric0XphkVd6lSjgSeM%2BTT%2BzSAxGDQ5XyHinqesE%2Bmrpzb2eFiwlY5QTCO%2F9jUi9ohi%2BaGAM9JWtnd6toCwolr9JYsfEOd1iIJG39Lmn5e%2BmBbx%2BamVYKYaDtBXhHKu7diqZj1hcVDZky6NFszdK%2BxNL21LSukREjzylOfh4NIV5SejCM1Xpx0xChjApB1lNiZbfelhGFV2X5taLB6ADsbvCDCR73i%2BAg6Ajo5w3oZwQ11fO4VgMmFRwDeMQAZrpf2qbga6CqLskfQ3T7o%2BFKNU5OuDcBy19tAAfgFxVaGyI%2BoO0%2FWQinm%2FaKu0us2HNSd7%2FfN%2BQGR9dv5QO3bW6tInCAapVwr7fqXwY%2FWXbdmMJlauu5XVIeDz5o5J2MOSXRULeTzCVD%2BlN5Wkw%2Fyp4fpVh19MOTNbP7AFqIz%2Fg8%3D)


<iframe frameborder="0" style="width:100%;height:848px;" src="https://viewer.diagrams.net/?tags=%7B%7D&highlight=0000ff&edit=_blank&layers=1&nav=1&title=diagramm.xml#R7V3bcptIGn6WuVDV7oUozodLW46SbM3UeOKkUnuVQqIlsUFqBpDszNNvNydB8yNAAoRkchHbTQMN%2Ff3nAxNptn376Jnu5g9sIWci8tbbRHqaiKKgqgb5QUd%2BRSOywUcDa8%2B24knHgRf7HxQPJtP2toX83MQAYyew3fzgEu92aBnkxkzPw6%2F5aSvs5O%2FqmmtUGHhZmk5x9LttBZtoVFf44%2FgnZK83yZ0FPj6yNZPJ8YC%2FMS38mhmSPkykmYdxEP22fZshh7685L1E581LjqYL89AuqHOC%2BYfh%2FTpIbwdNdP%2FGb389ffn0YRpf5WA6%2B%2FiBJ6LqkOs9rjC5LHlh5jI6oP69pyt9nOHtCnuBaR6H6PaagTmlp0x9b5mbvwkCulUPdGHinE7xuTXGaweZru1zS7wlw0ufTJmvzK3tUJyA91DX9Oc3H3nJEuPrJYfi1xz8SvaOvHGX%2FrrfOr%2FbK%2BTYO%2FLXo4s8e4sCchnpyYmHn49jj68bO0AvbvTcrwTV4UNsHfKXQH4lQAtMcgo9nSd%2FWx52v5reGgXxwBI7jun69iJcBh3x0HLv%2BfYBfUF%2BBHA6ivcBvfUsBS4ddMljkxE%2F8Mg96CAy%2FeAV%2BfT5duj1g0XAmTyg9nh8PSg9cHxj9J2nx50Fzpydea0zZlLJBQ7IC2xCFiWnkgc8ICt3Lp%2Bb4OH9zirM0J4yh6L3W4R1glGyAvSWGYph%2FhFhsnXeLzIlPipLnCBFJyVcJ6bB1yMJp4S6yZCvnhCrGbONdXrxI2WRX2LiakBoIkBoDFwzIHMx2f1wCcrjRHliUEswssFrvDOdLG5BPAUhMl8oIfhFSniHYCvC6yRfrI85JQc4DQJcEW%2Ba0hXelCrG3gPHrik8Iu79SGX1yNvvjNxa5%2B0iL3K6UMncxV6Zu3ZrxEbePyU1%2FjtafLbqklyGUBLEPDj2ekfGFjgIyErpha0HqnXTMQcvf%2BbgSxCfh2oCS%2BkphWAKt5i8Uvzwp%2FDj470XP%2F1J8RskxFw2T43m0TWfRKOHHDMgVJ%2B3NwBcxac%2BU4F%2BRLFg5MWFIDHQjB4oPuuITvJuzV%2BZabGeUHqfVOgk91EYG4Fdl8rMl0%2FPT2kPvj75JVoxc3ayfLxa%2BSiYsOSXvufzKVJvpG7VgTN20Y6KBNPfpPyMHI4tVv0KQN8j5eXpBf81fVlMZ%2Fajo60%2B4uTBuwawqDIAVhkAR5R2MYAlFpBqLYAlSplYscoSMjsDhac243bkQiwP%2BGcPH2zrqI0tvGTGw57IWJF%2FQQS7wGGiBUUILZ8yanh3rOHFb0g5ycQaWFWyxuVJGFL2JEjZ41tQ9kCiNhqJltGS78%2BSP8mDa2OOlThGPVNeEFoA3M7531ravL18M5T%2FPqFv%2F3w7vO6nYjNlZkRct4grIKkuCE8gjpPyJq0hJmyvAnWJ26l1LidADstB6y4e%2BntP8ZdatfwTXu63KFQx7t7AjQ3XrIF7kpn0biCwns6WLFxJOcsCLTMQ2FWWmDEdmbIwJd6cd%2BlIiX9%2BnpFTaHB2Za%2FPNwXujDBPqZWd06WWpxexK7pkPENihSdJUhRwXTXpmL3bEOm40htAbE%2Fs5YhsIkp8%2BC9Hi0fKSAYj4g9tL0LrvMC7bxF5NSXg%2FgL5Hz98nVCH%2FNxCWzx1MRHYc9fDK5tqffOl6ZEBaYuaegsIiZqOgxy89swto%2FvmjjVwGqzsN5Q4%2BqK%2FHWeW7pW00pdouaTGb%2BDhnyhzZKErclumsGjAgi6rIYpQTLsNuwQEtHiDgmlt%2B1HkY%2BbYGc1wFEkDEklSR8EQiYnRJ%2Fc5T1UsrHIAIqahp%2BCOwh5GEdLXjO%2BxTqRUG2o7PMJoTZLcCNISfHZf4ZGEgkaNKJQxz3%2B%2BxCoRh%2B0l1YeIvBr1nwTqYomUyKV9pOGCXjQgSaoCcBPtxN7ZgenQ10yAH6snvBnG2zrRSXY49OS2yL99wtYCVuUJB%2Be2kyyNrCD%2Ba1gpHTHOeI7n4yVcmuTBM4g1utFrROY%2BMl%2BR5CE2nK8p4HPUEzKiUvEWBqA3SVBMrz9BNDQzKX9XHzu2pXA%2BYSSmMyX35Sx0CA%2BNkimiH0HnGBKBYoZ6v7JJrpRN7wrTzLMP9ZmixXIcV0pdt%2B8vsxSkWzJElbq4kFT1QqpM%2FGPCAGhQbosGz4G675q7M6EeXSjYutO9701DCTCR%2Ba1pO1wgbLktPfMr8gNBlH7LIC6643lpXvcHxPjoVJBY8SDLxei%2BmJnWCzgVyPpuxaLox1mUMzZiN1XW1uDr2xqMbVDDqBDhLb%2FQYmCywuWaqnJTg0HQNPA%2BdRX6Qk5eTcOmLXVdaZaCdztuzu7dkuzW1Q2UNve1S%2BB96kFMYSzaukHq1vzorTkmh6MGV%2BsBRokakEZquVfy7qY%2Fd%2FiVSPk5JRnbmka5HXuP4BLvbkDo9xSt1QtCH0pd7ixgC%2Bd%2BVaH6ltCql6A1ahBAKxioaURYVL7IrXmkN5%2FgXgG%2BEpwAaCp3dKscE2rUgVzQEm2xjRJHEDuV3oQ7ws4HWh1JkfM7XtvUhJp5yPJvCEHF%2FgcqYAx3CKFTjvgxjX0QaeyXibeCDSCI9RLWRa2rjHV5LMy5Y3zJIsDBIOWJLfFsD18j%2F7obfCl8AV9gi6Be8VWZTtnECUwAsfO3Nj3NibUYqqF1lEoQxvq%2Fbuxdu46SnIsvdi6Wu%2FhOdhjp37Ui6zSVIPNPzV%2BzraxGNvofOxFLE%2B2ZlhCyXjHfYOc38%2BQUnYWMAjCA%2BL9y3Vjp0BLRyGTbcsn%2FMecYQ%2F3HVJsCnFVI7%2B01jlMZZOwyeFhmStM%2BEeQ9Ex4f0Hy2bNtG0IA%2BJ3R4hl19YV%2FB%2FN6nburs3iu97v1YUwQyrtAp7W1H1nWKdSlQhkRXrMvfLIld8pNoerP51zfnO7%2F7tAe7z16miA4yjpdPZgXfhNSRUlqIQiiSwWmMJto45leqLCp8rRt2HR%2BubH55DYn4Ja2KnkXcKY6Y8XNMnc5JQRr%2FsFwi%2F7S3%2BSaEpViEAuTD0cCUrDac0CChiW1CIxUz79SorizLSnyylaWGcm%2BFWUVU1mwK19gwl%2FKGs6JVVMDrOiugGzW7g3hvza58HdnaIAEOkjd%2F3tmBHdbeWGTWV6KF0ct9QZbtUUfszfNiRWBjyooG2K2i2FV6LAiFcoenZR9qb%2FHCXP5ch290Ghs7dKftaEcjAyQz7q0X%2F4o8zXzy49%2BxMUOhklgZD6GYTnB3PF4ja3aiPEq8RF3%2BSS2ftSQ%2Fo0IuYo%2FIE%2B0xKu%2F64cUA%2B7H3bOrbPo0qEH65N%2FXebBpB4TlVqsR1NqWin84ILX1Wo2UmZ1GHzIo8nmpu6XbuFj79kWqeX8L2CEkmF%2F9hZ7mRWLgpZifoeU4H9uKHvDRt6Jxw55cbasbfBryStk%2BffX9%2FSQfYayNHAJEDJTV0lXVVzkdccKtoVHiarCzcLLIyBRZT12dIrofoflGpbO4simR%2BohJhbaEfyw2VSjtiDFAVlieHo4PEUAnoGEfVtE2opS32tkOPJmHOvedcrrClw26b6Lwoq4tFpwQ0AAL5GtuopD0xN5YYMmSRJC9kBOkPdBSjxxOG7pO2TKSvQP1NXeposWqJ4WqFXFewdFbtrGwP5Lo31NfqfOb7gpzQtOXnYQUpMW%2BoF3LvOti0ehXZLSdLaxCA%2Bs6Wruwn0ISvuB7euuFJ0U754c5lqi067142oE4hhdBKfay019mDqdPTtG48iIIgc%2FCdyr%2FfYrBnNPIhFvMuC8%2FWrwsR1orfaeXqOdhvGel6PxWpWqMkMwC17DrbizjCiIRyg89m%2BDGfT2Qxb1Fwjry%2BtTZRoDPa6Et2sDxd7yj%2BxLaF0qu%2B%2FSXI4LpOyJpT129Ms4X3MIDEULHSc3dnlisNVqRuu9sxV1VZU3UNMlclQZYUOd6WbIAi%2FNeOGctSGvjJGOiLlJ1ZsNW9xocGQTuNwGZceRMxjN5BvRBvt2PzFbQ4oDghcRS23mOZFW%2FJnc6uF2BXOgQD5C4akAwDiGq97W3eGbkARPVCILIr7dimAIPIo5Xbg5UrCwaX9K%2FpuvXS8VZnd18CVttxgq1cmd%2FQWuCn2JRmcKpLU4vgOo%2FUQmvXpC15ksokzeM8phAsp5pWDsdS6CkxSSwk3MlAv3LR6DewlUiQdtxUx898HTX0XO766Jmq9kxdw%2BPUVcazYLDRB6XKhyQ0PkMrPM2FfqcrJ0nDhAplVoy6X8GTe73enErNuNilCiJTNdAc3TVjk60hF6pGvkPTuE7xi9oJEKkNrec%2FF6wkdTatG9ECl7StYG5V14qWyy7QASDLKyvHCmmgQnoZ6mxjhfSJCmnoG0P9Vkg3UwTGplMDbzrFMG4NBFgRX%2ByH2doLFDUT1yO%2BusXXRbmQRZe1XrOpWWfV1mNLxiGh6%2BKWjAz3UsEGIgC82uiZB%2BKrWUvGgRob59sWBUOlh4Cd0tFHKYoBO6XRZykUvmKdHdsZQke%2B1cSlygdRVfo7bTXRB9ALypnBaUItEDUFuyzznFR6t9IOEUrxLKX6LKF4llx9lnjWCjWt%2Bl6VhGywPgNoJwbQyqK6rvdd%2BRZy3y8Oex40Yll3718Q9CKwIQ1K7jMkCNl%2F91Nh7JF3ZKMDzSmcqLEo%2FRGhMlNXTLYwqlvPFBjT9xZK3U1YcmXSLyvSty%2FyC0RWnf3O4g0XFLPMWoU6dHXVKwHmqa19eukuMrPDDA1Cb6YTbKam6xZzNaJcWWlOy%2BYnYRYGTcOIGnxE%2F0eQvpUEjZ4qj4XCN1JUoJur1OFHGUF2XF5HdA%2Fs%2BNjwgXBY2iEYe%2FY%2FUbuZkOM%2Bmr69TNkyGXt6xs%2FRoU%2FItMKP8tw822WrBVQd4LmSAHYs6qxgGaoXuDlHypUSmlmzsWY%2ByTkOkpJb1faQVK616zrJ%2B25os%2FRQ1O7c3S%2BcDCdzPfsQHfjPd2oi%2FUS%2F7pGRQQ1p%2BmZkJ9xw94CwKgkaCsy7F6BigY9BFXe9Q68y1aQJYp6%2FUU7BloKnfuH34AZumoV1KgSZzRo8EQnvQEEwuJJC6dadysyXfZLob5mzVpZFToCXVu6BLp5S1dZYKJ5SkeMric0XphkVd6lSjgSeM%2BTT%2BzSAxGDQ5XyHinqesE%2Bmrpzb2eFiwlY5QTCO%2F9jUi9ohi%2BaGAM9JWtnd6toCwolr9JYsfEOd1iIJG39Lmn5e%2BmBbx%2BamVYKYaDtBXhHKu7diqZj1hcVDZky6NFszdK%2BxNL21LSukREjzylOfh4NIV5SejCM1Xpx0xChjApB1lNiZbfelhGFV2X5taLB6ADsbvCDCR73i%2BAg6Ajo5w3oZwQ11fO4VgMmFRwDeMQAZrpf2qbga6CqLskfQ3T7o%2BFKNU5OuDcBy19tAAfgFxVaGyI%2BoO0%2FWQinm%2FaKu0us2HNSd7%2FfN%2BQGR9dv5QO3bW6tInCAapVwr7fqXwY%2FWXbdmMJlauu5XVIeDz5o5J2MOSXRULeTzCVD%2BlN5Wkw%2Fyp4fpVh19MOTNbP7AFqIz%2Fg8%3D"></iframe>

## Jupyter Notebook

- For a hands-on demonstration of the reverse-engineered authentication using the Python `requests` library, access the Jupyter Notebook here: [Presentation: Reverse-Engineered Authentication Notebook](https://colab.research.google.com/drive/17H9eaPhxgqb4yonFpfEle39pQjWghL_o?usp=sharing)


<link rel="stylesheet" href="https://github.githubassets.com/assets/gist-embed-f65f23c5975e.css">
<div id="gist126680771" class="gist">
    <div class="gist-file" translate="no">
      <div class="gist-data">
        <div class="js-gist-file-update-container js-task-list-container file-box">
          <div id="file-copy-of-reverseengineeredsolidauthenticationbytimschupp-ipynb" class="file my-2">
            <div itemprop="text" class="Box-body p-0 blob-wrapper data type-jupyter-notebook">
              <div class="render-wrapper">
                <div class="render-container is-render-pending js-render-target"
                     data-identity="b896a2c7-130f-40d6-9e79-f19c42218f0a"
                     data-host="https://notebooks.githubusercontent.com"
                     data-type="ipynb">
                  <svg style="box-sizing: content-box; color: var(--color-icon-primary);" width="64" height="64" viewBox="0 0 16 16" fill="none" data-view-component="true" class="octospinner mx-auto anim-rotate">
                    <circle cx="8" cy="8" r="7" stroke="currentColor" stroke-opacity="0.25" stroke-width="2" vector-effect="non-scaling-stroke" fill="none"></circle>
                    <path d="M15 8a7.002 7.002 0 00-7-7" stroke="currentColor" stroke-width="2" stroke-linecap="round" vector-effect="non-scaling-stroke"></path>
                  </svg>
                  <div class="render-viewer-error">Sorry, something went wrong. <a class="Link--inTextBlock" href="https://gist.github.com/user-1024/00f76dfe47a63659de070d449c9946f9.js">Reload?</a></div>
                  <div class="render-viewer-fatal">Sorry, we cannot display this file.</div>
                  <div class="render-viewer-invalid">Sorry, this file is invalid so it cannot be displayed.</div>
                  <iframe class="render-viewer"
                          src="https://notebooks.githubusercontent.com/view/ipynb?bypass_fastly=true&amp;color_mode=auto&amp;commit=db9efc58d843777a23609c871a7478f9a282b6c4&amp;docs_host=https%3A%2F%2Fdocs.github.com&amp;enc_url=68747470733a2f2f7261772e67697468756275736572636f6e74656e742e636f6d2f676973742f757365722d313032342f30306637366466653437613633363539646530373064343439633939343666392f7261772f646239656663353864383433373737613233363039633837316137343738663961323832623663342f636f70792d6f662d72657665727365656e67696e6565726564736f6c696461757468656e7469636174696f6e627974696d7363687570702e6970796e62&amp;logged_in=false&amp;nwo=user-1024%2F00f76dfe47a63659de070d449c9946f9&amp;path=copy-of-reverseengineeredsolidauthenticationbytimschupp.ipynb&amp;repository_id=126680771&amp;repository_type=Gist#b896a2c7-130f-40d6-9e79-f19c42218f0a"
                          sandbox="allow-scripts allow-same-origin allow-top-navigation"
                          title="File display"
                          name="b896a2c7-130f-40d6-9e79-f19c42218f0a">
                    Viewer requires iframe.
                  </iframe>
                </div>
              </div>
            </div>
          </div>
        </div>
        <div class="gist-meta">
          <a href="https://gist.github.com/user-1024/00f76dfe47a63659de070d449c9946f9/raw/db9efc58d843777a23609c871a7478f9a282b6c4/copy-of-reverseengineeredsolidauthenticationbytimschupp.ipynb" style="float:right" class="Link--inTextBlock">view raw</a>
          <a href="https://gist.github.com/user-1024/00f76dfe47a63659de070d449c9946f9#file-copy-of-reverseengineeredsolidauthenticationbytimschupp-ipynb" class="Link--inTextBlock">
            copy-of-reverseengineeredsolidauthenticationbytimschupp.ipynb
          </a>
          hosted with ❤ by <a class="Link--inTextBlock" href="https://github.com">GitHub</a>
        </div>
      </div>
    </div>
</div>

Exploring the workings of OpenID Connect, the Solid Community Server implementation, and reverse-engineering client authentication with Python requests.

Understanding Solid Authentication


## Intro 

Sentry is an incredibly nice error tracking tool; it can significantly increase the speed of bug fixing and debugging. It also offers a bunch of other features. You should also check out [their service](https://sentry.io)

Since I like to know how the stuff works and willing to take it appart; I would always prefer to self-host if I can. This also gives me more comfort about controlling my own data which by the way also aids GDPR compliance (though this is also possible using Sentry's service directly).

Fortunately, there [still is a Helm chart](https://github.com/sentry-kubernetes/charts/tree/develop). 
'Still', because for example: [the deprecated Posthog chart](https://github.com/PostHog/charts-clickhouse), but also the originally (officially maintained?) [sentry chart](https://github.com/helm/charts/tree/master/stable/sentry) seems deprecated now.

## Setup

I prepared a script that will:

1. Configure an ingress with a cluster issuer annotation to serve under a specific `$HOSTNAME`
2. Configure a Postgres password
3. Enable session recordings and its interface

## Installation

Use the following script to install Sentry:

```bash
./install_sentry_microk8s.sh \
  K8_NAMESPACE="<installation-namespace>" \
  RELEASE_NAME="<release-name>" \
  SENTRY_HOSTNAME="<...>" \
  SENTRY_HOSTNAME="<...>" \
  POSTGRES_PASSWORD="<...>" \
  SMTP_USERNAME="<...>" \
  SMTP_PASSWORD="<...>" \
  SMTP_PORT="<...>" \
  SMTP_FROM_EMAIL="<...>" \
  SENTRY_ADMIN_EMAIL="<...>" \
  SENTRY_ADMIN_PASSWORD="<...>"
```

I found out about having to set and update the admin Postgres password using a secret; [in this Github issue](https://github.com/sentry-kubernetes/charts/issues/554).
That's why we create a simple secret for the Postgres-password here:

```bash
apiVersion: v1
kind: Secret
metadata:
  name: postgres
  namespace: $K8_NAMESPACE
stringData:
  postgres-password: "$POSTGRES_PASSWORD"
```

For enabling the session feature and directly setting the Python config, I read [over here](https://github.com/getsentry/self-hosted/issues/2057) and [here](https://github.com/getsentry/self-hosted/blob/master/sentry/sentry.conf.example.py) and [here](https://github.com/sentry-kubernetes/charts/issues/918), ending up with:

```yaml
config:
  sentryConfPy: |
    SENTRY_FEATURES["organizations:session-replay"] = True
    SENTRY_FEATURES["organizations:session-replay-ui"] = True
```

Ingress setup follows a pretty standard procedure:

```yaml
nginx:
  ingress:
    annotations:
      cert-manager.io/cluster-issuer: "letsencrypt-prod"
    enabled: true
    hostname: $SENTRY_HOSTNAME
    ingressClassName: "public"
    tls: true
    extraTls:
      - hosts:
          - $SENTRY_HOSTNAME
        secretName: sentry-tls
```

As in some of my other scripts, I like to assume the cluster issuer is named `"letsencrypt-prod"`. To me, that's a reasonable sacrifice for adding fewer inputs ;).

Also, **to get the install to succeed** I needed to increase the `hooks.activeDeadlineSeconds`; otherwise, it would time out and the installation would fail.

```yaml
hooks:
  activeDeadlineSeconds: 3500
```

## Open Issues

Currently, I still experience one of the metrics services failing. I'm not sure about the impact yet, but tracking seems to work just fine:

```bash
microk8s kubectl logs sentry-snuba-metrics-consumer -n sentry
snuba.clickhouse.errors.ClickhouseWriterError: Method write is not supported by storage Distributed with more than one shard and no sharding key provided (version 21.8.13.6 (official build))
```

seems like an issue with my storage provisioning possibly; no impact seen so-far.


Self Hosted Error Tracking and Reporting using Sentry Kubernetes and Helm

Self Host Sentry using Helm


Since launching [Msgmate.io](https://msgmate.io) at the end of March, we've seen a lot of activity and received a myriad of feedback regarding its services and development.

However, the development of the direct Msgmate feature has been quite restricted. As a result, the updates that are visible to the public have also been very limited. Therefore, I am writing this article to shed some light on what's in store for Msgmate's future.

Please note that this article is somewhat technical in nature!

### Statistics

<div style="
    display: flex;
    flex-wrap: wrap;
    justify-content: space-around;
">
    <div style="
        background-color: #2C333A;
        padding: 20px;
        border-radius: 10px;
        width: 20%;
        text-align: center;
        min-width: 320px;
    ">
        <h2 style="color: #E88388; font-size: 24px;">
            <strong>Registered Users:</strong>
        </h2>
        <p>300</p>
    </div>
    <div style="
        background-color: #2C333A;
        padding: 20px;
        border-radius: 10px;
        width: 20%;
        text-align: center;
        min-width: 320px;
    ">
        <h2 style="color: #E88388; font-size: 24px;">
            <strong>Messages Exchanged:</strong>
        </h2>
        <p>6,331</p>
    </div>
    <div style="
        background-color: #2C333A;
        padding: 20px;
        border-radius: 10px;
        width: 20%;
        text-align: center;
        min-width: 320px;
    ">
        <h2 style="color: #E88388; font-size: 24px;">
            <strong>Tokens Used:</strong>
        </h2>
        <p>579,807+</p>
    </div>
    <div style="
        background-color: #2C333A;
        padding: 20px;
        border-radius: 10px;
        width: 20%;
        text-align: center;
        min-width: 320px;
        margin-bottom: 0 !important;
    ">
        <h2 style="color: #E88388; font-size: 24px;">
            <strong>Images Generated:</strong>
        </h2>
        <p>192</p>
    </div>
</div>

Since the alpha launch in March, we've had 300 users register. A total of 6,331 messages have been exchanged, using over 579,807+ tokens for prompts and completions. Furthermore, a total of 192 images have been generated.

> I've received feedback indicating interest in such a service.
> However, there were also critical comments concerning the current reliability and limited range of features available.

Rest assured, I am taking all feedback seriously and am actively working to bring you a better and enhanced user experience.

<strong style="color: red;">For now we limited msgmate.io alpha registrations untill we scaled our bot service to handle more chats in paralel</strong>


### Development in the Software Stack

My interest in large language models and their immediate applications in everyday life led me to initiate Msgmate as an experiment. Having considerable experience in development operations and infrastructure, I've managed to evolve my stack through several iterations based on the backend + infrastructure definitions I've developed over the years.

**Here's a look at the evolution of the stack:**

|Iteration|Link|Description|
|--|--|--|
|1|[Django Clean Slate](https://github.com/tbscode/django-clean-slate)|First stack iteration which involved Django + Webpack + React.|
|2|[Django NextJS Setup](https://github.com/tbscode/django_nextjssetup)|First experimental combination of Django + NextJS + React.|
|3|[Tiny Django](https://github.com/tbscode/tiny-django)|Evolution into the first version of 'Tim's Stack' initially dubbed 'tiny-django'.|
|4|[AnySearch Stack](https://github.com/tbscode/anysearch-stack)|First Live application using the stack for a submission to the tum.ai hackathon. The stack proved to be instantly viable for rapid development and easy deployment to microk8s Kubernetes clusters.|
|5|[Tim's Stack Anystack](https://github.com/tbscode/tims-stack-anystack)|Clean re-write of the stack, adding automation documentation, and more. During the Bunnyshell Jamstack Hackathon, I had the unique opportunity to|
|6|[Tim's Stack V2](https://github.com/tbscode/tims-stack-v2)|Dawn of a new age ;) Finalization work on the stack has started with the objective of making it out-of-the-box ready, usable state.|

All these updated work towards shaping the backend that msgmate.io uses and making it more robust and scalable! The bigger beta update planned for msgmate will be using the update Tims Stack V2 and there are plans for launching an android and ios app along side the new backend and interface.

### The Stack Won 2. Place!

As mentioned in the Above list [I've participated in a Hackathon using my Stack](https://blog.t1m.me/blog/tims-stack). 

> 🥳 Checkout [the hackathon page](https://devpost.com/software/tim-s-stack-anystack).

My efforts in the Bunnyshell hackathon didn't go to waste, as we managed to secure second place and win `$4000`, which I'll be reinvesting in the development of Msgmate.io!

### Msgmate.io Vision

I envision msgmate.io as the premier platform for innovators and casual users a like - who wish to engage with large language models - automated agents - and seamlessly incorporate them into their applications and everyday life. 
Msgmate should provide user-friendly developer APIs, websockets, and client applications. Fundamentally, interaction with language models and agents should be as straightforward as possible. Yet beneath the surface, the platform should offer abundant interfaces and configuration options.

### Development in the Team

Given my commitments to other projects, the time I can invest in Msgmate development is limited.
So, to advance this service further, I need more development power!

Using the prize money from the hackathon, I've already hired a NextJS + React developer to help me revamp the Msgmate interface. 

**However, I still need more developers motivated to work on LLm tooling and software.**

> I am looking for motivated (self) taught developers or students who have experience working on their own projects.
> You need to be a fast learner. Having knowledge of Docker, NextJS, React, Django would be advantageous but not a requirement.
> If you are interested in collaborating with us, get in touch at tim.timschupp@gmail.com

### Want to Stay Updated? Have Feedback or Issues to Report?

We've set up a simple form over at [`msgmate.io/newsletter`](https://msgmate.io/newsletter/). By signing up, you'll receive the next update straight to your inbox.

Feel free to add a comment to the signup request for any feedback or bugs. You can fill out the newsletter request as often as you like! Happy messaging with Msgmate.io!

A progress and state update on the development of my msgmate.io plattform

Msgmate.io Development Update Oct 2023


Different command I've used over the year to quickly create pdfs from local note files.

## Raw Pandoc

```bash
pandoc -V geometry:margin=1cm <mardown-file> -o <pdf-file>
```

## Pandoc + `wkhtmltopdf`-engine

- html support, e.g.: tables etc
- dynmaic margin

```bash
pandoc --pdf-engine=wkhtmltopdf -V papersize=a4 -V margin-top=0.1 -V margin-left=0.1 -V margin-right=0.1 -V margin-bottom=0.1 --from markdown-markdown_in_html_blocks+raw_html <markdown-file> -o <pdf-file>
```

## Quick TXT file to dense readable pdf

```bash
pandoc --wrap=preserve $1 -V geometry:margin=1cm -V fontsize=8pt -o $1.pdf -f markdown+hard_line_breaks
```

## Github markdown using `grip`

```bash
sudo snap install grip
grip <markdown-file>
```

then visit the browser page and export the page as pdf.
Or `curl -X <page> > rendered.html`.

## HTML to pdf with

```bash
pandoc --pdf-engine=wkhtmltopdf rendered.html -o <pdf-file>
```

## TXT to pdf with `enscript` and `ps2pdf`

```bash
find . -type f -name $1 | while read ONELINE; do enscript "$ONELINE" -p "$(echo "$ONELINE" | sed 's/.txt/.ps/g')"; done
find . -type f -name "$(echo "$1" | sed 's/.txt/.ps/g')" | while read ONELINE; do ps2pdf "$ONELINE" "$(echo "$ONELINE" | sed 's/.ps/.pdf/g')"; done
```

A collection of strategies to quickly produce pdf's from markdown files

PDF's from markdown, pandoc and more


Simulate a webcam even in your deveice doesn't have one!
The video input can even work to simulate video inputs on remote development devices.

You'll need to install and anable `v4l2loopback`.

```bash
sudo apt install v4l2loopback-dkms
sudo modprobe v4l2loopback
```

## Live video from screen

```bash
sudo ffmpeg -f x11grab -r 15 -s 1280x720 -i :1 -vcodec rawvideo -pix_fmt yuv420p -threads 0 -f v4l2 /dev/video2
```

## From Video File


```bash
ffmpeg -re -i <some-video>.mp4 -map 0:v -f v4l2 /dev/video2
```

> Where to go from here? Loop a video of you listen patiently in your zoom meetings? ;)

How to use `ffmpeg` to funnel screen or video data as video input.

Simulate secondary webcam on linux


This guide illustrates the steps required to build, push, and deploy a Next.js app on Kubernetes using docker files for defining development and production Next.js images. These docker compose files allow for the building and pushing of images, with a Next.js config able to export static files. A simple helm chart will create deployment, service, and ingress for exposing our app.

> Template repo containing all the required code and configs can be [found on my github](https://github.com/tbscode/nextjs-helm-template).

### Prerequisites

This tutorial can also be adapted to work with any managed cluster as I've been using similar setups to quickly deploy Next.js apps for a while.

For your cluster you'll need:

- `cert-manger` cluster issuer with name `letsencrypt-prod` setup.
- `nginx-ingress` installed.
- `helm` installed.
- Public IP and Host Url connected.

Further, a private container registry is required.

> If you don't have a Kubernetes cluster ready follow [My Blog Post on Microk8s Private Cluster Setup](/blog/microk8s-on-vps).

### 1. Start a Next.js App

Begin by creating a simple Next.js app:

```
npx create-next-app@latest
What is your project named?  frontend
Would you like to use TypeScript? Yes
Would you like to use ESLint?  Yes
Would you like to use Tailwind CSS?  Yes
Would you like to use `src/` directory?  No
Would you like to use App Router? (recommended)  Yes
Would you like to customize the default import alias (@/*)?  No
```

We named the app `./frontend` so the respective folder is created.

### 2. Dockerize the Application

Two docker files can be created:

1. [`./frontend/Dockerfile`](https://github.com/tbscode/nextjs-helm-template/blob/main/frontend/Dockerfile): A development docker file that mounts frontend `./frontend` into the container for hot-reload.
2. [`./frontend/pro.dockerfile`](https://github.com/tbscode/nextjs-helm-template/blob/main/frontend/prod.dockerfile): A production docker file that statically builds the entire application and Next.js in a container.

The development docker file is straightforward:

```Dockerfile
FROM node:16-alpine

WORKDIR /frontend
COPY ./package.json .

RUN apk add curl

RUN npm i --save-dev

ENTRYPOINT ["npm", "run", "dev"]
```

Here, we execute `npm i` and then `npm run dev`. When mounted to the host, the `./node_modules` directory will also be present on the host.

The production docker file, which is based on the Next.js example docker file, can be found [here.](https://nextjs.org/docs/pages/building-your-application/deploying#docker-image)

```dockerfile
FROM node:16-alpine AS deps
RUN apk add --no-cache libc6-compat curl
WORKDIR /app

COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* ./
RUN \
    if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
    elif [ -f package-lock.json ]; then npm i; \
    elif [ -f pnpm-lock.yaml ]; then yarn global add pnpm && pnpm i --frozen-lockfile; \
    else echo "Lockfile not found." && exit 1; \
    fi

FROM node:16-alpine AS builder
WORKDIR /app
COPY . .
RUN apk add --no-cache curl
RUN npm i
RUN npm install --unsafe-perm -g sharp

ENV NEXT_TELEMETRY_DISABLED 1

RUN yarn build

FROM node:16-alpine AS runner
WORKDIR /app

ENV NODE_ENV production
ENV NEXT_TELEMETRY_DISABLED 1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app/styles ./styles
COPY --from=builder /app/public ./public

COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs

EXPOSE 3000

ENV PORT 3000

CMD ["node", "server.js"]
```

### 3. Compose Files for Development

Creating two compose files can pave the way for convenience:

1. [`docker-compose.yaml`](https://github.com/tbscode/nextjs-helm-template/blob/main/docker-compose.yaml): A development compose file.
2. [`docker-compose.pro.yaml`](https://github.com/tbscode/nextjs-helm-template/blob/main/docker-compose.pro.yaml): A production/ Local Infrastructure compose file.

Development is now always as straightforward as executing `docker-compose up`.

### 4. Creating a Helm Chart

To effectively manage the deployments and services we require, we prefer to create a simple Helm chart that can easily be adapted, installed, and updated. Our Helm chart will need the following components:

1. Frontend Deployment
2. Registry Image Pull Secrets
3. Frontend Service
4. Ingress

```
microk8s helm create ./helm
cd ./helm/template && rm -rf *
```

Everything will be deployed to a `rootNamespace`.

We have reset the template and created a simple setup for now [`./helm/template/frontend.yaml`](https://github.com/tbscode/nextjs-helm-template/blob/main/helm/templates/frontend.yaml).

```yaml
{{- if .Values.registryAuth.use }}
kind: Secret
type: kubernetes.io/dockerconfigjson
apiVersion: v1
metadata:
  name: dockerconfigjson-github-com
  namespace: {{ .Values.rootNamespace }}
stringData:
  .dockerconfigjson: >
    {{
      (
        dict "auths"
        (
          dict {{ .Values.registryAuth.registry }}
          (
            dict "auth" .Values.registryAuth.token
          )
        )
      )
      |
      toJson
    }}
{{- end }}
```

We've created a simple flag `.Values.registryAuth.use` to check if registry authentication should be used (as we wouldn't need it for local microk8s deployment).
We use `.Values.registryAuth.use` to define the registry that should be used.

Now we define a simple deployment for our nextjs backend:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend-container
  namespace: {{ .Values.rootNamespace }}
spec:
  replicas: 1
  selector:
    matchLabels:
      app: frontend-container
  template:
    metadata:
      labels:
        app: frontend-container
    spec:
      containers:
        - name: frontend-container
          image: {{ .Values.frontend.imageURL }}
          ports:
            - containerPort: 8000
          envFrom:
            - secretRef:
                name: frontend-secrets
      {{- if .Values.registryAuth.use }}
      imagePullSecrets:
        - name: dockerconfigjson-github-com
      {{- end }}
```

Note that we inject the `imageURL` and the registry pull secret if required. We go on to expose the port `3000` to the cluster through a simple service.

```yaml
apiVersion: v1
kind: Service
metadata:
  name: frontend-service
  namespace: {{ .Values.rootNamespace }}
  labels:
    app: frontend-container
spec:
  type: ClusterIP
  ports:
    - port: 3000
      targetPort: 3000
  selector:
    app: frontend-container

```

It's also advisable to create a secret that injects some general environment variables listed in `values.yaml:frontend.env.*`

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: frontend-secrets
  namespace: {{ .Values.rootNamespace }}
type: Opaque
data:
{{- range $key, $value := .Values.frontend.env }}
  {{ $key }}: {{ $value | b64enc }}
{{- end }}
```

Finally, we set up an ingress:

```yaml
{{- if .Values.ingress.use }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: frontend-ingress
  namespace: {{ .Values.rootNamespace }}
  annotations:
    nginx.ingress.kubernetes.io/proxy-connect-timeout: '3600'
    nginx.ingress.kubernetes.io/proxy-read-timeout: '3600'
    nginx.ingress.kubernetes.io/proxy-send-timeout: '3600'
    nginx.ingress.kubernetes.io/server-snippets: |
      location /ws/ {
        proxy_http_version 1.1;
        proxy_redirect off;
        proxy_buffering off;
      } 
    {{- if .Values.ingress.certManager }}
    cert-manager.io/cluster-issuer: letsencrypt-prod
    {{- end }}
    kubernetes.io/ingress.class: public
spec:
  {{- if .Values.ingress.certManager }}
  tls:
    - hosts:
      - {{ .Values.ingress.host }}
    secretName: frontend-ingress-tls
  {{- end }}
  rules:
  - host: {{ .Values.ingress.host }}
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: frontend-service
            port:
              number: 3000
{{- end }}
```

We've introduced some flags to set up `tls` if required and some annotations to facilitate websocket forwards.

#### Configuration via `Values.yaml`

Finally, to tie this all together, we create some default [`values.yaml`](https://github.com/tbscode/nextjs-helm-template/blob/main/helm/values.yaml)

```yaml
rootNamespace: "default"
registryAuth:
  registry: "localhost:32000"
  use: false
  token: ""
ingress:
  use: true
  certManager: false
  host: "localhost"
frontend:
  imageURL: "localhost:32000/frontend:registry"
  env:
    HOSTNAME: "localhost"
```

This can be used to control all the configuration options we just created.

### 5. Testing on Local Microk8s

It's always good to ensure that we can also deploy the entire infrastructure locally. For this, we can use a simple local `microk8s` installation.

```
microk8s enable ingress dns registry
docker-compose -f docker-compose.pro.yaml build
docker-compose -f docker-compose.pro.yaml push
```

Since we set up the template `.env` earlier, we're already configured to push and pull the image from `localhost:32000/frontend:registry`.
So now, all we need to do is install our newly created chart:

```
microk8s helm install next-js-frontend ./helm/ --set rootNamespace="nextjsfront"
```

Since we also set up `$ROOT_URL="http://localhost"`, the ingress will be configured to route to `http://localhost`. Now let's check the browser:

![Next page on localhost via microk8s](/static/assets/nextjs-page.jpg)

### 6. Deploy to Private Cluster

For the capability to deploy to a private cluster, you'll need a package registry available.

> If you quickly want a cheap registry, you can [check out my blog post](/blog/microk8s-gittea) on deploying Gitea in a private k8s cluster.

#### Modify Environment for Production

Firstly, we need to modify the environment to push to our private registry with a modified tag.

```bash
rm .env
echo "FRONTEND_IMAGE=\"<your-frontend-image-url>\"" >> .env
echo "ROOT_URL=\"<your-host-url>\"" >> .env
```

An example of an image URL is `my-gitea.example.com/gitea-private-user/package:latest`.
A `ROOT_URL` could be something like `nextjs.example.com`.

Note that you can restore the old default `.env` at any time using `git checkout .env`.

We also need to encode access credentials to our registry so we can inject them into our secret.

```bash
echo "<your-registry-user>:<your-registry-password>" | base64
```

```bash
docker-compose -f docker-compose.pro.yaml build
docker-compose -f docker-compose.pro.yaml push
KUBECONFIG="./kubeconfig.yaml" microk8s helm install next-js-frontend ./helm/ \
  --set rootNamespace="nextjsfront" \
  --set ingress.use=true \
  --set frontend.imageURL="<your-frontend-image-url>" \
  --set certManager.use=true \
  --set registryAuth.use=true \
  --set registryAuth.registry="<your-registry-host>" \
  --set registryAuth.token="<base64-encoded-token>"
```

> It works perfectly! In fact, this blog was deployed using this exact technique. 

### 7. Repository Automations

Now, we have a simple process to deploy the repository using GitHub Actions. 
These actions can also run on your own runners using Gitea Actions.

We create one simple workflow [`.github/workflow/deploy.yaml`](https://github.com/tbscode/nextjs-helm-template/blob/main/.github/workflow/deploy.yaml)

```yaml
name: Deploy NextJs App
    
on:
  workflow_dispatch:

permissions:
  contents: read
  pages: write
  id-token: write

jobs:
  deploy:
    runs-on: ubuntu-latest
    name: Deploy NextJS App
    steps:
      - uses: actions/checkout@master
        with:
          token: ${{ secrets.BOT_PAT }}
          submodules: recursive
          ref: ${{ github.event.pull_request.head.sha }}
      - name: Setup Kubeconfig
        id: setup_kubeconfig
        run: |
          touch .env; rm .env
          echo "${{ secrets.KUBE_CONFIG_BASE64 }}" | base64 -d > kubeconfig.yaml
      - name: Setup Environment
        id: setup_env
        run: |
          touch .env; rm .env; touch .env
          RANDOM_STRING="$( openssl rand -hex 3 )"
          FRONTEND_IMAGE="${{ secrets.CONTAINER_REGISTRY }}/${{ secrets.GITTEA_USER }}/nextjs-app:build-$RANDOM_STRING"
          echo "FRONTEND_IMAGE=\"$FRONTEND_IMAGE\"" >> .env
          echo "frontend_image=$FRONTEND_IMAGE" >> $GITHUB_OUTPUT
      - name: Build And Push Image
        run: |
          echo ${{ secrets.GITTEA_PASSWORD }} | docker login ${{ secrets.CONTAINER_REGISTRY }} -u ${{ secrets.GITTEA_USER }} --password-stdin
          DOCKER_BUILDKIT=1 docker-compose -f docker-compose.pro.yaml build
          DOCKER_BUILDKIT=1 docker-compose -f docker-compose.pro.yaml push
      - uses: azure/setup-helm@v3
        with:
           version: 'latest'
           token: ${{ secrets.BOT_PAT }}
        id: install-helm
      - uses: azure/setup-kubectl@v3
        with:
           version: 'latest'
        id: install-kubectl
      - name: Setup Cluster Connection
        id: cluster_connection
        run: |
          echo "Test"
          REGISTRY_AUTH=$(echo -n "${{ secrets.GITTEA_USER }}:${{ secrets.GITTEA_PASSWORD }}" | base64)
          KUBECONFIG=./kubeconfig.yaml kubectl create namespace nextjsnamespace || true
          cat << EOF | while read eval_command; do yq -i eval "$eval_command" ./helm/values.yaml; done
            .rootNamespace = "nextjsnamespace"
            .ingress.host = "${{ secrets.BOT_PAT }}"
            .frontend.imageURL = "${{ steps.setup_env.outputs.frontend_image }}"
            .ingress.certManager = true
            .registryAuth.token = "$REGISTRY_AUTH"
            .registryAuth.use = true
          EOF
          KUBECONFIG=./kubeconfig.yaml helm upgrade next-js-app ./helm/ --set rootNamespace="nextjsnamespace" --install
```

Using the following secrets configured in your GitHub account:

- `BOT_PAT`: GitHub bot access token
- `GITTEA_USER`: Private registry user
- `GITTEA_PASSWORD`: Private registry password or auth token
- `CONTAINER_REGISTRY`: Container registry host address
- `KUBE_CONFIG_BASE64`: Base64 encoded kubeconfig to connect to the k8s cluster

#### Automation Steps

1. Clone the repo and its sub-repos using `actions/checkout@master`.
2. Setup the `.env` with a random image tag
3. Build and push the production image using the generated tag
4. Setup Helm and kubectl using the Azure actions `azure/setup-helm@v3`, `azure/setup-kubectl@v3`
5. Modify `values.yaml` with our new configuration and update the Helm installation

> Opportunities from here are limitless; you can easily have any pull to the main deploy a feature environment or connect a backend to your Next.js app.

Simple steps to build, push, and deploy a Next.js app on Kubernetes

Latest Posts

Understanding Solid Authentication

Self Host Sentry using Helm

Msgmate.io Development Update Oct 2023

PDF's from markdown, pandoc and more

Simulate secondary webcam on linux

Quickly Deploy Any Next Js App On Kubernetes