Tribunal: testing LLMs in Elixir

News includes a groundbreaking effort to port Erlang to QNX for automotive systems, ExDoc 0.40.0 adding LLM-friendly features, Permit.Phoenix v0.4.0 improving authorization workflows, Oban Pro launching for Python with Elixir interop, new LLM evaluation tools Tribunal and Beamlens that bring AI-powered testing and self-diagnosing supervision trees to the BEAM, Tidewave crossing 100k ARR just 5 months after launch, Guillaume Duboc defending his PhD thesis on typing Elixir with set-theoretic types, and more!

Show Notes online - http://podcast.thinkingelixir.com/289

Elixir Community News

• https://paraxial.io/ – Paraxial.io is sponsoring today's show! Sign up for a free trial of Paraxial.io today and mention Thinking Elixir when you schedule a demo for a special offer.
• https://bsky.app/profile/theerlef.bsky.social/post/3mcd4nkapps2r – BlueSky announcement of new EEF blog post about porting Erlang to QNX
• https://erlef.org/blog/eef/otp-qnx-port – EEF blog post: 'Porting Erlang to QNX: a small nerd-snipe that actually worked'
• https://www.blackberry.com/en – BlackBerry QNX, a real-time operating system widely used in automotive and embedded applications
• https://github.com/erlang/otp/commit/d7e0b82835312a0458e276f4663bb432574a26d6.patch – The patch that successfully enabled Erlang to run on QNX after three days of focused work
• https://x.com/josevalim/status/2013649733579870534 – José Valim announces ExDoc 0.40.0 with LLM-friendly features
• https://hexdocs.pm/ex_doc/changelog.html#v0-40-0-2026-01-20 – ExDoc 0.40.0 changelog including markdown section copying and llms.txt support
• https://x.com/curiosum_dev/status/2013141946030522792 – Curiosum announces Permit.Phoenix v0.4.0 release
• https://permit.curiosum.com/ – Permit website with cheatsheet and articles about using the authorization library
• https://github.com/curiosum-dev/permit_phoenix/releases/tag/v0.4.0 – Permit.Phoenix v0.4.0 changelog with Phoenix Scopes support and LiveView improvements
• https://github.com/curiosum-dev/permit_phoenix – Permit.Phoenix GitHub project for Phoenix, Plug and LiveView authorization integrations
• https://oban.pro/articles/introducing-oban-python – Announcement of Oban Pro for Python with full Elixir/Python compatibility and interop
• https://x.com/georgeguimaraes/status/2011768435504791677 – George Guimarães announces Tribunal, an LLM evaluation framework for Elixir
• https://github.com/georgeguimaraes/tribunal – Tribunal GitHub project for testing RAG pipelines with assertions like assert_faithful and refute_hallucination
• https://github.com/georgeguimaraes/tribunal-juror – Tribunal Juror, a Phoenix LiveView UI for the Tribunal evaluation library
• https://github.com/beamlens/beamlens – Beamlens GitHub project providing adaptive runtime intelligence for the BEAM
• https://www.reddit.com/r/elixir/comments/1qd3oya/i_built_a_tool_that_lets_your_supervision_tree/ – Reddit discussion about Beamlens enabling supervision trees to self-diagnose incidents using LLMs
• https://screen.studio/share/w1qXNbUc – Beamlens demo video introducing the library
• https://github.com/ocean/ecto_libsql – ecto_libsql GitHub project, an Ecto adapter for libSQL/Turso databases
• https://github.com/ocean/ecto_libsql/releases – ecto_libsql releases page showing v0.8.7
• https://turso.tech/ – Turso, a lightweight SQLite-compatible database that scales to millions of instances
• https://github.com/tursodatabase/turso – Turso Database GitHub project, an in-process SQL database written in Rust
• https://github.com/Mati365/ckeditor5-phoenix – CKEditor 5 Phoenix integration library providing WYSIWYG editing with real-time backend sync
• https://x.com/josevalim/status/2011165398172078309 – José announces Tidewave's full integration with Supabase
• https://x.com/josevalim/status/2011485752484360442 – José announces Tidewave integration with TanStack Start
• https://x.com/josevalim/status/2012171326899843286 – Tidewave crosses 100k ARR milestone after 5 months since launch
• https://x.com/josevalim/status/2013327035439911031 – José shares update about Guillaume Duboc's PhD thesis defense
• https://x.com/duboc_guillaume/status/2013206550005870689 – Guillaume Duboc announces thesis defense on "Typing Dynamic Languages with Set-Theoretic Types - The Case of Elixir"
• https://gldubc.github.io/ – Guillaume Duboc's website with information on his PhD research on typing Elixir

Do you have some Elixir news to share? Tell us at @ThinkingElixir or email at show@thinkingelixir.com

Find us online

• Message the show - Bluesky
• Message the show - X
• Message the show on Fediverse - @ThinkingElixir@genserver.social
• Email the show - show@thinkingelixir.com
• Mark Ericksen on X - @brainlid
• Mark Ericksen on Bluesky - @brainlid.bsky.social
• Mark Ericksen on Fediverse - @brainlid@genserver.social
• David Bernheisel on Bluesky - @david.bernheisel.com
• David Bernheisel on Fediverse - @dbern@genserver.social

Sponsored By:

• Paraxial.io: Paraxial.io is sponsoring today's show! Sign up for a free trial of Paraxial.io today and mention Thinking Elixir when you schedule a demo for a special offer.