We're excited to introduce Braintrust, a platform for running and tracking AI evaluations (\u201cevals\u201d) [1].
At my previous startup Impira and leading AI at Figma, we had this recurring problem where we never knew if changes we made to our products would improve or regress key user scenarios. We built some tooling to solve this problem and after talking to other developers learned that it was a widespread issue. Specifically, it\u2019s challenging to establish a great dev loop that lets you systematically improve and ship high quality AI products.
We worked with the teams at Zapier, Coda, and Replit to refine Braintrust. We consistently heard that they were facing challenges with evaluation, so we built Braintrust to help them. Today we\u2019re releasing the product for anyone to use \u2014 including a free plan [3].
There are a lot of LLM tooling products on the market. Here are a few ways Braintrust is different:
- Rather than showing eval metrics in your observability tools, Braintrust offers an \u201cexperiment tracking\u201d workflow, meaning you can try out changes while developing them, and drill down into diffs between other experiments and git branches before you ship. Check out our docs [2] for more details.\n- We believe strongly that you should own your data and support on-premises and private VPC deployments.\n- We natively and equally support Typescript and Python.\n- We have a flexible free plan for builders and an unlimited free plan for academic and non-commercial open source projects [3]. We will introduce a self-service paid (\u201dPro\u201d) tier, hopefully with feedback from this community.
Our mission is to enable developers to build high quality, reliable AI products. We couldn\u2019t be doing this without Elad Gil, who helped me incubate the initial idea and team, which today includes founding designer, Coleen Baik, and founding engineer, Manu Goyal. Also big thanks to David Song, from Elad\u2019s team, who is also helping us.
We\u2019re excited to launch today [4], but we know there\u2019s a lot left to build and are excited to hear your feedback.
[1] https://www.braintrustdata.com\n[2] http://www.braintrustdata.com/docs/guides/evals\n[3] https://www.braintrustdata.com/pricing\n[4] https://www.braintrustdata.com/blog/reliable-ai"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Show HN: Braintrust \u2013 Eval platform for AI products"}},"_tags":["story","author_ankrgyl","story_37692239","show_hn"],"author":"ankrgyl","children":[37692553,37755113],"created_at":"2023-09-28T16:46:55Z","created_at_i":1695919615,"num_comments":2,"objectID":"37692239","points":8,"story_id":37692239,"story_text":"Hey HN,
We're excited to introduce Braintrust, a platform for running and tracking AI evaluations (\u201cevals\u201d) [1].
At my previous startup Impira and leading AI at Figma, we had this recurring problem where we never knew if changes we made to our products would improve or regress key user scenarios. We built some tooling to solve this problem and after talking to other developers learned that it was a widespread issue. Specifically, it\u2019s challenging to establish a great dev loop that lets you systematically improve and ship high quality AI products.
We worked with the teams at Zapier, Coda, and Replit to refine Braintrust. We consistently heard that they were facing challenges with evaluation, so we built Braintrust to help them. Today we\u2019re releasing the product for anyone to use \u2014 including a free plan [3].
There are a lot of LLM tooling products on the market. Here are a few ways Braintrust is different:
- Rather than showing eval metrics in your observability tools, Braintrust offers an \u201cexperiment tracking\u201d workflow, meaning you can try out changes while developing them, and drill down into diffs between other experiments and git branches before you ship. Check out our docs [2] for more details.\n- We believe strongly that you should own your data and support on-premises and private VPC deployments.\n- We natively and equally support Typescript and Python.\n- We have a flexible free plan for builders and an unlimited free plan for academic and non-commercial open source projects [3]. We will introduce a self-service paid (\u201dPro\u201d) tier, hopefully with feedback from this community.
Our mission is to enable developers to build high quality, reliable AI products. We couldn\u2019t be doing this without Elad Gil, who helped me incubate the initial idea and team, which today includes founding designer, Coleen Baik, and founding engineer, Manu Goyal. Also big thanks to David Song, from Elad\u2019s team, who is also helping us.
We\u2019re excited to launch today [4], but we know there\u2019s a lot left to build and are excited to hear your feedback.
[1] https://www.braintrustdata.com\n[2] http://www.braintrustdata.com/docs/guides/evals\n[3] https://www.braintrustdata.com/pricing\n[4] https://www.braintrustdata.com/blog/reliable-ai","title":"Show HN: Braintrust \u2013 Eval platform for AI products","updated_at":"2024-09-20T15:17:50Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Tawheed"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"I've been working very hard on a better landing page for my startup Braintrust.io.
Please give me your feedback."},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Ask HN: Feedback on Braintrust.io's new homepage"},"url":{"matchLevel":"none","matchedWords":[],"value":""}},"_tags":["story","author_Tawheed","story_1284052","ask_hn"],"author":"Tawheed","children":[1284053,1284065,1284072,1284365,1284755,1285761,1285835],"created_at":"2010-04-22T03:37:43Z","created_at_i":1271907463,"num_comments":8,"objectID":"1284052","points":5,"story_id":1284052,"story_text":"I've been working very hard on a better landing page for my startup Braintrust.io.
Please give me your feedback.","title":"Ask HN: Feedback on Braintrust.io's new homepage","updated_at":"2024-09-19T17:02:36Z","url":""},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"bahulneel"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Agent Brain Trust lets you summon a panel of real, named experts to critique your architecture, review your writing, pressure your product strategy, or debate your design patterns. 10 built-in trusts, an extensible roster, and a working turn-taking protocol that ensures nothing useful gets skipped. Guest experts are drafted via an MCP server that maps topics to real persona cards so the panel can reach into niche and novel territory without inventing expertise it does not have.
Wrote up the full thinking here: https://tinyurl.com/agent-brain-trust"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Show HN: Agent Brain Trust, customisable expert panels for AI agents"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://github.com/bahulneel/agent-brain-trust"}},"_tags":["story","author_bahulneel","story_47855932","show_hn"],"author":"bahulneel","created_at":"2026-04-21T23:03:25Z","created_at_i":1776812605,"num_comments":0,"objectID":"47855932","points":5,"story_id":47855932,"story_text":"Agent Brain Trust lets you summon a panel of real, named experts to critique your architecture, review your writing, pressure your product strategy, or debate your design patterns. 10 built-in trusts, an extensible roster, and a working turn-taking protocol that ensures nothing useful gets skipped. Guest experts are drafted via an MCP server that maps topics to real persona cards so the panel can reach into niche and novel territory without inventing expertise it does not have.
Wrote up the full thinking here: https://tinyurl.com/agent-brain-trust","title":"Show HN: Agent Brain Trust, customisable expert panels for AI agents","updated_at":"2026-04-22T11:45:06Z","url":"https://github.com/bahulneel/agent-brain-trust"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"hol"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"The secretive brain trust of Silicon Valley insiders who are helping Trump"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.washingtonpost.com/news/the-switch/wp/2016/11/21/the-secretive-brain-trust-of-silicon-valley-insiders-who-are-helping-trump/?tid=sm_tw"}},"_tags":["story","author_hol","story_13011214"],"author":"hol","created_at":"2016-11-22T02:45:52Z","created_at_i":1479782752,"num_comments":0,"objectID":"13011214","points":5,"story_id":13011214,"title":"The secretive brain trust of Silicon Valley insiders who are helping Trump","updated_at":"2024-09-19T23:58:11Z","url":"https://www.washingtonpost.com/news/the-switch/wp/2016/11/21/the-secretive-brain-trust-of-silicon-valley-insiders-who-are-helping-trump/?tid=sm_tw"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"zachdotai"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"AI evaluation startup Braintrust confirms breach"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://techcrunch.com/2026/05/06/ai-evaluation-startup-braintrust-confirms-breach-tells-every-customer-to-rotate-sensitive-keys/"}},"_tags":["story","author_zachdotai","story_48042494"],"author":"zachdotai","children":[48042843],"created_at":"2026-05-06T22:03:39Z","created_at_i":1778105019,"num_comments":1,"objectID":"48042494","points":4,"story_id":48042494,"title":"AI evaluation startup Braintrust confirms breach","updated_at":"2026-05-06T23:56:12Z","url":"https://techcrunch.com/2026/05/06/ai-evaluation-startup-braintrust-confirms-breach-tells-every-customer-to-rotate-sensitive-keys/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Bender"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Digital ID brain trust will meet behind closed doors"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.theregister.com/public-sector/2026/06/25/digital-id-brain-trust-will-meet-behind-closed-doors-as-minister-ducks-cost-questions/5261676"}},"_tags":["story","author_Bender","story_48675777"],"author":"Bender","created_at":"2026-06-25T16:26:49Z","created_at_i":1782404809,"num_comments":0,"objectID":"48675777","points":3,"story_id":48675777,"title":"Digital ID brain trust will meet behind closed doors","updated_at":"2026-06-25T18:06:38Z","url":"https://www.theregister.com/public-sector/2026/06/25/digital-id-brain-trust-will-meet-behind-closed-doors-as-minister-ducks-cost-questions/5261676"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Bender"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"UK digital ID gets brain trust to 'challenge' ministers on policy"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.theregister.com/public-sector/2026/06/12/uk-digital-id-gets-brain-trust-to-challenge-ministers-on-policy/5254650"}},"_tags":["story","author_Bender","story_48523036"],"author":"Bender","created_at":"2026-06-14T00:41:38Z","created_at_i":1781397698,"num_comments":0,"objectID":"48523036","points":3,"story_id":48523036,"title":"UK digital ID gets brain trust to 'challenge' ministers on policy","updated_at":"2026-06-14T01:34:24Z","url":"https://www.theregister.com/public-sector/2026/06/12/uk-digital-id-gets-brain-trust-to-challenge-ministers-on-policy/5254650"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"kenha"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Scale AI Eval with BrainTrust"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.braintrustdata.com/blog/reliable-ai"}},"_tags":["story","author_kenha","story_37692785"],"author":"kenha","created_at":"2023-09-28T17:23:49Z","created_at_i":1695921829,"num_comments":0,"objectID":"37692785","points":3,"story_id":37692785,"title":"Scale AI Eval with BrainTrust","updated_at":"2024-09-20T15:17:50Z","url":"https://www.braintrustdata.com/blog/reliable-ai"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Tawheed"},"story_text":{"matchLevel":"none","matchedWords":[],"value":""},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Ask HN - Feedback on homepage : Braintrust - organize your team's conversations"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://BraintrustHQ.com/"}},"_tags":["story","author_Tawheed","story_1313064"],"author":"Tawheed","children":[1313656],"created_at":"2010-05-02T20:04:19Z","created_at_i":1272830659,"num_comments":2,"objectID":"1313064","points":2,"story_id":1313064,"story_text":"","title":"Ask HN - Feedback on homepage : Braintrust - organize your team's conversations","updated_at":"2023-09-06T19:27:22Z","url":"http://BraintrustHQ.com/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tldl"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Launching AI Products with Braintrust's CEO Ankur Goyal"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://open.spotify.com/episode/4nBsFVLJoLWYlzHSGz2OU0"}},"_tags":["story","author_tldl","story_41789735"],"author":"tldl","children":[41789736],"created_at":"2024-10-09T16:28:16Z","created_at_i":1728491296,"num_comments":1,"objectID":"41789735","points":2,"story_id":41789735,"title":"Launching AI Products with Braintrust's CEO Ankur Goyal","updated_at":"2024-10-10T17:40:17Z","url":"https://open.spotify.com/episode/4nBsFVLJoLWYlzHSGz2OU0"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"spectruman"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Magic Leap Draws from the Bay Area to Assemble Its Brain Trust"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://spectrum.ieee.org/view-from-the-valley/consumer-electronics/audiovideo/magic-leap-draws-from-the-bay-area-to-assemble-its-brain-trust"}},"_tags":["story","author_spectruman","story_8575061"],"author":"spectruman","created_at":"2014-11-07T22:37:43Z","created_at_i":1415399863,"num_comments":0,"objectID":"8575061","points":2,"story_id":8575061,"title":"Magic Leap Draws from the Bay Area to Assemble Its Brain Trust","updated_at":"2024-09-19T21:20:08Z","url":"http://spectrum.ieee.org/view-from-the-valley/consumer-electronics/audiovideo/magic-leap-draws-from-the-bay-area-to-assemble-its-brain-trust"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jcabala"},"story_text":{"matchLevel":"none","matchedWords":[],"value":""},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Inside the Pixar Braintrust"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://m.fastcompany.com/3027135/inside-the-pixar-braintrust"}},"_tags":["story","author_jcabala","story_7396884"],"author":"jcabala","created_at":"2014-03-14T03:52:07Z","created_at_i":1394769127,"num_comments":0,"objectID":"7396884","points":2,"story_id":7396884,"story_text":"","title":"Inside the Pixar Braintrust","updated_at":"2023-09-06T22:24:39Z","url":"http://m.fastcompany.com/3027135/inside-the-pixar-braintrust"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"hepha1979"},"story_text":{"matchLevel":"none","matchedWords":[],"value":""},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Why You Need A Brain Trust "},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://www.hongkiat.com/blog/building-brain-trust/"}},"_tags":["story","author_hepha1979","story_6798010"],"author":"hepha1979","created_at":"2013-11-25T23:54:46Z","created_at_i":1385423686,"num_comments":0,"objectID":"6798010","points":2,"story_id":6798010,"story_text":"","title":"Why You Need A Brain Trust ","updated_at":"2023-09-06T22:45:33Z","url":"http://www.hongkiat.com/blog/building-brain-trust/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"handfuloflight"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Braintrust: LLM Eval as a Service"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.braintrust.dev"}},"_tags":["story","author_handfuloflight","story_44058161"],"author":"handfuloflight","created_at":"2025-05-22T02:21:28Z","created_at_i":1747880488,"num_comments":0,"objectID":"44058161","points":2,"story_id":44058161,"title":"Braintrust: LLM Eval as a Service","updated_at":"2025-05-22T06:51:42Z","url":"https://www.braintrust.dev"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"peytoncasper"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Evaluating Webpage Fact Extraction with Braintrust"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://coffeeblack.ai/blog/evaluating-webpage-fact-extraction-with-braintrust-part1"}},"_tags":["story","author_peytoncasper","story_41918546"],"author":"peytoncasper","created_at":"2024-10-22T20:48:35Z","created_at_i":1729630115,"num_comments":0,"objectID":"41918546","points":2,"story_id":41918546,"title":"Evaluating Webpage Fact Extraction with Braintrust","updated_at":"2024-10-22T21:10:46Z","url":"https://coffeeblack.ai/blog/evaluating-webpage-fact-extraction-with-braintrust-part1"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"luthfur"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"The Creative Meeting: Applying Lessons from Pixar Brain Trust"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://makingsmallercircles.com/articles/the-creative-meeting-applying-lessons-from-pixar-brain-trust-to-improve-how-we-solve-problem/"}},"_tags":["story","author_luthfur","story_39596062"],"author":"luthfur","created_at":"2024-03-04T21:02:53Z","created_at_i":1709586173,"num_comments":0,"objectID":"39596062","points":2,"story_id":39596062,"title":"The Creative Meeting: Applying Lessons from Pixar Brain Trust","updated_at":"2024-09-20T16:35:09Z","url":"https://makingsmallercircles.com/articles/the-creative-meeting-applying-lessons-from-pixar-brain-trust-to-improve-how-we-solve-problem/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"omarfarooq"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Braintrust AI"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.braintrustdata.com/"}},"_tags":["story","author_omarfarooq","story_38043131"],"author":"omarfarooq","created_at":"2023-10-27T19:54:47Z","created_at_i":1698436487,"num_comments":0,"objectID":"38043131","points":2,"story_id":38043131,"title":"Braintrust AI","updated_at":"2024-09-20T15:27:52Z","url":"https://www.braintrustdata.com/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"gennarro"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Q&A with Mart Noorma, the new director of NATO\u2019s cyber brain trust"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.c4isrnet.com/cyber/2022/11/11/qa-with-mart-noorma-the-new-director-of-natos-cyber-brain-trust/"}},"_tags":["story","author_gennarro","story_33586782"],"author":"gennarro","created_at":"2022-11-13T20:06:06Z","created_at_i":1668369966,"num_comments":0,"objectID":"33586782","points":2,"story_id":33586782,"title":"Q&A with Mart Noorma, the new director of NATO\u2019s cyber brain trust","updated_at":"2024-09-20T12:32:11Z","url":"https://www.c4isrnet.com/cyber/2022/11/11/qa-with-mart-noorma-the-new-director-of-natos-cyber-brain-trust/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"hhm"},"story_text":{"matchLevel":"none","matchedWords":[],"value":""},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"The BlackBerry Brain Trust (old but interesting)"},"url":{"matchLevel":"none","matchedWords":[],"value":"http://www.wired.com/wired/archive/13.01/perimeter.html"}},"_tags":["story","author_hhm","story_293448"],"author":"hhm","created_at":"2008-09-03T02:06:51Z","created_at_i":1220407611,"num_comments":0,"objectID":"293448","points":2,"story_id":293448,"story_text":"","title":"The BlackBerry Brain Trust (old but interesting)","updated_at":"2024-09-19T16:26:07Z","url":"http://www.wired.com/wired/archive/13.01/perimeter.html"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Tawheed"},"story_text":{"matchLevel":"none","matchedWords":[],"value":""},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"1-click demo login for Braintrust"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://braintrust.io/demo"}},"_tags":["story","author_Tawheed","story_1352607"],"author":"Tawheed","created_at":"2010-05-16T19:26:11Z","created_at_i":1274037971,"num_comments":0,"objectID":"1352607","points":2,"story_id":1352607,"story_text":"","title":"1-click demo login for Braintrust","updated_at":"2023-09-06T19:30:28Z","url":"http://braintrust.io/demo"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"127001brewer"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"The secretive brain trust of Silicon Valley insiders who are helping Trump"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.washingtonpost.com/news/the-switch/wp/2016/11/21/the-secretive-brain-trust-of-silicon-valley-insiders-who-are-helping-trump/"}},"_tags":["story","author_127001brewer","story_13014031"],"author":"127001brewer","created_at":"2016-11-22T13:43:38Z","created_at_i":1479822218,"num_comments":0,"objectID":"13014031","points":2,"story_id":13014031,"title":"The secretive brain trust of Silicon Valley insiders who are helping Trump","updated_at":"2024-09-19T23:58:31Z","url":"https://www.washingtonpost.com/news/the-switch/wp/2016/11/21/the-secretive-brain-trust-of-silicon-valley-insiders-who-are-helping-trump/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Tawheed"},"story_text":{"matchLevel":"none","matchedWords":[],"value":""},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Rate my startup: BrainTrust.io - a private communication tool for groups"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://braintrust.io"}},"_tags":["story","author_Tawheed","story_1213803"],"author":"Tawheed","children":[1213900,1214042,1215141],"created_at":"2010-03-23T19:22:04Z","created_at_i":1269372124,"num_comments":4,"objectID":"1213803","points":1,"story_id":1213803,"story_text":"","title":"Rate my startup: BrainTrust.io - a private communication tool for groups","updated_at":"2024-09-19T16:55:40Z","url":"http://braintrust.io"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ValentineC"},"story_text":{"matchLevel":"none","matchedWords":[],"value":""},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Inside the Pixar braintrust"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://www.fastcompany.com/3027135/inside-the-pixar-braintrust"}},"_tags":["story","author_ValentineC","story_7392476"],"author":"ValentineC","created_at":"2014-03-13T15:35:52Z","created_at_i":1394724952,"num_comments":0,"objectID":"7392476","points":1,"story_id":7392476,"story_text":"","title":"Inside the Pixar braintrust","updated_at":"2024-09-19T20:39:42Z","url":"http://www.fastcompany.com/3027135/inside-the-pixar-braintrust"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"julien"},"story_text":{"matchLevel":"none","matchedWords":[],"value":""},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Dear NYC Journo Brain Trust"},"url":{"matchLevel":"none","matchedWords":[],"value":"http://threads2.scripting.com/2013/february/dearNycJournoBrainTrust"}},"_tags":["story","author_julien","story_5233025"],"author":"julien","created_at":"2013-02-16T21:04:26Z","created_at_i":1361048666,"num_comments":0,"objectID":"5233025","points":1,"story_id":5233025,"story_text":"","title":"Dear NYC Journo Brain Trust","updated_at":"2023-09-06T20:50:44Z","url":"http://threads2.scripting.com/2013/february/dearNycJournoBrainTrust"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"JoelMarsh"},"story_text":{"matchLevel":"none","matchedWords":[],"value":""},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"UX Brainstorming: The Brain Trust Method"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://thehipperelement.com/articles/the-brain-trust-method-of-ux-brainstorming/"}},"_tags":["story","author_JoelMarsh","story_4899562"],"author":"JoelMarsh","created_at":"2012-12-10T15:57:58Z","created_at_i":1355155078,"num_comments":0,"objectID":"4899562","points":1,"story_id":4899562,"story_text":"","title":"UX Brainstorming: The Brain Trust Method","updated_at":"2024-09-19T19:11:47Z","url":"http://thehipperelement.com/articles/the-brain-trust-method-of-ux-brainstorming/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"benocodes"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Custom LLM as a Judge to Detect Hallucinations with Braintrust"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://cookbook.openai.com/examples/custom-llm-as-a-judge"}},"_tags":["story","author_benocodes","story_41993105"],"author":"benocodes","created_at":"2024-10-30T09:10:27Z","created_at_i":1730279427,"num_comments":0,"objectID":"41993105","points":1,"story_id":41993105,"title":"Custom LLM as a Judge to Detect Hallucinations with Braintrust","updated_at":"2024-10-30T09:13:02Z","url":"https://cookbook.openai.com/examples/custom-llm-as-a-judge"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mooreds"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Applying lessons from Pixar Brain Trust to improve how we solve problems"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://makingsmallercircles.com/articles/the-creative-meeting-applying-lessons-from-pixar-brain-trust-to-improve-how-we-solve-problem/"}},"_tags":["story","author_mooreds","story_39603843"],"author":"mooreds","created_at":"2024-03-05T14:26:39Z","created_at_i":1709648799,"num_comments":0,"objectID":"39603843","points":1,"story_id":39603843,"title":"Applying lessons from Pixar Brain Trust to improve how we solve problems","updated_at":"2024-09-20T16:32:41Z","url":"https://makingsmallercircles.com/articles/the-creative-meeting-applying-lessons-from-pixar-brain-trust-to-improve-how-we-solve-problem/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ankrgyl"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Braintrust raises $5M from Greylock to solve evals"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.braintrustdata.com/blog/seed-round"}},"_tags":["story","author_ankrgyl","story_38631649"],"author":"ankrgyl","created_at":"2023-12-13T18:16:05Z","created_at_i":1702491365,"num_comments":0,"objectID":"38631649","points":1,"story_id":38631649,"title":"Braintrust raises $5M from Greylock to solve evals","updated_at":"2024-09-20T15:50:51Z","url":"https://www.braintrustdata.com/blog/seed-round"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"fzliu"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Sembly: Collaborative Braintrust"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://sembly.com/"}},"_tags":["story","author_fzliu","story_38500733"],"author":"fzliu","created_at":"2023-12-02T18:28:36Z","created_at_i":1701541716,"num_comments":0,"objectID":"38500733","points":1,"story_id":38500733,"title":"Sembly: Collaborative Braintrust","updated_at":"2024-09-20T15:44:38Z","url":"https://sembly.com/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jwilliams"},"story_text":{"matchLevel":"none","matchedWords":[],"value":""},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Brain Trust: 100 Ivy League Business & Entrepreneurship Courses You Can Take for Free"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://www.bschool.com/blog/2008/brain-trust-100-ivy-league-business-entrepreneurship-courses-you-can-take-for-free/"}},"_tags":["story","author_jwilliams","story_342168"],"author":"jwilliams","created_at":"2008-10-24T07:32:49Z","created_at_i":1224833569,"num_comments":0,"objectID":"342168","points":1,"story_id":342168,"story_text":"","title":"Brain Trust: 100 Ivy League Business & Entrepreneurship Courses You Can Take for Free","updated_at":"2023-09-06T19:26:19Z","url":"http://www.bschool.com/blog/2008/brain-trust-100-ivy-league-business-entrepreneurship-courses-you-can-take-for-free/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"productivetom"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"What Is Btrst? Braintrust Explained"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.squadroneer.com/braintrust/"}},"_tags":["story","author_productivetom","story_29990552"],"author":"productivetom","created_at":"2022-01-19T06:11:09Z","created_at_i":1642572669,"num_comments":0,"objectID":"29990552","points":1,"story_id":29990552,"title":"What Is Btrst? Braintrust Explained","updated_at":"2024-09-20T10:19:24Z","url":"https://www.squadroneer.com/braintrust/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ozten"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Braintrust (BTRST) is now available on Coinbase"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://blog.coinbase.com/braintrust-btrst-is-now-available-on-coinbase-5c64a363c9f0"}},"_tags":["story","author_ozten","story_28547112"],"author":"ozten","created_at":"2021-09-16T02:09:48Z","created_at_i":1631758188,"num_comments":0,"objectID":"28547112","points":1,"story_id":28547112,"title":"Braintrust (BTRST) is now available on Coinbase","updated_at":"2024-09-20T09:26:41Z","url":"https://blog.coinbase.com/braintrust-btrst-is-now-available-on-coinbase-5c64a363c9f0"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Semirhage"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Here\u2019s the Brain Trust of Advisers Steering Mark Zuckerberg Through This Crisis"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.fastcompany.com/40557164/heres-the-brain-trust-of-advisers-steering-mark-zuckerberg-through-this-crisis"}},"_tags":["story","author_Semirhage","story_16806369"],"author":"Semirhage","created_at":"2018-04-10T22:54:17Z","created_at_i":1523400857,"num_comments":0,"objectID":"16806369","points":1,"story_id":16806369,"title":"Here\u2019s the Brain Trust of Advisers Steering Mark Zuckerberg Through This Crisis","updated_at":"2024-09-20T02:16:36Z","url":"https://www.fastcompany.com/40557164/heres-the-brain-trust-of-advisers-steering-mark-zuckerberg-through-this-crisis"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"rbanffy"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"The Co-Inventor of BlackBerry Is Building Canada\u2019s Quantum Brain Trust"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"https://www.bloomberg.com/news/articles/2018-02-09/the-co-inventor-of-blackberry-is-building-canada-s-quantum-brain-trust"}},"_tags":["story","author_rbanffy","story_16359464"],"author":"rbanffy","created_at":"2018-02-12T16:00:08Z","created_at_i":1518451208,"num_comments":0,"objectID":"16359464","points":1,"story_id":16359464,"title":"The Co-Inventor of BlackBerry Is Building Canada\u2019s Quantum Brain Trust","updated_at":"2024-09-20T02:04:45Z","url":"https://www.bloomberg.com/news/articles/2018-02-09/the-co-inventor-of-blackberry-is-building-canada-s-quantum-brain-trust"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"rmason"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Big Blue Machine: Inside the brain trust that created the unstoppable LA Dodgers"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.usatoday.com/story/sports/mlb/columnist/bob-nightengale/2017/08/14/los-angeles-dodgers-best-season-ever/564337001/"}},"_tags":["story","author_rmason","story_15030131"],"author":"rmason","created_at":"2017-08-16T18:57:19Z","created_at_i":1502909839,"num_comments":0,"objectID":"15030131","points":1,"story_id":15030131,"title":"Big Blue Machine: Inside the brain trust that created the unstoppable LA Dodgers","updated_at":"2024-09-20T01:12:47Z","url":"https://www.usatoday.com/story/sports/mlb/columnist/bob-nightengale/2017/08/14/los-angeles-dodgers-best-season-ever/564337001/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Tawheed"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Braintrust - The TCDisrupt Braintrust"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://braintrust.io/braintrusts/217/shares"}},"_tags":["story","author_Tawheed","story_1377882"],"author":"Tawheed","created_at":"2010-05-25T16:48:09Z","created_at_i":1274806089,"num_comments":0,"objectID":"1377882","points":1,"story_id":1377882,"title":"Braintrust - The TCDisrupt Braintrust","updated_at":"2024-09-19T17:03:51Z","url":"http://braintrust.io/braintrusts/217/shares"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"dwaxe"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Brain Trust: How AI Is Helping Surgeons Improve Tumor Diagnosis"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://blogs.nvidia.com/blog/2017/02/17/ai-helps-improve-brain-tumor-diagnosis/"}},"_tags":["story","author_dwaxe","story_13670091"],"author":"dwaxe","created_at":"2017-02-17T17:37:14Z","created_at_i":1487353034,"num_comments":0,"objectID":"13670091","points":1,"story_id":13670091,"title":"Brain Trust: How AI Is Helping Surgeons Improve Tumor Diagnosis","updated_at":"2024-09-20T00:27:09Z","url":"https://blogs.nvidia.com/blog/2017/02/17/ai-helps-improve-brain-tumor-diagnosis/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Tawheed"},"story_text":{"matchLevel":"none","matchedWords":[],"value":""},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"BrainTrust.io - Create a Private Community for your audience"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://braintrust.io/about/private_community/"}},"_tags":["story","author_Tawheed","story_1229681"],"author":"Tawheed","created_at":"2010-03-30T17:34:45Z","created_at_i":1269970485,"num_comments":0,"objectID":"1229681","points":1,"story_id":1229681,"story_text":"","title":"BrainTrust.io - Create a Private Community for your audience","updated_at":"2024-09-19T16:57:10Z","url":"http://braintrust.io/about/private_community/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"MaysonL"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Apple's Brain Trust \u2013 Iovine, Reznor, Cue and Kondrk \u2013 Talk Apple Music"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://www.billboard.com/biz/articles/news/digital-and-mobile/7407889/apples-brain-trust-iovine-reznor-cue-and-kondrk-on"}},"_tags":["story","author_MaysonL","story_11904697"],"author":"MaysonL","created_at":"2016-06-14T19:35:20Z","created_at_i":1465932920,"num_comments":0,"objectID":"11904697","points":1,"story_id":11904697,"title":"Apple's Brain Trust \u2013 Iovine, Reznor, Cue and Kondrk \u2013 Talk Apple Music","updated_at":"2024-09-19T23:17:08Z","url":"http://www.billboard.com/biz/articles/news/digital-and-mobile/7407889/apples-brain-trust-iovine-reznor-cue-and-kondrk-on"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Tawheed"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Why Should You Have a Business BrainTrust?"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"http://www.47hats.com/2009/12/why-should-you-have-a-business-braintrust/"}},"_tags":["story","author_Tawheed","story_1003311"],"author":"Tawheed","created_at":"2009-12-18T16:04:44Z","created_at_i":1261152284,"num_comments":0,"objectID":"1003311","points":1,"story_id":1003311,"title":"Why Should You Have a Business BrainTrust?","updated_at":"2024-09-19T16:46:26Z","url":"http://www.47hats.com/2009/12/why-should-you-have-a-business-braintrust/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"calcsam"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Hi HN, we're Sam, Shane, and Abhi.
Almost a year ago, we first shared Mastra here (https://news.ycombinator.com/item?id=43103073). It\u2019s kind of fun looking back since we were only a few months into building at the time. The HN community gave a lot of enthusiasm and some helpful feedback.
Today, we released Mastra 1.0 in stable, so we wanted to come back and talk about what\u2019s changed.
If you\u2019re new to Mastra, it's an open-source TypeScript agent framework that also lets you create multi-agent workflows, run evals, inspect in a local studio, and emit observability.
Since our last post, Mastra has grown to over 300k weekly npm downloads and 19.4k GitHub stars. It\u2019s now Apache 2.0 licensed and runs in prod at companies like Replit, PayPal, and Sanity.
Agent development is changing quickly, so we\u2019ve added a lot since February:
- Native model routing: You can access 600+ models from 40+ providers by specifying a model string (e.g., `openai/gpt-5.2-codex`) with TS autocomplete and fallbacks.
- Guardrails: Low-latency input and output processors for prompt injection detection, PII redaction, and content moderation. The tricky thing here was the low-latency part.
- Scorers: An async eval primitive for grading agent outputs. Users were asking how they should do evals. We wanted to make it easy to attach to Mastra agents, runnable in Mastra studio, and save results in Mastra storage.
- Plus a few other features like AI tracing (per-call costing for Langfuse, Braintrust, etc), memory processors, a `.network()` method that turns any agent into a routing agent, and server adapters to integrate Mastra within an existing Express/Hono server.
(That last one took a bit of time, we went down the ESM/CJS bundling rabbithole, ran into lots of monorepo issues, and ultimately opted for a more explicit approach.)
Anyway, we'd love for you to try Mastra out and let us know what you think. You can get started with `npm create mastra@latest`.
We'll be around and happy to answer any questions!"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Mastra 1.0, open-source JavaScript agent framework from the Gatsby devs"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/mastra-ai/mastra"}},"_tags":["story","author_calcsam","story_46693959","show_hn"],"author":"calcsam","children":[46697168,46697330,46697336,46697346,46697549,46697554,46697582,46697822,46698748,46698861,46699007,46699322,46699323,46699748,46699990,46700367,46700788,46701008,46701238,46701350,46701702,46703051,46704176,46706505,46707830,46708111,46708234,46709744],"created_at":"2026-01-20T16:38:56Z","created_at_i":1768927136,"num_comments":70,"objectID":"46693959","points":213,"story_id":46693959,"story_text":"Hi HN, we're Sam, Shane, and Abhi.
Almost a year ago, we first shared Mastra here (https://news.ycombinator.com/item?id=43103073). It\u2019s kind of fun looking back since we were only a few months into building at the time. The HN community gave a lot of enthusiasm and some helpful feedback.
Today, we released Mastra 1.0 in stable, so we wanted to come back and talk about what\u2019s changed.
If you\u2019re new to Mastra, it's an open-source TypeScript agent framework that also lets you create multi-agent workflows, run evals, inspect in a local studio, and emit observability.
Since our last post, Mastra has grown to over 300k weekly npm downloads and 19.4k GitHub stars. It\u2019s now Apache 2.0 licensed and runs in prod at companies like Replit, PayPal, and Sanity.
Agent development is changing quickly, so we\u2019ve added a lot since February:
- Native model routing: You can access 600+ models from 40+ providers by specifying a model string (e.g., `openai/gpt-5.2-codex`) with TS autocomplete and fallbacks.
- Guardrails: Low-latency input and output processors for prompt injection detection, PII redaction, and content moderation. The tricky thing here was the low-latency part.
- Scorers: An async eval primitive for grading agent outputs. Users were asking how they should do evals. We wanted to make it easy to attach to Mastra agents, runnable in Mastra studio, and save results in Mastra storage.
- Plus a few other features like AI tracing (per-call costing for Langfuse, Braintrust, etc), memory processors, a `.network()` method that turns any agent into a routing agent, and server adapters to integrate Mastra within an existing Express/Hono server.
(That last one took a bit of time, we went down the ESM/CJS bundling rabbithole, ran into lots of monorepo issues, and ultimately opted for a more explicit approach.)
Anyway, we'd love for you to try Mastra out and let us know what you think. You can get started with `npm create mastra@latest`.
We'll be around and happy to answer any questions!","title":"Show HN: Mastra 1.0, open-source JavaScript agent framework from the Gatsby devs","updated_at":"2026-03-05T23:29:06Z","url":"https://github.com/mastra-ai/mastra"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"skull8888888"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["braintrust"],"value":"Hey HN, we\u2019re Robert, Din and Temirlan from Laminar (https://www.lmnr.ai), an open-source observability and analytics platform for complex LLM apps. It\u2019s designed to be fast, reliable, and scalable. The stack is RabbitMQ for message queues, Postgres for storage, Clickhouse for analytics, Qdrant for semantic search - all powered by Rust.
How is Laminar different from the swarm of other \u201cLLM observability\u201d platforms?
On the observability part, we\u2019re focused on handling full execution traces, not just LLM calls. We built a Rust ingestor for OpenTelemetry (Otel) spans with GenAI semantic conventions. As LLM apps get more complex (think Agents with hundreds of LLM and function calls, or complex RAG pipelines), full tracing is critical. With Otel spans, we can: 1. Cover the entire execution trace. 2. Keep the platform future-proof 3. Leverage an amazing OpenLLMetry (https://github.com/traceloop/openllmetry), open-source package for span production.
The key difference is that we tie text analytics directly to execution traces. Rich text data makes LLM traces unique, so we let you track \u201csemantic metrics\u201d (like what your AI agent is actually saying) and connect those metrics to where they happen in the trace. If you want to know if your AI drive-through agent made an upsell, you can design an LLM extraction pipeline in our builder (more on it later), host it on Laminar, and handle everything from event requests to output logging. Processing requests simply come as events in the Otel span.
We think it\u2019s a win to separate core app logic from LLM event processing. Most devs don\u2019t want to manage background queues for LLM analytics processing but still want insights into how their Agents or RAGs are working.
Our Pipeline Builder uses graph UI where nodes are LLM and util functions, and edges showing data flow. We built a custom task execution engine with support of parallel branch executions, cycles and branches (it\u2019s overkill for simple pipelines, but it\u2019s extremely cool and we\u2019ve spent a lot of time designing a robust engine). You can also call pipelines directly as API endpoints. We found them to be extremely useful for iterating on and separating LLM logic. Laminar also traces pipeline directly, which removes the overhead of sending large outputs over the network.
One thing missing from all LLM observability platforms right now is an adequate search over traces. We\u2019re attacking this problem by indexing each span in a vector DB and performing hybrid search at query time. This feature is still in beta, but we think it\u2019s gonna be crucial part of our platform going forward.
We also support evaluations. We loved the \u201crun everything locally, send results to a server\u201d approach from Braintrust and Weights & Biases, so we did that too: a simple SDK and nice dashboards to track everything. Evals are still early, but we\u2019re pushing hard on them.
Our goal is to make Laminar the Supabase for LLMOps - the go-to open-source comprehensive platform for all things LLMs / GenAI. In it\u2019s current shape, Laminar is just few weeks old and developing rapidly, we\u2019d love any feedback or for you to give Laminar a try in your LLM projects!"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Laminar \u2013 Open-Source DataDog + PostHog for LLM Apps, Built in Rust"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/lmnr-ai/lmnr"}},"_tags":["story","author_skull8888888","story_41451698","show_hn"],"author":"skull8888888","children":[41452424,41452994,41453001,41453058,41453087,41453241,41453566,41453832,41455101,41455103,41456518,41459224,41459548,41459620,41462581],"created_at":"2024-09-04T22:52:19Z","created_at_i":1725490339,"num_comments":45,"objectID":"41451698","points":203,"story_id":41451698,"story_text":"Hey HN, we\u2019re Robert, Din and Temirlan from Laminar (https://www.lmnr.ai), an open-source observability and analytics platform for complex LLM apps. It\u2019s designed to be fast, reliable, and scalable. The stack is RabbitMQ for message queues, Postgres for storage, Clickhouse for analytics, Qdrant for semantic search - all powered by Rust.
How is Laminar different from the swarm of other \u201cLLM observability\u201d platforms?
On the observability part, we\u2019re focused on handling full execution traces, not just LLM calls. We built a Rust ingestor for OpenTelemetry (Otel) spans with GenAI semantic conventions. As LLM apps get more complex (think Agents with hundreds of LLM and function calls, or complex RAG pipelines), full tracing is critical. With Otel spans, we can: 1. Cover the entire execution trace. 2. Keep the platform future-proof 3. Leverage an amazing OpenLLMetry (https://github.com/traceloop/openllmetry), open-source package for span production.
The key difference is that we tie text analytics directly to execution traces. Rich text data makes LLM traces unique, so we let you track \u201csemantic metrics\u201d (like what your AI agent is actually saying) and connect those metrics to where they happen in the trace. If you want to know if your AI drive-through agent made an upsell, you can design an LLM extraction pipeline in our builder (more on it later), host it on Laminar, and handle everything from event requests to output logging. Processing requests simply come as events in the Otel span.
We think it\u2019s a win to separate core app logic from LLM event processing. Most devs don\u2019t want to manage background queues for LLM analytics processing but still want insights into how their Agents or RAGs are working.
Our Pipeline Builder uses graph UI where nodes are LLM and util functions, and edges showing data flow. We built a custom task execution engine with support of parallel branch executions, cycles and branches (it\u2019s overkill for simple pipelines, but it\u2019s extremely cool and we\u2019ve spent a lot of time designing a robust engine). You can also call pipelines directly as API endpoints. We found them to be extremely useful for iterating on and separating LLM logic. Laminar also traces pipeline directly, which removes the overhead of sending large outputs over the network.
One thing missing from all LLM observability platforms right now is an adequate search over traces. We\u2019re attacking this problem by indexing each span in a vector DB and performing hybrid search at query time. This feature is still in beta, but we think it\u2019s gonna be crucial part of our platform going forward.
We also support evaluations. We loved the \u201crun everything locally, send results to a server\u201d approach from Braintrust and Weights & Biases, so we did that too: a simple SDK and nice dashboards to track everything. Evals are still early, but we\u2019re pushing hard on them.
Our goal is to make Laminar the Supabase for LLMOps - the go-to open-source comprehensive platform for all things LLMs / GenAI. In it\u2019s current shape, Laminar is just few weeks old and developing rapidly, we\u2019d love any feedback or for you to give Laminar a try in your LLM projects!","title":"Show HN: Laminar \u2013 Open-Source DataDog + PostHog for LLM Apps, Built in Rust","updated_at":"2026-02-18T07:06:12Z","url":"https://github.com/lmnr-ai/lmnr"}],"hitsPerPage":50,"nbHits":132,"nbPages":3,"page":0,"params":"query=Braintrust&tags=story&hitsPerPage=50&advancedSyntax=true&analyticsTags=backend","processingTimeMS":17,"processingTimingsMS":{"_request":{"roundTrip":20},"afterFetch":{"format":{"highlighting":1,"total":1},"merge":{"mergeLoop":{"prepareNextHit":8,"total":8},"total":8},"total":8},"fetch":{"query":6,"scanning":2,"total":9},"total":17},"query":"Braintrust","serverTimeMS":19}