{"exhaustive":{"nbHits":false,"typo":false},"exhaustiveNbHits":false,"exhaustiveTypo":false,"hits":[{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"dot_treo"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["berriai","litellm"],"value":"About an hour ago new versions have been deployed to PyPI.<p>I was just setting up a new project, and things behaved weirdly. My laptop ran out of RAM, it looked like a forkbomb was running.<p>I've investigated, and found that a base64 encoded blob has been added to proxy_server.py.<p>It writes and decodes another file which it then runs.<p>I'm in the process of reporting this upstream, but wanted to give everyone here a headsup.<p>It is also reported in this issue:\n<a href=\"https://github.com/BerriAI/litellm/issues/24512\" rel=\"nofollow\">https://github.com/<em>BerriAI</em>/<em>litellm</em>/issues/24512</a>"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["litellm"],"value":"Tell HN: <em>Litellm</em> 1.82.7 and 1.82.8 on PyPI are compromised"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["berriai","litellm"],"value":"https://github.com/<em>BerriAI</em>/<em>litellm</em>/issues/24512"}},"_tags":["story","author_dot_treo","story_47501426"],"author":"dot_treo","children":[47501432,47501658,47501856,47501928,47501993,47502002,47502008,47502034,47502063,47502085,47502109,47502237,47502260,47502272,47502296,47502297,47502301,47502309,47502319,47502346,47502350,47502353,47502355,47502380,47502402,47502412,47502434,47502440,47502447,47502454,47502459,47502493,47502529,47502542,47502548,47502549,47502568,47502586,47502604,47502619,47502731,47502769,47502785,47502798,47502839,47502856,47502858,47502906,47502920,47502925,47502955,47502980,47503031,47503065,47503106,47503114,47503196,47503270,47503281,47503343,47503373,47503580,47503590,47503685,47503702,47503725,47503842,47504102,47504118,47504121,47504191,47504387,47504719,47504732,47504933,47505214,47505215,47505713,47505844,47505992,47506076,47506099,47506165,47506200,47506347,47506350,47506356,47506618,47506658,47506701,47506858,47506974,47507089,47507545,47507548,47507550,47507593,47507836,47508055,47508120,47508281,47508315,47508422,47508540,47509479,47510199,47510246,47510301,47510350,47510471,47510846,47511149,47511188,47511296,47511671,47511696,47511845,47511907,47512080,47512083,47512165,47512178,47512863,47513129,47513787,47513932,47514873,47515393,47515479,47515544,47515626,47516215,47516771,47517825,47518203,47518256,47519380,47519747,47522157,47522817,47526751,47529617,47529741,47530192,47595397],"created_at":"2026-03-24T12:06:29Z","created_at_i":1774353989,"num_comments":500,"objectID":"47501426","points":938,"story_id":47501426,"story_text":"About an hour ago new versions have been deployed to PyPI.<p>I was just setting up a new project, and things behaved weirdly. My laptop ran out of RAM, it looked like a forkbomb was running.<p>I&#x27;ve investigated, and found that a base64 encoded blob has been added to proxy_server.py.<p>It writes and decodes another file which it then runs.<p>I&#x27;m in the process of reporting this upstream, but wanted to give everyone here a headsup.<p>It is also reported in this issue:\n<a href=\"https:&#x2F;&#x2F;github.com&#x2F;BerriAI&#x2F;litellm&#x2F;issues&#x2F;24512\" rel=\"nofollow\">https:&#x2F;&#x2F;github.com&#x2F;BerriAI&#x2F;litellm&#x2F;issues&#x2F;24512</a>","title":"Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised","updated_at":"2026-08-12T16:45:03Z","url":"https://github.com/BerriAI/litellm/issues/24512"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"santiago-pl"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["berriai","litellm"],"value":"Hi, I\u2019m Jakub, a solo founder based in Warsaw.<p>I\u2019ve been building GoModel since December with a couple of contributors. It's an open-source AI gateway that sits between your app and model providers like OpenAI, Anthropic or others.<p>I built it for my startup to solve a few problems:<p><pre><code>  - track AI usage and cost per client or team\n  - switch models without changing app code\n  - debug request flows more easily\n  - reduce AI spendings with exact and semantic caching\n</code></pre>\nHow is it different?<p><pre><code>  - ~17MB docker image\n    - <em>LiteLLM</em>'s image is more than 44x bigger (&quot;docker.<em>litellm</em>.ai/<em>berriai</em>/<em>litellm</em>:latest&quot; ~ 746 MB on amd64)\n  - request workflow is visible and easy to inspect    \n  - config is environment-variable-first by default\n</code></pre>\nI'm posting now partly because of the recent <em>LiteLLM</em> supply-chain attack. Their team handled it impressively well, but some people are looking at alternatives anyway, and GoModel is one.<p>Website: <a href=\"https://gomodel.enterpilot.io\" rel=\"nofollow\">https://gomodel.enterpilot.io</a><p>Any feedback is appreciated."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: GoModel \u2013 an open-source AI gateway in Go"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/ENTERPILOT/GOModel/"}},"_tags":["story","author_santiago-pl","story_47849097","show_hn"],"author":"santiago-pl","children":[47849790,47849997,47850001,47850517,47850697,47850860,47851312,47851437,47851452,47851488,47851752,47851816,47851918,47851960,47852763,47853211,47853600,47853948,47854725,47861263,47861532,47862492,47872649,47888441,47927922],"created_at":"2026-04-21T14:11:53Z","created_at_i":1776780713,"num_comments":75,"objectID":"47849097","points":217,"story_id":47849097,"story_text":"Hi, I\u2019m Jakub, a solo founder based in Warsaw.<p>I\u2019ve been building GoModel since December with a couple of contributors. It&#x27;s an open-source AI gateway that sits between your app and model providers like OpenAI, Anthropic or others.<p>I built it for my startup to solve a few problems:<p><pre><code>  - track AI usage and cost per client or team\n  - switch models without changing app code\n  - debug request flows more easily\n  - reduce AI spendings with exact and semantic caching\n</code></pre>\nHow is it different?<p><pre><code>  - ~17MB docker image\n    - LiteLLM&#x27;s image is more than 44x bigger (&quot;docker.litellm.ai&#x2F;berriai&#x2F;litellm:latest&quot; ~ 746 MB on amd64)\n  - request workflow is visible and easy to inspect    \n  - config is environment-variable-first by default\n</code></pre>\nI&#x27;m posting now partly because of the recent LiteLLM supply-chain attack. Their team handled it impressively well, but some people are looking at alternatives anyway, and GoModel is one.<p>Website: <a href=\"https:&#x2F;&#x2F;gomodel.enterpilot.io\" rel=\"nofollow\">https:&#x2F;&#x2F;gomodel.enterpilot.io</a><p>Any feedback is appreciated.","title":"Show HN: GoModel \u2013 an open-source AI gateway in Go","updated_at":"2026-08-06T15:01:25Z","url":"https://github.com/ENTERPILOT/GOModel/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ij23"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["berriai","litellm"],"value":"Hello hacker news,<p>I\u2019m the maintainer of <em>liteLLM</em>() - package to simplify input/output to OpenAI, Azure, Cohere, Anthropic, Hugging face API Endpoints: <a href=\"https://github.com/BerriAI/litellm/\">https://github.com/<em>BerriAI</em>/<em>litellm</em>/</a><p>We\u2019re open sourcing our implementation of <em>liteLLM</em> proxy: <a href=\"https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-server/readme.md\">https://github.com/<em>BerriAI</em>/<em>litellm</em>/blob/main/cookbook/proxy-...</a><p>TLDR: It has one API endpoint /chat/completions and standardizes input/output for 50+ LLM models + handles logging, error tracking, caching, streaming<p>What can <em>liteLLM</em> proxy do?\n- It\u2019s a central place to manage all LLM provider integrations<p>- Consistent Input/Output Format\n    - Call all models using the OpenAI format: completion(model, messages)\n    - Text responses will always be available at ['choices'][0]['message']['content']<p>- Error Handling Using Model Fallbacks (if GPT-4 fails, try llama2)<p>- Logging - Log Requests, Responses and Errors to Supabase, Posthog, Mixpanel, Sentry, Helicone<p>- Token Usage &amp; Spend - Track Input + Completion tokens used + Spend/model<p>- Caching - Implementation of Semantic Caching<p>- Streaming &amp; Async Support - Return generators to stream text responses<p>You can deploy <em>liteLLM</em> to your own infrastructure using Railway, GCP, AWS, Azure<p>Happy completion() !"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["litellm"],"value":"Show HN: <em>liteLLM</em> Proxy Server: 50+ LLM Models, Error Handling, Caching"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["berriai","litellm"],"value":"https://github.com/<em>BerriAI</em>/<em>litellm</em>/blob/main/cookbook/proxy-server/readme.md"}},"_tags":["story","author_ij23","story_37095542","show_hn"],"author":"ij23","children":[37096039,37096625,37096662,37096686,37096881,37097078,37097106,37098103,37099449,37100300,37105236,37128926],"created_at":"2023-08-12T00:08:13Z","created_at_i":1691798893,"num_comments":34,"objectID":"37095542","points":140,"story_id":37095542,"story_text":"Hello hacker news,<p>I\u2019m the maintainer of liteLLM() - package to simplify input&#x2F;output to OpenAI, Azure, Cohere, Anthropic, Hugging face API Endpoints: <a href=\"https:&#x2F;&#x2F;github.com&#x2F;BerriAI&#x2F;litellm&#x2F;\">https:&#x2F;&#x2F;github.com&#x2F;BerriAI&#x2F;litellm&#x2F;</a><p>We\u2019re open sourcing our implementation of liteLLM proxy: <a href=\"https:&#x2F;&#x2F;github.com&#x2F;BerriAI&#x2F;litellm&#x2F;blob&#x2F;main&#x2F;cookbook&#x2F;proxy-server&#x2F;readme.md\">https:&#x2F;&#x2F;github.com&#x2F;BerriAI&#x2F;litellm&#x2F;blob&#x2F;main&#x2F;cookbook&#x2F;proxy-...</a><p>TLDR: It has one API endpoint &#x2F;chat&#x2F;completions and standardizes input&#x2F;output for 50+ LLM models + handles logging, error tracking, caching, streaming<p>What can liteLLM proxy do?\n- It\u2019s a central place to manage all LLM provider integrations<p>- Consistent Input&#x2F;Output Format\n    - Call all models using the OpenAI format: completion(model, messages)\n    - Text responses will always be available at [&#x27;choices&#x27;][0][&#x27;message&#x27;][&#x27;content&#x27;]<p>- Error Handling Using Model Fallbacks (if GPT-4 fails, try llama2)<p>- Logging - Log Requests, Responses and Errors to Supabase, Posthog, Mixpanel, Sentry, Helicone<p>- Token Usage &amp; Spend - Track Input + Completion tokens used + Spend&#x2F;model<p>- Caching - Implementation of Semantic Caching<p>- Streaming &amp; Async Support - Return generators to stream text responses<p>You can deploy liteLLM to your own infrastructure using Railway, GCP, AWS, Azure<p>Happy completion() !","title":"Show HN: liteLLM Proxy Server: 50+ LLM Models, Error Handling, Caching","updated_at":"2026-03-08T20:37:54Z","url":"https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-server/readme.md"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"cauchyk"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["berriai","litellm"],"value":"Hi all,<p>I\u2019m hacking on new features for the ClickHouse native client and wanted the same \u201cjust call the model\u201d ergonomics JavaScript and Python now enjoy. It didn\u2019t exist for modern C++, so I wrote one.<p>ai\u2011sdk\u2011cpp (Apache\u20112.0) gives you:<p>- Unified calls to OpenAI (GPT\u20114o) and Anthropic (Claude 3.5) with a single C++20 API.\n - Streaming, multi\u2011turn chat, error handling\u2014all std::optional/std::variant, no macros.\n - Tool calling (function\u2011calling) so the model can hit real APIs; sync or async, runs in parallel.<p>The tricky bit: C++ still lacks real reflection, so mapping plain functions \u2192 JSON schemas isn\u2019t as automatic as, say, TypeScript decorators. I\u2019d love fresh eyes on that part. Try the examples and tell me where it feels clunky. This is inspired by Vercel's AI SDK [1], and <em>litellm</em> [2].<p>Repo live here: <a href=\"https://github.com/ClickHouse/ai-sdk-cpp\">https://github.com/ClickHouse/ai-sdk-cpp</a>, feedback welcome!<p>[1] <a href=\"https://github.com/vercel/ai\">https://github.com/vercel/ai</a>\n[2] <a href=\"https://github.com/BerriAI/litellm\">https://github.com/<em>BerriAI</em>/<em>litellm</em></a>"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: A modern C++20 AI SDK (GPT\u20114o, Claude 3.5, tool\u2011calling)"}},"_tags":["story","author_cauchyk","story_44412726","show_hn"],"author":"cauchyk","children":[44441031,44441401,44441804,44442336,44442828],"created_at":"2025-06-29T12:52:42Z","created_at_i":1751201562,"num_comments":6,"objectID":"44412726","points":56,"story_id":44412726,"story_text":"Hi all,<p>I\u2019m hacking on new features for the ClickHouse native client and wanted the same \u201cjust call the model\u201d ergonomics JavaScript and Python now enjoy. It didn\u2019t exist for modern C++, so I wrote one.<p>ai\u2011sdk\u2011cpp (Apache\u20112.0) gives you:<p>- Unified calls to OpenAI (GPT\u20114o) and Anthropic (Claude 3.5) with a single C++20 API.\n - Streaming, multi\u2011turn chat, error handling\u2014all std::optional&#x2F;std::variant, no macros.\n - Tool calling (function\u2011calling) so the model can hit real APIs; sync or async, runs in parallel.<p>The tricky bit: C++ still lacks real reflection, so mapping plain functions \u2192 JSON schemas isn\u2019t as automatic as, say, TypeScript decorators. I\u2019d love fresh eyes on that part. Try the examples and tell me where it feels clunky. This is inspired by Vercel&#x27;s AI SDK [1], and litellm [2].<p>Repo live here: <a href=\"https:&#x2F;&#x2F;github.com&#x2F;ClickHouse&#x2F;ai-sdk-cpp\">https:&#x2F;&#x2F;github.com&#x2F;ClickHouse&#x2F;ai-sdk-cpp</a>, feedback welcome!<p>[1] <a href=\"https:&#x2F;&#x2F;github.com&#x2F;vercel&#x2F;ai\">https:&#x2F;&#x2F;github.com&#x2F;vercel&#x2F;ai</a>\n[2] <a href=\"https:&#x2F;&#x2F;github.com&#x2F;BerriAI&#x2F;litellm\">https:&#x2F;&#x2F;github.com&#x2F;BerriAI&#x2F;litellm</a>","title":"Show HN: A modern C++20 AI SDK (GPT\u20114o, Claude 3.5, tool\u2011calling)","updated_at":"2025-07-05T20:21:34Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"theanonymousone"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["litellm"],"value":"Malicious <em>litellm</em>_init.pth in <em>litellm</em> 1.82.8 PyPI package \u2013 credential stealer"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["berriai","litellm"],"value":"https://github.com/<em>BerriAI</em>/<em>litellm</em>/issues/24512"}},"_tags":["story","author_theanonymousone","story_47501729"],"author":"theanonymousone","children":[47507320,47508414],"created_at":"2026-03-24T12:36:20Z","created_at_i":1774355780,"num_comments":1,"objectID":"47501729","points":739,"story_id":47501729,"title":"Malicious litellm_init.pth in litellm 1.82.8 PyPI package \u2013 credential stealer","updated_at":"2026-04-24T17:45:46Z","url":"https://github.com/BerriAI/litellm/issues/24512"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ij23"},"story_text":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["litellm"],"value":"I built this library because langchain was too bloated and I needed a simple abstraction to call multiple LLM APIs. <em>litellm</em> has two functions - completion(), embedding()"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["litellm"],"value":"Show HN: <em>Litellm</em> \u2013 Simple library to standardize OpenAI, Cohere, Azure LLM I/O"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["berriai","litellm"],"value":"https://github.com/<em>BerriAI</em>/<em>litellm</em>"}},"_tags":["story","author_ij23","story_36887711","show_hn"],"author":"ij23","children":[36888761,36889561,36890114,36892376,36892443,36896206,36900108],"created_at":"2023-07-27T01:31:35Z","created_at_i":1690421495,"num_comments":17,"objectID":"36887711","points":62,"story_id":36887711,"story_text":"I built this library because langchain was too bloated and I needed a simple abstraction to call multiple LLM APIs. litellm has two functions - completion(), embedding()","title":"Show HN: Litellm \u2013 Simple library to standardize OpenAI, Cohere, Azure LLM I/O","updated_at":"2026-08-04T05:56:14Z","url":"https://github.com/BerriAI/litellm"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Bluestein"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["litellm"],"value":"<em>LiteLLM</em> Python SDK Proxy Server LLM Gateway Call 100 LLM APIs in OpenAI Format"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["berriai","litellm"],"value":"https://github.com/<em>BerriAI</em>/<em>litellm</em>"}},"_tags":["story","author_Bluestein","story_44674162"],"author":"Bluestein","created_at":"2025-07-24T18:19:40Z","created_at_i":1753381180,"num_comments":0,"objectID":"44674162","points":5,"story_id":44674162,"title":"LiteLLM Python SDK Proxy Server LLM Gateway Call 100 LLM APIs in OpenAI Format","updated_at":"2025-07-24T20:17:18Z","url":"https://github.com/BerriAI/litellm"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ij23"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["litellm"],"value":"<em>LiteLLM</em> Agent Platform: Run Claude Code/Codex On-Prem Sandboxes and Vaults"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["berriai","litellm"],"value":"https://github.com/<em>BerriAI</em>/<em>litellm</em>-agent-platform"}},"_tags":["story","author_ij23","story_48155595"],"author":"ij23","children":[48155596,48238144],"created_at":"2026-05-16T00:22:04Z","created_at_i":1778890924,"num_comments":0,"objectID":"48155595","points":3,"story_id":48155595,"title":"LiteLLM Agent Platform: Run Claude Code/Codex On-Prem Sandboxes and Vaults","updated_at":"2026-05-22T16:31:10Z","url":"https://github.com/BerriAI/litellm-agent-platform"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ij23"},"title":{"matchLevel":"none","matchedWords":[],"value":"Llama2 on Replicate faster than ChatGPT?"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["berriai","litellm"],"value":"https://github.com/<em>BerriAI</em>/<em>litellm</em>/blob/main/cookbook/Evaluating_LLMs.ipynb"}},"_tags":["story","author_ij23","story_37153342"],"author":"ij23","children":[37153343,37153393],"created_at":"2023-08-16T20:30:37Z","created_at_i":1692217837,"num_comments":2,"objectID":"37153342","points":1,"story_id":37153342,"title":"Llama2 on Replicate faster than ChatGPT?","updated_at":"2024-09-20T14:50:54Z","url":"https://github.com/BerriAI/litellm/blob/main/cookbook/Evaluating_LLMs.ipynb"}],"hitsPerPage":50,"nbHits":9,"nbPages":1,"page":0,"params":"query=BerriAI+%28LiteLLM%29&tags=story&hitsPerPage=50&advancedSyntax=true&analyticsTags=backend","processingTimeMS":13,"processingTimingsMS":{"_request":{"queue":1,"roundTrip":15},"fetch":{"query":11,"total":12},"total":13},"query":"BerriAI (LiteLLM)","serverTimeMS":15}