A brief outline of the system:
- Android app listens for incoming SMS events and forwards them over MQTT to a server running Ollama which generates responses\n- Conversations are whitelisted and manually assigned a persona. The LLM has access to the last N messages of the conversation for additional context.
[0]: https://github.com/evidlo/sms_llm
I'm aware that replying can encourage/allow the sender to send more spam. Hopefully reporting the numbers after the conversation is a reasonable compromise."},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Show HN: Trolling SMS spammers with Ollama"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://evan.widloski.com/software/sms_llm/"}},"_tags":["story","author_Evidlo","story_42796496","show_hn"],"author":"Evidlo","children":[42796731,42796788,42796804,42810166,42817601,42817953,42817985,42818044,42818050,42818071,42818552,42818560,42818578,42818860,42819010,42819144,42819303,42819356,42819391,42819712,42820060,42820317,42820467,42820533,42820584,42821037,42821836,42821959,42824628,42828067,42828163],"created_at":"2025-01-22T19:23:48Z","created_at_i":1737573828,"num_comments":126,"objectID":"42796496","points":318,"story_id":42796496,"story_text":"I've been working on a side project to generate responses to spam with various funny LLM personas, such as a millenial gym bro and a 19th century British gentleman. By request, I've made a write-up on my website which has some humorous screenshots and made the code available on Github for others to try out [0].
A brief outline of the system:
- Android app listens for incoming SMS events and forwards them over MQTT to a server running Ollama which generates responses\n- Conversations are whitelisted and manually assigned a persona. The LLM has access to the last N messages of the conversation for additional context.
[0]: https://github.com/evidlo/sms_llm
I'm aware that replying can encourage/allow the sender to send more spam. Hopefully reporting the numbers after the conversation is a reasonable compromise.","title":"Show HN: Trolling SMS spammers with Ollama","updated_at":"2026-01-29T17:27:21Z","url":"https://evan.widloski.com/software/sms_llm/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Dontizi"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Show HN: Open-Source DocumentAI with Ollama"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://rlama.dev/"}},"_tags":["story","author_Dontizi","story_43296918","show_hn"],"author":"Dontizi","children":[43297430,43297435,43297466,43297885,43298080,43298104,43298273,43298520,43298814,43298918,43298997,43299285,43300020,43300263,43300305,43302039,43303993],"created_at":"2025-03-08T02:12:13Z","created_at_i":1741399933,"num_comments":33,"objectID":"43296918","points":295,"story_id":43296918,"title":"Show HN: Open-Source DocumentAI with Ollama","updated_at":"2025-03-16T22:33:42Z","url":"https://rlama.dev/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"eadz"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Phi 4 available on Ollama"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://ollama.com/library/phi4"}},"_tags":["story","author_eadz","story_42642971"],"author":"eadz","children":[42643022,42643388,42671010,42671067,42671070,42671088,42671283,42671855,42671932,42672580,42673129,42673227,42673346,42673819,42674730,42675448,42676687,42682546,42685362],"created_at":"2025-01-09T08:12:57Z","created_at_i":1736410377,"num_comments":135,"objectID":"42642971","points":291,"story_id":42642971,"title":"Phi 4 available on Ollama","updated_at":"2025-12-30T13:21:33Z","url":"https://ollama.com/library/phi4"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"yujonglee"},"story_text":{"matchLevel":"none","matchedWords":[],"value":"Hello everyone. This is Yujong from the Hyprnote team (https://github.com/fastrepl/hyprnote).
We built OWhisper for 2 reasons:\n(Also outlined in https://docs.hyprnote.com/owhisper/what-is-this)
(1). While working with on-device, realtime speech-to-text, we found there isn't tooling that exists to download / run the model in a practical way.
(2). Also, we got frequent requests to provide a way to plug in custom STT endpoints to the Hyprnote desktop app, just like doing it with OpenAI-compatible LLM endpoints.
The (2) part is still kind of WIP, but we spent some time writing docs so you'll get a good idea of what it will look like if you skim through them.
For (1) - You can try it now. (https://docs.hyprnote.com/owhisper/cli/get-started)
bash\n brew tap fastrepl/hyprnote && brew install owhisper\n owhisper pull whisper-cpp-base-q8-en\n owhisper run whisper-cpp-base-q8-en\n\nWe're here and looking forward to your comments!"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Show HN: OWhisper \u2013 Ollama for realtime speech-to-text"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://docs.hyprnote.com/owhisper/what-is-this"}},"_tags":["story","author_yujonglee","story_44901853","show_hn"],"author":"yujonglee","children":[44904204,44904267,44904907,44905003,44905187,44905812,44906643,44906779,44907040,44907329,44907533,44907758,44908078,44908231,44908961,44909249,44909985,44910601,44910667,44911648,44911726,44911752,44912216,44914759,44915232,44915327,44917458],"created_at":"2025-08-14T15:47:43Z","created_at_i":1755186463,"num_comments":75,"objectID":"44901853","points":289,"story_id":44901853,"story_text":"Hello everyone. This is Yujong from the Hyprnote team (https://github.com/fastrepl/hyprnote).
We built OWhisper for 2 reasons:\n(Also outlined in https://docs.hyprnote.com/owhisper/what-is-this)
(1). While working with on-device, realtime speech-to-text, we found there isn't tooling that exists to download / run the model in a practical way.
(2). Also, we got frequent requests to provide a way to plug in custom STT endpoints to the Hyprnote desktop app, just like doing it with OpenAI-compatible LLM endpoints.
The (2) part is still kind of WIP, but we spent some time writing docs so you'll get a good idea of what it will look like if you skim through them.
For (1) - You can try it now. (https://docs.hyprnote.com/owhisper/cli/get-started)
bash\n brew tap fastrepl/hyprnote && brew install owhisper\n owhisper pull whisper-cpp-base-q8-en\n owhisper run whisper-cpp-base-q8-en\n\nWe're here and looking forward to your comments!","title":"Show HN: OWhisper \u2013 Ollama for realtime speech-to-text","updated_at":"2026-03-05T22:27:47Z","url":"https://docs.hyprnote.com/owhisper/what-is-this"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jmorgan"},"story_text":{"matchLevel":"none","matchedWords":[],"value":"Hi HN
A few folks and I have been working on this project for a couple weeks now. After previously working on the Docker project for a number of years (both on the container runtime and image registry side), the recent rise in open source language models made us think something similar needed to exist for large language models too.
While not exactly the same as running linux containers, running LLMs shares quite a few of the same challenges. There are "base layers" (e.g. models like Llama 2), specific configuration to run correctly (parameters, temperature, context window sizes etc). There's also embeddings that a model can use at runtime to look up data \u2013 we don't support this yet but it's something we're looking at doing soon.
It's an early project, and there's still lots to do!"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Show HN: Ollama \u2013 Run LLMs on your Mac"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://github.com/jmorganca/ollama"}},"_tags":["story","author_jmorgan","story_36802582","show_hn"],"author":"jmorgan","children":[36802913,36802952,36803034,36803083,36804104,36804190,36804514,36805148,36805479,36806203,36806300,36806309,36806411,36806448,36807169,36807388,36807830,36808304,36808368,36809319,36809364,36809394,36809470,36809715,36809885,36810270,36810415,36811123,36812192,36813328,36832090,36832489,36860520,36876874],"created_at":"2023-07-20T16:06:44Z","created_at_i":1689869204,"num_comments":94,"objectID":"36802582","points":284,"story_id":36802582,"story_text":"Hi HN
A few folks and I have been working on this project for a couple weeks now. After previously working on the Docker project for a number of years (both on the container runtime and image registry side), the recent rise in open source language models made us think something similar needed to exist for large language models too.
While not exactly the same as running linux containers, running LLMs shares quite a few of the same challenges. There are "base layers" (e.g. models like Llama 2), specific configuration to run correctly (parameters, temperature, context window sizes etc). There's also embeddings that a model can use at runtime to look up data \u2013 we don't support this yet but it's something we're looking at doing soon.
It's an early project, and there's still lots to do!","title":"Show HN: Ollama \u2013 Run LLMs on your Mac","updated_at":"2025-01-12T11:55:59Z","url":"https://github.com/jmorganca/ollama"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Patrick_Devine"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Structured Outputs with Ollama"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://ollama.com/blog/structured-outputs"}},"_tags":["story","author_Patrick_Devine","story_42346344"],"author":"Patrick_Devine","children":[42346507,42346509,42346676,42346699,42346837,42346878,42346948,42347000,42347096,42347131,42347605,42347766,42347874,42347927,42348568,42349310,42349562,42354556],"created_at":"2024-12-07T01:12:32Z","created_at_i":1733533952,"num_comments":70,"objectID":"42346344","points":265,"story_id":42346344,"title":"Structured Outputs with Ollama","updated_at":"2025-09-24T16:52:31Z","url":"https://ollama.com/blog/structured-outputs"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"excerionsforte"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Official DeepSeek R1 Now on Ollama"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://ollama.com/library/deepseek-r1"}},"_tags":["story","author_excerionsforte","story_42772983"],"author":"excerionsforte","children":[42776645,42776754,42776772,42776938,42776974,42777095,42777334,42777649,42777908,42777953,42778017,42778980,42779450,42779589,42781918,42783565,42812178],"created_at":"2025-01-20T21:00:24Z","created_at_i":1737406824,"num_comments":80,"objectID":"42772983","points":234,"story_id":42772983,"title":"Official DeepSeek R1 Now on Ollama","updated_at":"2025-10-20T20:38:54Z","url":"https://ollama.com/library/deepseek-r1"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"HenryNdubuaku"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Hey HN, Henry and Roman here - we've been building a cross-platform framework for deploying LLMs, VLMs, Embedding Models and TTS models locally on smartphones.
Ollama enables deploying LLMs models locally on laptops and edge severs, Cactus enables deploying on phones. Deploying directly on phones facilitates building AI apps and agents capable of phone use without breaking privacy, supports real-time inference with no latency, we have seen personalised RAG pipelines for users and more.
Apple and Google actively went into local AI models recently with the launch of Apple Foundation Frameworks and Google AI Edge respectively. However, both are platform-specific and only support specific models from the company. To this end, Cactus:
- Is available in Flutter, React-Native & Kotlin Multi-platform for cross-platform developers, since most apps are built with these today.
- Supports any GGUF model you can find on Huggingface; Qwen, Gemma, Llama, DeepSeek, Phi, Mistral, SmolLM, SmolVLM, InternVLM, Jan Nano etc.
- Accommodates from FP32 to as low as 2-bit quantized models, for better efficiency and less device strain.
- Have MCP tool-calls to make them performant, truly helpful (set reminder, gallery search, reply messages) and more.
- Fallback to big cloud models for complex, constrained or large-context tasks, ensuring robustness and high availability.
It's completely open source. Would love to have more people try it out and tell us how to make it great!
Repo: https://github.com/cactus-compute/cactus"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Show HN: Cactus \u2013 Ollama for Smartphones"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/cactus-compute/cactus"}},"_tags":["story","author_HenryNdubuaku","story_44524544","show_hn"],"author":"HenryNdubuaku","children":[44524769,44524805,44524812,44524873,44524965,44524990,44525158,44525242,44525270,44525273,44525324,44525807,44525814,44526013,44526027,44526188,44526192,44526590,44527019,44527020,44527318,44527324,44528566,44528773,44529079,44529772,44530017,44530312,44530591,44530738,44530897,44532973,44535678,44537399,44564462],"created_at":"2025-07-10T19:20:59Z","created_at_i":1752175259,"num_comments":82,"objectID":"44524544","points":231,"story_id":44524544,"story_text":"Hey HN, Henry and Roman here - we've been building a cross-platform framework for deploying LLMs, VLMs, Embedding Models and TTS models locally on smartphones.
Ollama enables deploying LLMs models locally on laptops and edge severs, Cactus enables deploying on phones. Deploying directly on phones facilitates building AI apps and agents capable of phone use without breaking privacy, supports real-time inference with no latency, we have seen personalised RAG pipelines for users and more.
Apple and Google actively went into local AI models recently with the launch of Apple Foundation Frameworks and Google AI Edge respectively. However, both are platform-specific and only support specific models from the company. To this end, Cactus:
- Is available in Flutter, React-Native & Kotlin Multi-platform for cross-platform developers, since most apps are built with these today.
- Supports any GGUF model you can find on Huggingface; Qwen, Gemma, Llama, DeepSeek, Phi, Mistral, SmolLM, SmolVLM, InternVLM, Jan Nano etc.
- Accommodates from FP32 to as low as 2-bit quantized models, for better efficiency and less device strain.
- Have MCP tool-calls to make them performant, truly helpful (set reminder, gallery search, reply messages) and more.
- Fallback to big cloud models for complex, constrained or large-context tasks, ensuring robustness and high availability.
It's completely open source. Would love to have more people try it out and tell us how to make it great!
Repo: https://github.com/cactus-compute/cactus","title":"Show HN: Cactus \u2013 Ollama for Smartphones","updated_at":"2026-05-15T12:28:43Z","url":"https://github.com/cactus-compute/cactus"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mchiang"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Bringing K/V context quantisation to Ollama"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://smcleod.net/2024/12/bringing-k/v-context-quantisation-to-ollama/"}},"_tags":["story","author_mchiang","story_42323953"],"author":"mchiang","children":[42324030,42324163,42324392,42324808,42329756,42334827],"created_at":"2024-12-05T01:40:30Z","created_at_i":1733362830,"num_comments":32,"objectID":"42323953","points":220,"story_id":42323953,"title":"Bringing K/V context quantisation to Ollama","updated_at":"2025-01-28T01:40:29Z","url":"https://smcleod.net/2024/12/bringing-k/v-context-quantisation-to-ollama/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"buyucu"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Llama.cpp supports Vulkan. why doesn't Ollama?"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://github.com/ollama/ollama/pull/5059"}},"_tags":["story","author_buyucu","story_42886680"],"author":"buyucu","children":[42886681,42886861,42886871,42886906,42886918,42886922,42886933,42886940,42887012,42887047,42887118,42887271,42887289,42887564,42887619,42887850,42888206,42888302,42889966,42892833,42894893,42916446],"created_at":"2025-01-31T11:30:48Z","created_at_i":1738323048,"num_comments":228,"objectID":"42886680","points":217,"story_id":42886680,"title":"Llama.cpp supports Vulkan. why doesn't Ollama?","updated_at":"2026-01-20T12:16:23Z","url":"https://github.com/ollama/ollama/pull/5059"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"kossisoroyce"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Show HN: Timber \u2013 Ollama for classical ML models, 336x faster than Python"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/kossisoroyce/timber"}},"_tags":["story","author_kossisoroyce","story_47212576","show_hn"],"author":"kossisoroyce","children":[47213239,47213467,47213507,47213614,47213671,47213724,47214198,47214612,47214831,47214906,47218145,47227711,47254775],"created_at":"2026-03-02T00:57:40Z","created_at_i":1772413060,"num_comments":33,"objectID":"47212576","points":207,"story_id":47212576,"title":"Show HN: Timber \u2013 Ollama for classical ML models, 336x faster than Python","updated_at":"2026-03-07T12:18:47Z","url":"https://github.com/kossisoroyce/timber"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Jabrov"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama violating llama.cpp license for over a year"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://github.com/ollama/ollama/issues/3185"}},"_tags":["story","author_Jabrov","story_44003741"],"author":"Jabrov","children":[44004169,44004372,44004376,44004538,44005203,44005280,44005300,44005365,44005555,44005832,44006472,44006740,44007086,44007117,44008020,44008583,44010524,44010731],"created_at":"2025-05-16T10:36:38Z","created_at_i":1747391798,"num_comments":68,"objectID":"44003741","points":202,"story_id":44003741,"title":"Ollama violating llama.cpp license for over a year","updated_at":"2026-07-19T16:38:50Z","url":"https://github.com/ollama/ollama/issues/3185"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"maxloh"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Jan \u2013 Ollama alternative with local UI"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/menloresearch/jan"}},"_tags":["story","author_maxloh","story_44845272"],"author":"maxloh","children":[44845613,44846161,44846208,44846219,44846300,44846431,44846669,44847191,44847197,44847724,44848144],"created_at":"2025-08-09T09:54:13Z","created_at_i":1754733253,"num_comments":73,"objectID":"44845272","points":197,"story_id":44845272,"title":"Jan \u2013 Ollama alternative with local UI","updated_at":"2026-06-14T18:48:43Z","url":"https://github.com/menloresearch/jan"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"alexzeitler"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama is now available as an official Docker image"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://ollama.ai/blog/ollama-is-now-available-as-an-official-docker-image"}},"_tags":["story","author_alexzeitler","story_37786525"],"author":"alexzeitler","children":[37786752,37786851,37786939,37786962,37786973,37787076,37787088,37787119,37787245,37787837,37788641,37793279,37796629,37799451],"created_at":"2023-10-06T02:18:27Z","created_at_i":1696558707,"num_comments":47,"objectID":"37786525","points":197,"story_id":37786525,"title":"Ollama is now available as an official Docker image","updated_at":"2026-01-27T14:56:27Z","url":"https://ollama.ai/blog/ollama-is-now-available-as-an-official-docker-image"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ashvardanian"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://github.com/ollama/ollama/releases/tag/v0.1.33-rc5"}},"_tags":["story","author_ashvardanian","story_40191723"],"author":"ashvardanian","children":[40192725,40192730,40193020,40193138,40194890,40195901,40197784],"created_at":"2024-04-28T20:48:42Z","created_at_i":1714337322,"num_comments":64,"objectID":"40191723","points":192,"story_id":40191723,"title":"Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B","updated_at":"2024-09-20T16:57:20Z","url":"https://github.com/ollama/ollama/releases/tag/v0.1.33-rc5"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"BUFU"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama 0.4 is released with support for Meta's Llama 3.2 Vision models locally"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://ollama.com/blog/llama3.2-vision"}},"_tags":["story","author_BUFU","story_42069453"],"author":"BUFU","children":[42070613,42070824,42070973,42071467,42071635,42074494,42074781,42074823,42075703],"created_at":"2024-11-06T21:10:39Z","created_at_i":1730927439,"num_comments":25,"objectID":"42069453","points":182,"story_id":42069453,"title":"Ollama 0.4 is released with support for Meta's Llama 3.2 Vision models locally","updated_at":"2024-12-14T00:07:21Z","url":"https://ollama.com/blog/llama3.2-vision"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"indigodaddy"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama and gguf"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://github.com/ollama/ollama/issues/11714"}},"_tags":["story","author_indigodaddy","story_44867238"],"author":"indigodaddy","children":[44867259,44868084,44869432,44869466,44869785,44870586,44871237,44871769,44872421,44872931,44873099,44873204,44874089],"created_at":"2025-08-11T17:54:08Z","created_at_i":1754934848,"num_comments":91,"objectID":"44867238","points":173,"story_id":44867238,"title":"Ollama and gguf","updated_at":"2026-03-05T22:33:20Z","url":"https://github.com/ollama/ollama/issues/11714"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jmorgan"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Hi HN,
Over the last few months I've been working with some folks on a tool named Ollama (https://github.com/jmorganca/ollama) to run open-source LLMs like Llama 2, Code Llama and Falcon locally, starting with macOS.
The biggest ask since then has been "how can I run Ollama on Linux?" with GPU support out of the box. Setting up and configuring CUDA and then compiling and running llama.cpp (which is a fantastic library and runs under the hood) can be quite painful on different combinations of linux distributions and Nvidia GPUs. The goal for Ollama's linux version was to automate this process to make it easy to get up and running.
The is the first Linux release! There's still lots to do, but I wanted to share it here for to see what everyone thinks. Thanks for anyone who has given it a try and sent feedback!"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama for Linux \u2013 Run LLMs on Linux with GPU Acceleration"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://github.com/jmorganca/ollama/releases/tag/v0.1.0"}},"_tags":["story","author_jmorgan","story_37661755"],"author":"jmorgan","children":[37662105,37662341,37662521,37662785,37662914,37662915,37663220,37663423,37663568,37663634,37663791,37663809,37663942,37664186,37666330,37668668,37670936,37767267,37767357],"created_at":"2023-09-26T16:29:56Z","created_at_i":1695745796,"num_comments":54,"objectID":"37661755","points":173,"story_id":37661755,"story_text":"Hi HN,
Over the last few months I've been working with some folks on a tool named Ollama (https://github.com/jmorganca/ollama) to run open-source LLMs like Llama 2, Code Llama and Falcon locally, starting with macOS.
The biggest ask since then has been "how can I run Ollama on Linux?" with GPU support out of the box. Setting up and configuring CUDA and then compiling and running llama.cpp (which is a fantastic library and runs under the hood) can be quite painful on different combinations of linux distributions and Nvidia GPUs. The goal for Ollama's linux version was to automate this process to make it easy to get up and running.
The is the first Linux release! There's still lots to do, but I wanted to share it here for to see what everyone thinks. Thanks for anyone who has given it a try and sent feedback!","title":"Ollama for Linux \u2013 Run LLMs on Linux with GPU Acceleration","updated_at":"2024-09-21T17:02:33Z","url":"https://github.com/jmorganca/ollama/releases/tag/v0.1.0"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"EarlyOom"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Run structured extraction on documents/images locally with Ollama and Pydantic"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/vlm-run/vlmrun-hub"}},"_tags":["story","author_EarlyOom","story_43110173"],"author":"EarlyOom","children":[43110174,43110666,43111653,43111654,43111690,43111791,43111932,43112396,43112884,43113928,43115087,43115613],"created_at":"2025-02-20T01:54:10Z","created_at_i":1740016450,"num_comments":29,"objectID":"43110173","points":170,"story_id":43110173,"title":"Run structured extraction on documents/images locally with Ollama and Pydantic","updated_at":"2025-02-27T00:53:09Z","url":"https://github.com/vlm-run/vlmrun-hub"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mezba"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/mezbaul-h/june"}},"_tags":["story","author_mezba","story_40744293","show_hn"],"author":"mezba","children":[40745394,40745755,40745806,40746596,40747469,40747609,40749183,40749503,40749837,40750696,40751084,40751264,40751802,40791106],"created_at":"2024-06-20T22:48:40Z","created_at_i":1718923720,"num_comments":26,"objectID":"40744293","points":167,"story_id":40744293,"title":"Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit","updated_at":"2024-09-20T17:16:12Z","url":"https://github.com/mezbaul-h/june"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"rldjbpin"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Finding thousands of exposed Ollama instances using Shodan"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://blogs.cisco.com/security/detecting-exposed-llm-servers-shodan-case-study-on-ollama"}},"_tags":["story","author_rldjbpin","story_45113418"],"author":"rldjbpin","children":[45114004,45114029,45114122,45114124,45114142,45114192,45114304,45114573,45114696,45114847,45114958,45115360,45115503,45116913,45117420,45117644,45118929,45125568],"created_at":"2025-09-03T08:18:07Z","created_at_i":1756887487,"num_comments":72,"objectID":"45113418","points":166,"story_id":45113418,"title":"Finding thousands of exposed Ollama instances using Shodan","updated_at":"2026-05-14T13:50:24Z","url":"https://blogs.cisco.com/security/detecting-exposed-llm-servers-shodan-case-study-on-ollama"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"rexec"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"This lets you talk to local LLMs in Apple Notes. I saw Obsidian Ollama (https://github.com/hinterdupfinger/obsidian-ollama) and thought it was handy, but I'm too lazy to migrate away from the Apple ecosystem, so I quickly hacked this together. I tend to use Notes as a scratchpad for prompts, so it's nice to do some quick inference without leaving the app.
Notes doesn't really support plugins so I'm using the macOS accessibility API for reading selections and then stream responses using the clipboard (not ideal but it works)."},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Show HN: NotesOllama \u2013 I added local LLM support to Apple Notes (through Ollama)"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://smallest.app/notesollama/"}},"_tags":["story","author_rexec","story_39456113","show_hn"],"author":"rexec","children":[39456483,39456645,39456735,39457233,39457250,39457682,39459342,39459478,39461087,39461728,39461970,39462205,39463147,39463195,39463875,39465785,39499086],"created_at":"2024-02-21T16:46:19Z","created_at_i":1708533979,"num_comments":31,"objectID":"39456113","points":156,"story_id":39456113,"story_text":"This lets you talk to local LLMs in Apple Notes. I saw Obsidian Ollama (https://github.com/hinterdupfinger/obsidian-ollama) and thought it was handy, but I'm too lazy to migrate away from the Apple ecosystem, so I quickly hacked this together. I tend to use Notes as a scratchpad for prompts, so it's nice to do some quick inference without leaving the app.
Notes doesn't really support plugins so I'm using the macOS accessibility API for reading selections and then stream responses using the clipboard (not ideal but it works).","title":"Show HN: NotesOllama \u2013 I added local LLM support to Apple Notes (through Ollama)","updated_at":"2025-06-26T19:05:00Z","url":"https://smallest.app/notesollama/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"inferhaven"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama: All Aboard Open Models"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://ollama.com/blog/all-aboard-open-models"}},"_tags":["story","author_inferhaven","story_48965880"],"author":"inferhaven","children":[48966710,48966776,48967415,48968173,48968635,48968808,48968898,48968948,48969087,48969240,48969297,48969301,48969496,48969861,48969993,48970512,48973963,48974318,48975451],"created_at":"2026-07-19T07:59:44Z","created_at_i":1784447984,"num_comments":57,"objectID":"48965880","points":138,"story_id":48965880,"title":"Ollama: All Aboard Open Models","updated_at":"2026-07-25T12:17:40Z","url":"https://ollama.com/blog/all-aboard-open-models"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"flybird"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Building a Local Perplexity Alternative with Perplexica, Ollama, and SearXNG"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://jointerminus.medium.com/building-a-local-perplexity-alternative-with-perplexica-ollama-and-searxng-71602523e256"}},"_tags":["story","author_flybird","story_41125919"],"author":"flybird","children":[41126510,41126517,41126609,41126675,41126908,41126996,41127251,41127793,41128532,41129689,41130350],"created_at":"2024-08-01T03:36:54Z","created_at_i":1722483414,"num_comments":49,"objectID":"41125919","points":134,"story_id":41125919,"title":"Building a Local Perplexity Alternative with Perplexica, Ollama, and SearXNG","updated_at":"2025-10-06T20:30:28Z","url":"https://jointerminus.medium.com/building-a-local-perplexity-alternative-with-perplexica-ollama-and-searxng-71602523e256"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tcsenpai"},"story_text":{"matchLevel":"none","matchedWords":[],"value":"Source: https://github.com/tcsenpai/spacellama"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Show HN: I made an Ollama summarizer for Firefox"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://addons.mozilla.org/en-US/firefox/addon/spacellama/"}},"_tags":["story","author_tcsenpai","story_41810507","show_hn"],"author":"tcsenpai","children":[41812624,41812744,41813283,41814310,41815472,41816189,41816225,41829546],"created_at":"2024-10-11T15:45:48Z","created_at_i":1728661548,"num_comments":33,"objectID":"41810507","points":132,"story_id":41810507,"story_text":"Source: https://github.com/tcsenpai/spacellama","title":"Show HN: I made an Ollama summarizer for Firefox","updated_at":"2024-12-15T01:12:47Z","url":"https://addons.mozilla.org/en-US/firefox/addon/spacellama/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"bswamina"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Quick Primer on MCP Using Ollama and LangChain"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.polarsparc.com/xhtml/MCP.html"}},"_tags":["story","author_bswamina","story_43676084"],"author":"bswamina","children":[43676998,43677029,43677152,43677301,43677445,43680853,43683345],"created_at":"2025-04-13T21:43:36Z","created_at_i":1744580616,"num_comments":20,"objectID":"43676084","points":131,"story_id":43676084,"title":"Quick Primer on MCP Using Ollama and LangChain","updated_at":"2025-07-19T05:33:55Z","url":"https://www.polarsparc.com/xhtml/MCP.html"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"punnerud"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Knowledge graphs using Ollama and Embeddings to answer and visualizing queries"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/punnerud/Local_Knowledge_Graph"}},"_tags":["story","author_punnerud","story_41578446"],"author":"punnerud","children":[41580489,41580563,41582326,41591563,41592307,41592609,41592656,41592722],"created_at":"2024-09-18T11:37:09Z","created_at_i":1726659429,"num_comments":9,"objectID":"41578446","points":122,"story_id":41578446,"title":"Knowledge graphs using Ollama and Embeddings to answer and visualizing queries","updated_at":"2025-08-14T22:32:02Z","url":"https://github.com/punnerud/Local_Knowledge_Graph"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"lijunhao"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama v0.1.45"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://github.com/ollama/ollama/releases/tag/v0.1.45-rc1"}},"_tags":["story","author_lijunhao","story_40692970"],"author":"lijunhao","children":[40693189,40693209,40693258,40693319,40693337,40693373,40693391,40693940,40694185,40694236,40694480,40694760,40695448,40697801,40698036],"created_at":"2024-06-15T21:54:02Z","created_at_i":1718488442,"num_comments":64,"objectID":"40692970","points":111,"story_id":40692970,"title":"Ollama v0.1.45","updated_at":"2026-07-27T11:45:01Z","url":"https://github.com/ollama/ollama/releases/tag/v0.1.45-rc1"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tosh"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama 0.1.32: WizardLM 2, Mixtral 8x22B, macOS CPU/GPU model split"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://github.com/ollama/ollama/releases/tag/v0.1.32"}},"_tags":["story","author_tosh","story_40061831"],"author":"tosh","children":[40062316,40062475,40062616,40062972,40063632,40063681,40063847,40063851,40064443,40065288],"created_at":"2024-04-17T08:15:36Z","created_at_i":1713341736,"num_comments":55,"objectID":"40061831","points":107,"story_id":40061831,"title":"Ollama 0.1.32: WizardLM 2, Mixtral 8x22B, macOS CPU/GPU model split","updated_at":"2026-01-20T17:50:04Z","url":"https://github.com/ollama/ollama/releases/tag/v0.1.32"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tosh"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Google announces Firebase Genkit with Ollama support"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://ollama.com/blog/firebase-genkit"}},"_tags":["story","author_tosh","story_40426711"],"author":"tosh","children":[40426822,40426823,40426917,40426942,40427017,40427135,40427160,40427311,40427324,40427401,40427642,40427985,40473139],"created_at":"2024-05-21T10:55:54Z","created_at_i":1716288954,"num_comments":31,"objectID":"40426711","points":102,"story_id":40426711,"title":"Google announces Firebase Genkit with Ollama support","updated_at":"2024-09-20T17:02:05Z","url":"https://ollama.com/blog/firebase-genkit"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"fangpenlin"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Nvidia on NixOS WSL \u2013 Ollama up 24/7 on your gaming PC"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://yomaq.github.io/posts/nvidia-on-nixos-wsl-ollama-up-24-7-on-your-gaming-pc/"}},"_tags":["story","author_fangpenlin","story_43649711"],"author":"fangpenlin","children":[43649734,43650426,43650428,43650440,43650612,43650673,43650681,43650771,43651067,43651283,43652018,43652239,43652267,43652380,43652547,43652594,43653127,43654505,43654908],"created_at":"2025-04-11T01:50:09Z","created_at_i":1744336209,"num_comments":49,"objectID":"43649711","points":95,"story_id":43649711,"title":"Nvidia on NixOS WSL \u2013 Ollama up 24/7 on your gaming PC","updated_at":"2025-04-24T16:58:29Z","url":"https://yomaq.github.io/posts/nvidia-on-nixos-wsl-ollama-up-24-7-on-your-gaming-pc/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Nik0912"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Exploring Microsoft's Phi-3-Mini and its integration with tool like Ollama"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://pieces.app/blog/phi-3-mini-integrations"}},"_tags":["story","author_Nik0912","story_42515238"],"author":"Nik0912","children":[42533000,42533144,42533355,42533715,42536643,42537442],"created_at":"2024-12-26T14:09:37Z","created_at_i":1735222177,"num_comments":31,"objectID":"42515238","points":84,"story_id":42515238,"title":"Exploring Microsoft's Phi-3-Mini and its integration with tool like Ollama","updated_at":"2025-08-14T22:26:12Z","url":"https://pieces.app/blog/phi-3-mini-integrations"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tymscar"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"When Curl Works but IntelliJ Doesn't: The Ollama Connection Mystery"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://blog.tymscar.com/posts/intellijollamaconnectionmystery/"}},"_tags":["story","author_tymscar","story_45477983"],"author":"tymscar","children":[45490257,45518010,45518264,45518924,45519180,45519251,45520356,45521340,45521874,45523407],"created_at":"2025-10-05T00:39:26Z","created_at_i":1759624766,"num_comments":27,"objectID":"45477983","points":82,"story_id":45477983,"title":"When Curl Works but IntelliJ Doesn't: The Ollama Connection Mystery","updated_at":"2026-03-05T22:50:25Z","url":"https://blog.tymscar.com/posts/intellijollamaconnectionmystery/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"thor-rodrigues"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama now supports tool calling with popular models in local LLM"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://ollama.com/blog/tool-support"}},"_tags":["story","author_thor-rodrigues","story_41291425"],"author":"thor-rodrigues","children":[41291831,41291861,41291866,41292029,41292366,41292694,41292856],"created_at":"2024-08-19T14:35:42Z","created_at_i":1724078142,"num_comments":24,"objectID":"41291425","points":81,"story_id":41291425,"title":"Ollama now supports tool calling with popular models in local LLM","updated_at":"2024-09-24T19:38:16Z","url":"https://ollama.com/blog/tool-support"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ibobev"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Gemma, Ollama and LangChainGo"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://eli.thegreenplace.net/2024/gemma-ollama-and-langchaingo/"}},"_tags":["story","author_ibobev","story_39480398"],"author":"ibobev","children":[39501132,39501356,39501743,39502293,39502861,39505341],"created_at":"2024-02-23T13:37:25Z","created_at_i":1708695445,"num_comments":22,"objectID":"39480398","points":79,"story_id":39480398,"title":"Gemma, Ollama and LangChainGo","updated_at":"2024-11-20T22:49:26Z","url":"https://eli.thegreenplace.net/2024/gemma-ollama-and-langchaingo/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"sgreenlay"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama-Swift"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://nshipster.com/ollama/"}},"_tags":["story","author_sgreenlay","story_43065006"],"author":"sgreenlay","children":[43065224,43065430,43065468,43065602,43065969,43067334,43067454],"created_at":"2025-02-16T03:19:48Z","created_at_i":1739675988,"num_comments":16,"objectID":"43065006","points":66,"story_id":43065006,"title":"Ollama-Swift","updated_at":"2025-03-04T20:22:01Z","url":"https://nshipster.com/ollama/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"heresie-dabord"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"175K+ publicly-exposed Ollama AI instances discovered"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://www.techradar.com/pro/security/over-175-000-publicly-exposed-ollama-ai-servers-discovered-worldwide-so-fix-now"}},"_tags":["story","author_heresie-dabord","story_46831784"],"author":"heresie-dabord","children":[46831833,46831884,46832127,46832246,46832422,46832464,46832602,46832728,46833360,46833670,46833746,46833765],"created_at":"2026-01-31T00:12:22Z","created_at_i":1769818342,"num_comments":38,"objectID":"46831784","points":65,"story_id":46831784,"title":"175K+ publicly-exposed Ollama AI instances discovered","updated_at":"2026-03-05T23:27:33Z","url":"https://www.techradar.com/pro/security/over-175-000-publicly-exposed-ollama-ai-servers-discovered-worldwide-so-fix-now"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tosh"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"ollama launch"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://ollama.com/blog/launch"}},"_tags":["story","author_tosh","story_46745138"],"author":"tosh","children":[46757594],"created_at":"2026-01-24T16:52:46Z","created_at_i":1769273566,"num_comments":3,"objectID":"46745138","points":56,"story_id":46745138,"title":"ollama launch","updated_at":"2026-07-19T15:21:19Z","url":"https://ollama.com/blog/launch"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mdp2021"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Ollama Models Atom Feed"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"https://simonwillison.net/2025/Mar/22/ollama-models-atom-feed/"}},"_tags":["story","author_mdp2021","story_43451579"],"author":"mdp2021","children":[43451615,43451633],"created_at":"2025-03-23T08:32:40Z","created_at_i":1742718760,"num_comments":12,"objectID":"43451579","points":50,"story_id":43451579,"title":"Ollama Models Atom Feed","updated_at":"2025-03-29T11:56:35Z","url":"https://simonwillison.net/2025/Mar/22/ollama-models-atom-feed/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"atjamielittle"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["ollama"],"value":"Show HN: Elelem, a tool-calling CLI for Ollama and DeepSeek in C"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://codeberg.org/politebot/elelem"}},"_tags":["story","author_atjamielittle","story_44378254","show_hn"],"author":"atjamielittle","children":[44380552],"created_at":"2025-06-25T15:10:52Z","created_at_i":1750864252,"num_comments":3,"objectID":"44378254","points":45,"story_id":44378254,"title":"Show HN: Elelem, a tool-calling CLI for Ollama and DeepSeek in C","updated_at":"2025-06-28T22:28:22Z","url":"https://codeberg.org/politebot/elelem"}],"hitsPerPage":50,"nbHits":4385,"nbPages":20,"page":0,"params":"query=Ollama&tags=story&hitsPerPage=50&advancedSyntax=true&analyticsTags=backend","processingTimeMS":7,"processingTimingsMS":{"_request":{"roundTrip":19},"afterFetch":{"format":{"highlighting":1,"total":2},"merge":{"mergeLoop":{"prepareNextHit":1,"total":1},"total":1},"total":1},"fetch":{"query":2,"scanning":2,"total":5},"total":7},"query":"Ollama","serverTimeMS":10}