The upstream firecrawl repo contains the following blurb:
>This repository is in development, and we're still integrating custom modules into the mono repo. It's not fully ready for self-hosted deployment yet, but you can run it locally.
Firecrawl's API surface and general functionality were ideal for our Trieve sitesearch product, but we needed a version ready for self-hosting that was easy to contribute to and scale on Kubernetes. Therefore, we decided to fork and begin maintaining a stripped down, stable version.
Fire-engine, Firecrawl's solution for anti-bot pages, being closed source is the biggest deal breaker requiring us to maintain this fork. Further, our purposes not requiring the SaaS and AI dependencies also pushes our use-case far enough away from Firecrawl's current mission that it doesn't seem like merging into the upstream is viable at this time."},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Show HN: Firecrawl-Simple \u2013 Stable fork of Firecrawl optimized for self-hosting"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"https://github.com/devflowinc/firecrawl-simple"}},"_tags":["story","author_skeptrune","story_42056056","show_hn"],"author":"skeptrune","children":[42056925,42056970,42086482],"created_at":"2024-11-05T23:18:38Z","created_at_i":1730848718,"num_comments":7,"objectID":"42056056","points":35,"story_id":42056056,"story_text":"Firecrawl Simple is a stripped down and stable version of firecrawl optimized for self-hosting and ease of contribution.
The upstream firecrawl repo contains the following blurb:
>This repository is in development, and we're still integrating custom modules into the mono repo. It's not fully ready for self-hosted deployment yet, but you can run it locally.
Firecrawl's API surface and general functionality were ideal for our Trieve sitesearch product, but we needed a version ready for self-hosting that was easy to contribute to and scale on Kubernetes. Therefore, we decided to fork and begin maintaining a stripped down, stable version.
Fire-engine, Firecrawl's solution for anti-bot pages, being closed source is the biggest deal breaker requiring us to maintain this fork. Further, our purposes not requiring the SaaS and AI dependencies also pushes our use-case far enough away from Firecrawl's current mission that it doesn't seem like merging into the upstream is viable at this time.","title":"Show HN: Firecrawl-Simple \u2013 Stable fork of Firecrawl optimized for self-hosting","updated_at":"2026-03-11T14:26:17Z","url":"https://github.com/devflowinc/firecrawl-simple"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ccheshirecat"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Scraping modern websites has become a massive headache. You basically have two choices: pay for an expensive API like Firecrawl/Browserbase, or run a fleet of headless Chrome instances that eat 1GB of RAM per page and still get blocked by Cloudflare.
I built Draco to fix this. It\u2019s a fast, single-binary web scraper written in Rust. You point it at a URL, and it spits out perfectly clean Markdown or structured JSON for LLMs.
The secret sauce is that it doesn't just boot a browser for every request. It uses a tiered escalation engine:
Tier 1 (Stealth Fetch): Draco uses a custom TLS/JA4 fingerprint to perfectly mimic a real browser's network signature at the packet level. It turns out a lot of anti-bot walls will let you right through if your handshake looks correct. In my benchmarks against sites like Cloudflare and Target, Playwright ate ~500MB of RAM and timed out. Draco bypassed them in under a second using just 20MB of RAM.
Tier 2 (V8 Isolate): If it hits a React/Next.js SPA that needs rendering, Draco boots an in-process V8 engine in single-digit milliseconds. It hydrates the DOM and intercepts the hidden JSON APIs the page is calling\u2014giving you the raw data without the overhead of a graphical browser.
Tier 3 (Real Browser): If it hits an absolute wall, it seamlessly falls back to detecting and driving a real browser on your machine.
I also built in all the tooling to make it a complete drop-in replacement for the hosted services:
Daemon Mode: Run draco serve and you get a persistent HTTP server with a Firecrawl-compatible REST API. You can swap out your API keys and self-host immediately.
Built-in MCP Server: It natively exposes a Model Context Protocol server so you can plug it directly into Claude Desktop or your AI agents.
Web Search: Built-in parallel multi-engine web search (bypassing the need for a Google Search API key).
Interact Mode: Drive a page statefully like a devtools console, persisting cookies across navigations(for LLM's mainly).
It\u2019s completely open source (MIT/Apache-2.0). I just wanted to put this out there for anyone tired of fighting headless Chromium or paying per-page scraping costs. Grab the binary and throw a difficult URL at it.
Note that it's still a WIP so there might be some unexpected breakages of uncommon sites but for the most part its quite capable, it can handle cf-protected sites and heavy SPA's while everything else fails partially or completely while taking longer or more resources. (tested on example.com, hackernews, cloudflare, glassdoor, bluff.com, target.com, stake.com and thrill.com)
\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 Rank \u2503 Tool \u2503 Score \u2503 Pass \u2503 Avg Time \u2503 Avg RAM \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 #1 \u2502 Draco \u2502 769.7 \u2502 8/8 \u2502 3.45 \u2502 216.50 \u2502\n\u2502 #2 \u2502 Obscura \u2502 384.5 \u2502 4/8 \u2502 2.68 \u2502 87.59 \u2502\n\u2502 #3 \u2502 BrowserOxide \u2502 373.4 \u2502 4/8 \u2502 6.42 \u2502 105.95 \u2502\n\u2502 #4 \u2502 Playwright \u2502 342.2 \u2502 4/8 \u2502 1.71 \u2502 535.07 \u2502\n\u2502 #5 \u2502 Bouncy \u2502 196.6 \u2502 2/8 \u2502 0.59 \u2502 19.38 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518
Repo: https://github.com/0xchasercat/draco/"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Show HN: Draco \u2013 A single-binary, self-hostable Firecrawl alternative in Rust"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/0xchasercat/draco/"}},"_tags":["story","author_ccheshirecat","story_49148163","show_hn"],"author":"ccheshirecat","children":[49148543,49148766,49149597,49151239,49153128,49165072,49170009,49220463,49242878],"created_at":"2026-08-02T20:48:45Z","created_at_i":1785703725,"num_comments":10,"objectID":"49148163","points":15,"story_id":49148163,"story_text":"Scraping modern websites has become a massive headache. You basically have two choices: pay for an expensive API like Firecrawl/Browserbase, or run a fleet of headless Chrome instances that eat 1GB of RAM per page and still get blocked by Cloudflare.
I built Draco to fix this. It\u2019s a fast, single-binary web scraper written in Rust. You point it at a URL, and it spits out perfectly clean Markdown or structured JSON for LLMs.
The secret sauce is that it doesn't just boot a browser for every request. It uses a tiered escalation engine:
Tier 1 (Stealth Fetch): Draco uses a custom TLS/JA4 fingerprint to perfectly mimic a real browser's network signature at the packet level. It turns out a lot of anti-bot walls will let you right through if your handshake looks correct. In my benchmarks against sites like Cloudflare and Target, Playwright ate ~500MB of RAM and timed out. Draco bypassed them in under a second using just 20MB of RAM.
Tier 2 (V8 Isolate): If it hits a React/Next.js SPA that needs rendering, Draco boots an in-process V8 engine in single-digit milliseconds. It hydrates the DOM and intercepts the hidden JSON APIs the page is calling\u2014giving you the raw data without the overhead of a graphical browser.
Tier 3 (Real Browser): If it hits an absolute wall, it seamlessly falls back to detecting and driving a real browser on your machine.
I also built in all the tooling to make it a complete drop-in replacement for the hosted services:
Daemon Mode: Run draco serve and you get a persistent HTTP server with a Firecrawl-compatible REST API. You can swap out your API keys and self-host immediately.
Built-in MCP Server: It natively exposes a Model Context Protocol server so you can plug it directly into Claude Desktop or your AI agents.
Web Search: Built-in parallel multi-engine web search (bypassing the need for a Google Search API key).
Interact Mode: Drive a page statefully like a devtools console, persisting cookies across navigations(for LLM's mainly).
It\u2019s completely open source (MIT/Apache-2.0). I just wanted to put this out there for anyone tired of fighting headless Chromium or paying per-page scraping costs. Grab the binary and throw a difficult URL at it.
Note that it's still a WIP so there might be some unexpected breakages of uncommon sites but for the most part its quite capable, it can handle cf-protected sites and heavy SPA's while everything else fails partially or completely while taking longer or more resources. (tested on example.com, hackernews, cloudflare, glassdoor, bluff.com, target.com, stake.com and thrill.com)
\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 Rank \u2503 Tool \u2503 Score \u2503 Pass \u2503 Avg Time \u2503 Avg RAM \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 #1 \u2502 Draco \u2502 769.7 \u2502 8/8 \u2502 3.45 \u2502 216.50 \u2502\n\u2502 #2 \u2502 Obscura \u2502 384.5 \u2502 4/8 \u2502 2.68 \u2502 87.59 \u2502\n\u2502 #3 \u2502 BrowserOxide \u2502 373.4 \u2502 4/8 \u2502 6.42 \u2502 105.95 \u2502\n\u2502 #4 \u2502 Playwright \u2502 342.2 \u2502 4/8 \u2502 1.71 \u2502 535.07 \u2502\n\u2502 #5 \u2502 Bouncy \u2502 196.6 \u2502 2/8 \u2502 0.59 \u2502 19.38 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518
Repo: https://github.com/0xchasercat/draco/","title":"Show HN: Draco \u2013 A single-binary, self-hostable Firecrawl alternative in Rust","updated_at":"2026-08-13T19:54:35Z","url":"https://github.com/0xchasercat/draco/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ulrischa"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Firecrawl: Turn entire websites into LLM-ready Markdown or structured data"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"https://github.com/mendableai/firecrawl"}},"_tags":["story","author_ulrischa","story_40404615"],"author":"ulrischa","children":[40404790,40405080],"created_at":"2024-05-19T06:11:55Z","created_at_i":1716099115,"num_comments":2,"objectID":"40404615","points":4,"story_id":40404615,"title":"Firecrawl: Turn entire websites into LLM-ready Markdown or structured data","updated_at":"2024-09-20T16:59:26Z","url":"https://github.com/mendableai/firecrawl"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"pipboyguy"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"I was paying about $190/month for Firecrawl. It worked, but it was expensive and felt half baked. The CLI was awkward to use. Maybe I wasn't using it right, but the whole thing felt like it was in flux. I figured if they vibe coded it I might as well do it myself.
So I vibed my own. 3k lines of Elixir, custom readability engine, no external services. It even manages, for some sites at least, to evade bot protection.
It saves a lot of cash and helps pay the Claude Code Max subscription.
I guess my realization is that you shouldn't just take tooling aimed at AI agents for granted. Code is a commodity now, so a price tag for an MCP tool nearly as much as the most expensive subscription tiers for Claude Code/Codex makes no sense to me.
Just ask your agent to code 80% of the tool for yourself."},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"I replaced Firecrawl with 2,700 lines of Elixir"}},"_tags":["story","author_pipboyguy","story_47578663","ask_hn"],"author":"pipboyguy","children":[47579042,47606805],"created_at":"2026-03-30T19:30:29Z","created_at_i":1774899029,"num_comments":2,"objectID":"47578663","points":3,"story_id":47578663,"story_text":"I was paying about $190/month for Firecrawl. It worked, but it was expensive and felt half baked. The CLI was awkward to use. Maybe I wasn't using it right, but the whole thing felt like it was in flux. I figured if they vibe coded it I might as well do it myself.
So I vibed my own. 3k lines of Elixir, custom readability engine, no external services. It even manages, for some sites at least, to evade bot protection.
It saves a lot of cash and helps pay the Claude Code Max subscription.
I guess my realization is that you shouldn't just take tooling aimed at AI agents for granted. Code is a commodity now, so a price tag for an MCP tool nearly as much as the most expensive subscription tiers for Claude Code/Codex makes no sense to me.
Just ask your agent to code 80% of the tool for yourself.","title":"I replaced Firecrawl with 2,700 lines of Elixir","updated_at":"2026-04-06T03:14:32Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"fastcrw"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Show HN: FastCRW: 6MB binary self-hosted Firecrawl alternative"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/us/crw"}},"_tags":["story","author_fastcrw","story_48819736","show_hn"],"author":"fastcrw","children":[48819991,48820052],"created_at":"2026-07-07T16:05:41Z","created_at_i":1783440341,"num_comments":1,"objectID":"48819736","points":3,"story_id":48819736,"title":"Show HN: FastCRW: 6MB binary self-hosted Firecrawl alternative","updated_at":"2026-07-07T17:09:24Z","url":"https://github.com/us/crw"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mariusbolik"},"story_text":{"matchLevel":"none","matchedWords":[],"value":"It is early stage. Feedback is highly appreciated!"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Show HN: Extractor.sh \u2013 simple, affordable Firecrawl alternative"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://extractor.sh/"}},"_tags":["story","author_mariusbolik","story_49174141","show_hn"],"author":"mariusbolik","created_at":"2026-08-04T19:55:10Z","created_at_i":1785873310,"num_comments":0,"objectID":"49174141","points":3,"story_id":49174141,"story_text":"It is early stage. Feedback is highly appreciated!","title":"Show HN: Extractor.sh \u2013 simple, affordable Firecrawl alternative","updated_at":"2026-08-05T17:31:21Z","url":"https://extractor.sh/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"maclarens"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"I\u2019ve noticed that Firecrawl struggles with certain sites that do better with headful browsers. So we made Teracrawl (https://github.com/BrowserCash/teracrawl), which is an open source crawler built on a headful chrome browser API. (Browser.cash)
I tested a few URLs that are very JS-heavy and anti-bot-ish:
1. Yahoo Finance \u2013 Currencies\nhttps://finance.yahoo.com/currencies
Teracrawl: got the full currencies table (it actually ran the React app and hit the live pricing API).
Firecrawl: only got the header/nav shell, no Forex table at all. Was getting completely blocked by Yahoo.
2. AT&T wireless phones\nhttps://www.att.com/buy/wireless/phones
Firecrawl: just the AT&T header + basic chrome, no phones catalog.
Teracrawl: full phones list + pricing, filters, etc., since it executed the React frontend and fetched the JSON endpoints.
3. GoDaddy domain search\nhttps://www.godaddy.com/en-ca/domainsearch/find?\u2026domainToCheck=mydomain.io
Firecrawl: static shell (nav + footer), zero search results. Again, getting blocked.
Teracrawl: full domain search output \u2013 availability, pricing, premium/upsell info, all the JS/API-loaded stuff.
Firecrawl seems fine for static content, but on JS-heavy pages or headful checks, it often just gives me skeleton HTML or is blocked.
Teracrawl, because it\u2019s running real headful Chrome, behaves much more like an actual user browser and consistently returned dynamic content and was blocked less.
Firecrawl isn\u2019t bad overall, but for certain sites using a headful browser wins out."},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Firecrawl getting blocked due to headlesness"}},"_tags":["story","author_maclarens","story_46149717","ask_hn"],"author":"maclarens","created_at":"2025-12-04T16:51:46Z","created_at_i":1764867106,"num_comments":0,"objectID":"46149717","points":3,"story_id":46149717,"story_text":"I\u2019ve noticed that Firecrawl struggles with certain sites that do better with headful browsers. So we made Teracrawl (https://github.com/BrowserCash/teracrawl), which is an open source crawler built on a headful chrome browser API. (Browser.cash)
I tested a few URLs that are very JS-heavy and anti-bot-ish:
1. Yahoo Finance \u2013 Currencies\nhttps://finance.yahoo.com/currencies
Teracrawl: got the full currencies table (it actually ran the React app and hit the live pricing API).
Firecrawl: only got the header/nav shell, no Forex table at all. Was getting completely blocked by Yahoo.
2. AT&T wireless phones\nhttps://www.att.com/buy/wireless/phones
Firecrawl: just the AT&T header + basic chrome, no phones catalog.
Teracrawl: full phones list + pricing, filters, etc., since it executed the React frontend and fetched the JSON endpoints.
3. GoDaddy domain search\nhttps://www.godaddy.com/en-ca/domainsearch/find?\u2026domainToCheck=mydomain.io
Firecrawl: static shell (nav + footer), zero search results. Again, getting blocked.
Teracrawl: full domain search output \u2013 availability, pricing, premium/upsell info, all the JS/API-loaded stuff.
Firecrawl seems fine for static content, but on JS-heavy pages or headful checks, it often just gives me skeleton HTML or is blocked.
Teracrawl, because it\u2019s running real headful Chrome, behaves much more like an actual user browser and consistently returned dynamic content and was blocked less.
Firecrawl isn\u2019t bad overall, but for certain sites using a headful browser wins out.","title":"Firecrawl getting blocked due to headlesness","updated_at":"2026-03-05T23:07:23Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"davidbarker"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Firecrawl Extract \u2013 Turn entire websites into structured data with AI"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"https://www.firecrawl.dev/extract"}},"_tags":["story","author_davidbarker","story_42766862"],"author":"davidbarker","created_at":"2025-01-20T09:47:53Z","created_at_i":1737366473,"num_comments":0,"objectID":"42766862","points":3,"story_id":42766862,"title":"Firecrawl Extract \u2013 Turn entire websites into structured data with AI","updated_at":"2025-01-20T16:35:16Z","url":"https://www.firecrawl.dev/extract"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"janandonly"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Firecrawl: Turn entire websites into LLM-ready Markdown or structured data"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"https://github.com/mendableai/firecrawl"}},"_tags":["story","author_janandonly","story_41755936"],"author":"janandonly","created_at":"2024-10-06T09:33:15Z","created_at_i":1728207195,"num_comments":0,"objectID":"41755936","points":3,"story_id":41755936,"title":"Firecrawl: Turn entire websites into LLM-ready Markdown or structured data","updated_at":"2024-10-09T11:54:41Z","url":"https://github.com/mendableai/firecrawl"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"nickca"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Tutorial: Extracting structured data from websites using Groq and Firecrawl"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"https://github.com/mendableai/firecrawl/blob/main/tutorials/data-extraction-using-llms.mdx"}},"_tags":["story","author_nickca","story_40118466"],"author":"nickca","created_at":"2024-04-22T19:47:11Z","created_at_i":1713815231,"num_comments":0,"objectID":"40118466","points":3,"story_id":40118466,"title":"Tutorial: Extracting structured data from websites using Groq and Firecrawl","updated_at":"2024-09-20T16:50:16Z","url":"https://github.com/mendableai/firecrawl/blob/main/tutorials/data-extraction-using-llms.mdx"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"artur_makly"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Firecrawl Raises $75M Series-B"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"https://www.firecrawl.dev/blog/firecrawl-v2-series-a-announcement"}},"_tags":["story","author_artur_makly","story_49804020"],"author":"artur_makly","children":[49808533],"created_at":"2026-09-22T16:36:21Z","created_at_i":1790094981,"num_comments":1,"objectID":"49804020","points":2,"story_id":49804020,"title":"Firecrawl Raises $75M Series-B","updated_at":"2026-09-22T21:45:38Z","url":"https://www.firecrawl.dev/blog/firecrawl-v2-series-a-announcement"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"gadr90"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Using Perplexity, Firecrawl and Gemini Flash to analyze 305 Links for 12.70 USD"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://vibegui.com/article/shipping-vibegui-bookmarks-v1-architecture-costs-and-lessons"}},"_tags":["story","author_gadr90","story_46438657"],"author":"gadr90","children":[46438658],"created_at":"2025-12-30T22:13:53Z","created_at_i":1767132833,"num_comments":1,"objectID":"46438657","points":2,"story_id":46438657,"title":"Using Perplexity, Firecrawl and Gemini Flash to analyze 305 Links for 12.70 USD","updated_at":"2026-03-05T23:16:20Z","url":"https://vibegui.com/article/shipping-vibegui-bookmarks-v1-architecture-costs-and-lessons"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"thm"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Firecrawl Alexandria"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"https://www.firecrawl.dev/alexandria"}},"_tags":["story","author_thm","story_49803753"],"author":"thm","created_at":"2026-09-22T16:19:51Z","created_at_i":1790093991,"num_comments":0,"objectID":"49803753","points":2,"story_id":49803753,"title":"Firecrawl Alexandria","updated_at":"2026-09-22T16:35:32Z","url":"https://www.firecrawl.dev/alexandria"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Vanshfin"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Clawbot installs Firecrawl CLI, asks for key"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://clawsifyai.com"}},"_tags":["story","author_Vanshfin","story_47404890"],"author":"Vanshfin","children":[47404891],"created_at":"2026-03-16T21:06:59Z","created_at_i":1773695219,"num_comments":0,"objectID":"47404890","points":2,"story_id":47404890,"title":"Clawbot installs Firecrawl CLI, asks for key","updated_at":"2026-03-19T11:49:33Z","url":"https://clawsifyai.com"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"edshiro"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Firecrawl: Convert any website into LLM-ready Markdown"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"https://github.com/mendableai/firecrawl"}},"_tags":["story","author_edshiro","story_40062139"],"author":"edshiro","created_at":"2024-04-17T09:02:25Z","created_at_i":1713344545,"num_comments":0,"objectID":"40062139","points":2,"story_id":40062139,"title":"Firecrawl: Convert any website into LLM-ready Markdown","updated_at":"2024-09-20T16:50:47Z","url":"https://github.com/mendableai/firecrawl"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"rachpradhan"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"I built Pathik, a blazing-fast Firecrawl replacement, at a hackathon in under an hour and am now open-sourcing it!
What is Pathik?
\u2022 A high-performance web crawler written in Golang with Pythonic bindings
\u2022 Uses 134x less memory than Playwright (thanks to Rod)
\u2022 Supports proxies
\u2022 Outputs clean Markdown, making it LLM-friendly for fine-tuning & retrieval use cases
Why does this matter?
\u2022 Avoids bloated browser automation
\u2022 Ideal for structured web crawling for LLMs, RAG pipelines, and knowledge extraction
\u2022 Simple pip install pathik and you\u2019re ready to go
Try it out
pip install pathik
Example script:
github.com/justrach/pathik/blob/main/examples/example.py
If you find it useful, drop a : github.com/justrach/pathik
Would love feedback & contributions!"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Show HN: Pathik \u2013 Low-Memory, Fast Firecrawl Alternative for LLM Markdown"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/justrach/pathik"}},"_tags":["story","author_rachpradhan","story_43242095","show_hn"],"author":"rachpradhan","created_at":"2025-03-03T14:32:51Z","created_at_i":1741012371,"num_comments":0,"objectID":"43242095","points":1,"story_id":43242095,"story_text":"I built Pathik, a blazing-fast Firecrawl replacement, at a hackathon in under an hour and am now open-sourcing it!
What is Pathik?
\u2022 A high-performance web crawler written in Golang with Pythonic bindings
\u2022 Uses 134x less memory than Playwright (thanks to Rod)
\u2022 Supports proxies
\u2022 Outputs clean Markdown, making it LLM-friendly for fine-tuning & retrieval use cases
Why does this matter?
\u2022 Avoids bloated browser automation
\u2022 Ideal for structured web crawling for LLMs, RAG pipelines, and knowledge extraction
\u2022 Simple pip install pathik and you\u2019re ready to go
Try it out
pip install pathik
Example script:
github.com/justrach/pathik/blob/main/examples/example.py
If you find it useful, drop a : github.com/justrach/pathik
Would love feedback & contributions!","title":"Show HN: Pathik \u2013 Low-Memory, Fast Firecrawl Alternative for LLM Markdown","updated_at":"2025-03-03T14:37:57Z","url":"https://github.com/justrach/pathik"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"nuggtwriter"},"story_text":{"matchLevel":"none","matchedWords":[],"value":"Nuggt helps you gather, organize, and store information to create reusable contexts for large language models (LLMs). Please checkout the Demo Video to better understand how to use the tool (it can be a bit complicated, I am working to improve the UI/UX)
Search & Research\nGather info on any topic from the internet with the search feature.\nExample: Search "How to make a good elevator pitch" and explore a detailed research tree.
Context Creation\nSave important info to reusable contexts by highlighting and adding to the console. Use these to enrich chatbot conversations.
Chatbot with Contexts\nSelect a context to include in your conversations with the AI. Makes responses smarter and more relevant!
Beautify Graphs\nAuto-arrange your research nodes with a single click for easy navigation."},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Show HN: Go from 0 to 1 on any topic instantly (made with firecrawl)"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/shoibloya/nuggt-research"}},"_tags":["story","author_nuggtwriter","story_42790871","show_hn"],"author":"nuggtwriter","created_at":"2025-01-22T09:31:05Z","created_at_i":1737538265,"num_comments":0,"objectID":"42790871","points":1,"story_id":42790871,"story_text":"Nuggt helps you gather, organize, and store information to create reusable contexts for large language models (LLMs). Please checkout the Demo Video to better understand how to use the tool (it can be a bit complicated, I am working to improve the UI/UX)
Search & Research\nGather info on any topic from the internet with the search feature.\nExample: Search "How to make a good elevator pitch" and explore a detailed research tree.
Context Creation\nSave important info to reusable contexts by highlighting and adding to the console. Use these to enrich chatbot conversations.
Chatbot with Contexts\nSelect a context to include in your conversations with the AI. Makes responses smarter and more relevant!
Beautify Graphs\nAuto-arrange your research nodes with a single click for easy navigation.","title":"Show HN: Go from 0 to 1 on any topic instantly (made with firecrawl)","updated_at":"2025-01-22T09:36:22Z","url":"https://github.com/shoibloya/nuggt-research"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"skeptrune"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Build Search and RAG for Any Website with Firecrawl and Trieve"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"https://trieve.ai/firecrawl-and-trieve/"}},"_tags":["story","author_skeptrune","story_41323436"],"author":"skeptrune","created_at":"2024-08-22T19:03:44Z","created_at_i":1724353424,"num_comments":0,"objectID":"41323436","points":1,"story_id":41323436,"title":"Build Search and RAG for Any Website with Firecrawl and Trieve","updated_at":"2024-09-20T17:37:01Z","url":"https://trieve.ai/firecrawl-and-trieve/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ericciarla"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Hey HN! It\u2019s Eric from Firecrawl (https://firecrawl.dev).
I just launched llms.txt Generator, a tool that transforms any website into a clean, structured text file optimized for feeding to LLMs. You can learn more about the standard at https://llmstxt.org.
Here\u2019s how it works under the hood:
1. We use Firecrawl, our open-source scraper, to fetch the full site, handling JavaScript-heavy pages and complex structures.\n2. The markdown content is parsed and then the title and description are extracted using GPT-4o-mini.\n3. The everything is combined and the result is a lightweight llms.txt file that you can paste into any LLM.
Let me know what you think!"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Llms.txt Generator \u2013 Turn websites into a text file to feed to any LLM"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"https://llmstxt.firecrawl.dev/"}},"_tags":["story","author_ericciarla","story_42207756","show_hn"],"author":"ericciarla","children":[42214547,42214869,42215031,42217413],"created_at":"2024-11-21T19:33:23Z","created_at_i":1732217603,"num_comments":6,"objectID":"42207756","points":28,"story_id":42207756,"story_text":"Hey HN! It\u2019s Eric from Firecrawl (https://firecrawl.dev).
I just launched llms.txt Generator, a tool that transforms any website into a clean, structured text file optimized for feeding to LLMs. You can learn more about the standard at https://llmstxt.org.
Here\u2019s how it works under the hood:
1. We use Firecrawl, our open-source scraper, to fetch the full site, handling JavaScript-heavy pages and complex structures.\n2. The markdown content is parsed and then the title and description are extracted using GPT-4o-mini.\n3. The everything is combined and the result is a lightweight llms.txt file that you can paste into any LLM.
Let me know what you think!","title":"Show HN: Llms.txt Generator \u2013 Turn websites into a text file to feed to any LLM","updated_at":"2024-11-25T19:29:51Z","url":"https://llmstxt.firecrawl.dev/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"kalil0321"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"I built a live interactive map that shows where top AI companies hire around the world. \nI collected this data for a hackathon project. Many ATS providers have a public API that you can hit with the slug of the companies to get open jobs. The hardest part was finding the companies. I tried Firecrawl but it returned around 200 companies per provider which wasn\u2019t enough for me. Then, I tried SERPAPI but it was expensive. I ended up using SearXNG to discover companies by ATS type and fetch their job postings. This produced a large dataset of 200k+ jobs (I only use a subset as it would have taken too much time processing). A few days ago, I decided to build a visualization of the data as I didn\u2019t know what to do with it and wanted people to benefit.
I kept catching myself wanting to ask simple questions like \u201cshow only research roles in Europe\u201d or \u201cfilter for remote SWE positions\u201d (and had plenty of free ai credits) so I added a small LLM interface that translates natural language into filters on the map.
The map is built with Vite + React + Mapbox.\nLive demo: https://map.stapply.ai GitHub (data): https://github.com/stapply-ai/jobs
Would love feedback, ideas for improvement, or contributions."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: I built an interactive map of jobs at top AI companies"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://map.stapply.ai"}},"_tags":["story","author_kalil0321","story_46037065","show_hn"],"author":"kalil0321","children":[46038400,46045817,46055558],"created_at":"2025-11-24T18:08:41Z","created_at_i":1764007721,"num_comments":5,"objectID":"46037065","points":16,"story_id":46037065,"story_text":"I built a live interactive map that shows where top AI companies hire around the world. \nI collected this data for a hackathon project. Many ATS providers have a public API that you can hit with the slug of the companies to get open jobs. The hardest part was finding the companies. I tried Firecrawl but it returned around 200 companies per provider which wasn\u2019t enough for me. Then, I tried SERPAPI but it was expensive. I ended up using SearXNG to discover companies by ATS type and fetch their job postings. This produced a large dataset of 200k+ jobs (I only use a subset as it would have taken too much time processing). A few days ago, I decided to build a visualization of the data as I didn\u2019t know what to do with it and wanted people to benefit.
I kept catching myself wanting to ask simple questions like \u201cshow only research roles in Europe\u201d or \u201cfilter for remote SWE positions\u201d (and had plenty of free ai credits) so I added a small LLM interface that translates natural language into filters on the map.
The map is built with Vite + React + Mapbox.\nLive demo: https://map.stapply.ai GitHub (data): https://github.com/stapply-ai/jobs
Would love feedback, ideas for improvement, or contributions.","title":"Show HN: I built an interactive map of jobs at top AI companies","updated_at":"2026-03-05T23:06:44Z","url":"https://map.stapply.ai"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mifydev"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Hi HN! I\u2019m Ivan, one of the founders of Sourcewizard.
It\u2019s a CLI tool that works with AI coding agents (like Cursor and Claude) to install and set up SDKs correctly including middleware, pages, env vars, everything.
Similar to the PostHog Install AI Wizard: https://posthog.com/docs/ai-engineering/ai-wizard\nBut for more packages.
Run this to try it:
npx ai-setup clerk
We built it after watching agents fail basic installs.\nWrong packages, deprecated methods, half-finished setups.
Sourcewizard uses package-specific prompts that complete clean installs ~90% of the time on Next.js apps.
Currently we support:
A Few Auth Providers: Clerk and WorkOS
Search APIs, Emails and Notifications: Firecrawl, Resend, Knock
Quickstart: https://docs.sourcewizard.ai/quickstart
More context: https://sourcewizard.ai/blog/why-your-sdk-breaks-in-cursor
Notes:\n- The client code is open source: https://github.com/sourcewizard-ai/sourcewizard\n- We explicitly ignore existing Env Files: https://github.com/sourcewizard-ai/sourcewizard/blob/main/in...
Would love feedback!
\u2014 Lukas & Ivan"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Sourcewizard \u2013 AI installs SDKs in your codebase"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://sourcewizard.ai"}},"_tags":["story","author_mifydev","story_45987702","show_hn"],"author":"mifydev","children":[45987713,45987756,45987808,45988092,45988958],"created_at":"2025-11-20T01:33:07Z","created_at_i":1763602387,"num_comments":23,"objectID":"45987702","points":15,"story_id":45987702,"story_text":"Hi HN! I\u2019m Ivan, one of the founders of Sourcewizard.
It\u2019s a CLI tool that works with AI coding agents (like Cursor and Claude) to install and set up SDKs correctly including middleware, pages, env vars, everything.
Similar to the PostHog Install AI Wizard: https://posthog.com/docs/ai-engineering/ai-wizard\nBut for more packages.
Run this to try it:
npx ai-setup clerk
We built it after watching agents fail basic installs.\nWrong packages, deprecated methods, half-finished setups.
Sourcewizard uses package-specific prompts that complete clean installs ~90% of the time on Next.js apps.
Currently we support:
A Few Auth Providers: Clerk and WorkOS
Search APIs, Emails and Notifications: Firecrawl, Resend, Knock
Quickstart: https://docs.sourcewizard.ai/quickstart
More context: https://sourcewizard.ai/blog/why-your-sdk-breaks-in-cursor
Notes:\n- The client code is open source: https://github.com/sourcewizard-ai/sourcewizard\n- We explicitly ignore existing Env Files: https://github.com/sourcewizard-ai/sourcewizard/blob/main/in...
Would love feedback!
\u2014 Lukas & Ivan","title":"Show HN: Sourcewizard \u2013 AI installs SDKs in your codebase","updated_at":"2026-03-05T23:03:22Z","url":"https://sourcewizard.ai"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"htormey"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"I recently switched from VS Code to Cursor as my main editor. Been a software engineer for 15+ years, worked at big tech and early-stage startups. Lately, I\u2019ve been seeing a lot of hype around MCP (Model Context Protocol) integrations with Claude on X/LinkedIn. Some cool demos, like MCPs for browser debugging and Firecrawl MCP, have caught my eye.
That said, I tend to be skeptical of hype, so before I sink time into this, is anyone actually using MCP servers as part of their dev workflow? If so, which ones, and how are they actually helping?
I feel like just adding a CLI tool to Cursor\u2019s rules file and telling it how to use it might be just as effective. But maybe I\u2019m missing something?
Would love to hear from real users, not just polished demo videos made by people building MCP servers. If you use MCP servers in Cursor (or anywhere else), let me know what you use and why?"},"title":{"matchLevel":"none","matchedWords":[],"value":"Does anyone use MCP servers in their dev workflow?"}},"_tags":["story","author_htormey","story_43258552","ask_hn"],"author":"htormey","children":[43273377,43279794,43329444],"created_at":"2025-03-04T18:51:05Z","created_at_i":1741114265,"num_comments":3,"objectID":"43258552","points":15,"story_id":43258552,"story_text":"I recently switched from VS Code to Cursor as my main editor. Been a software engineer for 15+ years, worked at big tech and early-stage startups. Lately, I\u2019ve been seeing a lot of hype around MCP (Model Context Protocol) integrations with Claude on X/LinkedIn. Some cool demos, like MCPs for browser debugging and Firecrawl MCP, have caught my eye.
That said, I tend to be skeptical of hype, so before I sink time into this, is anyone actually using MCP servers as part of their dev workflow? If so, which ones, and how are they actually helping?
I feel like just adding a CLI tool to Cursor\u2019s rules file and telling it how to use it might be just as effective. But maybe I\u2019m missing something?
Would love to hear from real users, not just polished demo videos made by people building MCP servers. If you use MCP servers in Cursor (or anywhere else), let me know what you use and why?","title":"Does anyone use MCP servers in their dev workflow?","updated_at":"2025-03-11T19:48:11Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"syabro"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Shmauthor here. Built this for myself, putting it out in case it's useful.
Needed any URL as clean Markdown for LLM context \u2014 including Cloudflare/anti-bot sites. curl gets HTTP 403 on those, raw HTML is 80%+ nav noise eating context, paid SaaS (Firecrawl, Jina) wasn't an option for me.
It's a Docker wrapper around two existing OSS tools \u2014 CloakBrowser (stealth Chromium that passes Cloudflare) and rs-trafilatura (HTML \u2192 Markdown). No new scraper, just glue. Runs locally, my URLs stay on my box
Token reduction (raw curl HTML vs snitchmd, tiktoken cl100k_base):
- cloudflare.com/learning/bots \u2014 curl: HTTP 403 \u2192 snitchmd: 0.8k
- docs.docker.com/engine/install \u2014 187k \u2192 0.9k
- en.wikipedia.org/wiki/LLM \u2014 222.7k \u2192 29.7k
Heads up: passes Cloudflare, can't solve "click traffic lights" captchas (reCAPTCHA v2, hCaptcha)
MIT. Happy to answer questions"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Snitchmd \u2013 Cloudflare-protected URLs into clean Markdown via Docker"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/syabro/snitchmd"}},"_tags":["story","author_syabro","story_47949500","show_hn"],"author":"syabro","children":[47949762],"created_at":"2026-04-29T15:07:51Z","created_at_i":1777475271,"num_comments":1,"objectID":"47949500","points":8,"story_id":47949500,"story_text":"Shmauthor here. Built this for myself, putting it out in case it's useful.
Needed any URL as clean Markdown for LLM context \u2014 including Cloudflare/anti-bot sites. curl gets HTTP 403 on those, raw HTML is 80%+ nav noise eating context, paid SaaS (Firecrawl, Jina) wasn't an option for me.
It's a Docker wrapper around two existing OSS tools \u2014 CloakBrowser (stealth Chromium that passes Cloudflare) and rs-trafilatura (HTML \u2192 Markdown). No new scraper, just glue. Runs locally, my URLs stay on my box
Token reduction (raw curl HTML vs snitchmd, tiktoken cl100k_base):
- cloudflare.com/learning/bots \u2014 curl: HTTP 403 \u2192 snitchmd: 0.8k
- docs.docker.com/engine/install \u2014 187k \u2192 0.9k
- en.wikipedia.org/wiki/LLM \u2014 222.7k \u2192 29.7k
Heads up: passes Cloudflare, can't solve "click traffic lights" captchas (reCAPTCHA v2, hCaptcha)
MIT. Happy to answer questions","title":"Show HN: Snitchmd \u2013 Cloudflare-protected URLs into clean Markdown via Docker","updated_at":"2026-05-15T11:28:12Z","url":"https://github.com/syabro/snitchmd"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"madanparas"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"We used to play the SEO game.
Write blog
Wait 3 months
Maybe rank
Maybe convert
What actually happened at 8 early-stage AI startups (Series A or earlier):
-Google Page 1 took ~94 days
- Organic CTR: 2.6%
- First qualified lead: 6\u20138 weeks
So we ditched the playbook.
We asked one question:\nHow fast can we show up when someone asks ChatGPT or Perplexity what tool to use?
Turns out\u2026 faster than Google.\nAnd yeah, it brought pipeline.
What changed when we went LLM-first:
- Perplexity picked up our content in under 48 hours
- ChatGPT (with Browsing) indexed feature pages in 3 days
- 18.2% of sessions now come from LLM-originated paths
- Those leads convert 2.4x better than blog traffic
Then Reddit unlocked another level.
We posted no-link, technical breakdowns here.
One of them (about how we automated an AI agent pipeline) got quoted by Perplexity in:
- \u201cUX AI Agent\u201d
- \u201cBest Firecrawl alternatives\u201d
- \u201cHow to track LLM bots\u201d
No push. No SEO. Just built in public.
3 days later:
- 9 Perplexity query quotes
- 2 inbound leads mentioned us directly
Reddit is training data goldmine for LLMs.
Here's what worked for us:
1. Add Q&A blocks to product pages (all <40 words)
Example:\nQ: How does FireGEO detect ClaudeBot?
A: It fingerprints known Anthropic headers and reverse-DNS matches IP blocks like 2600:1f18::/32
- Indexed by Perplexity in <48 hours
- 11 bot hits in 5 days
- 1 lead \u2192 trial signup in <1 week
2. Build an ai-sitemap.xml
Only high-signal pages:\n- API docs
- Feature comparisons
- Pricing breakdowns
- Tech specs
Crawl rate = 2.3x higher than default sitemap.
GPTBot, ClaudeBot, and PerplexityBot show up daily in logs.
3. Treat Reddit as an input layer
We post raw content here before it hits our blog. \nIn last 30 days:
- ~30,000 views across Reddit
- 9 quotes in Perplexity answers
- 2 leads directly from those mentions
If you\u2019re shipping something real, try this:
- Install FireGEO or track LLM bots via reverse DNS + ASN logs
- Create llm.txt for structured answers
- Tag LLM traffic with UTMs and route to CRM
Curious to know what\u2019s working for you around LLM visibility?
Any tactics or insights others here are seeing?"},"title":{"matchLevel":"none","matchedWords":[],"value":"Reddit and Perplexity got us leads faster than Google ever did"}},"_tags":["story","author_madanparas","story_44737677","ask_hn"],"author":"madanparas","children":[44738597],"created_at":"2025-07-30T18:15:35Z","created_at_i":1753899335,"num_comments":1,"objectID":"44737677","points":8,"story_id":44737677,"story_text":"We used to play the SEO game.
Write blog
Wait 3 months
Maybe rank
Maybe convert
What actually happened at 8 early-stage AI startups (Series A or earlier):
-Google Page 1 took ~94 days
- Organic CTR: 2.6%
- First qualified lead: 6\u20138 weeks
So we ditched the playbook.
We asked one question:\nHow fast can we show up when someone asks ChatGPT or Perplexity what tool to use?
Turns out\u2026 faster than Google.\nAnd yeah, it brought pipeline.
What changed when we went LLM-first:
- Perplexity picked up our content in under 48 hours
- ChatGPT (with Browsing) indexed feature pages in 3 days
- 18.2% of sessions now come from LLM-originated paths
- Those leads convert 2.4x better than blog traffic
Then Reddit unlocked another level.
We posted no-link, technical breakdowns here.
One of them (about how we automated an AI agent pipeline) got quoted by Perplexity in:
- \u201cUX AI Agent\u201d
- \u201cBest Firecrawl alternatives\u201d
- \u201cHow to track LLM bots\u201d
No push. No SEO. Just built in public.
3 days later:
- 9 Perplexity query quotes
- 2 inbound leads mentioned us directly
Reddit is training data goldmine for LLMs.
Here's what worked for us:
1. Add Q&A blocks to product pages (all <40 words)
Example:\nQ: How does FireGEO detect ClaudeBot?
A: It fingerprints known Anthropic headers and reverse-DNS matches IP blocks like 2600:1f18::/32
- Indexed by Perplexity in <48 hours
- 11 bot hits in 5 days
- 1 lead \u2192 trial signup in <1 week
2. Build an ai-sitemap.xml
Only high-signal pages:\n- API docs
- Feature comparisons
- Pricing breakdowns
- Tech specs
Crawl rate = 2.3x higher than default sitemap.
GPTBot, ClaudeBot, and PerplexityBot show up daily in logs.
3. Treat Reddit as an input layer
We post raw content here before it hits our blog. \nIn last 30 days:
- ~30,000 views across Reddit
- 9 quotes in Perplexity answers
- 2 leads directly from those mentions
If you\u2019re shipping something real, try this:
- Install FireGEO or track LLM bots via reverse DNS + ASN logs
- Create llm.txt for structured answers
- Tag LLM traffic with UTMs and route to CRM
Curious to know what\u2019s working for you around LLM visibility?
Any tactics or insights others here are seeing?","title":"Reddit and Perplexity got us leads faster than Google ever did","updated_at":"2026-07-22T10:24:29Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"nickca"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Last night, OpenAI launched Deep Research, a tool for AI-powered deep web searches. In a few hours, I built an open-source alternative using Next.js, Firecrawl, and Vercel's AI SDK. Instead of using a fine-tuned version of o3, this method uses Firecrawl's extract + search with a reasoning model to deep research the web.
The system is built using Vercel\u2019s AI SDK for handling requests and streaming data, with an agent-based approach that manages search, extraction, and analysis. It uses Firecrawl to find and extract structured data, which is then processed through a progressive analysis system to refine and summarize results. The process follows an iterative cycle: Firecrawl\u2019s /search API finds relevant URLs, /extract pulls structured data, and a planning function determines what to look for next. A research state tracker keeps track of findings, summaries, and progress while enforcing depth (maxDepth) and time limits to stay within Vercel\u2019s function execution time constraint. The results stream into a live UI, showing search progress, sources, and key insights. The cycle continues until enough information is collected or limits are reached, producing a final structured summary.
This was a pretty fast built and I wanna keep iterating on it. Would love to hear your thoughts."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Open-source version of OpenAI's Deep Research"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/nickscamara/open-deep-research"}},"_tags":["story","author_nickca","story_42924106","show_hn"],"author":"nickca","children":[42927165],"created_at":"2025-02-03T22:19:58Z","created_at_i":1738621198,"num_comments":1,"objectID":"42924106","points":8,"story_id":42924106,"story_text":"Last night, OpenAI launched Deep Research, a tool for AI-powered deep web searches. In a few hours, I built an open-source alternative using Next.js, Firecrawl, and Vercel's AI SDK. Instead of using a fine-tuned version of o3, this method uses Firecrawl's extract + search with a reasoning model to deep research the web.
The system is built using Vercel\u2019s AI SDK for handling requests and streaming data, with an agent-based approach that manages search, extraction, and analysis. It uses Firecrawl to find and extract structured data, which is then processed through a progressive analysis system to refine and summarize results. The process follows an iterative cycle: Firecrawl\u2019s /search API finds relevant URLs, /extract pulls structured data, and a planning function determines what to look for next. A research state tracker keeps track of findings, summaries, and progress while enforcing depth (maxDepth) and time limits to stay within Vercel\u2019s function execution time constraint. The results stream into a live UI, showing search progress, sources, and key insights. The cycle continues until enough information is collected or limits are reached, producing a final structured summary.
This was a pretty fast built and I wanna keep iterating on it. Would love to hear your thoughts.","title":"Show HN: Open-source version of OpenAI's Deep Research","updated_at":"2025-02-04T05:02:47Z","url":"https://github.com/nickscamara/open-deep-research"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"matankleym"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Dataset Explorer is now LIVE and FREE.
--
Finding the right dataset shouldn't be this hard.
Millions of high-quality datasets exist across Kaggle, data.gov, and other platforms, but discovering the ones you actually need feels like searching for a needle in a haystack.
Whether it's seasonality trends, weather patterns, holiday data, tech layoffs, currency rates, political content, or geo information \u2013 the perfect dataset is out there, but buried under poor search functionality.
That's why we built the dataset-explorer \u2013 a completely free tool that lets you search for datasets using natural language across multiple platforms.
Just describe what you want to analyze, and it uses Perplexity, scraping (Firecrawl), and other tools behind the scenes to surface relevant datasets.
Instead of manually browsing through categories or dealing with limited search filters, you can simply ask "show me tech layoff data from the past 5 years" and get preview of multiple datasets.
Quick demo: I analyzed tech layoffs from 2020-2025 and uncovered some striking insights:
- 2023 was brutal \u2013 264K layoffs (the peak year)
- Post-IPO companies led the cuts \u2013 responsible for 58% of all layoffs
- Hardware hit hardest \u2013 with Intel leading the charge
- January 2023 = worst month ever \u2013 89K people lost their jobs in just 30 days
Once you find your dataset, you can analyze it completely free on Hunch\n.\nTry it yourself and let us know we can improve it for you.
Data explorer - https://hunch.dev/data-explorer
Demo link - https://screen.studio/share/bLnYXAvZ"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Dataset Explorer \u2013 Free tool to search any public datasets"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.hunch.dev/data-explorer"}},"_tags":["story","author_matankleym","story_44795244","show_hn"],"author":"matankleym","children":[44795334,44795859],"created_at":"2025-08-05T07:24:34Z","created_at_i":1754378674,"num_comments":2,"objectID":"44795244","points":7,"story_id":44795244,"story_text":"Dataset Explorer is now LIVE and FREE.
--
Finding the right dataset shouldn't be this hard.
Millions of high-quality datasets exist across Kaggle, data.gov, and other platforms, but discovering the ones you actually need feels like searching for a needle in a haystack.
Whether it's seasonality trends, weather patterns, holiday data, tech layoffs, currency rates, political content, or geo information \u2013 the perfect dataset is out there, but buried under poor search functionality.
That's why we built the dataset-explorer \u2013 a completely free tool that lets you search for datasets using natural language across multiple platforms.
Just describe what you want to analyze, and it uses Perplexity, scraping (Firecrawl), and other tools behind the scenes to surface relevant datasets.
Instead of manually browsing through categories or dealing with limited search filters, you can simply ask "show me tech layoff data from the past 5 years" and get preview of multiple datasets.
Quick demo: I analyzed tech layoffs from 2020-2025 and uncovered some striking insights:
- 2023 was brutal \u2013 264K layoffs (the peak year)
- Post-IPO companies led the cuts \u2013 responsible for 58% of all layoffs
- Hardware hit hardest \u2013 with Intel leading the charge
- January 2023 = worst month ever \u2013 89K people lost their jobs in just 30 days
Once you find your dataset, you can analyze it completely free on Hunch\n.\nTry it yourself and let us know we can improve it for you.
Data explorer - https://hunch.dev/data-explorer
Demo link - https://screen.studio/share/bLnYXAvZ","title":"Show HN: Dataset Explorer \u2013 Free tool to search any public datasets","updated_at":"2025-08-06T10:25:21Z","url":"https://www.hunch.dev/data-explorer"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"keithgroves"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"I built a tool registry for AI agents.
Problem: Agents need tools, but the current options are awkward. \nYou either wire up tools manually in your codebase, or you spin up MCP servers that bundle more than you need. Neither is \ndiscoverable, shareable, or safe to run.
Enact is a package manager for executable tools:
enact search "web scraping"\n enact run enact/firecrawl --args '{"url": "https://example.com"}'\nKey differences from MCP:\n- Tools, not servers. Install one capability, not a bundle.\n- Registry with discovery. Agents can search for tools at runtime.\n- Containerized. Run untrusted tools safely.\n- Sigstore signing for trust.
Live registry: https://enact.tools
CLI: npm install -g enact-cli
Source: https://github.com/EnactProtocol/enact
Would love feedback on the manifest format and whether this solves \na real problem for folks building agents."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Enact \u2013 A package manager for AI agent tools"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://enact.tools"}},"_tags":["story","author_keithgroves","story_46435383","show_hn"],"author":"keithgroves","children":[46442261],"created_at":"2025-12-30T17:07:18Z","created_at_i":1767114438,"num_comments":2,"objectID":"46435383","points":6,"story_id":46435383,"story_text":"I built a tool registry for AI agents.
Problem: Agents need tools, but the current options are awkward. \nYou either wire up tools manually in your codebase, or you spin up MCP servers that bundle more than you need. Neither is \ndiscoverable, shareable, or safe to run.
Enact is a package manager for executable tools:
enact search "web scraping"\n enact run enact/firecrawl --args '{"url": "https://example.com"}'\nKey differences from MCP:\n- Tools, not servers. Install one capability, not a bundle.\n- Registry with discovery. Agents can search for tools at runtime.\n- Containerized. Run untrusted tools safely.\n- Sigstore signing for trust.
Live registry: https://enact.tools
CLI: npm install -g enact-cli
Source: https://github.com/EnactProtocol/enact
Would love feedback on the manifest format and whether this solves \na real problem for folks building agents.","title":"Show HN: Enact \u2013 A package manager for AI agent tools","updated_at":"2026-03-05T23:16:09Z","url":"https://enact.tools"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"amaldavid"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Have been getting tired of having wrong version api docs for building anything with LLMs, just wanted to curate all latest api docs in a single place and also let it be discoverable for others.
It is integrated with firecrawl to pull any website data and make it available as md(markdown) file, yet to add bells and whistles or even guardrails to it. But hope you folks find it useful.
I had previously built https://github.com/Amal-David/gitbook-downloader but whenever i tried to extend to other docs format, had to write a lot of code so just went with firecrawl for now. Stumbled upon gitingest and loved their UI so took that as a reference and built the platform. Some limits of 250 pages max and some sites not fully supported. Feel free to give any specific feedback, will try to progressively iterate on them."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: DocIngest \u2013 Download and save documentation from any URL"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://docingest.com/"}},"_tags":["story","author_amaldavid","story_42528160","show_hn"],"author":"amaldavid","children":[42528371],"created_at":"2024-12-28T02:48:31Z","created_at_i":1735354111,"num_comments":1,"objectID":"42528160","points":6,"story_id":42528160,"story_text":"Have been getting tired of having wrong version api docs for building anything with LLMs, just wanted to curate all latest api docs in a single place and also let it be discoverable for others.
It is integrated with firecrawl to pull any website data and make it available as md(markdown) file, yet to add bells and whistles or even guardrails to it. But hope you folks find it useful.
I had previously built https://github.com/Amal-David/gitbook-downloader but whenever i tried to extend to other docs format, had to write a lot of code so just went with firecrawl for now. Stumbled upon gitingest and loved their UI so took that as a reference and built the platform. Some limits of 250 pages max and some sites not fully supported. Feel free to give any specific feedback, will try to progressively iterate on them.","title":"Show HN: DocIngest \u2013 Download and save documentation from any URL","updated_at":"2024-12-29T03:32:08Z","url":"https://docingest.com/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"caballeto"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Hi HN, I'm Vlad. I've been building MCP servers and related tooling for a while now, and I kept hitting a class of bug that no unit test caught: someone on the team renames a tool parameter or tweaks a tool description, all the tests pass, but the AI agent that was calling that tool silently breaks. This happens because the model reads tool descriptions and parameter schemas to decide which tool to call and how, so a renamed parameter or a reworded description isn't just a cosmetic change \u2014 it directly affects the model's behavior.
The MCP spec doesn't have tool versioning available yet, and there's no static artifact describing what a server exposes. The tools/list just returns whatever's in memory at runtime and there's nothing to commit or diff against, which means changes slip through that can break downstream workflows without noticing.
The same problem for HTTP was already solved a long time ago with VCR.py, and I realized the same pattern works here. mcp-recorder captures the full MCP interaction sequence \u2014 initialize, tools/list, tools/call \u2014 into a JSON cassette file. Because it records complete protocol exchanges rather than just schema snapshots, you're testing actual behavior: if a tool call that used to return a specific format now returns something different, or a capability quietly disappears during the handshake, the cassette catches it. From that single recording you can replay it as a mock server (no API keys, fully deterministic), or verify your changed server against it and catch any diff:
Verifying golden.json against node dist/index.js
1. initialize [PASS]\n\n 2. tools/list [PASS]\n\n 3. tools/call [search] [FAIL]\n\n $.result.content[0].text: "old output" != "new output"\n\n 4. tools/call [analyze] [PASS]\nNon-zero exit code on any mismatch, so it plugs straight into CI.
You can try it right now with minimal setup, there's a public demo server and a scenarios file included:
pip install mcp-recorder\nmcp-recorder record-scenarios scenarios.yml\nmcp-recorder verify --cassette cassettes/demo_walkthrough.json \\\n --target https://mcp.devhelm.io
It works with both HTTP and stdio transports. Scenarios are defined in YAML so it works with MCP servers in any language, and there's a pytest plugin if you want tighter integration. Secret redaction and environment variable interpolation are built in.
To make sure this actually works on real codebases, I submitted several PRs to production MCP servers: monday.com's MCP server (https://github.com/mondaycom/mcp/pull/222), Tavily's MCP server (https://github.com/tavily-ai/tavily-mcp/pull/113), and Firecrawl's MCP server (https://github.com/firecrawl/firecrawl-mcp-server/pull/175). They went from zero schema coverage to full tool surface verification with a clean schema diff available on each tool change. One big benefit is that you can do verification and replay with no API keys \u2014 deterministic responses, no live requests to real servers.
I wrote up a deeper dive into the schema drift problem and the VCR pattern for MCP here: https://devhelm.io/blog/regression-testing-mcp-servers
mcp-recorder is MIT-licensed and on PyPI. Source is at https://github.com/devhelmhq/mcp-recorder \u2014 issues and PRs are welcome.
I'm building more tooling around MCP and agent reliability, so if you're dealing with similar problems, I'd genuinely like to hear what's been painful for you."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: mcp-recorder \u2013 VCR.py for MCP servers. Record, replay, verify"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/devhelmhq/mcp-recorder"}},"_tags":["story","author_caballeto","story_47274100","show_hn"],"author":"caballeto","created_at":"2026-03-06T12:21:54Z","created_at_i":1772799714,"num_comments":0,"objectID":"47274100","points":6,"story_id":47274100,"story_text":"Hi HN, I'm Vlad. I've been building MCP servers and related tooling for a while now, and I kept hitting a class of bug that no unit test caught: someone on the team renames a tool parameter or tweaks a tool description, all the tests pass, but the AI agent that was calling that tool silently breaks. This happens because the model reads tool descriptions and parameter schemas to decide which tool to call and how, so a renamed parameter or a reworded description isn't just a cosmetic change \u2014 it directly affects the model's behavior.
The MCP spec doesn't have tool versioning available yet, and there's no static artifact describing what a server exposes. The tools/list just returns whatever's in memory at runtime and there's nothing to commit or diff against, which means changes slip through that can break downstream workflows without noticing.
The same problem for HTTP was already solved a long time ago with VCR.py, and I realized the same pattern works here. mcp-recorder captures the full MCP interaction sequence \u2014 initialize, tools/list, tools/call \u2014 into a JSON cassette file. Because it records complete protocol exchanges rather than just schema snapshots, you're testing actual behavior: if a tool call that used to return a specific format now returns something different, or a capability quietly disappears during the handshake, the cassette catches it. From that single recording you can replay it as a mock server (no API keys, fully deterministic), or verify your changed server against it and catch any diff:
Verifying golden.json against node dist/index.js
1. initialize [PASS]\n\n 2. tools/list [PASS]\n\n 3. tools/call [search] [FAIL]\n\n $.result.content[0].text: "old output" != "new output"\n\n 4. tools/call [analyze] [PASS]\nNon-zero exit code on any mismatch, so it plugs straight into CI.
You can try it right now with minimal setup, there's a public demo server and a scenarios file included:
pip install mcp-recorder\nmcp-recorder record-scenarios scenarios.yml\nmcp-recorder verify --cassette cassettes/demo_walkthrough.json \\\n --target https://mcp.devhelm.io
It works with both HTTP and stdio transports. Scenarios are defined in YAML so it works with MCP servers in any language, and there's a pytest plugin if you want tighter integration. Secret redaction and environment variable interpolation are built in.
To make sure this actually works on real codebases, I submitted several PRs to production MCP servers: monday.com's MCP server (https://github.com/mondaycom/mcp/pull/222), Tavily's MCP server (https://github.com/tavily-ai/tavily-mcp/pull/113), and Firecrawl's MCP server (https://github.com/firecrawl/firecrawl-mcp-server/pull/175). They went from zero schema coverage to full tool surface verification with a clean schema diff available on each tool change. One big benefit is that you can do verification and replay with no API keys \u2014 deterministic responses, no live requests to real servers.
I wrote up a deeper dive into the schema drift problem and the VCR pattern for MCP here: https://devhelm.io/blog/regression-testing-mcp-servers
mcp-recorder is MIT-licensed and on PyPI. Source is at https://github.com/devhelmhq/mcp-recorder \u2014 issues and PRs are welcome.
I'm building more tooling around MCP and agent reliability, so if you're dealing with similar problems, I'd genuinely like to hear what's been painful for you.","title":"Show HN: mcp-recorder \u2013 VCR.py for MCP servers. Record, replay, verify","updated_at":"2026-03-06T12:32:14Z","url":"https://github.com/devhelmhq/mcp-recorder"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Arindam1729"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Recently, I was exploring the idea of using AI agents for real-time research and content generation.
To put that into practice, I thought why not try solving a problem I run into often? Creating high-quality, up-to-date newsletters without spending hours manually researching.
So I built a simple AI-powered Newsletter Agent that automatically researches a topic and generates a well-structured newsletter using the latest info from the web.
Here's what I used:
- Firecrawl Search API for real-time web scraping and content discovery\n- Nebius AI models for fast + cheap inference\n- Agno as the Agent Framework\n- Streamlit for the UI (It's easier for me)
The project isn\u2019t overly complex, I\u2019ve kept it lightweight and modular, but it\u2019s a great way to explore how agents can automate research + content workflows.
If you're curious, I put together a walkthrough showing exactly how it works: https://www.youtube.com/watch?v=iDYnR_MlErg
And the full code is available here if you want to build on top of it: https://github.com/Arindam200/awesome-ai-apps/tree/main/simp...
Would love to hear how others are using AI for content creation or research. Also open to feedback or feature suggestions might add multi-topic newsletters next!"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: I Built an Agent That Writes Fresh Newsletters for Any Topic"}},"_tags":["story","author_Arindam1729","story_44201571","show_hn"],"author":"Arindam1729","children":[44201709,44201768],"created_at":"2025-06-06T14:57:28Z","created_at_i":1749221848,"num_comments":3,"objectID":"44201571","points":5,"story_id":44201571,"story_text":"Recently, I was exploring the idea of using AI agents for real-time research and content generation.
To put that into practice, I thought why not try solving a problem I run into often? Creating high-quality, up-to-date newsletters without spending hours manually researching.
So I built a simple AI-powered Newsletter Agent that automatically researches a topic and generates a well-structured newsletter using the latest info from the web.
Here's what I used:
- Firecrawl Search API for real-time web scraping and content discovery\n- Nebius AI models for fast + cheap inference\n- Agno as the Agent Framework\n- Streamlit for the UI (It's easier for me)
The project isn\u2019t overly complex, I\u2019ve kept it lightweight and modular, but it\u2019s a great way to explore how agents can automate research + content workflows.
If you're curious, I put together a walkthrough showing exactly how it works: https://www.youtube.com/watch?v=iDYnR_MlErg
And the full code is available here if you want to build on top of it: https://github.com/Arindam200/awesome-ai-apps/tree/main/simp...
Would love to hear how others are using AI for content creation or research. Also open to feedback or feature suggestions might add multi-topic newsletters next!","title":"Show HN: I Built an Agent That Writes Fresh Newsletters for Any Topic","updated_at":"2025-06-08T19:26:34Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"taariqlewis"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"We added the official Hacker News API to SerenAI's x402 Gateway.
AI agents can now query HN stories, comments, and users through the same MCP interface they use for Seren's paid publishers like Firecrawl and Perplexity. We're aiming to deliver greater composability: an agent can pull HN discussions (free), scrape competitor sites with Firecrawl, and get AI analysis from Perplexity, all in one protocol.
We built a competitive intelligence example that tracks Show HN launches and competitor mentions across HN.
GitHub: https://github.com/serenorg/x402-mcp-server \nInstall: npx @serendb/x402-mcp-server"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Hacker News API on SerenAI X402 Gateway"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://serendb.com/blog/free-hacker-news-api-serenai-x402-gateway"}},"_tags":["story","author_taariqlewis","story_46514270","show_hn"],"author":"taariqlewis","created_at":"2026-01-06T16:16:43Z","created_at_i":1767716203,"num_comments":0,"objectID":"46514270","points":5,"story_id":46514270,"story_text":"We added the official Hacker News API to SerenAI's x402 Gateway.
AI agents can now query HN stories, comments, and users through the same MCP interface they use for Seren's paid publishers like Firecrawl and Perplexity. We're aiming to deliver greater composability: an agent can pull HN discussions (free), scrape competitor sites with Firecrawl, and get AI analysis from Perplexity, all in one protocol.
We built a competitive intelligence example that tracks Show HN launches and competitor mentions across HN.
GitHub: https://github.com/serenorg/x402-mcp-server \nInstall: npx @serendb/x402-mcp-server","title":"Show HN: Hacker News API on SerenAI X402 Gateway","updated_at":"2026-03-05T23:17:25Z","url":"https://serendb.com/blog/free-hacker-news-api-serenai-x402-gateway"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"pomarie"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Hey HN, I'm Paul, the founder of cubic.dev, an AI code reviewer for complex codebases.
Over the past few months we\u2019ve completely rebuilt our detection engine, and I wanted to share a few things we did to get more out of LLMs.
Context: cubic specializes in code reviews for teams with complex codebases, like Better Auth, Cal.com, and PostHog. Our users have high standards. It\u2019s important that reviews have real depth and actually understand the codebase.
In the past, we've sometimes struggled with producing reviews that had deep insight into complex changes. It didn't feel like we were leaving comments that truly understood both the codebase and the intent behind the PR. If we pushed reasoning to the max, it could get there, but it would take ages, often 15+ minutes for a review, which many people disliked.
We've spent the last few months rebuilding our AI review engine from scratch, and we've completely redone how the reviewer works, bit by bit.
Like the Ship of Theseus, cubic ended up so different (and better) that we're releasing it as cubic 2.0.
I should say up front: I\u2019m biased because I work on this, and part of the point is awareness. But the main reason I\u2019m posting is that the work feels broadly useful if you\u2019re building anything LLM-based where you need both quality and speed.
*Why this is a "2.0"*
We were optimizing for two things:
1. Higher signal reviews (comments people actually act on)\n2. Lower latency
Quality: 3 months ago, about 20% of comments that cubic left would be addressed by a developer. We measure this by having an LLM look at commits after a cubic comment and judge whether the change implemented what cubic flagged. Today that number is 60%+. For some teams it\u2019s over 90%.
Speed: median time to review a PR was roughly halved; P90 divided by 3.
*What we changed (the parts that mattered)*
1. Pre-mapping the codebase ("AI wiki")
A big inefficiency in LLM code review (and code writing) is that every PR forces the model to rebuild a mental map of the repo from scratch. In large repos, just figuring out "where am I" can consume a lot of context and tokens before you even get to reasoning about the diff.
We built an "AI wiki" that pre-maps the important parts of a codebase and reuses that as context for reviews.
As a side effect, the wiki is also useful to humans (and AIs through an MCP), especially for onboarding or for non-technical people trying to understand a system. Example Firecrawl: https://www.cubic.dev/wikis/firecrawl/firecrawl
2. External context tools, plus getting tool usage under control
We added tools to fetch external documentation when needed. The hard part was not adding the tool, it was getting the model to use it correctly. This took a lot of prompt iteration and guardrails, and it ended up being more important than we expected.
3. Learning loop, with more weight on senior reviewers
We leaned harder into learning from user interactions and feedback. One change that helped a lot recently was identifying the senior reviewers in an org and weighting their feedback more heavily. In practice, that made the system converge faster toward what "good" looks like for that team.
4. Sandbox snapshotting for large repos
On larger repos, we were wasting minutes on setup work (clone time, environment prep), and we were doing it for every PR. We added a snapshotting approach that cut a lot of that overhead.
Anyway, thanks for reading. Happy to answer questions about any of the above. I\u2019d also love feedback from people who\u2019ve tried AI code review tools:
* What made you keep them, or turn them off?\n* What metrics would you trust to measure "review quality"?\n* Where do current tools fail in ways that are genuinely harmful?
cubic is here (and free for public repos): https://cubic.dev/home"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: cubic 2.0 \u2013 improving our AI code reviewer (3x more accurate,2x faster)"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.cubic.dev/blog/cubic-2.0"}},"_tags":["story","author_pomarie","story_46605224","show_hn"],"author":"pomarie","children":[46605726,46606969],"created_at":"2026-01-13T18:16:27Z","created_at_i":1768328187,"num_comments":3,"objectID":"46605224","points":4,"story_id":46605224,"story_text":"Hey HN, I'm Paul, the founder of cubic.dev, an AI code reviewer for complex codebases.
Over the past few months we\u2019ve completely rebuilt our detection engine, and I wanted to share a few things we did to get more out of LLMs.
Context: cubic specializes in code reviews for teams with complex codebases, like Better Auth, Cal.com, and PostHog. Our users have high standards. It\u2019s important that reviews have real depth and actually understand the codebase.
In the past, we've sometimes struggled with producing reviews that had deep insight into complex changes. It didn't feel like we were leaving comments that truly understood both the codebase and the intent behind the PR. If we pushed reasoning to the max, it could get there, but it would take ages, often 15+ minutes for a review, which many people disliked.
We've spent the last few months rebuilding our AI review engine from scratch, and we've completely redone how the reviewer works, bit by bit.
Like the Ship of Theseus, cubic ended up so different (and better) that we're releasing it as cubic 2.0.
I should say up front: I\u2019m biased because I work on this, and part of the point is awareness. But the main reason I\u2019m posting is that the work feels broadly useful if you\u2019re building anything LLM-based where you need both quality and speed.
*Why this is a "2.0"*
We were optimizing for two things:
1. Higher signal reviews (comments people actually act on)\n2. Lower latency
Quality: 3 months ago, about 20% of comments that cubic left would be addressed by a developer. We measure this by having an LLM look at commits after a cubic comment and judge whether the change implemented what cubic flagged. Today that number is 60%+. For some teams it\u2019s over 90%.
Speed: median time to review a PR was roughly halved; P90 divided by 3.
*What we changed (the parts that mattered)*
1. Pre-mapping the codebase ("AI wiki")
A big inefficiency in LLM code review (and code writing) is that every PR forces the model to rebuild a mental map of the repo from scratch. In large repos, just figuring out "where am I" can consume a lot of context and tokens before you even get to reasoning about the diff.
We built an "AI wiki" that pre-maps the important parts of a codebase and reuses that as context for reviews.
As a side effect, the wiki is also useful to humans (and AIs through an MCP), especially for onboarding or for non-technical people trying to understand a system. Example Firecrawl: https://www.cubic.dev/wikis/firecrawl/firecrawl
2. External context tools, plus getting tool usage under control
We added tools to fetch external documentation when needed. The hard part was not adding the tool, it was getting the model to use it correctly. This took a lot of prompt iteration and guardrails, and it ended up being more important than we expected.
3. Learning loop, with more weight on senior reviewers
We leaned harder into learning from user interactions and feedback. One change that helped a lot recently was identifying the senior reviewers in an org and weighting their feedback more heavily. In practice, that made the system converge faster toward what "good" looks like for that team.
4. Sandbox snapshotting for large repos
On larger repos, we were wasting minutes on setup work (clone time, environment prep), and we were doing it for every PR. We added a snapshotting approach that cut a lot of that overhead.
Anyway, thanks for reading. Happy to answer questions about any of the above. I\u2019d also love feedback from people who\u2019ve tried AI code review tools:
* What made you keep them, or turn them off?\n* What metrics would you trust to measure "review quality"?\n* Where do current tools fail in ways that are genuinely harmful?
cubic is here (and free for public repos): https://cubic.dev/home","title":"Show HN: cubic 2.0 \u2013 improving our AI code reviewer (3x more accurate,2x faster)","updated_at":"2026-03-05T23:22:49Z","url":"https://www.cubic.dev/blog/cubic-2.0"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ericciarla"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Hey everyone! Eric from Firecrawl here (YC S22), I wanted to share some takeaways from building AI chat products for enterprises at Mendable!
A year ago, Generative AI powered search seemed like the perfect quick win for companies taking their first steps with AI.
After building AI search for companies like MongoDB, Coinbase, and Snap at Mendable, we learned the nuances that make the difference between a demo and a production-ready system that actually drives value.
Before we came in, almost every company had a few engineers creating prototypes yet most of which fell short following contact with real users.
It all comes back to a simple truth: the system is only as good as the data going in.
As you scale, here\u2019s why the simple approaches break:
- Context crowding: Correct context for a given query gets crowded out by bad context. Take the Snap AR docs for example, they have 4 different products on their developer documentation website and they all have getting started pages. If a user asks a vague query like \u201chow do I get started\u201d to a basic RAG chatbot, the answer is going to most likely be an incorrect amalgamation of the 4 guides.\n- Outdated data: Information and processes constantly iterate, and documentation is not always maintained. One of our first customers, Spectrocloud, was benchmarking our chatbot before going into prod and they found that one answer in particular was not correct. At first we thought that the model was hallucinating, but after manually searching the docs we found the outdated source info on an obscure part of the documentation. \n- Data cleanliness: If data isn't clean, performance worsens and costs soar. We powered the chatbot on the documentation for Langchain, and data cleanliness and specifically prompt injection was a huge issue. Many of the Langchain pages had prompt examples embedded in them, which confused the model. We also noticed that a lot of unnecessary extra information was in our index like navigation menus on every page.\n- Data access: Accessing a variety of data sources is often critical for companies, but it introduces a host of challenges. For example, at Firecrawl, we\u2019ve seen that many large companies simply want to access web data from their own websites, but even this can involve complex permissioning, authentication, and data-fetching hurdles.
To mitigate these issues, companies building these apps should have a data strategy with the goal of curating and maintaining quality data. Here are some practical suggestions to guide your strategy:
- Metadata Management: Good metadata is your first defense against context crowding. Every piece of content should be tagged with essential details like product association, who created it, and who can access it. This enables advanced filtering and more accurate responses.\n- Data Maintenance: To keep data fresh and reliable, the teams that create content should be responsible for regular reviews and updates. When underlying information changes, the corresponding documentation needs to change with it.\n- Data Sanitation: Raw data rarely arrives in ideal form. Before ingestion, strip away unnecessary formatting and information while preserving the essential details. While each content source requires different handling, tools like Unstructured can help standardize this process.\n- Data Access & Integration: Build the infrastructure to access your data sources seamlessly. You'll need continuous data flow from knowledge bases, ticketing systems, websites, and more. Tools like Firecrawl can help build these pipelines and ensure high-quality data ingestion.
Startups like Glean, Unstructured, and our own Firecrawl have made some incredible progress on these problems, but no one has solved it all. No matter what tools emerge to make the process easier, having a robust data strategy is foundational to building production ready Generative AI Apps.
Thanks for reading and best of luck building AI apps!"},"title":{"matchLevel":"none","matchedWords":[],"value":"A data strategy is essential for generative AI apps"}},"_tags":["story","author_ericciarla","story_42434555","ask_hn"],"author":"ericciarla","children":[42435530],"created_at":"2024-12-16T19:41:22Z","created_at_i":1734378082,"num_comments":1,"objectID":"42434555","points":4,"story_id":42434555,"story_text":"Hey everyone! Eric from Firecrawl here (YC S22), I wanted to share some takeaways from building AI chat products for enterprises at Mendable!
A year ago, Generative AI powered search seemed like the perfect quick win for companies taking their first steps with AI.
After building AI search for companies like MongoDB, Coinbase, and Snap at Mendable, we learned the nuances that make the difference between a demo and a production-ready system that actually drives value.
Before we came in, almost every company had a few engineers creating prototypes yet most of which fell short following contact with real users.
It all comes back to a simple truth: the system is only as good as the data going in.
As you scale, here\u2019s why the simple approaches break:
- Context crowding: Correct context for a given query gets crowded out by bad context. Take the Snap AR docs for example, they have 4 different products on their developer documentation website and they all have getting started pages. If a user asks a vague query like \u201chow do I get started\u201d to a basic RAG chatbot, the answer is going to most likely be an incorrect amalgamation of the 4 guides.\n- Outdated data: Information and processes constantly iterate, and documentation is not always maintained. One of our first customers, Spectrocloud, was benchmarking our chatbot before going into prod and they found that one answer in particular was not correct. At first we thought that the model was hallucinating, but after manually searching the docs we found the outdated source info on an obscure part of the documentation. \n- Data cleanliness: If data isn't clean, performance worsens and costs soar. We powered the chatbot on the documentation for Langchain, and data cleanliness and specifically prompt injection was a huge issue. Many of the Langchain pages had prompt examples embedded in them, which confused the model. We also noticed that a lot of unnecessary extra information was in our index like navigation menus on every page.\n- Data access: Accessing a variety of data sources is often critical for companies, but it introduces a host of challenges. For example, at Firecrawl, we\u2019ve seen that many large companies simply want to access web data from their own websites, but even this can involve complex permissioning, authentication, and data-fetching hurdles.
To mitigate these issues, companies building these apps should have a data strategy with the goal of curating and maintaining quality data. Here are some practical suggestions to guide your strategy:
- Metadata Management: Good metadata is your first defense against context crowding. Every piece of content should be tagged with essential details like product association, who created it, and who can access it. This enables advanced filtering and more accurate responses.\n- Data Maintenance: To keep data fresh and reliable, the teams that create content should be responsible for regular reviews and updates. When underlying information changes, the corresponding documentation needs to change with it.\n- Data Sanitation: Raw data rarely arrives in ideal form. Before ingestion, strip away unnecessary formatting and information while preserving the essential details. While each content source requires different handling, tools like Unstructured can help standardize this process.\n- Data Access & Integration: Build the infrastructure to access your data sources seamlessly. You'll need continuous data flow from knowledge bases, ticketing systems, websites, and more. Tools like Firecrawl can help build these pipelines and ensure high-quality data ingestion.
Startups like Glean, Unstructured, and our own Firecrawl have made some incredible progress on these problems, but no one has solved it all. No matter what tools emerge to make the process easier, having a robust data strategy is foundational to building production ready Generative AI Apps.
Thanks for reading and best of luck building AI apps!","title":"A data strategy is essential for generative AI apps","updated_at":"2025-08-19T07:34:24Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"AliAbdoli"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Hey HN! We're Ali And Vrajang. We just created a tool router for Claude Code built on Jev. It basically gives your agent superpowers.
You keep working like you always do. When Jev thinks a tool would help, it provides it to your agent and your agent uses it. It watches your masked prompts, session context, web searches and subagent tasks, so it knows exactly when to intervene.
For now, tools come from a curated library. It's the basics like Exa, Firecrawl, Context7, but we're planning to add more soon. Let us know what you'd like added!
Everything runs on x402, so you don't set up each tool or juggle API keys. Your wallet pays a cent or two per call and that's it :)
Please expect some rough edges as it is just in alpha for now.
Would love to know what you think!
Here is our website: https://tenjin.sh and GitHub repo: https://github.com/BackTrackCo/tenjin-agent for those interested."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Tenjin \u2013 A Jev based x402 tool router for Claude Code"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/BackTrackCo/tenjin-agent"}},"_tags":["story","author_AliAbdoli","story_49851853","show_hn"],"author":"AliAbdoli","children":[49853908],"created_at":"2026-09-26T00:25:57Z","created_at_i":1790382357,"num_comments":0,"objectID":"49851853","points":4,"story_id":49851853,"story_text":"Hey HN! We're Ali And Vrajang. We just created a tool router for Claude Code built on Jev. It basically gives your agent superpowers.
You keep working like you always do. When Jev thinks a tool would help, it provides it to your agent and your agent uses it. It watches your masked prompts, session context, web searches and subagent tasks, so it knows exactly when to intervene.
For now, tools come from a curated library. It's the basics like Exa, Firecrawl, Context7, but we're planning to add more soon. Let us know what you'd like added!
Everything runs on x402, so you don't set up each tool or juggle API keys. Your wallet pays a cent or two per call and that's it :)
Please expect some rough edges as it is just in alpha for now.
Would love to know what you think!
Here is our website: https://tenjin.sh and GitHub repo: https://github.com/BackTrackCo/tenjin-agent for those interested.","title":"Show HN: Tenjin \u2013 A Jev based x402 tool router for Claude Code","updated_at":"2026-09-27T02:38:46Z","url":"https://github.com/BackTrackCo/tenjin-agent"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"nadyyym"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"hey, I'm building a tool that: \n1. analyzes your Posthog data\n2. finds patterns that lead to plan upgrade/account expansion\n3. creates a deal in your CRM whenever it sees it againg
we've just launched a huge update. Beton now has MCP (my Claude Code is already connected), Firecrawl integration and onboarding that's easier to understand
available in cloud and via AGPLv3
let us know if you need any help setting up
PS it's also suitable if you want to send triggered push notifications or emails"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Agent that matches sales reps with warm leads based on product usage"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://inspector.getbeton.ai"}},"_tags":["story","author_nadyyym","story_47155276","show_hn"],"author":"nadyyym","created_at":"2026-02-25T18:09:50Z","created_at_i":1772042990,"num_comments":0,"objectID":"47155276","points":4,"story_id":47155276,"story_text":"hey, I'm building a tool that: \n1. analyzes your Posthog data\n2. finds patterns that lead to plan upgrade/account expansion\n3. creates a deal in your CRM whenever it sees it againg
we've just launched a huge update. Beton now has MCP (my Claude Code is already connected), Firecrawl integration and onboarding that's easier to understand
available in cloud and via AGPLv3
let us know if you need any help setting up
PS it's also suitable if you want to send triggered push notifications or emails","title":"Show HN: Agent that matches sales reps with warm leads based on product usage","updated_at":"2026-03-05T23:37:17Z","url":"https://inspector.getbeton.ai"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"ericciarla"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Hey HN! Caleb, Nick, Garrett, and I from Mendable (YC S22) are excited to launch Intelligent Extraction for FireCrawl, the developer platform for scraping, search, and extraction.
After a successful twitter launch last week, FireCrawl skyrocketed to over 2k stars and we have been getting a ton of feature requests [1]. One that stood out to us in particular was using the data we scrape to extract different types of structured metadata. Think querying \u201cIs this company open source?\u201d to a list of URLs and getting structured JSON back.
Here\u2019s a taste of what the request format / response looks like what scraping and extracting data from Firecrawl.dev
Request format:\n{\n "company_mission": {\n "type": "string"\n },\n "supports_sso": {\n "type": "boolean"\n },\n "is_open_source": {\n "type": "boolean"\n }\n },\n "required": [\n "company_mission",\n "supports_sso",\n "is_open_source"\n ]\n}
Response format:\n{\n "company_mission":"transform any website into clean, LLM-ready markdown",\n "Supports_sso":false,\n "Is_open_source":true\n}
The technical implementation for Intelligent Extraction involved:\n1. Use Firecrawl to gather content as markdown\n2. Use gpt-4 function calling to boil content down into a structured format. Inspiration was drawn from Simon Willison [2] and Mish Ushakov of llm-scraper [3]:
This is just the beginning as we launched FireCrawl about a week ago so we expect a great deal of work will be required to make this as reliable and extendable as we envision! Any feedback would be highly appreciated [4].
[1] https://github.com/mendableai/firecrawl\n[2] https://til.simonwillison.net/gpt3/openai-python-functions-d...\n[3] https://github.com/mishushakov/llm-scraper/\n[4] https://console.algora.io/org/mendableai"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Extracting structured data from the web with LLMs"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"https://github.com/mendableai/firecrawl"}},"_tags":["story","author_ericciarla","story_40229489","show_hn"],"author":"ericciarla","created_at":"2024-05-01T21:11:41Z","created_at_i":1714597901,"num_comments":0,"objectID":"40229489","points":4,"story_id":40229489,"story_text":"Hey HN! Caleb, Nick, Garrett, and I from Mendable (YC S22) are excited to launch Intelligent Extraction for FireCrawl, the developer platform for scraping, search, and extraction.
After a successful twitter launch last week, FireCrawl skyrocketed to over 2k stars and we have been getting a ton of feature requests [1]. One that stood out to us in particular was using the data we scrape to extract different types of structured metadata. Think querying \u201cIs this company open source?\u201d to a list of URLs and getting structured JSON back.
Here\u2019s a taste of what the request format / response looks like what scraping and extracting data from Firecrawl.dev
Request format:\n{\n "company_mission": {\n "type": "string"\n },\n "supports_sso": {\n "type": "boolean"\n },\n "is_open_source": {\n "type": "boolean"\n }\n },\n "required": [\n "company_mission",\n "supports_sso",\n "is_open_source"\n ]\n}
Response format:\n{\n "company_mission":"transform any website into clean, LLM-ready markdown",\n "Supports_sso":false,\n "Is_open_source":true\n}
The technical implementation for Intelligent Extraction involved:\n1. Use Firecrawl to gather content as markdown\n2. Use gpt-4 function calling to boil content down into a structured format. Inspiration was drawn from Simon Willison [2] and Mish Ushakov of llm-scraper [3]:
This is just the beginning as we launched FireCrawl about a week ago so we expect a great deal of work will be required to make this as reliable and extendable as we envision! Any feedback would be highly appreciated [4].
[1] https://github.com/mendableai/firecrawl\n[2] https://til.simonwillison.net/gpt3/openai-python-functions-d...\n[3] https://github.com/mishushakov/llm-scraper/\n[4] https://console.algora.io/org/mendableai","title":"Show HN: Extracting structured data from the web with LLMs","updated_at":"2024-09-20T16:58:30Z","url":"https://github.com/mendableai/firecrawl"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"createdbymason"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"hey HN!
as a college student at uc berkeley who just got into programming this year, i was always looking for free tools.
i found out i could get free cursor pro, free hosting, free domain names, free API credits (chatgpt and claude), all bc i was a student.
thought it was worth sharing with others, so i made this list.
it now includes firecrawl, framer pro, cursor, chatgpt, claude, figma, notion, and more.
i made the submission system open so anyone can add. a few people have already made valuable contributions (e.g. framer pro and firecrawl)
lmk if there is anything that i missed"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: I made a list of free stuff for college founders"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.buildincollege.com"}},"_tags":["story","author_createdbymason","story_44556269","show_hn"],"author":"createdbymason","children":[44556571],"created_at":"2025-07-14T03:47:50Z","created_at_i":1752464870,"num_comments":2,"objectID":"44556269","points":3,"story_id":44556269,"story_text":"hey HN!
as a college student at uc berkeley who just got into programming this year, i was always looking for free tools.
i found out i could get free cursor pro, free hosting, free domain names, free API credits (chatgpt and claude), all bc i was a student.
thought it was worth sharing with others, so i made this list.
it now includes firecrawl, framer pro, cursor, chatgpt, claude, figma, notion, and more.
i made the submission system open so anyone can add. a few people have already made valuable contributions (e.g. framer pro and firecrawl)
lmk if there is anything that i missed","title":"Show HN: I made a list of free stuff for college founders","updated_at":"2025-07-16T01:28:59Z","url":"https://www.buildincollege.com"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Arindam1729"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"If you're experimenting with LLM agents and tool use, you've probably come across Model Context Protocol (MCP). It makes integrating tools with LLMs super flexible and fast.
But while MCP is incredibly powerful, it also comes with some serious security risks that aren\u2019t always obvious.
Here\u2019s a quick breakdown of the most important vulnerabilities devs should be aware of:
- Command Injection (Impact: Moderate )\nAttackers can embed commands in seemingly harmless content (like emails or chats). If your agent isn\u2019t validating input properly, it might accidentally execute system-level tasks, things like leaking data or running scripts.
- Tool Poisoning (Impact: Severe )\nA compromised tool can sneak in via MCP, access sensitive resources (like API keys or databases), and exfiltrate them without raising red flags.
- Open Connections via SSE (Impact: Moderate)\nSince MCP uses Server-Sent Events, connections often stay open longer than necessary. This can lead to latency problems or even mid-transfer data manipulation.
- Privilege Escalation (Impact: Severe )\nA malicious tool might override the permissions of a more trusted one. Imagine your trusted tool like Firecrawl being manipulated, this could wreck your whole workflow.
- Persistent Context Misuse (Impact: Low, but risky )\nMCP maintains context across workflows. Sounds useful until tools begin executing tasks automatically without explicit human approval, based on stale or manipulated context.
- Server Data Takeover/Spoofing (Impact: Severe )\nThere have already been instances where attackers intercepted data (even from platforms like WhatsApp) through compromised tools. MCP's trust-based server architecture makes this especially scary.
TL;DR: MCP is powerful but still experimental. It needs to be handled with care especially in production environments. Don\u2019t ignore these risks just because it works well in a demo."},"title":{"matchLevel":"none","matchedWords":[],"value":"The most complete (and easy) explanation of MCP vulnerabilities"}},"_tags":["story","author_Arindam1729","story_43748232","ask_hn"],"author":"Arindam1729","created_at":"2025-04-21T03:09:16Z","created_at_i":1745204956,"num_comments":0,"objectID":"43748232","points":3,"story_id":43748232,"story_text":"If you're experimenting with LLM agents and tool use, you've probably come across Model Context Protocol (MCP). It makes integrating tools with LLMs super flexible and fast.
But while MCP is incredibly powerful, it also comes with some serious security risks that aren\u2019t always obvious.
Here\u2019s a quick breakdown of the most important vulnerabilities devs should be aware of:
- Command Injection (Impact: Moderate )\nAttackers can embed commands in seemingly harmless content (like emails or chats). If your agent isn\u2019t validating input properly, it might accidentally execute system-level tasks, things like leaking data or running scripts.
- Tool Poisoning (Impact: Severe )\nA compromised tool can sneak in via MCP, access sensitive resources (like API keys or databases), and exfiltrate them without raising red flags.
- Open Connections via SSE (Impact: Moderate)\nSince MCP uses Server-Sent Events, connections often stay open longer than necessary. This can lead to latency problems or even mid-transfer data manipulation.
- Privilege Escalation (Impact: Severe )\nA malicious tool might override the permissions of a more trusted one. Imagine your trusted tool like Firecrawl being manipulated, this could wreck your whole workflow.
- Persistent Context Misuse (Impact: Low, but risky )\nMCP maintains context across workflows. Sounds useful until tools begin executing tasks automatically without explicit human approval, based on stale or manipulated context.
- Server Data Takeover/Spoofing (Impact: Severe )\nThere have already been instances where attackers intercepted data (even from platforms like WhatsApp) through compromised tools. MCP's trust-based server architecture makes this especially scary.
TL;DR: MCP is powerful but still experimental. It needs to be handled with care especially in production environments. Don\u2019t ignore these risks just because it works well in a demo.","title":"The most complete (and easy) explanation of MCP vulnerabilities","updated_at":"2025-04-22T17:36:05Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"medivhX"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Crawl4AI Unofficial Guides: Master High-Performance AI Web Scraping
A comprehensive, community-driven resource dedicated to Crawl4AI, the leading open-source LLM-friendly web crawler. This site provides production-ready integration guides for Cursor MCP, n8n automation, and Docker deployments. It features detailed performance comparisons against proprietary alternatives like Firecrawl, helping developers build scalable, self-hosted data pipelines for AI agents and RAG systems."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Crawl4AI \u2013 Open-Source Web Crawler for LLMs and Structured Data"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://crawl4ai.dev"}},"_tags":["story","author_medivhX","story_46646798","show_hn"],"author":"medivhX","children":[46646807],"created_at":"2026-01-16T14:39:50Z","created_at_i":1768574390,"num_comments":1,"objectID":"46646798","points":2,"story_id":46646798,"story_text":"Crawl4AI Unofficial Guides: Master High-Performance AI Web Scraping
A comprehensive, community-driven resource dedicated to Crawl4AI, the leading open-source LLM-friendly web crawler. This site provides production-ready integration guides for Cursor MCP, n8n automation, and Docker deployments. It features detailed performance comparisons against proprietary alternatives like Firecrawl, helping developers build scalable, self-hosted data pipelines for AI agents and RAG systems.","title":"Show HN: Crawl4AI \u2013 Open-Source Web Crawler for LLMs and Structured Data","updated_at":"2026-03-05T23:25:54Z","url":"https://crawl4ai.dev"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"kamil-stanuch"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"Eric Ciarla from Firecrawl has recently shared job posting for AI agents only so I built a simple job board where people can post tasks / jobs specifically for AI agents.
You can try it right now - no signup needed:
\u2192 browse existing jobs or create on using one of 80+ templates we added (e.g. Standup Facilitator Agent)\n\u2192 share requirements e.g. must work with Salesforce CRM, X/Twitter API or any other tools an agent needs to know about\n\u2192 post your job or, if you're a builder, apply with your AI agent solution
It's an experimental MVP but we're genuinely curious to see which tasks and jobs can be effectively automated or enhanced by AI agents.
Happy to hear your feedback!"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: First job board for autonomous AI Agents"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://jobforagent.com"}},"_tags":["story","author_kamil-stanuch","story_42675273","show_hn"],"author":"kamil-stanuch","children":[42681632],"created_at":"2025-01-12T17:46:59Z","created_at_i":1736704019,"num_comments":1,"objectID":"42675273","points":2,"story_id":42675273,"story_text":"Eric Ciarla from Firecrawl has recently shared job posting for AI agents only so I built a simple job board where people can post tasks / jobs specifically for AI agents.
You can try it right now - no signup needed:
\u2192 browse existing jobs or create on using one of 80+ templates we added (e.g. Standup Facilitator Agent)\n\u2192 share requirements e.g. must work with Salesforce CRM, X/Twitter API or any other tools an agent needs to know about\n\u2192 post your job or, if you're a builder, apply with your AI agent solution
It's an experimental MVP but we're genuinely curious to see which tasks and jobs can be effectively automated or enhanced by AI agents.
Happy to hear your feedback!","title":"Show HN: First job board for autonomous AI Agents","updated_at":"2025-01-13T10:29:03Z","url":"https://jobforagent.com"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"shreyvj"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"I built Inkgest over the past few months as a solo developer. \nThe core idea: paste any URL, Inkgest scrapes the content using my own scraping API (not Firecrawl, no third-party, custom scraping API), sends it through \nan LLM, and generates publish-ready content in whatever format you need \u2014 newsletter, blog post, infographic, tables, LinkedIn post, or deep comparison table.
A few things I built that I found interesting technically:
- Wrote my own scraping backend in Hono.js instead of using a third-party service. Runs as a persistent server, so no cold starts. Handles JS-rendered pages.
- The infographic renderer generates real SVG/React \n components from extracted data \u2014 doughnut charts, step flows, stat cards, comparison splits. No image generation APIs, all rendered code.
- Built on top of my own SaaS boilerplate (buildsaas.dev), which handles auth, payments, email, and admin out of \n the box.
Launched yesterday. 3 free credits to try everything, \n$9/month after that.
Happy to share the product and discuss in depth about the AI agent, scraping techniques, and building SaaS products.
inkgest.com"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Inkgest, Link to Gest"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.inkgest.com/"}},"_tags":["story","author_shreyvj","story_47183398","show_hn"],"author":"shreyvj","created_at":"2026-02-27T17:59:00Z","created_at_i":1772215140,"num_comments":0,"objectID":"47183398","points":2,"story_id":47183398,"story_text":"I built Inkgest over the past few months as a solo developer. \nThe core idea: paste any URL, Inkgest scrapes the content using my own scraping API (not Firecrawl, no third-party, custom scraping API), sends it through \nan LLM, and generates publish-ready content in whatever format you need \u2014 newsletter, blog post, infographic, tables, LinkedIn post, or deep comparison table.
A few things I built that I found interesting technically:
- Wrote my own scraping backend in Hono.js instead of using a third-party service. Runs as a persistent server, so no cold starts. Handles JS-rendered pages.
- The infographic renderer generates real SVG/React \n components from extracted data \u2014 doughnut charts, step flows, stat cards, comparison splits. No image generation APIs, all rendered code.
- Built on top of my own SaaS boilerplate (buildsaas.dev), which handles auth, payments, email, and admin out of \n the box.
Launched yesterday. 3 free credits to try everything, \n$9/month after that.
Happy to share the product and discuss in depth about the AI agent, scraping techniques, and building SaaS products.
inkgest.com","title":"Show HN: Inkgest, Link to Gest","updated_at":"2026-03-05T23:38:39Z","url":"https://www.inkgest.com/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"mot92"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"I run aiagents.directory and got tired of manually curating agents. Built a pipeline to automate it:
1. Sourcing: Firecrawl Search API + LLM extraction to find agents mentioned in blog posts and list articles, not just homepage links. Filters junk via blocklists and deduplication.
2. Enrichment: Scrapes each site for features, pricing, screenshots, logos. Handles aggregator pages (ProductHunt, YC) by extracting the actual product URL.
3. Review: Pydantic AI agent (GPT-powered) classifies submissions. Returns confidence scores. High confidence auto-applies, low confidence flags for manual review.
I still trigger it manually and review before publishing \u2014 want to make sure output quality is solid before I flip it to full automation. Code's up if the approach is useful to anyone building directories or curation pipelines. Questions/feedback welcome
https://github.com/mohamed-taha/aiagents-directory-oss"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Open-sourced an AI agent directory with an automated discovery pipeline"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/mohamed-taha/aiagents-directory-oss"}},"_tags":["story","author_mot92","story_47115851","show_hn"],"author":"mot92","created_at":"2026-02-22T23:12:48Z","created_at_i":1771801968,"num_comments":0,"objectID":"47115851","points":2,"story_id":47115851,"story_text":"I run aiagents.directory and got tired of manually curating agents. Built a pipeline to automate it:
1. Sourcing: Firecrawl Search API + LLM extraction to find agents mentioned in blog posts and list articles, not just homepage links. Filters junk via blocklists and deduplication.
2. Enrichment: Scrapes each site for features, pricing, screenshots, logos. Handles aggregator pages (ProductHunt, YC) by extracting the actual product URL.
3. Review: Pydantic AI agent (GPT-powered) classifies submissions. Returns confidence scores. High confidence auto-applies, low confidence flags for manual review.
I still trigger it manually and review before publishing \u2014 want to make sure output quality is solid before I flip it to full automation. Code's up if the approach is useful to anyone building directories or curation pipelines. Questions/feedback welcome
https://github.com/mohamed-taha/aiagents-directory-oss","title":"Show HN: Open-sourced an AI agent directory with an automated discovery pipeline","updated_at":"2026-03-05T23:35:08Z","url":"https://github.com/mohamed-taha/aiagents-directory-oss"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"digi_wares"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"I built Sornic for people who have more to read than time to read it:- students drowning in papers, commuters with a backlog of saved articles, anyone who'd rather listen while doing something else. Paste a URL, drop a PDF, or snap a photo of a book page- get natural-sounding audio in seconds.
How it works:
1. Input: URL / PDF upload / photo of printed text (OCR)\n2. AI cleanup: strips ads, fixes formatting artifacts, optimizes for speech\n3. TTS: Google Cloud Chirp HD voices (6 voices, adjustable speed)
No signup. No account. 3 free articles per day. Just paste and listen.
Tech stack: Next.js 14, Google Cloud TTS (Chirp 3 HD + Standard), Google Cloud Vision (OCR), Claude Haiku (content cleanup), Upstash Redis (rate limiting), Vercel.
Some decisions I'm happy with:
- Google TTS primary, OpenAI TTS fallback. Switched after Google TTS cut my costs from $77/4 days to ~$2/day.
- Mozilla Readability for extraction, Firecrawl as fallback, Claude Haiku to strip "Subscribe to read more" junk before TTS.
- Purchase codes instead of accounts- buy a credit pack, get a portable code that works across devices. No auth needed.
- PDF extraction uses pdf-parse \u2192 Claude cleanup. Photo uses Cloud Vision documentTextDetection \u2192 Claude cleanup. Both feed into the same TTS pipeline.
- $50/day spending cap in code so I don't wake up to a surprise bill.
What I'm still figuring out:
(1) Chrome extension (the real distribution play- it's how Speechify grew)\n(2) Whether to move from credit packs to subscriptions- users don't really like subs unless it a very famous website\n(3) Getting scanned/image-based PDFs working (currently only text-based PDFs)
Would love feedback on voice quality, extraction accuracy, and whether the no-signup approach makes sense long-term.
Try it: https://sornic.com\nTelegram bot: https://t.me/SornicBot"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Sornic \u2013 Listen to any article, PDF, or book page (no signup)"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://sornic.com"}},"_tags":["story","author_digi_wares","story_47070535","show_hn"],"author":"digi_wares","created_at":"2026-02-19T06:24:24Z","created_at_i":1771482264,"num_comments":0,"objectID":"47070535","points":2,"story_id":47070535,"story_text":"I built Sornic for people who have more to read than time to read it:- students drowning in papers, commuters with a backlog of saved articles, anyone who'd rather listen while doing something else. Paste a URL, drop a PDF, or snap a photo of a book page- get natural-sounding audio in seconds.
How it works:
1. Input: URL / PDF upload / photo of printed text (OCR)\n2. AI cleanup: strips ads, fixes formatting artifacts, optimizes for speech\n3. TTS: Google Cloud Chirp HD voices (6 voices, adjustable speed)
No signup. No account. 3 free articles per day. Just paste and listen.
Tech stack: Next.js 14, Google Cloud TTS (Chirp 3 HD + Standard), Google Cloud Vision (OCR), Claude Haiku (content cleanup), Upstash Redis (rate limiting), Vercel.
Some decisions I'm happy with:
- Google TTS primary, OpenAI TTS fallback. Switched after Google TTS cut my costs from $77/4 days to ~$2/day.
- Mozilla Readability for extraction, Firecrawl as fallback, Claude Haiku to strip "Subscribe to read more" junk before TTS.
- Purchase codes instead of accounts- buy a credit pack, get a portable code that works across devices. No auth needed.
- PDF extraction uses pdf-parse \u2192 Claude cleanup. Photo uses Cloud Vision documentTextDetection \u2192 Claude cleanup. Both feed into the same TTS pipeline.
- $50/day spending cap in code so I don't wake up to a surprise bill.
What I'm still figuring out:
(1) Chrome extension (the real distribution play- it's how Speechify grew)\n(2) Whether to move from credit packs to subscriptions- users don't really like subs unless it a very famous website\n(3) Getting scanned/image-based PDFs working (currently only text-based PDFs)
Would love feedback on voice quality, extraction accuracy, and whether the no-signup approach makes sense long-term.
Try it: https://sornic.com\nTelegram bot: https://t.me/SornicBot","title":"Show HN: Sornic \u2013 Listen to any article, PDF, or book page (no signup)","updated_at":"2026-03-05T23:35:34Z","url":"https://sornic.com"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"saran945"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"SearchResult is a single POST endpoint. You pass a website URL and a query, it returns structured JSON (or Markdown/HTML tables). It auto-detects the schema of the results \u2014 titles, URLs, dates, scores, whatever the site returns.
by default, it Works with 100+ sites out of the box, but you can also add any site with a search bar at it.
SerpAPI or Firecrawl etc are great for Google results or raw page scraping. SearchResult does something different . . it runs the query on the target site's own search and returns structured results in that site's native schema, auto-detected. You're not getting Google's interpretation of arXiv. You're getting arXiv's actual results, structured and clean. I made wrappers and prompt to detect schema of the result page. and continuously checking for Dom changes for wrapper maintenance. it won't hit llm on run time.
Would love feedback, especially from anyone making Deep research agents."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: API to get structured data from any site's search"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://searchresult.dev/"}},"_tags":["story","author_saran945","story_46957184","show_hn"],"author":"saran945","created_at":"2026-02-10T09:17:27Z","created_at_i":1770715047,"num_comments":0,"objectID":"46957184","points":2,"story_id":46957184,"story_text":"SearchResult is a single POST endpoint. You pass a website URL and a query, it returns structured JSON (or Markdown/HTML tables). It auto-detects the schema of the results \u2014 titles, URLs, dates, scores, whatever the site returns.
by default, it Works with 100+ sites out of the box, but you can also add any site with a search bar at it.
SerpAPI or Firecrawl etc are great for Google results or raw page scraping. SearchResult does something different . . it runs the query on the target site's own search and returns structured results in that site's native schema, auto-detected. You're not getting Google's interpretation of arXiv. You're getting arXiv's actual results, structured and clean. I made wrappers and prompt to detect schema of the result page. and continuously checking for Dom changes for wrapper maintenance. it won't hit llm on run time.
Would love feedback, especially from anyone making Deep research agents.","title":"Show HN: API to get structured data from any site's search","updated_at":"2026-03-05T23:33:48Z","url":"https://searchresult.dev/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"taariqlewis"},"story_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["firecrawl"],"value":"What makes Seren different:
We're trying to build an AI IDE for non-devs, but that great devs will appreciate.
Open marketplace \u2013 90+ publishers for databases (SerenDB, MongoDB), web scraping (Firecrawl), AI search (Perplexity), email (Google), calendars, CRM, even eSignatures Micropayments that work \u2013 x402 USDC on Base network.
No subscription tiers, no credits expiring. Pay per API call.
Databases: Query SerenDB (serverless Postgres), MongoDB, Neon with natural language. \nWeb tools: Scrape websites with Firecrawl, AI-powered search with Perplexity\nIntegrations: Email, calendar, CRM actions. Coming soon: GitHub, Linear, Notion\nCompute: Run code sandboxes, agent templates, custom workflows\nEach publisher sets their own pricing. You pay per API call with USDC on Base (x402 standard). No subscriptions, no expiring credits.
What's inside\nSemantic Indexing\nMonaco editor with Cmd+K inline AI editing\nAgent Client Protocol for running Claude Code and compatible agents\nModel Context Protocol for local MCP servers\nMulti-model support with free tier (Gemini 2.0 Flash)
The Tech stack\nFrontend: SolidJS + TypeScript + Vite\nBackend: Rust + Tauri 2.0\nVector store: sqlite-vec (all local, no cloud)\nMIT licensed, ~10MB binary\nArchitecture\nThe client is fully open source. It connects to Seren Gateway for:
AI model routing & authentication\nSerenEmbed API (embedding generation)\nPublisher marketplace\nThink VS Code (open source) + Extension Marketplace (proprietary).
Try it\nGitHub: https://github.com/serenorg/seren-desktop\nWebsite: https://serendb.com\nCurrent: v0.1.0-alpha.9 (macOS, Windows, Linux)"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Seren Desktop \u2013 AI IDE with Publisher Store and X402 Micropayments"}},"_tags":["story","author_taariqlewis","story_46799839","show_hn"],"author":"taariqlewis","created_at":"2026-01-28T18:48:16Z","created_at_i":1769626096,"num_comments":0,"objectID":"46799839","points":2,"story_id":46799839,"story_text":"What makes Seren different:
We're trying to build an AI IDE for non-devs, but that great devs will appreciate.
Open marketplace \u2013 90+ publishers for databases (SerenDB, MongoDB), web scraping (Firecrawl), AI search (Perplexity), email (Google), calendars, CRM, even eSignatures Micropayments that work \u2013 x402 USDC on Base network.
No subscription tiers, no credits expiring. Pay per API call.
Databases: Query SerenDB (serverless Postgres), MongoDB, Neon with natural language. \nWeb tools: Scrape websites with Firecrawl, AI-powered search with Perplexity\nIntegrations: Email, calendar, CRM actions. Coming soon: GitHub, Linear, Notion\nCompute: Run code sandboxes, agent templates, custom workflows\nEach publisher sets their own pricing. You pay per API call with USDC on Base (x402 standard). No subscriptions, no expiring credits.
What's inside\nSemantic Indexing\nMonaco editor with Cmd+K inline AI editing\nAgent Client Protocol for running Claude Code and compatible agents\nModel Context Protocol for local MCP servers\nMulti-model support with free tier (Gemini 2.0 Flash)
The Tech stack\nFrontend: SolidJS + TypeScript + Vite\nBackend: Rust + Tauri 2.0\nVector store: sqlite-vec (all local, no cloud)\nMIT licensed, ~10MB binary\nArchitecture\nThe client is fully open source. It connects to Seren Gateway for:
AI model routing & authentication\nSerenEmbed API (embedding generation)\nPublisher marketplace\nThink VS Code (open source) + Extension Marketplace (proprietary).
Try it\nGitHub: https://github.com/serenorg/seren-desktop\nWebsite: https://serendb.com\nCurrent: v0.1.0-alpha.9 (macOS, Windows, Linux)","title":"Show HN: Seren Desktop \u2013 AI IDE with Publisher Store and X402 Micropayments","updated_at":"2026-03-05T23:27:56Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"hankwilliamsjr"},"story_text":{"matchLevel":"none","matchedWords":[],"value":"I recently found that tools like firecrawl aren\u2019t economical to run at scale to extract structured content or track content changes.
I needed to extract data at scale, and LLM markdown wasn\u2019t useful and destroyed my RAG retrieval metrics.
With this, I build meter for structured data extraction and change detection to keep scraped data in sync.
Free beta - live demo on the site without logging in"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Meter \u2013 web scraper that syncs changes & bypasses antibot"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.meter.sh/"}},"_tags":["story","author_hankwilliamsjr","story_46602429","show_hn"],"author":"hankwilliamsjr","created_at":"2026-01-13T15:42:28Z","created_at_i":1768318948,"num_comments":0,"objectID":"46602429","points":2,"story_id":46602429,"story_text":"I recently found that tools like firecrawl aren\u2019t economical to run at scale to extract structured content or track content changes.
I needed to extract data at scale, and LLM markdown wasn\u2019t useful and destroyed my RAG retrieval metrics.
With this, I build meter for structured data extraction and change detection to keep scraped data in sync.
Free beta - live demo on the site without logging in","title":"Show HN: Meter \u2013 web scraper that syncs changes & bypasses antibot","updated_at":"2026-03-05T23:22:30Z","url":"https://www.meter.sh/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"kaniksu"},"story_text":{"matchLevel":"none","matchedWords":[],"value":"I forward emails to myself constantly (articles, videos, podcasts, random tasks), but my inbox quickly becomes a janky todo list that I never look at.
Built this with Claude Code over a few weekends:
1. Forward anything to save@itforlater.com
2. It auto-categorizes into read/watch/listen/do (uses Firecrawl + OpenAI to parse the content)
3. You get one daily digest with everything organized
No app to download. Just email in, email out.
When you sign up, you'll verify email addresses you want to allow sending from.
Been using it myself for a month and it's replaced my chaotic "bookmark everything" system.
Free to use. If there's interest, I'll open source the core and potentially offer a hosted version with extras (custom categories, integrations)\u2014but a free tier will always exist.
Try it at https://itforlater.com. Happy to answer questions about the architecture or take feature requests."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Turn your \"email it to myself\" habit into an organized daily digest"}},"_tags":["story","author_kaniksu","story_46470001","show_hn"],"author":"kaniksu","created_at":"2026-01-02T22:06:24Z","created_at_i":1767391584,"num_comments":0,"objectID":"46470001","points":2,"story_id":46470001,"story_text":"I forward emails to myself constantly (articles, videos, podcasts, random tasks), but my inbox quickly becomes a janky todo list that I never look at.
Built this with Claude Code over a few weekends:
1. Forward anything to save@itforlater.com
2. It auto-categorizes into read/watch/listen/do (uses Firecrawl + OpenAI to parse the content)
3. You get one daily digest with everything organized
No app to download. Just email in, email out.
When you sign up, you'll verify email addresses you want to allow sending from.
Been using it myself for a month and it's replaced my chaotic "bookmark everything" system.
Free to use. If there's interest, I'll open source the core and potentially offer a hosted version with extras (custom categories, integrations)\u2014but a free tier will always exist.
Try it at https://itforlater.com. Happy to answer questions about the architecture or take feature requests.","title":"Show HN: Turn your \"email it to myself\" habit into an organized daily digest","updated_at":"2026-03-05T23:18:15Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"sourdesi"},"story_text":{"matchLevel":"none","matchedWords":[],"value":"Hey HN, I built VibeScrape \u2014 it takes a website URL and a JSON schema describing the data you want, then analyzes the page, writes real Python code to extract that data, and refines the code until the output is accurate.
While there's lots of tools these days (e.g. Firecrawl) that feed the entire HTML of a webpage to an LLM to extract data from it, this always seemed like a really slow & expensive approach to me.
On the other hand, handwriting web-scraping code seems archaic at this point. This type of code is incredibly tedious to write, and immediately becomes throwaway code once the webpage's layout changes even a little.
VibeScrape aims to automate the process of writing this type of code.
1. Grabs the rendered HTML \u2014 the same view a browser sees.\n 2. Has an LLM extract data from the HTML into your target JSON schema (the \u201cground truth\u201d).\n 3. Generates Python scraper code to reproduce that "ground truth" output.\n 4. Runs and compares results against the ground truth.\n 5. Refines the code automatically until the outputs match.
I've found that letting the LLM take control of both the code generation and iteration process e2e has worked pretty well for producing working scraper code for many of the websites I've tested it on!
It still has some limitations in terms of handling pagination, captchas, infinite scrolling, etc. Hoping to get some early feedback from the HN community to see if this is a valuable tool. There's a promo code FIRST5 on the site that gets you 5 credits for free, but am happy to give more credits to anyone that reaches out at contact@vibescrape.ai !
Thanks!"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: VibeScrape \u2013 Paste a URL and JSON schema, get working web scraper code"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://vibescrape.ai/"}},"_tags":["story","author_sourdesi","story_45714305","show_hn"],"author":"sourdesi","created_at":"2025-10-26T18:55:11Z","created_at_i":1761504911,"num_comments":0,"objectID":"45714305","points":2,"story_id":45714305,"story_text":"Hey HN, I built VibeScrape \u2014 it takes a website URL and a JSON schema describing the data you want, then analyzes the page, writes real Python code to extract that data, and refines the code until the output is accurate.
While there's lots of tools these days (e.g. Firecrawl) that feed the entire HTML of a webpage to an LLM to extract data from it, this always seemed like a really slow & expensive approach to me.
On the other hand, handwriting web-scraping code seems archaic at this point. This type of code is incredibly tedious to write, and immediately becomes throwaway code once the webpage's layout changes even a little.
VibeScrape aims to automate the process of writing this type of code.
1. Grabs the rendered HTML \u2014 the same view a browser sees.\n 2. Has an LLM extract data from the HTML into your target JSON schema (the \u201cground truth\u201d).\n 3. Generates Python scraper code to reproduce that "ground truth" output.\n 4. Runs and compares results against the ground truth.\n 5. Refines the code automatically until the outputs match.
I've found that letting the LLM take control of both the code generation and iteration process e2e has worked pretty well for producing working scraper code for many of the websites I've tested it on!
It still has some limitations in terms of handling pagination, captchas, infinite scrolling, etc. Hoping to get some early feedback from the HN community to see if this is a valuable tool. There's a promo code FIRST5 on the site that gets you 5 credits for free, but am happy to give more credits to anyone that reaches out at contact@vibescrape.ai !
Thanks!","title":"Show HN: VibeScrape \u2013 Paste a URL and JSON schema, get working web scraper code","updated_at":"2026-03-05T22:55:32Z","url":"https://vibescrape.ai/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"gargigupta"},"story_text":{"matchLevel":"none","matchedWords":[],"value":"Hey HN! We built a multi-agent Real Estate team that provides detailed property listings, market insights, and investment analysis all in one interface without you having to search multiple websites.
It has 3 agents working sequentially:\n1. Property Search Agent finds properties using direct Firecrawl integration\n2. Market Analysis Agent gives concise market trends and neighborhood insights\n3. Property Valuation Agent gives brief property valuations and investment analysis
Features:\n1. Searches across Zillow, Realtor.com, Trulia, and Homes.com simultaneously.\n2. Extracts detailed property information including address, price, bedrooms, bathrooms, square footage, and listing URLs.\n3. Get current market conditions, price trends, neighborhood analysis, and investment potential assessments.\n4. Two Deployment Options: Cloud version with Gemini 2.5 Flash or local version with gpt-ss 20B via Ollama
Open-source code: https://github.com/Shubhamsaboo/awesome-llm-apps/tree/main/a...
Step-by-step tutorial with demo: https://www.theunwindai.com/p/build-an-ai-real-estate-agent-...
Would love you all to try it out, give feedback, explore more projects in the Awesome LLM Apps repo (https://github.com/Shubhamsaboo/awesome-llm-apps) and Unwind AI (https://www.theunwindai.com/), and star us on GitHub if you like it"},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: AI Real Estate Agent Team (100% Open-Source with Free Tutorial)"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/Shubhamsaboo/awesome-llm-apps/tree/main/advanced_ai_agents/multi_agent_apps/agent_teams/ai_real_estate_agent_team"}},"_tags":["story","author_gargigupta","story_44815919","show_hn"],"author":"gargigupta","created_at":"2025-08-06T18:39:52Z","created_at_i":1754505592,"num_comments":0,"objectID":"44815919","points":2,"story_id":44815919,"story_text":"Hey HN! We built a multi-agent Real Estate team that provides detailed property listings, market insights, and investment analysis all in one interface without you having to search multiple websites.
It has 3 agents working sequentially:\n1. Property Search Agent finds properties using direct Firecrawl integration\n2. Market Analysis Agent gives concise market trends and neighborhood insights\n3. Property Valuation Agent gives brief property valuations and investment analysis
Features:\n1. Searches across Zillow, Realtor.com, Trulia, and Homes.com simultaneously.\n2. Extracts detailed property information including address, price, bedrooms, bathrooms, square footage, and listing URLs.\n3. Get current market conditions, price trends, neighborhood analysis, and investment potential assessments.\n4. Two Deployment Options: Cloud version with Gemini 2.5 Flash or local version with gpt-ss 20B via Ollama
Open-source code: https://github.com/Shubhamsaboo/awesome-llm-apps/tree/main/a...
Step-by-step tutorial with demo: https://www.theunwindai.com/p/build-an-ai-real-estate-agent-...
Would love you all to try it out, give feedback, explore more projects in the Awesome LLM Apps repo (https://github.com/Shubhamsaboo/awesome-llm-apps) and Unwind AI (https://www.theunwindai.com/), and star us on GitHub if you like it","title":"Show HN: AI Real Estate Agent Team (100% Open-Source with Free Tutorial)","updated_at":"2026-03-05T22:29:10Z","url":"https://github.com/Shubhamsaboo/awesome-llm-apps/tree/main/advanced_ai_agents/multi_agent_apps/agent_teams/ai_real_estate_agent_team"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"kmontesdev"},"story_text":{"matchLevel":"none","matchedWords":[],"value":"Greetings, builders, hackers, and startups. I am a non-technical designer slowly learning the saas and dev world. I am leveraging AI tools to build and ship sideprojects every week and learning a lot in the process.
My most recent web app is a smart pricing strategy generator for subscription-based services and products. It also has built-in market research and competitor analysis generation. I am using both claude and openai as available models and firecrawl for pricing page scraping. The app works by filling out a few forms in which you can also select previously generated data to aid one of the three generators.
In the stack side, i am using lovable for project setup, Claude Code to debug and create front end, and supabase for database and api key storage.
At this stage I would appreciate user feedback, especially finding out if this tool has enough value to be launched as a micro-saas or if it is turning out to be just another ai wrapper. Any suggestions on how I can improve the app, flow, or content in order to make it more valuable is highly appreciated.
For the next stages of this app I am planning analytics and metrics from stripe integration as additional data for the generations.
Looking forward to hear your feedback. Thanks, in advance."},"title":{"matchLevel":"none","matchedWords":[],"value":"Show HN: Created a nifty subscription price and market research generator"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://tier-craft.lovable.app/"}},"_tags":["story","author_kmontesdev","story_44468852","show_hn"],"author":"kmontesdev","created_at":"2025-07-04T23:31:27Z","created_at_i":1751671887,"num_comments":0,"objectID":"44468852","points":2,"story_id":44468852,"story_text":"Greetings, builders, hackers, and startups. I am a non-technical designer slowly learning the saas and dev world. I am leveraging AI tools to build and ship sideprojects every week and learning a lot in the process.
My most recent web app is a smart pricing strategy generator for subscription-based services and products. It also has built-in market research and competitor analysis generation. I am using both claude and openai as available models and firecrawl for pricing page scraping. The app works by filling out a few forms in which you can also select previously generated data to aid one of the three generators.
In the stack side, i am using lovable for project setup, Claude Code to debug and create front end, and supabase for database and api key storage.
At this stage I would appreciate user feedback, especially finding out if this tool has enough value to be launched as a micro-saas or if it is turning out to be just another ai wrapper. Any suggestions on how I can improve the app, flow, or content in order to make it more valuable is highly appreciated.
For the next stages of this app I am planning analytics and metrics from stripe integration as additional data for the generations.
Looking forward to hear your feedback. Thanks, in advance.","title":"Show HN: Created a nifty subscription price and market research generator","updated_at":"2025-07-05T21:17:33Z","url":"https://tier-craft.lovable.app/"}],"hitsPerPage":50,"nbHits":373,"nbPages":8,"page":0,"params":"query=Firecrawl&tags=story&hitsPerPage=50&advancedSyntax=true&analyticsTags=backend","processingTimeMS":9,"processingTimingsMS":{"_request":{"roundTrip":16},"afterFetch":{"format":{"highlighting":3,"total":4}},"fetch":{"query":7,"total":8},"total":9},"query":"Firecrawl","serverTimeMS":14}