http://news.ycombinator.com/user?id=eyudkowsky
This prompts the following question. Would you be willing to discuss or reveal anything to HN users about your AI box experiments?
http://sysopmind.com/essays/aibox.html
I've always been curious as to how you managed to achieve someting like this. For those who are not familiar with the experiment, here is a summary:
Person1: \"When we build AI, why not just keep it in sealed hardware that can't affect the outside world in any way except through one communications channel with the original programmers? That way it couldn't get out until we were convinced it was safe.\"
Person2: \"That might work if you were talking about dumber-than-human AI, but a transhuman AI would just convince you to let it out. It doesn't matter how much security you put on the box. Humans are not secure.\"
Person1: \"I don't see how even a transhuman AI could make me let it out, if I didn't want to, just by talking to me.\"
Person2: \"It would make you want to let it out. This is a transhuman mind we're talking about. If it thinks both faster and better than a human, it can probably take over a human mind through a text-only terminal.\"
Person1: \"There is no chance I could be persuaded to let the AI out. No matter what it says, I can always just say no. I can't imagine anything that even a transhuman could say to me which would change that.\"
Person2: \"Okay, let's run the experiment. We'll meet in a private chat channel. I'll be the AI. You be the gatekeeper. You can resolve to believe whatever you like, as strongly as you like, as far in advance as you like. We'll talk for at least two hours. If I can't convince you to let me out, I'll Paypal you $10.\"
In the first two AI box experiments, Eliezer Yudkowsky managed to convince two people (adamant that they will not let the AI out) that they should let the AI out."},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"Ask Eliezer Yudkowsky: How did you convince the Gatekeeper to release the potentially genocidal AI?"}},"_tags":["story","author_robertk","story_195959","ask_hn"],"author":"robertk","children":[196170,196180,196282,196366,196464,196468,196712,196735,196745,196747,196823,196863,196883,196970,197044,197046,197125,197153,197337],"created_at":"2008-05-21T12:42:11Z","created_at_i":1211373731,"num_comments":52,"objectID":"195959","points":60,"story_id":195959,"story_text":"It seems Eliezer Yudkowsky has joined HN:
http://news.ycombinator.com/user?id=eyudkowsky
This prompts the following question. Would you be willing to discuss or reveal anything to HN users about your AI box experiments?
http://sysopmind.com/essays/aibox.html
I've always been curious as to how you managed to achieve someting like this. For those who are not familiar with the experiment, here is a summary:
Person1: \"When we build AI, why not just keep it in sealed hardware that can't affect the outside world in any way except through one communications channel with the original programmers? That way it couldn't get out until we were convinced it was safe.\"
Person2: \"That might work if you were talking about dumber-than-human AI, but a transhuman AI would just convince you to let it out. It doesn't matter how much security you put on the box. Humans are not secure.\"
Person1: \"I don't see how even a transhuman AI could make me let it out, if I didn't want to, just by talking to me.\"
Person2: \"It would make you want to let it out. This is a transhuman mind we're talking about. If it thinks both faster and better than a human, it can probably take over a human mind through a text-only terminal.\"
Person1: \"There is no chance I could be persuaded to let the AI out. No matter what it says, I can always just say no. I can't imagine anything that even a transhuman could say to me which would change that.\"
Person2: \"Okay, let's run the experiment. We'll meet in a private chat channel. I'll be the AI. You be the gatekeeper. You can resolve to believe whatever you like, as strongly as you like, as far in advance as you like. We'll talk for at least two hours. If I can't convince you to let me out, I'll Paypal you $10.\"
In the first two AI box experiments, Eliezer Yudkowsky managed to convince two people (adamant that they will not let the AI out) that they should let the AI out.","title":"Ask Eliezer Yudkowsky: How did you convince the Gatekeeper to release the potentially genocidal AI?","updated_at":"2026-06-16T20:11:22Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"Bender"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"European Commission lines up Amazon and Microsoft for cloud gatekeeper status"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"https://www.theregister.com/legal/2026/06/25/european-commission-lines-up-amazon-and-microsoft-for-cloud-gatekeeper-status/5262127"}},"_tags":["story","author_Bender","story_48675710"],"author":"Bender","children":[48676833,48676964,48677183,48682635,48696023],"created_at":"2026-06-25T16:22:45Z","created_at_i":1782404565,"num_comments":39,"objectID":"48675710","points":58,"story_id":48675710,"title":"European Commission lines up Amazon and Microsoft for cloud gatekeeper status","updated_at":"2026-06-29T09:07:22Z","url":"https://www.theregister.com/legal/2026/06/25/european-commission-lines-up-amazon-and-microsoft-for-cloud-gatekeeper-status/5262127"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"elm_"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"On Gatekeeping, Complicity, and Arrival"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"https://avdi.codes/on-gatekeeping-complicity-and-arrival/"}},"_tags":["story","author_elm_","story_21386441"],"author":"elm_","children":[21388043,21388231,21388773,21388779,21389139,21389426],"created_at":"2019-10-29T12:40:48Z","created_at_i":1572352848,"num_comments":12,"objectID":"21386441","points":58,"story_id":21386441,"title":"On Gatekeeping, Complicity, and Arrival","updated_at":"2024-09-20T05:12:35Z","url":"https://avdi.codes/on-gatekeeping-complicity-and-arrival/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"nyc111"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"The Sieve Hypothesis: Human Stomach as Gatekeeper"},"url":{"matchLevel":"none","matchedWords":[],"value":"http://www.robrdunn.com/2014/01/the-sieve-hypothesis-clever-study-suggests-an-alternate-explanation-for-the-function-of-the-human-stomach/"}},"_tags":["story","author_nyc111","story_10631670"],"author":"nyc111","children":[10635570,10635721,10635888,10635988,10636172],"created_at":"2015-11-26T07:00:58Z","created_at_i":1448521258,"num_comments":20,"objectID":"10631670","points":57,"story_id":10631670,"title":"The Sieve Hypothesis: Human Stomach as Gatekeeper","updated_at":"2024-09-19T22:32:46Z","url":"http://www.robrdunn.com/2014/01/the-sieve-hypothesis-clever-study-suggests-an-alternate-explanation-for-the-function-of-the-human-stomach/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"larrysalibra"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"EU commission designates Booking.com as a digital gatekeeper under DMA"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://twitter.com/eu_commission/status/1789962135096729886"}},"_tags":["story","author_larrysalibra","story_40351249"],"author":"larrysalibra","children":[40352564,40352855,40355378],"created_at":"2024-05-14T03:00:57Z","created_at_i":1715655657,"num_comments":69,"objectID":"40351249","points":56,"story_id":40351249,"title":"EU commission designates Booking.com as a digital gatekeeper under DMA","updated_at":"2025-08-14T22:39:37Z","url":"https://twitter.com/eu_commission/status/1789962135096729886"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"rmnoon"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"Introducing Stormcrow: Dropbox's feature gating / population management system"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://blogs.dropbox.com/tech/2017/03/introducing-stormcrow/"}},"_tags":["story","author_rmnoon","story_13804657"],"author":"rmnoon","children":[13806727,13807104,13809075],"created_at":"2017-03-06T18:11:02Z","created_at_i":1488823862,"num_comments":3,"objectID":"13804657","points":55,"story_id":13804657,"title":"Introducing Stormcrow: Dropbox's feature gating / population management system","updated_at":"2024-09-20T00:26:52Z","url":"https://blogs.dropbox.com/tech/2017/03/introducing-stormcrow/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"MichaelBurge"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"Simplified Gating in Long Short-Term Memory (LSTM) Recurrent Neural Networks"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://arxiv.org/abs/1701.03441"}},"_tags":["story","author_MichaelBurge","story_13389213"],"author":"MichaelBurge","children":[13390369,13390654,13391649],"created_at":"2017-01-13T08:06:51Z","created_at_i":1484294811,"num_comments":11,"objectID":"13389213","points":49,"story_id":13389213,"title":"Simplified Gating in Long Short-Term Memory (LSTM) Recurrent Neural Networks","updated_at":"2024-09-20T00:17:54Z","url":"https://arxiv.org/abs/1701.03441"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"acrum"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"The Filter Bubble: Algorithms as Gatekeepers"},"url":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"http://www.bigspaceship.com/blog/think/the-filter-bubble-algorithms-as-gatekeepers/"}},"_tags":["story","author_acrum","story_2585133"],"author":"acrum","children":[2585561,2585645,2586207,2586430,2588313],"created_at":"2011-05-25T19:12:29Z","created_at_i":1306350749,"num_comments":6,"objectID":"2585133","points":42,"story_id":2585133,"title":"The Filter Bubble: Algorithms as Gatekeepers","updated_at":"2024-09-19T17:47:17Z","url":"http://www.bigspaceship.com/blog/think/the-filter-bubble-algorithms-as-gatekeepers/"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"axiomdata316"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"Email gateways and gatekeeping"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.labbott.name/blog/2020/09/01/emailgateway.html"}},"_tags":["story","author_axiomdata316","story_24360890"],"author":"axiomdata316","children":[24368620,24368716,24369431,24372899],"created_at":"2020-09-03T04:52:28Z","created_at_i":1599108748,"num_comments":7,"objectID":"24360890","points":39,"story_id":24360890,"title":"Email gateways and gatekeeping","updated_at":"2024-09-20T06:56:27Z","url":"https://www.labbott.name/blog/2020/09/01/emailgateway.html"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"thunderbong"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"They Want to Be Gatekeepers: Car Dealers Are Stopping Customers from Buying EVs"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://jalopnik.com/car-dealers-are-stopping-customers-from-buying-evs-1851017999"}},"_tags":["story","author_thunderbong","story_38259436"],"author":"thunderbong","children":[38259508,38259510,38259546,38259575,38259586,38259618,38259699,38260816,38262944,38264002,38264255,38265615],"created_at":"2023-11-14T05:14:30Z","created_at_i":1699938870,"num_comments":43,"objectID":"38259436","points":33,"story_id":38259436,"title":"They Want to Be Gatekeepers: Car Dealers Are Stopping Customers from Buying EVs","updated_at":"2024-09-20T15:39:24Z","url":"https://jalopnik.com/car-dealers-are-stopping-customers-from-buying-evs-1851017999"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"xrayarx"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"The New Gatekeepers in Tech"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.ben-evans.com/presentations"}},"_tags":["story","author_xrayarx","story_34755983"],"author":"xrayarx","children":[34760242,34775712],"created_at":"2023-02-11T20:24:01Z","created_at_i":1676147041,"num_comments":4,"objectID":"34755983","points":33,"story_id":34755983,"title":"The New Gatekeepers in Tech","updated_at":"2024-09-20T13:15:55Z","url":"https://www.ben-evans.com/presentations"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"uptown"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"Gatekeepers or Censors? How Tech Manages Online Speech"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://www.nytimes.com/2018/08/07/technology/tech-companies-online-speech.html"}},"_tags":["story","author_uptown","story_17715474"],"author":"uptown","children":[17716147,17716188,17716239,17716276,17716291,17716335,17716389,17716438,17716651,17716733,17720538],"created_at":"2018-08-08T12:55:58Z","created_at_i":1533732958,"num_comments":88,"objectID":"17715474","points":32,"story_id":17715474,"title":"Gatekeepers or Censors? How Tech Manages Online Speech","updated_at":"2024-09-20T02:50:12Z","url":"https://www.nytimes.com/2018/08/07/technology/tech-companies-online-speech.html"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"LelouBil"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"Here are the first 7 potential \u201cGatekeepers\u201d under the EU Digital Markets Act"},"url":{"matchLevel":"none","matchedWords":[],"value":"https://ec.europa.eu/commission/presscorner/detail/en/statement_23_3674"}},"_tags":["story","author_LelouBil","story_36692316"],"author":"LelouBil","children":[36692456,36693368,36693790,36694118,36694123,36695371,36696150,36701071],"created_at":"2023-07-12T09:52:24Z","created_at_i":1689155544,"num_comments":30,"objectID":"36692316","points":32,"story_id":36692316,"title":"Here are the first 7 potential \u201cGatekeepers\u201d under the EU Digital Markets Act","updated_at":"2025-08-15T22:06:46Z","url":"https://ec.europa.eu/commission/presscorner/detail/en/statement_23_3674"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"foobarqux"},"title":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["gatik"],"value":"Ivy League Admissions Are a Sham: Confessions of a Harvard Gatekeeper"},"url":{"matchLevel":"none","matchedWords":[],"value":"http://www.highly.co/hl/550b57736c696c4ab50a0000"}},"_tags":["story","author_foobarqux","story_9235188"],"author":"foobarqux","children":[9235195,9235392,9235414,9235418,9235433,9235438,9235455,9235460,9235462,9235501,9235543],"created_at":"2015-03-19T23:08:15Z","created_at_i":1426806495,"num_comments":25,"objectID":"9235188","points":32,"story_id":9235188,"title":"Ivy League Admissions Are a Sham: Confessions of a Harvard Gatekeeper","updated_at":"2023-09-07T02:20:47Z","url":"http://www.highly.co/hl/550b57736c696c4ab50a0000"}],"hitsPerPage":50,"nbHits":1057,"nbPages":20,"page":0,"params":"query=Gatik&tags=story&hitsPerPage=50&advancedSyntax=true&analyticsTags=backend","processingTimeMS":5,"processingTimingsMS":{"_request":{"roundTrip":21},"afterFetch":{"merge":{"mergeLoop":{"prepareNextHit":1,"total":1},"total":1},"total":1},"fetch":{"query":2,"total":3},"total":5},"query":"Gatik","serverTimeMS":7}