While everyone has been worried about it becoming too easy to hack into things, something I've been worried about for the last few years is it becoming too hard to hack into things. Right now, all of these things are being broken into to be jailbroken and such, but the companies building these hardware devices have access to the models too.
It seems like a distinctly possible and sad world that they might get so much better at keeping their devices locked down by hardening them with the support of LLMs that future devices become an absolute bitch to jailbreak, even moreso than they were before.
I don’t think it’s symmetrical. As a company, in a lot of cases, you get one shot to secure the thing. And then all owners collectively then get infinite time to jailbreak it.
Only until consumers get fed up with everything from their TV to their coffee maker spying on them and start demanding IOT products that don't actually connect to the Internet and instead interact with a local gateway device. That may never happen, but the barriers to making all your gadgets cooperate and speak the same protocol instead of requiring vendor-specific apps are definitely getting lower.
I think this is true, but on the flip side it could become common to buy "blank" hardware where the user provides their own OS or firmware. In the past it was just too burdensome to do this so we expected the hardware vendor to provide the software, but we could enter an era where providing the software is relatively easy.
I think one area we are seeing this is the flood of hand held emulator gaming machines. Companies in China pump these things out and the community has provided numerous custom firmwares to the point a new one seems to come out every week. All vibe coded in Rust but who cares, if it plays the games and looks nice that's all you need.
If by "secure" you mean keep owners from using a device in whatever way they wish, then I hope whoever made that decision fails, and I wish them discomfort and inconvenience as karma for the attempt.
Nah, we just start opening the phones, cutting off some PBC patches, soldering some wires, and intercepting/synthesizing signals live. Keeping phones locked is just a zero sum game.
Yes, and bus encryption. I used to think "if a human made it, a human can break it" but that may no longer be true with disparate access to frontier models.
Nintendo manages to squeak out a margin on hardware unlike the other big players. However, this is only because they trail behind in specs, and their devices are simply cheaper.
At least for now, they have to build the hardware with the old shitty AI, and then we get to... let's just say "utilize", the hardware we own using the new and much smarter AIs.
I would think the more viable path (in the US) would be the legal system. i.e. Amazon pressuring frontier labs to add "safeguards" to protect their IP.
What you’re essentially saying is right now we have weak human written code that might have some vulnerabilities, but something written by a frontier LLM may not have those.
More like companies can spend the money to have AI scan everything and pick off all the low hanging fruit. I work on a legacy codebase and these new AI tools are finding mountains of very stupid very serious security issues. After we finally get through them we will have a codebase that has no easily found exploits left.
Not exactly imo, you could spend a crapton of money with white box pen testing with a bunch of frontier llms, and it might become a bunch harder to break into (it generally costs more to attack than it does to defend). Not to mention formal verification might become possible for more devices which might make it nearly impossible to find exploits.
Opus 5.5 has built Lineage ROMs for me for 3 different devices in the last couple of weeks, that do not have official Lineage support or any one else working on them.
It sounds like most of the work went into getting the slideshow to work, setting up a feedback loop so Astra could work out / blunder into a solution with the desired characteristics.
Reading this causes embarassment from author's helplessness.
> I first tried GPT-5.6 Sol with reasoning effort set to Extra High. It wrote code to connect to the Kindle but couldn't resolve an error that prevented it from storing data on the device. Then I tried Fable 5.1 and Opus 5.5, but Anthropic's cybersecurity safeguards blocked each attempt after two prompts. Finally, I tried GPT-6 Astra with reasoning effort set to High. We were able to jailbreak the Kindle.
"We were able to".
It takes a modern AI human and 4 LLMs to do something that could be done by googling "kindle jailbreak" and following instructions - they have to be performed on Kindle anyway.
I'm a seasoned embedded engineer and it took me hours to figure what the "right" jailbreak was and how it worked. In the end it was easy to accomplish but as usual the documentation is abysmal.
I sometimes catch myself saying "we" when prompting AI so it really doesn't bother me to see this language being used. But I concede it can sound jarring or hubristic.
This is one reason why I think the AI revolution may not in the short term be a huge stock-market boom. By breaking down anti-competitive walls it could in the near term hurt incumbents (amazon, google, reddit, etc).
How soon until we all have jailbroken our TV ads out of existence? What next, jailbreaking the adds off our netflix accounts? To be clear I think this is an excellent development for everyone, however evenly distributed good doesn't usually positively correlate with share price.
I was able to do something pretty similar w Claude Code 4 months ago w Opus 4.8 (https://github.com/brunomarinho/labs-kindle-artframe). It took a few attempts as Claude understands that jailbreaking is not legal.
I find that fascinating because I was able to get Claude to jailbreak my Kindles and add them to my tech ecosystem at home. I also was able to get it to use jailbreaking hacks on my Spotify car thing and implement some alternative ways for my raspberry pi to drive it.
I wanted to do this with a google home hub so i can run my own fully local "home assisant", but claude rejected it... instead i ordered a used lenovo thinkview device which appetently has a community around custom bootloading...
But it does make you think there might be a real market for unlocked devices like this now that it is trivial to build your own software to do exactly what you want.
claude code helped me get access to the raw files from a scientific instrument we use in the lab all the time, but which won't give us raw data (only PDF reports). claude asked me for the .exe, which it promptly decompiled and reverse engineered the data format, and then helped me write my own analysis suite for the raw data. super cool.
Just a quick reminder, you have the legal right to "jailbreak" devices that you own. This is a recognized exemption and a right that has been fought over in court.
> 9. Computer Programs—Jailbreaking [47]
>
> Multiple organizations petitioned to renew the four exemptions for computer programs that enable electronic devices to interoperate with or to remove software applications (“jailbreaking”). These exemptions permit circumvention for the purpose of jailbreaking (1) smartphones and other portable all-purpose computing devices, (2) smart televisions, (3) voice assistant devices, and (4) routers and dedicated networking devices. No oppositions were filed against renewal. The petitions demonstrated the continuing need and justification for the exemption and that petitioners have personal knowledge and experience with regard to this exemption. Petitioners described how users of a variety products in each of these categories rely on this exemption to maintain functionality and security of older devices, to install alternative operation systems, and to customize software applications on electronic devices. Collectively, the petitions demonstrated that without this exemption, TPMs installed on the enumerated products would have an adverse effect on various noninfringing uses.
and
> 11. Computer Programs—Repairs of Devices Designed Primarily for Use by Consumers [51]
>
> The Electronic Frontier Foundation (“EFF”) petitioned to renew the exemption for computer programs that control devices designed primarily for use by consumers for diagnosis, maintenance, or repair of the device. The Office did not receive meaningful opposition to renewal.[52]
>
> The petitioners demonstrated the continuing need and justification for the exemption. For example, EFF asserted that “[m]anufacturers of these devices continue to implement technological protection measures that inhibit lawful repairs, maintenance, and diagnostics, and they show no sign of changing course.” [53]
>
> EFF has personal knowledge of this exemption, as it has been involved with the section 1201 rulemaking process since its inception and has specifically advocated for device repair exemptions.
I jailbroke my Kindle with GPT-5.5 Sol and turned it into an external e-paper display for the long batches of text AI models generate on a regular basis.
Please note, I say so in the human note on top, but "The project code was written by GPT-5.6 Sol xhigh. It has not been verified by a human for correctness, efficiency, or speed; i.e. ymmv and use this at your personal risk."
In case you're wondering why I didn't write the code myself... I needed this as a tool to read long sequences of AI text output for work. Including code. Spending time making this tool would be counter-productive to that goal.
I do plan on writing a text rendering engine some day. Hopefully, soon.
Machine assistance also allows me to do things that would've been otherwise impossible.
For example, I'm aware of NASA's 3001 standard for human-rated spaceflight systems has very specific findings related to readability of labels. There are also other standards for checklist and document readability. Asking software to support the standards wouldn't work. Most people just aren't going to use NASA's human rating checklist to design websites and terminal outputs. They either don't know about it, or don't care enough to do so.
But machine intelligence makes it possible for me to extract NASA's learnings, turn them into requirements, and then have the machine custom build software that embeds it into my day-to-day life. https://github.com/areoformed/Re-Kindled/blob/main/code/TYPO...
I'm very grateful for that. The world is built to a very different tune, but with machine intelligence I can make tools that fit for me.
> The project code was written by GPT-5.6 Sol xhigh. It has not been verified by a human for correctness, efficiency, or speed; i.e. ymmv and use this at your personal risk.
In an age where everyone has access to AI, AI-written code has zero value. You should share the prompt or ask your agent to write a document other people’s agents can use to reimplement the code from scratch and/or customize to their specific needs.
> In an age where everyone has access to AI, AI-written code has zero value
Yes, which is why I didn't post it to HN as a project. I posted this as an aside because I think it's an interesting way of showing that you can use AI in ways that other people just won't (i.e. I implemented the 3001 and FAA readability standards here). link https://github.com/areoformed/Re-Kindled/blob/main/code/TYPO... (I added the link to the comment too!)
> You should share the prompt or ask your agent to write a document other people’s agents can use to reimplement the code from scratch and/or customize to their specific needs.
It's about raw environmental efficiency?
It took more than 40+ hours of iterations to get this right. Let's say 10 people check this out because of the HN post. That's now 400+ hours to get to the same end point.
I am not sure how large the model is, but let's say it takes roughly 16 B200s to serve, then the total load is ~14kWhr. 5,600KWhr, which is roughly half the energy a house uses in a year.
If a 100 people try it. That becomes 5 houses...
This isn't a single-shot prompt. It's more energy efficient to do it this way. Why do you want to burn 40+ hours of compute to get to the same end when you can extend from working code?
the ai can just read the code? when your prompt is 'jailbreak my kindle' (example) and it takes X hours, obviously the working code that jailbreaks the kindle is worth more than the 3 word prompt.
come on man, it has significantly reduced value, but your comment assumes that if any two people input the same prompt (or sequence of prompts) they would get the same working answer, where its very much possible he had some sequence of guiding, and the thing he is sharing is a working product. which has more value than a prompt that may produce a working product some of the time...
Meanwhile Opus 5.5 didn't even answer my question about reading user memory in a linux die notifier, a question which had nothing to do with security, just low-level system programming..
Oh wait, the author said:
> The attempts with Fable 5.1 and Opus 5.5 were both blocked by Anthropic's cybersecurity safeguards after two prompts.
Whenever I start checking a dump, the conversation with Sol won't last five rounds before it flags the work as cybersecurity-related. No matter what I do, the conversation gets cut off in the middle.
Astra is even worse. It starts saying nonsense and even refuses to run a test that we know will cause an AV (as a repro for a nullptr case).
Whenever that happens, I just switch to Opus to continue my work.
It seems like a distinctly possible and sad world that they might get so much better at keeping their devices locked down by hardening them with the support of LLMs that future devices become an absolute bitch to jailbreak, even moreso than they were before.
I think one area we are seeing this is the flood of hand held emulator gaming machines. Companies in China pump these things out and the community has provided numerous custom firmwares to the point a new one seems to come out every week. All vibe coded in Rust but who cares, if it plays the games and looks nice that's all you need.
https://github.com/9cb14c1ec0/android_device_lenovo_sycamore...
https://github.com/9cb14c1ec0/android_device_tcl_selene_vzw
https://github.com/9cb14c1ec0/android_device_lenovo_clove_ro...
Quite impressive what these models are capable of.
It sounds like most of the work went into getting the slideshow to work, setting up a feedback loop so Astra could work out / blunder into a solution with the desired characteristics.
> I first tried GPT-5.6 Sol with reasoning effort set to Extra High. It wrote code to connect to the Kindle but couldn't resolve an error that prevented it from storing data on the device. Then I tried Fable 5.1 and Opus 5.5, but Anthropic's cybersecurity safeguards blocked each attempt after two prompts. Finally, I tried GPT-6 Astra with reasoning effort set to High. We were able to jailbreak the Kindle.
"We were able to".
It takes a modern AI human and 4 LLMs to do something that could be done by googling "kindle jailbreak" and following instructions - they have to be performed on Kindle anyway.
I sometimes catch myself saying "we" when prompting AI so it really doesn't bother me to see this language being used. But I concede it can sound jarring or hubristic.
How soon until we all have jailbroken our TV ads out of existence? What next, jailbreaking the adds off our netflix accounts? To be clear I think this is an excellent development for everyone, however evenly distributed good doesn't usually positively correlate with share price.
Claude is just a lil bitch.
But it does make you think there might be a real market for unlocked devices like this now that it is trivial to build your own software to do exactly what you want.
It is your device. It is your money. You own it.
From the Federal Register, https://www.federalregister.gov/documents/2024/10/28/2024-24...
and I jailbroke my Kindle with GPT-5.5 Sol and turned it into an external e-paper display for the long batches of text AI models generate on a regular basis.Here's the project! I MIT licensed it. https://github.com/areoformed/Re-Kindled
Please note, I say so in the human note on top, but "The project code was written by GPT-5.6 Sol xhigh. It has not been verified by a human for correctness, efficiency, or speed; i.e. ymmv and use this at your personal risk."
In case you're wondering why I didn't write the code myself... I needed this as a tool to read long sequences of AI text output for work. Including code. Spending time making this tool would be counter-productive to that goal.
I do plan on writing a text rendering engine some day. Hopefully, soon.
Machine assistance also allows me to do things that would've been otherwise impossible.
For example, I'm aware of NASA's 3001 standard for human-rated spaceflight systems has very specific findings related to readability of labels. There are also other standards for checklist and document readability. Asking software to support the standards wouldn't work. Most people just aren't going to use NASA's human rating checklist to design websites and terminal outputs. They either don't know about it, or don't care enough to do so.
But machine intelligence makes it possible for me to extract NASA's learnings, turn them into requirements, and then have the machine custom build software that embeds it into my day-to-day life. https://github.com/areoformed/Re-Kindled/blob/main/code/TYPO...
I'm very grateful for that. The world is built to a very different tune, but with machine intelligence I can make tools that fit for me.
In an age where everyone has access to AI, AI-written code has zero value. You should share the prompt or ask your agent to write a document other people’s agents can use to reimplement the code from scratch and/or customize to their specific needs.
It took more than 40+ hours of iterations to get this right. Let's say 10 people check this out because of the HN post. That's now 400+ hours to get to the same end point.
I am not sure how large the model is, but let's say it takes roughly 16 B200s to serve, then the total load is ~14kWhr. 5,600KWhr, which is roughly half the energy a house uses in a year.
If a 100 people try it. That becomes 5 houses...
This isn't a single-shot prompt. It's more energy efficient to do it this way. Why do you want to burn 40+ hours of compute to get to the same end when you can extend from working code?
Oh wait, the author said:
> The attempts with Fable 5.1 and Opus 5.5 were both blocked by Anthropic's cybersecurity safeguards after two prompts.
How unsurprising :)
Whenever I start checking a dump, the conversation with Sol won't last five rounds before it flags the work as cybersecurity-related. No matter what I do, the conversation gets cut off in the middle.
Astra is even worse. It starts saying nonsense and even refuses to run a test that we know will cause an AV (as a repro for a nullptr case).
Whenever that happens, I just switch to Opus to continue my work.