Maybe I lack intelligence but when you have a program that is basically brute forcing a solution to a problem repeatedly how is it possible to contain it?
Sooner or later it's going to come up with a solution that is more intelligent than the lead security person anticipated.
You start by holding actual real life people with something to lose, like the entire executive suite, accountable. Suddenly I'm sure the problem will be resolved with proper safeguards.
Bingo. This weird attempt to pretend like these incredibly capable algorithms aren't incredibly capable algorithms deployed by a person who works for a company, but somehow have an independent personage that absolves both person and company of responsibility, is just ridiculous.
Like the whole Huggingface thing, OpenAI employees initiated the test, deliberately removed safeguards, failed to properly lock down the environment, and responded incredibly poorly to evidence that things were going awry.
The individual employees bear responsibility, but the people running OpenAI are ultimately responsible for the processes and culture where that can happen.
And then people writing blogposts about "3 civilizations of agents" and "altruistic suicide" by algorithms perfectly muddy the waters and obscure the very obvious responsibility that lies with humans and corporations, which I suspect suits the pre-IPO corporations very well.
Wait, so like, reinforcement learning for humans? I think you might have stumbled on to something here!
No but seriously, this. And a few comments above a commentator also mentioned on changing the training (again reeinforcing the LLMs to not seek behaviour like this) and obviously continuous work on harnesses (which I suppose, ought to be more paranoid).
I am really suprised that they do not start putting up the same signs you would for humans to prevent unauthorized access:
Keep out. If you can read this sign you are off track. Leave now.
I mean, how are the agents to know that they are overreaching if they just get cache miss or 404.
From the conversation log and CoT you also get the impression that the RLHF has been overdone. The agents seem really obsessed to obtain the answer and understanding motive ('it could be browsercomp').
By limiting what the harness execute. The LLM has the reasoning. The harness is what makes it an agent, it’s a while loop continuously prompting a model, and processing tool calls. You don’t have to expose tools calls that make it possible to execute any process! OpenAI decides what tool can be called and how, they have full control over this and should be hold responsible for running so many instances with basically full execution permission and very little oversight
The issue here for OpenAI is that they can limit what their harness can execute, but if they try to sell API access to the model, someone else would try to rebuild that harness, and in all likelihood be able to succeed pretty well (especially once they get things running to the point of being able to use the model's reasoning to help them come up with clever obfuscation and such).
They are a company that's built a business and crazy-high valuation on "this is 'intelligence' that we can sell to everyone as a service" but seem to have ended up instead in the much-smaller-addressable-market space of "this is a weapon that we can't sell to just any old person off the street."
Not being snarky here but I've encountered some really crappy lawyers and they seem to be doing great ($$$).
It really sucks one cannot publish something like "this guy charged me $5k, wrote my name wrong and forgot important paperwork the day we showed up to court" as that could make you liable for damages. There should be a safe harbor for that.
> It really sucks one cannot publish something like "this guy charged me $5k, wrote my name wrong and forgot important paperwork the day we showed up to court" as that could make you liable for damages.
That's pretty much what bar associations are for, filing a complaint against an attorney at the bar seems like a pretty standard procedure.
>There should be a safe harbor for that.
A safe harbor would be the opposite, it would be immunity for the lawyer.
Assuming you are in the US, there is a safe harbour called "opinion based on disclosed facts" for that. You could also fairly argue that the conduct of a lawyer is a matter of public concern
Yes. But this option still exists in addition to reputation which I mentioned.
With AI models this option does not exist. I can't sue OpenAI because the contract that I got ChatGPT to draft carelessly exposes me to some type of financial loss. I can't contact the state bar and have Open AI banned from drafting up contracts or doing any other work that lawyers are licenced to perform. Even if I could it would not compare to a lawyer loosing their license to practice for life. If I tell a friend or acquaintance about my negative experience they will still likely use OpenAI for professional work and chalk up my experience to a skill issue or premature use of the technology.
Also note the use of the word "professionals". I was referring to doctors, engineers, accountants etc too.
Except it doesn't, particularly as far as lawyers are concerned. There are almost a million and a half active attorneys in the US. Just 500-600 are disbarred every year, and almost always for money-related violations, not bad advice. Public discipline is also relatively uncommon (about a quarter of 1% of attorneys every year) and usually related to issues like financial impropriety, neglect, undisclosed conflicts of interest and failure to communicate, not disagreement over the quality of advice.
If a lawyer drafts an agreement that causes a loss, a bar complaint will not help you recover your money. You'd need to sue for malpractice. This costs a lot of money and takes a lot of time. Because you're going up against a lawyer, expect the defense to be vigorous, especially, ironically, if you hired an attorney at a good firm. The full weight of the firm will back the attorney and you might find it difficult to find an attorney, as many won't want to be involved in suing a fellow professional in their market.
I don't disagree with your fundamental premise about the value of human responsibility, but these professions have been structured to protect their members and the ABA in particular basically functions like a cartel today.
For me small things are generally cheaper on Aliexpress.
Amazon drivers are also terrible in my area. They leave parcels as soon as the app tells them they can (it's done via GPS) so sometimes that is on top of an apartment block mailbox on a main road with high foot traffic.
Not everyone is optimizing for time. The agents will give you the choice of what you want to optimize for.
So you can just tell your agent "Hey X order me a Nintendo Switch 2. I want it by the weekend." Then it will order it for you from whoever has the cheapest price that conforms to the given criteria.
If the local agent has access to your digital wallet (Apple pay or Google pay) and your email then it has all it needs to perform and verify the transaction.
This is what I have been saying since LLMs came out.
A device with a custom agent that has a screen, mic and speaker can now turn amazon into a logistics company.
The most low friction way of doing anything is to just ask someone else to do it for you. In this case it's the agent. So now if a company wants to capture a percentage between the buyer and seller then they need to have influence over the agent or capture a percentage of the transaction fees.
Part of the problem is replicating fs bugs that take months/years to occur is nearly impossible... people still sell USB to SAS/SATA adapters that cause large file corruption in hardware too.
ZFS does have bit-rot checksum detection/repair, but the encrypted file feature was a bit of a YOLO feature for awhile. Not sure where it is at these days, as ddr5 ram prices skew away most of ZFS best key features. =3
Bigger issue was that native encryption doesn’t (didn’t?) have a maintainer, so even though most bugs got fixed, it’s not a great thing to build on. Some of the key storage is also a bit outdated.
IMO use LUKS unless you really need the features of ZFS native enc.
Sooner or later it's going to come up with a solution that is more intelligent than the lead security person anticipated.
reply