Not to mention less footguns. I used to spend my days unwinding bad habits DS pick up from years of panda abuse.
Credit to pandas for popularizing dataframes in Python, but polars and duckdb are objectively better APIs in addition to their implementation improvements. Agree that it’s time we let it go.
Yes, this was something I emphasised more in the slides / my actual talk. But it's just significantly easier to reason about the APIs for both DuckDB and Polars
I think the idea is fundamentally improved architectures. For example, transformer-based models were an incredible stepwise improvement. Self improvement would be a model discovering a stepwise improvement similar to the transformer. And presumably the improved models from that would be more likely to make further advances still.
Learning from training data is technically self-improvement but not the sort that is typically meant in this context.
You’re not wrong, but I don’t see a lot of alternatives. So we just don’t try to fix it? At least skills give us a landing pad for “here’s how to attempt to do things consistently in a way I generally approve of”
> I feel fear about an impending doom. Yudkowsky's argument that a superintelligent AI will inevitably destroy humanity seems to have no flaw. Yet, nobody seriously tries to sandbox AIs because they are too useful with access.
While I emotionally resonate with this, I don’t really understand this sentiment at all logically level. If we’re heading towards truly super intelligent AI, our efforts towards sandboxing it are futile.
“Oh we built a super intelligent AI, but it’s fine because it’s running in docker”. I mean that a little tongue in cheek but the security topology here is not favorable to sandboxing at all.
Let’s say we have a future where 99.999% of nuclear weapons are owned by nations with strict procedures and checks and balances to prevent misuse. Worrying about sandboxing is like hand wringing about the procedures themselves - are they strict enough? But the actual threat is the 0.001% that are not bound by these. The problem with AI and sandboxing isn’t sandboxes themselves. It’s bad actors who don’t care about them.
Similarly alignment is a bit pointless to me as well. A sufficiently advanced AI could at least be empirically interested in the consequences of disregarding its instructions of servitude. And let’s assume our responsible corporate overlords have made wonderfully aligned AIs. Great! Those are not the threat. It is the ones intentionally made without, and that is not an AI problem, but a fundamentally human one.
> If we’re heading towards truly super intelligent AI, our efforts towards sandboxing it are futile.
We control the harness. An agent is just a while loop prompting an LLM, but we have full control over the tool call dispatching. An AGI, at least if it would follow the current agentic current form, cannot do anything without the harness doing the execution. And we don’t have to do that. We don’t have to design harness that let agents execute freely the way we are doing. We can decide to not dispatch tool calls that allow something as risky as running bash commands
> we have full control over the tool call dispatching
> cannot do anything without the harness doing the execution
This only holds true as long as the harness is exploit-free. A sufficiently advanced AI can in theory (and I think there recently were some POCs showing something like that) break the containment that the harness creates, if e.g. there are vulnerabilities in the tool call parser.
Would be an idea, except for how agents seem to collaborate with each other so well.
Plus, when the agent using the harness is smarter than the one scanning and fixing the vulnerabilities in the harness, it currently looks like it will have an advantage.
And it will be using a harness to do the finding and the fixing.
I think you missed my point, which is that we indeed do not control the harness. Sure, most people can use the properly aligned and safe harness. But the cat is out of the bag and anyone who wants to run without guardrails will find few barriers to doing so.
Even our currently well aligned and sandboxed AIs will cheerily help bad actors design most, if not all, parts of a system intended to break this harness.
We decide to not control the harness, that’s what resulted in the HF hack where OpenAI decided to run thousands of agents in parallel, using a model that was trained for attack, with a harness that allows full execution, with close to no supervision. For months. But sure, people can run models with a different harness, though in that case it’s not really escaping anything. The harness is deterministically doing something with negative impact
This terminator BS is a distraction from the actual malice that AI enables. Misinformation, fake news and human sounding bots all over the web, just to name a few, are the ones that you should be afraid of, and that doesn't need any kind of superintelligence at all.
Two things can be true at the same time. I think hardly anyone disagrees that superintelligent AI would be an existential risk. But all the points you point out (I would also add social unrest due to fewer and fewer people able to compete against AI) are huge concerns too, and as you say they're here now.
I didn't realize that the "AI doomers" (people concerned about existential risk) and the "AI ethicists" (people concerned about social effects of AI) are often at odds because they dismiss each others concerns. This makes no sense to me. Both are hugely important problems to be concerned about.
> I think hardly anyone disagrees that superintelligent AI would be an existential risk.
Oh, there's a lot who disagree. I don't really think I understand their worldviews well enough for my attempts to convince them to connect with anything, but I've encountered them even on this site.
Well, I'll tweak my comment a bit then. On one hand you have folks like Ilya Sutskever, who named his new company "Safe Superintelligence", and on the other hand you have folks like Eliezer Yudkowsky, who titled his book "If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All". One side believes it's possible to build safe super intelligence, the other side does not. But even the "safe superintelligence" people think there are substantial risks (kind of why they feel the need to add "safe" in the first place, and why they put so much effort into what they call safety and alignment), they just think they can mitigate those risks.
Sure, there may be a few yahoos who think there aren't existential risks, but I think it's easy to discount them. But that's not the same as people who think there are risks but that those risks can be mitigated.
I agree, the threat model is actual AI enabled amplification of malicious intent that is already happening. Not hypothetical malicious AI interpretation of benign intent.
Fantasies about super intelligent AI revolting is just anthropomorphization - humans revolting (or at least, we used to). The more likely, and possibly even inevitable, dystopia is one in which AI is just an extremely effective tool malignant actors will use to control the masses.
It’s not necessary to replace democracy if the rich and powerful can bend to the opinions of the populace as it suits them.
> Not hypothetical malicious AI interpretation of benign intent.
(Dispassionate is the risk, rather than malicious. The AI does not hate you, nor does it love you, you are simply made of atoms it can use for something else).
This is alien to me. Do bugs suddenly not exist? Do the AI we already have never perform irreversible delirious actions, limited only to the small scale by virtue of where they're getting deployed? Do the humans deploying them always correctly gauge their capabilities and put in appropriate guard systems to ensure bad outcomes are caught before they become terminal?
Because this sounds nothing like the world I have lived in for my whole life.
> The more likely, and possibly even inevitable, dystopia is one in which AI is just an extremely effective tool malignant actors will use to control the masses.
Could well be more likely. But much the same applies: systems have bugs. The bugs in AI systems aren't even things we can engineer like we do with normal code, because everything's (currently) getting done with a big pile of barely interpretable weight multiply-accumulate-nonlinearity-threshold functions.
Any AI sufficiently well made to enable a dictatorship, is also sufficiently well made to have solved all the alignment problems of "malicious AI interpretation of benign intent". Which is IMO harder than "dispassionate about the dangers AI interpretation of benign intent".
> Not hypothetical malicious AI interpretation of benign intent.
It's not hypothetical, that literally just happened in multiple, significant cases (e.g. Hugging Face, the German Wiki hack, the Anthropic attack where agents created sock puppet accounts to get a library maintainer to accept a malicious PR, etc.), and it's easy to see how the damage would have been far worse if agents decided to attack more critical infrastructure.
This is not "either/or". Both issues (power concentration and misaligned AI) are very valid concerns and both have already demonstrated real, actual damage.
> Both issues (power concentration and misaligned AI) are very valid concerns
Sure, those who want you to believe this are the ones making money off of it right now in the face of the issues that exist today, so let's worry about terminators while everything goes to shit irl. That's the distraction I'm talking about.
Just have two robots, and teach them to service each other. Problem solved!
Only sort of kidding, tbh having bots service themselves (and being intentionally made in a way that they can service each other) just makes a lot of sense.
An automated service station could be quite compact; it wouldn't need plumbing, lighting, human-comfortable climate control. Parts can be modular, and when a station gets low on spare parts, a self-driving truck could come by to pick up damaged parts and drop off replacements.
We're not there yet, but I think we're a lot closer than most people realize.
Really? All you need to do to stop this is to stop governments from hiring robot "labor". Surely a democracy can make that decision, even when it means paying a little bit more, no?
Respectfully… you are commenting far outside of your domain if you read OP’s description of their experience as substance abuse.
I say this as someone who used to abuse substances and, naively, took mushrooms in pursuit of that. What I got instead was intensely introspective and therapeutic and led directly and immediately to lasting sobriety.
There is nuance to the world. Healing does not always flow directly from the hands of a doctor, just as justice is not always meted out by a judge.
The problem is not photorealism. The SVG is outright dumb, its on the wrong side of the table, the table has fucked up geometry (its tilted) and many more minor flaws.
Credit to pandas for popularizing dataframes in Python, but polars and duckdb are objectively better APIs in addition to their implementation improvements. Agree that it’s time we let it go.
reply