I don't know if this is a good framing. "Too much" is subjective, and every heavy AI user will assert that they're just unlocking their potential, that calculators didn't make us dumber, etc.
But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you. On the flip side, if you use an LLM do most of your thinking, what's left? We have people here who use LLMs to raise their children, to manage relationships, to design products. So what's your unique contribution to this world - is it the prompt you once wrote? You're standing in front of a token-generating machine, pulling a lever, sometimes receiving gifts. Is that your edge, your unique experience, your purpose in life?
Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?
In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share. I want to support that. And I want to be a human who can write novels, the old-fashioned way. I'm not good at lifting weights or running, so my thinking is the only thing I have.
Have you read the "Whispering earring" essay? I love it for the LLM era.[1]
You can treat AI as a whispering earring - "What should we do now? How do we fix this? What do you think?" Or you can treat it like an exoskelton - "Implement kd-tree with metric space xyz for this problem, mapping this to that blah blah".
That's pre-thought execution automation that makes review much simpler - you already know the shape of the desired output. The whispering earring is atrophy.
It cracks me up that you bring up the exoskeleton metaphor, because I'm pretty sure it originated from a 100% AI-generated essay that made it to the top of HN a while back. So I guess, AI is whispering things into our ears whether we notice or not.
Exoskeleton is such an obvious metaphor for AI (at least, one potential ways of using it) that surely no one, human or AI, could possibly claim credit for it.
It feels more like a BMX bike to me. The quality of what I get out seems to be a direct function of what I put in. The threshold of what I can achieve is raised slightly, the speed with which I can achieve it is raised quite a lot - but it's not a magic rocket ship that whisks me to far off hostile environments and protects me while I'm there. I'm still "naked". The code is still mine, and I can get off the bike at any point and walk.
Thats... not how most of the world is using llms. Students are getting measurably dumber (as in lacking actual topic understanding, since actually learning sonething properly is much harder than shallow knowledge), every other day I read about somebody losing their wealth due to listening to llms with finances.
Other recent articles - psychologists start seeing people losing their uniqueness and many people slowly veering towards same middle bland opinions.
I think top parent is correct - we are in the process of visibly losing a lot from our humanity, much more than we are gaining. We shouldnt be listening too much to loudest mouths here that praise efficiency gains, many of those are 'semi-autistic' folks with tunnel vision focused on 1 aspect and ignoring larger picture.
I dont see any change of course possible, the sweet nectar of efficiency and offloading decisions and work is too strong a drug to ignore. But everything has its set of benefits and drawbacks, and drawbacks here seem pretty brutal so far.
>So I guess, AI is whispering things into our ears whether we notice or not.
Humanity is a weak hive mind species. You can, with effort, keep it from whispering it into your ears, if you really want that. But it's impossible to keep it from whispering in another ear attached to your hivemind. And the only difference in those ears is the propagation delay before it's rattling around in your own skull.
Something I never considered much is what happens when everyone else is using the Whispering Earring.
You may be more free and independent, but you may also be unable to compete as everyone else easily gains wealth and success. Natural selection doesn't particularly care about freedom of consciousness.
Eventually, that becomes interesting. Before we get to that stage, I think we pass through (almost) everyone following almost the exact same advice as each other.
Such people are extremely predictable.
You may have already noticed, before LLMs, cliché? Talking points that make a group identifiable? Words and phrases that act as applause lights or cognitive stop signs? (That last sentence itself being a pair of clichés that you can use to identify where I hang out online).
Anyway, point is, LLMs will give us a memetic monoculture before they turn us all into a world of correctly personalised Whispering Earring wearers. That makes them predictable, that makes them exploitable. It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.
(Of course, the fact I've said this in writing means this is in the training data; in the general case this means the LLMs will know that and account for that, but I suspect comments like this won't shift the needle all that much compared to the aggregate output of 3 billion people reacting to short-form emotional manipulation A/B slop)
My dog often gets misidentified as a restricted breed. This used to make apartment hunting difficult because, occasionally, the property manager would visually ID the dog breed as banned, I’d have to go to the vet and get paperwork, potentially gene testing, arguing she wasn't, it was a whole thing.
But, recently, the apartment I moved into had an online portal where I had to upload a photo and it would identify the breed to determine if it was approved.
I correctly assumed the portal was using an LLM for this purpose. I wrote a script which submitted different photos of my dog to the major LLM providers until it found a photo which all the LLMs would identify as the correct breed.
I simply submitted that photo and, as expected, passed with flying colors.
> It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.
I don't believe this is how chess works, and I don't believe this is how Stockfish works, and I don't believe this is how AI works.
Stockfish isn't winning because it's playing a better sequence of programmed steps, and having access to "the next version of Stockfish" doesn't mean it can "guess the next move" and play against that.
You have Stockfish version n, see board state s. I have Stockfish version n, see board state s. I want to know what you're about to do, so I put Stockfish into state s, ask it what the best move is, and I know you'll make that move because I know you'll ask Stockfish version n the same question of the same state. I now know board state s+1.
The steps are not pre-programmed, but the program itself is (modulo hardware imprecision) deterministic. If there's a RNG in there then sure, this doesn't work as easily as I wrote it; and there may be randomness in the thing that this is a metaphor for, regardless of if there's one in Stockfish or not, but that's not hard to work with when you want to win against an aggregate: we invented the field of statistics to deal with random numbers because they come up so often.
There is deliberate randomness in stockfish. The easiest way to see this is from the fact that, when playing the white pieces, it won't play the same opening move every time. Often it's e5, but it also goes for e4 or Nf3 or something else entirely.
This is by design, and very much necessary for a competitive chess engine. Otherwise, people could do basically what you say: Run an offline (as in, ahead of time, with ample compute resources) search against stockfish that finds a line where it loses, then make an engine that plays that every time.
As a consequence, even if you know that your opponent is running stockfish, you can't really use that against them. Your best bet is also just running stockfish.
Jacek Dukaj in his book "Perfect Imperfection" (Polish hard sci-fi about alternative physics, mind uploading and enclosed universes) wrote about entities vastly superior to human-level intelligence having an ability to simulate humans to such a degree of precision that they were able to determine what a given human would say or do based on his own internal experience and knowledge in real time. The superintelligent beings didn't scan his brain patterns, they just had multiple simulations of him and chose the one that would best match his actions based only on external cues like position of his hands, body language, speech patterns, etc.
A solution to avoid being read to such a degree that your free will effectively is destroyed was to introduce randomness as a program in your own mind processed in an equivalent of a "cloud VM" - random changes in your body posture, hand movements, weird phrases introduced while you spoke could confuse the internal models of you enough to avoid hyper-predictability.
That's really good, I'll have to read some of his work (Jacek Dukaj). It makes me think about the role of indeterminacy and unpredictability in physics. And reminds me of..who was it, Gurdjieff maybe, who taught that humans are like sleeping machines, and that constant conscious effort is necessary to "stay awake". Many of his exercises for students had the purpose of recognizing one's own habits, routine patterns of thinking and behaving, and breaking out of them.
The sci-fi story also reminds me of, I think it was related to Karl Popper's book about "objective knowledge". There was a thought experiment about Mozart, if a scientist could quantify everything external about the composer, like the brain, the body, the room he's in, his life experiences - is it possible to deterministically compose new works exactly as the real Mozart would. And if not, what is the difference between the perfectly simulated Mozart and the real one?
This "randomness" is related to the ancient Greek concept of clinamen.
> Clinamen, derived from clinare, to incline: the unpredictable swerve of atoms in the atomistic doctrine of Epicurus. This swerving, according to Lucretius, provides the "free will which living things throughout the world have".
Stockfish already does that. You run it as long as you want. When it's calculating evaluations for move k+1, it already knows what move k was, because it has that evaluation. It can explore that tree first, before exploring other trees to try to improve on move k.
Stockfish never settles down to an exact move, unless it sees a forced mate sequence. You could leave it running indefinitely, but the exponential blowup will soon grind you to a halt.
What new versions of Stockfish buy you are optimizations and better heuristics. An older version of Stockfish can still beat a new one, if given enough of a compute advantage.
Your whole strategy basically involves having more compute than the other guy so you can look one step further. And yeah in an AI world having more compute or some exclusive data seem like obvious ways to get a leg up.
It is literally impossible for everyone to obtain wealth. Wealth isn't a number or physical circumstances. Wealth is having economic power over the people around you. Success is a little more nebulous, but when thought of as a relative of wealth it's similarly contextual.
The danger isn't everyone but you getting wealthy. The danger is that wealth tends towards concentration. And it tends to concentrate around people who are already wealthy. The danger is, bluntly, that things will get worse for all but a few and most people will be so caught up in a red queen's race that they can't see how to stop.
No. The world has absolutely gotten wealthier over the last few centuries.
Wealth is obviously not zero sum. Humanity is far wealthier today than before the industrial revolution, and the trend is still towards increasing wealth.
As Led Zeppelin once said, "you know sometimes words have two meanings". That definition of wealth is all well and good for books about how markets are magical fix-everything pixie dust. But it's not the definition of wealth that applies when used to describe a person. A wealthy person is not someone with a lot of things. That's a hoarder. A wealthy person is someone who can use their economic power to shape the world around them.
That definition of wealth is all well and good for books about how markets are magical fix-everything pixie dust
No, it actually means something. We have access to refrigeration, climate control, near-limitless computation, entertainment, transportation, knowledge, communication, and medicine. All of this stuff would BLOW THE MINDS of medieval kings. Even if you're living in a 1BR apartment, working at Walmart and struggling to pay rent, you enjoy many luxuries Charlemagne could scarcely dream of.
Go to the store, grab a pineapple off the shelf, take it home, and eat it. Then read about the great lengths [1] the wealthy people of the past went to in order to try to grow pineapples in Europe, and how obsessed the culture became with this fruit.
If you're in that one bedroom, struggling at Walmart, you probably don't have a dishwasher or a washing machine and dryer, and possibly not air conditioning or heating, or at least, your electricity bill is enough that you have to think twice before setting climate control to a comfortable level. Your social life probably isn't anywhere near what Charlemagne had. You have a smartphone and that has brought you news from afar. But that news is terrible! Local politics, things are fucked, climate change is out of control, geopolitics is a whole other shit show. Charlemagne didn't have to deal with Israel and Donald Trump. His world was very different. Sure, I have a fridge and a microwave and can use that to sate my hunger in mere minutes, a luxury inconceivable to Charlemagne, but all the trappings of modern society, with TikTok and Reels hasn't made us globally happier. Seeing the emotional damage from whichever online platform surfaces drone videos out of Ukraine today for me as a form of wealth is one perspective. We've lost the plot, or at least I have. We're solving the problems in front of our faces without asking if that's the right problem to be solved.
>A wealthy person is not someone with a lot of things.
No one made that claim.
By nearly any sane metric the world is wealthier. You can verify this on sites like Our World in Data.
We are healthier, have more leisure time, are less likely to die in natural disasters, etc.
It's willful ignorance to define wealth in a way that wouldn't show humanity as having gotten more wealthy. You're doing harm by overlooking massive gains we've made and understanding why we made them and instead redefining the common sense understanding of basic terms.
My father's generation had much more leisure time than my generation. When he was out of the office at 5pm work stopped. There were no laptops, slack pings, etc, etc. Vacations were longer, pensions were a thing. You could get a job supporting a family right out of high school. Things have absolutely gotten worse in the west no matter how much you want to twist the stats.
You are talking about access to resources and technologies, which is basically constantly rising, we couldn't stop that if we wanted to, so that's a given and more importantly orthogonal to the wealth that buys you goods and services.
The latter is absolutely zero sum, because of everybody has a billion, a billion isn't that much any more. E.g. human trafficking exists because some people are rich and others are poor, and the fact that even those poor people might have access to things an emperor 1000 years ago would have dreamed of doesn't change that, because the people who treat them like slaves have even more. It's all about the relative difference between people who are alive today, the absolute increase compared to people who lived N years ago is a red herring.
> ... human trafficking exists because some people are rich and others are poor ...
No, it doesn't. This is an absurd claim, and nobody intelligent should take it seriously. Human trafficking exists because some people choose to do evil things. There are plenty of relatively-rich people who are actively trying to stop the evil of human trafficking.
Furthermore, just being wealthy has not, historically, made you immune to being made a slave. For just one example, look at the story of Joseph in the Bible. He was his father's favorite son, the heir presumptive to fantastic wealth for his time (that "coat of many colors" his father gave him cost the equivalent of what a sports car would today), and yet his brothers who hated him sold him into slavery while his father wasn't around to protect him. Throughout history, cross-border slave raiding has been a thing, too: it didn't matter how much gold you had tucked under your bed if the Vikings (or whoever) caught you alone, or outnumbered and overpowered your hired guards: once they hauled you away and slapped chains on you, you no longer had access to your wealth, and your life depended on making your captors happy with your work.
To think that slavery, ancient or modern, is caused by relative wealth differentials reveals an abysmal ignorance of both history and human nature.
I'm deliberately giving an illustration that shows that relative power throughout most of history had to do with physical strength and weapons, rather than money.
And if you didn't mean that slavery is caused by relative wealth differentials, then saying "... the people who treat them like slaves have even more. It's all about the relative difference between people who are alive today ..." was a very poor way of making your point. So what was your point that I missed, then?
That in this context, it's about the wealth that leads to "having economic power over the people around you". That is zero sum, and the general rise in absolute living standards is as relevant to that as a comment about apples is in a discussion about Apple.
Ah. When that one commenter said "Wealth is having economic power over the people around you", I ignored him, because that's a ridiculous definition, not to be taken seriously. Any argument that arises out of that definition is inherently flawed, since one of its premises is false; it would be like saying "Since the moon is made of green cheese, therefore X". The definition of wealth that nearly everyone agrees with is, well, I'll just cite the opening line of https://en.wikipedia.org/wiki/Wealth because it's a pretty good summary: "Wealth is the abundance of valuable financial assets or physical possessions which can be converted into a form that can be used for transactions." Or, to put it in simpler terms, wealth is the stuff you have that you can use to satisfy your desires. Anyone trying to make it about economic power over people is trying to redefine the standard definition for his own purposes... and that's usually a big red flag warning you of a bad argument.
>You are talking about access to resources and technologies, which is basically constantly rising, we couldn't stop that if we wanted to
We could very obviously stop our progress in resources and technology. It has happened throughout history many times and we see plenty of modern examples.
You seem to be treating the accumulated products of civilization as though they are geological facts rather than fragile outputs of maintained institutions.
Maybe you should ask yourself why you believe that 1) access to resources and technology are "basically constantly rising" and 2) why the relative difference in wealth today is more important than the (magical?) existence of a system were everyone has continually gotten more access to resources and technology.
> Maybe you should ask yourself why you believe that 1) access to resources and technology are "basically constantly rising"
Because people have and share ideas, and I was using that as a shorthand. Even if they aren't, if you completely strike that part -- my point remains completely untouched.
> why the relative difference in wealth today is more important than the (magical?) existence of a system were everyone has continually gotten more access to resources and technology
Because in the context of "Wealth is having economic power over the people around you.", that is the definition of wealth that is used. To quote a randomly picked one, emphasis mine:
"Large possessions; a comparative abundance of things"
> You are aware that the default state of humanity hasn't been one of continually increasing access to resources and technology?
There was no default "state", at least by the time you're talking about humans, we probably already learned from each other by observation. Sure, politics and wars and religion and feudal dynasties and whatever make bumps in the line, but generally it goes up. Once things get out, they're out, roughly speaking (that's the keyword here), and even in the animal kingdom.
And as I said, even if that's completely false, doesn't matter for the wider point about wealth, and that it can only buy you power over others if they have less of it.
>There was no default "state", at least by the time you're talking about humans, we probably already learned from each other by observation. Sure, politics and wars and religion and feudal dynasties and whatever make bumps in the line, but generally it goes up.
No, that's not the shape of the line. You can view it here:
The hockey stick upturn in recent centuries needs explanation.
I'm pushing back so hard on this comparative definition of wealth (which I notice is the secondary definition you found behind large possessions) because I think it's silly to do that when we have an unbelievably dramatic increase in wealth in recent centuries which needs understanding first.
Hand-waving away human progress and forcing zero sum definitions of wealth feels unserious. You have terms for things like inequality, why not use those instead of distorting what wealth means?
> The hockey stick upturn in recent centuries needs explanation.
Why? It goes up. As I said.
> I think it's silly to do that when we have an unbelievably dramatic increase in wealth in recent centuries which needs understanding first.
That's not pushing back, that's wanting to talk about something else first. That's just not something I'm interested in.
> distorting what wealth means?
It simply means both things, from context it was obvious how it was used in this instance, you ignoring that and going on tells me explaining it yet another time won't help.
This is such a good point. It matters HOW you use the LLM.
Come to it with knowledge and understanding of a subject matter, asking for an implementation? That's different from going to it with no knowledge, asking for guidance on everything.
I did a code review for a really, let's just say "challenging", co-worker a few years ago. There was one block of code that was really obtuse and I couldn't figure out why it was there and what it did. So I asked him.
He sighed like he was taking responsibility for a grave sin and I should admire him for it, and said, "I don't know. I copied that from StackOverflow."
I've felt that AI is just an amplification of what we've all done and been through with SO answers.
I have learned the hard way that "corpo code" is ephemeral. I spent at least 2 man-years on Ruby/Rails code for Desk.com, once spending an entire month to solve a really bad and intermittent session-logout issue that stumped 8 developers before me (I could tell because they left a bunch of commented-out debug code and I had the git-blame lines), and after I left, that whole stack got retired and its clients moved to some garbage Salesforce offering as part of their crufty Java monolith. (Sorry, SFDC.)
So if we move from "bespoke handcrafted code" to "roughly-engineered, unstylish but still quite working LLM-generated code" in that space, there is LITERALLY no love lost.
Spend your handcrafted love-energy on open-source or library code that will stick around. Leave the corpo code to the LLMs, I say, especially if you don't own it.
Way too many people like to reach for the easiest path. They will often assume or have blind faith that the AI or online answer is correct. Coming up with the solution themselves or taking time to understand why something might be the correct answer, can be just too much work to be bothered with.
This has a nice, Borgesian, charm to it but I feel like it's premise is confused. If it's just about the here near-empty signifier of "happiness," what does the magic tech of the earring actually contribute to the parable that some hypothetically perfect narcotic wouldn't?
The earring doesn't make you feel better, it actually produces a better result. It's never wrong. You might regret addiction, for example? Also, it's a parable. Take it too literally and it loses its charm.
> It always tells you what will make you happiest. If it would make you happiest to succeed at your work, it will tell you how best to complete it. If it would make you happiest to do a half-assed job at your work and then go home and spend the rest of the day in bed having vague sexual fantasies, the earring will tell you to do that. The earring is never wrong.
It just has to be this in the story because it doesn't want to engage with why what the earring indicates is so good except that it provides unfailable "happiness". It implies a static world where "the right decision" always exists, but doesn't want to engage with the details. It also implies a kind of akrasia[1] that I don't think resonates with lived experience: "happiness" is a state we pass in and out of, not the accumulation of correct decisions. We undergo unhappiness for the sake of future happiness. "Happiness" is a questionable end-all goal first of all, but either way not even the kind of thing ontologically compatible with the author's framing.
It just feels half-baked, it doesn't actually say anything even when it has all the pretenses of that. Maybe that's too harsh though.
You're getting too hung up on the word "happiness". The earring gives you good advice. Good according to whom? According to you. It helps you succeed at your goals and live a fulfilling life.
This doesn't make sense with the story. If it's good advice according to you, why do you need it at all? Doesn't it provide you something that you otherwise don't have?
Maybe I am just not smart enough for this.. What is the parable supposed to be about? What are we supposed to learn from it?
Except it's not really a mechsuit; inasmuch as it is, it's more like an Evangelion: it's got bits of people's souls floating around in there; it only works well with a high synch rate with the user, which is difficult to achieve consistently; and sometimes it pops off and does its own thing anyway.
And the way you achieve and maintain deep knowledge of the shape of the output you want is... well, to write and test it yourself. LLMs are a cognitohazard. If you aren't accustomed to programming without them (or even if you are) it's hard to see the line between using them to remove drudgery and relying on them to think for you.
The Whispering Earring story reminds me of that episode of Rick and Morty in which Morty gets a piece of crystal embedded in his forehead which allows him to perceive the moment of his death; he then begins steering his actions towards the future in which an elderly Jessica (his crush) tells him she loves him as he dies, down to individual articulated phonemes and muscle actions. In the end, it's revealed this future is not what Morty had hoped for.
That Evangelion analogy is a good one. I don't necessarily agree with all of it (for example, I think the high synch rate is the weakest part of the analogy, and LLMs can be used as tools with some decent success without needing to plunge into them too deeply), but every piece of it can be coherently argued for (for example, perhaps I'm missing an even higher threshold of success that I could achieve with a higher "synch rate" equivalent, and I just don't know about it).
But kudos for coming up with such an evocative analogy.
"Synch rate" is roughly "you know how to manipulate the model to get it to do precisely what you want". This is nonobvious even for open-weights models and can change as new models are published, often under the same name as the old models.
"Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?"
Many of the LLM maximalists I know don't have the skills or knowledge to excel in technology and need to use LLMs to do their job. It's seen as a cheat code to get work done.
As an example, A person I went to high school with that could barely figure out how to setup a Drupal site a few years ago, is now a frontier engineer at an AI startup. His Linkedin posts are filled with AI buzz words on a daily basis.
"It's inherently worth more because it's hard-earned and anchored to experiences we share. "
At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.
> At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.
So what if its indistinguishable - its not a product of human intellect or effort. I feel there is a large disconnect where people look at artful output such as music or writing as a thing no different than a box of paper clips. To them, "It's just there." They don't care how it got there, they just like the feeling they get from the consumption.
That's fast food thinking; It's engineered to be "tasty" in the sense that they put the right amount of chemicals into the food to tickle the right nerve endings. It's junk food that exists to turn a profit. Whereas even the local diner puts effort into its food and has a damn fine Greek menu and the best mozzarella sticks.
I’m not sure it’s so simple. I went on a trip with friends this weekend and cut a video from all the short clips we filmed on our phones. Before AI I would have used some song from my music library as the background tune. This time I created a song with Suno that fits the clips and includes some funny anecdotes from our vacation. If I was very talented and had the time, I could have written the lyrics and recorded the song myself (which undoubtedly would be even more awesome), but I can’t do this. So I used AI as a tool to do it for me.
Everyone agrees it’s more personal and a better conserved memory of our shared experiences.
I don't get how people can listen to and share AI generated "music" and not feel legitimately embarrassed to their core. Like the AI Charlie Kirk song, but unironically. I'm truly baffled how anyone can enjoy it.
> If I was very talented and had the time, I could have written the lyrics and recorded the song myself (which undoubtedly would be even more awesome), but I can’t do this.
I promise you that any art you produced would be better than slop. There is not a single crayon scribble or off-tune melody that would be worse than the generic garbage that comes out of an AI model
You have to remember that the vast majority of tech workers have open disdain towards humans and share zero solidarity with their fellow workers.
Just look at the leadership of tech workers, some of the most deeply anti-human and anti-democratic people on the planet. Not hard to see how these terrible people rub off of their workers.
> If you knew how the modern music industry and ...
This just assumes I'm listening to mainstream music or whatever when I mostly buy grungy metal and whatever else I enjoy off Bandcamp. Many times, directly supporting the artist.
> ... local diners work, you might not be so impressed.
I already stated the homemade gyro platter and they're mozzarella sticks w/sauce are slammin and they are proud of it. Sure it's business but they crafted some damn fine mozzarella sticks and they always come out crispy and oozing with hot but not roof melting cheese. That's giving a damn even if its slammin a few in the fryer then tossin em on a plate and sendin em into the human ether. I hope they stay around as they're a great Sunday afternoon lunch spot with friends.
I love gyros; you're making me want to go visit that diner. Do you mind revealing its name and the city it's in? (If you don't want to because that would reveal too much about your own location, I understand perfectly well and have no objections). Odds are I'm nowhere close to it, but sometimes my job takes me to unexpected locations, so I'm always happy to have a list like "If you're ever in Topeka, Kansas, check out (name of hole-in-the-wall burrito joint), they make the best darn burritos I've ever had". (For clarity: that's a made up example, I don't know anything about restaurants in Topeka).
But the AI would not be able to perform live on a stage, producing emotions. I can't imagine staying 3 hours in a crowded room looking at a robot or a computer screen, even if it produces great music. Or if we get to that point, what gives the right and permission for humans to exist anymore?
>But the AI would not be able to perform live on a stage
The vast majority of music consumption is by people that have never seen the artist live. Hell, I have no idea myself if there is a human behind it.
>what gives the right and permission for humans to exist anymore?
Nothing. We are just a mote in incomprehensibly small fraction of eternity. Eventually the process of equalizing entropy will erase all traces of our existence. Enjoy it while you have it.
One of my college buddies, when getting set up for a marathon viewing of the trilogy (extended editions, naturally), referred to it as "twelve hours of butt-numbing Tolkien goodness". Exact words: it was more than 20 years ago but that phrase was so evocative it stuck in my memory.
If it's indistinguishable, that could be because the user doesn't care to look closely, or it could be because it's just that well made now. For simple profile pictures, I genuinely stopped being able to tell if I'm looking at a real photo or not last year.
> That's fast food thinking; It's engineered to be "tasty" in the sense that they put the right amount of chemicals into the food to tickle the right nerve endings. It's junk food that exists to turn a profit. Whereas even the local diner puts effort into its food and has a damn fine Greek menu and the best mozzarella sticks.
The digital version fast food we already have and is a little different, in that it's the tuning of "the algorithm" to addict us, while text and image models* seem to be trying to actually fool us.
* I suspect video and music models are trying to addict, but I'm not super sure either way.
Real programmers write the code and throw it away after compilation. All the fixes happen in the binary. You are not a real programmer unless you debug hex dumps and add changes directly to the compiled program.
Text editor? No. You punch the program on cards and then wait 1 week to get your turn and get a compiler error.
> You're simply drawing the line where it suits you.
I'm drawing the line where the human effort is no longer there.
There was a similar argument against electronic music. No effort because no instrument is played. But the fact is the instrument just changed shape, drastically.
Like there's an artist I am sure a few here would be familiar with: Master Boot Record, a video game heavy metal electronic blend fully composed on a DAW. MBR live streams his composition sessions while chillin in his own irc channel he's hosting himself so you can chat with him. It's impressive to hear how he lays down notes, loops it, tweaks and expands it, then little by little builds a song passage by passage. Often with the irc channel reacting to the process itself and perhaps influencing the output. You know when it finally clicks and you're left feeling "Oh that was cool as fuck!" That's uniquely human, a life form that evolved over billions of years on a mud ball floating in a vast vacuum - at random. Talking, thinking, playing, laughing meat. Amazing huh?
An AI shitting out a similar pattern because it was trained on the same and similar patterns is not interesting or original. It's just a really good illusion from a well tuned stochastic parrot. It's interesting in the technical sense, but not in the artistic one. There was no creative process involved. Prompting is just a well formed set of instructions and questions which a human could understand - a spec. Nothing special there. It's like driving to McDonalds and ordering a quarter pounder with cheese meal. As a search engine, cool. As an artist, no thanks.
Perhaps you underestimated the person you went to high school with? The fact he or she "could barely figure out how to set up a Drupal site a few years ago" doesn't mean much. Lots of people are capable of immense progress when they apply themselves.
I consider myself an LLM maximalist, at least in the context of software engineering. Even after delegating everything you can to LLMs, there's still work left for you. And it's far more interesting. Even the delegation to LLMs is interesting - it's not that easy to make it work well.
In my domain (web app CRUD), we build the same thing over and over and over. And it's not beautiful or interesting like furniture. It feels like with LLMs, all that repetitive stuff is gone and I can focus myself on the tricky problems.
If we're contributing anecdata I have used LLMs to tremendous effect to learn all kinds of interesting stuff because I like learning interesting stuff and LLMs can tailor the level of instruction to exactly where you're at and don't mind questions interrupting ever 6 seconds which is better than both textbooks and teachers, mostly.
Actually yes, I've used an LLM to make a better old fashioned (whisky based drink). Turns out skimping on the orange oil/zest makes a huge difference. And, no now that I've learned it I don't expect I'll have to go back for this express need.
This is where LLMs and even things like youtube videos muddle the difference between what you are and things you have to do as a human. The number of things you can be presented with in the modern world that you'd like, or have to do, is simply endless. If it's something small, with little opportunity cost if failure occurs, then just going to an LLM for an answer doesn't bother me. Now if it's something that's more costly like DIY remodeling an LLM isn't a primary driver, but in as a device that can push back when I ask about mistakes that can be made and what to watch out for.
> that’s like teachers saying “you won’t always have a calculator
They were right, though. For one, you still need to now what kind of equation to put into the calculator. If you don't know the concepts, what are you going to calculate. Real life situations don't actually spell out neat textbook questions.
And just take a look throughout the day how many quick maths you do subconsciously. Just things like cooking or grocery shopping have tons of moments where you just do the calculation in your head. And you can do that because you've learned how and when to do that.
It's not just a matter of "What if it's inaccessible", but also "Do you want to be dependent on your phone even more". Are just going to walk around with your phone on voice mode narrating your life into your LLM to let it tell you what to do?
They were not. I'm not dead yet so I suppose there's still a chance, but so far in decades of living life I've always had calculators when I've needed them.
> For one, you still need to now what kind of equation to put into the calculator.
That's a very different construct. Math concepts don't care how the arithmetic is performed.
> If you don't know the concepts, what are you going to calculate.
If you don't know the concepts, then what kind of arithmetic are you going to perform with anything -- whether the mind's eye, pencil and paper, a sliderule, or a pocket supercomputer?
I have a calculator (and many other tools, of course) in my pocket at all times, now... and yet I don't pull it out very often. Because I can add $1.99 + $3.49 faster in my head ($2 plus $3.50 minus two cents = $5.48) than I could get the result by pulling out the phone, swiping my access pattern, tapping the calculator app, and punching in the results. Heck, even if I had a dedicated calculator that only required pulling it out and pressing an On button, I'd still come up with the result faster than I could punch the buttons.
It's definitely worth it for kids to invest the time it takes to learn the little shortcuts involved in mental math: that effort gets repaid many times over during the course of a normal human lifetime.
> I reject all notions that these are mutually-exclusive concepts.
Agreed; just because I "rarely" pull out the calculator doesn't mean I never use it. Sometimes my wife asks me "If this cake recipe calls for filling a 9-inch pan with batter, and makes 3 layers, but I have an 8.5-inch pan and I'm making two layers, then how much should I reduce the recipe?" The π terms cancel each other out in that math, but I still can't do (4.25×4.25×2) / (4.5×4.5×3) in my head faster than I can punch it into the calculator. (It comes out to about 0.59, BTW, meaning she should use either half or two-thirds of the recipe depending on if she wants a slightly thinner or thicker cake).
Bringing this back to the broader LLM-related discussion, the lesson I took away from the Fable kerfluffle was "the model will usually, but not always, be available." You might have an Internet connection hiccup, you might have hit your subscription usage credits for the week and not be willing to pay API pricing, the model might even be taken away from you. (Most of which are problems you won't face if you run a local model, but that's a different discussion). So just as it's important to learn mental math even when you have a calculator as backup, it's important to retain the skill to solve many categories of problems without the LLM, even if you still reach for the LLM for the categories of problems that it can solve faster than you can.
I don't think I'd want to rely upon having a rental pocket calculator to use. It could become unavailable at any time, for any reason. For something as common, inexpensive, and easy to find as pocket calculators have been for decades, renting one sounds crazy.
But that's the way that clown-based LLMs are: They're rented. They could disappear at any time.
And sure, I do use the bot to get some stuff done, anyway. I think of it as somewhat akin of the time-sharing computer systems of yore, where people absolutely rented (often far-away) computing resources to accomplish their work.
Those time-sharing systems were only temporary. The giants like Control Data Corp fell out of favor as computers became smaller, faster, and cheaper. (And then they came back in the shape of things like AWS, but that has a lot of non-technical things driving it; AWS is, strictly-speaking, completely optional.)
Unless we stop making computers smaller/faster/cheaper (which we haven't, despite the present-day supply/demand market blip) I think it is certain that we will get there again with LLMs, where the giants fall. Not this year, or next, but small systems will catch up.
And these little machines don't even have to win the race. They just have to be Good Enough.
So it seems inevitable that people like you and I will live to see fantastic local LLMs happening with affordable hardware that fits on a desk.
And then, eventually, into our pockets -- like a calculator.
Given the power draw of the GPUs and RAM needed to run those local LLMs, I don't see them being comfortable to hold in one's pocket anytime soon. :-) Not until some not-yet-imagined breakthrough in heat dissipation is made. But a thin client that talks to the local LLM on your private network, that's already possible today. So yes, within the next 5-10 years I fully expect I'll actually want to use an AI assistant on my phone. Currently I go through the settings and turn everything related to Google Assistant off, and do that again after every Android update. But once it's talking to a model that I control, instead of a model controlled by the world's largest advertising company, I'll feel the opposite way about it.
P.S. If "clown-based LLMs" was an autocorrect-assisted typo for cloud-based, it was inspired. :-) If you typed that on purpose, it was also inspired. Have an upvote.
Power consumption per unit of work keeps going down, too. Our pocket supercomputers are astoundingly efficient compared to what it used to take to get the same work done at relatable points in the past. We'll get there.
Before we get there, we'll have network connectivity to our desktop AI boxes at home. That's pretty good, too.
And if this clown-bot[1] boom is a bubble (as I believe it is), then it's just a matter of time before it pops. The blast radius is unknown, but at one end it seems likely to result in something between potentially-idle fabs that have already been mostly paid for. Idle factories are bad and it makes sense to avoid that even if it is expensive, so this means cheap hardware.
At the other end, it means failed companies with fabs that are sold for pennies on the dollar alongside a resolute unwillingness amongst the investors (who just had their asses handed to them) to start on Boom 2.0. This latter scenario would mean golden age of cheap hardware.
I think we'll be fine.
[1]: Yeah, clown is on purpose. It fits as a replacement in any technical parlance where "cloud" would be used. :)
I'm of the same opinion re: the bubble. The dotcom bubble held on longer than I thought it would (I had been predicting that it would burst in 1999), so I'm not sure I should be making predictions as to when the rental-LLM bubble will burst. But I'm sticking with the hardware I have until memory prices finally fall again. Whether that's only because of spare fab capacity as you mentioned, or because one of the major companies has gone bankrupt and their already-purchased hardware is being sold off at fire-sale prices (perhaps to other companies, who then reduce their new-hardware orders accordingly, which again results in spare fab capacity), I'm going to buy more RAM when it's cheap.
... And I've just been agreeing with you on all points with this comment, haven't I? Time to wrap up the discussion, then: once total agreement is reached then there's not much point in continuing the discussion unless someone has something new to say, and I don't have anything new to say on this topic. See you around.
I feel like you missed the point I was trying to make.
Sure, a teacher framing the necessity of learning math as "you don't always have calculator" is pretty lazy answer that doesn't really get to the point. In my experience, teachers rarely even made this point, students did that tried to argue that Mathematics was pointless when you have calculators.
But my point was that actually knowing these concepts is good for complex reasoning you can do in all aspects of life. Whether you have the calculator with you or not.
Sure it does. I still nees to know the correct mathematics to type into the calculator and the calculator is deterministic. If you don't understand it without the llm explaining it, how can you be sure you actually understand it? How can you catch mistakes made due to it being non-deterministic? Ask another llm? Same problem
>> "It's inherently worth more because it's hard-earned and anchored to experiences we share."
> At some point, it will be impossible to tell the difference. Many people already can't tell if something was generated by AI.
Nit: it won't be impossible, just so hard so most people won't bother or give up if they try, and society will settle into a mediocre, regressed state. Then wait a little while, and the next generation will justify their mediocrity as actually some kind of progress, and the people who knew better will be dead and unable to challenge that.
More technology != more progress. Just look what social media had done. At its best, it's like what came before, just more isolating.
I mean, you act if society is something that always progresses into something you in particular want. Studying history shows us that society commonly progresses in to horrific terrors that defy reason. And so we may be entering that state again.
"Gatekeeping" is a derogatory way of saying "maintaining standards of quality", usually by those who stand to benefit from lowered standards. I heartily approve of "gatekeeping" because I like having competent co-workers.
The thing with calculator argument that always gets me. I do math unconsciously all the time. Even something as simple as adjusting a recipe when cooking. I don't need to grab a device to do that, I just do the quick math in my head. And the only way I can do that is because I learned how to do math. And even with a calculator, I still needed to know what to calculate. The argument "we don't need math, we have a calculator" assumes you always get a textbook question that lays it out for you.
Same goes for LLMs. I can use them for programming, and they're very convenient. But I still need to know what to ask it and make sure it stays within the confines of what I want. And without my knowledge I would have no clue if what it's trying do is correct, or safe.
Naturally, this assumes a workflow where you do actually look and modify the output yourself. But I'd argue that any non tech person is inevitably going to hit a wall where they can't debug themselves out of without getting a human involved.
> I just do the quick math in my head. And the only way I can do that is because I learned how to do math.
A lot of Hacker news commenters tend to overestimate human ability without education. That is, the tend to believe that people are able to do a lot more without regular training than they actually are. They believe that "math" is some naturally ordained eventuality that humans just do, or likewise that "reasoning" is some immutable natural behavior. In truth these things are "unnatural" human reinforced structures that we have to learn and adapt our brains into.
I'm abhorrent at arithmetic and I think it affects me greatly. I know quite a few people who are barely literate and manage to get by in life, but it obviously makes things hard for them and strips them of opportunities (and they forego holding any identity that seems to "smart" for themselves). I think numeracy is similar.
I wonder how much better I could become as an adult at mental math. I did a math course a few years ago (Math Academy) and put some effort into improving my addition, subtraction, division, and multiplication, but I wish I'd done more up front. Even one-figure arithmetic is so fuzzy to me, like it's a hashtable where the keys don't line up to the right values. Can I unlearn that now? O(1) doesn't matter if the result is wrong.
Underrated and paradoxical element of this, to which, as far as I can tell, LLM boosters offer no solution:
> And without my knowledge I would have no clue if what it's trying do is correct, or safe.
I would contend you got the knowledge by typing the code yourself, that there's no other way to get it, and that if you stop typing the code yourself--and the slogging that entails--you'll lose the ability to prompt LLMs effectively.
It's not that I think the physiokinetic aspects of typing as an input mechanism hold some metaphysical distinction, but rather the level of engagement it forces with the code, and the units in which it does so. I'm not aware as yet of any viable replacement for that.
It's easy to trade on decades of software engineering experience with LLMs: with sufficient experience, everything goes around and comes around, almost any pattern is recognisable, the gratification is immediate, the benefits are now, while the costs and disasters are down the road.
However, the technology world is not static, and if you don't keep up with new frameworks, libraries, languages and other tech in that physical-mechanical "mind-body-keyboard" way that typing--or something substantially close to it--accomplishes, you will lose the ability to navigate that world fluently. To say it's just another abstraction layer and that the world didn't crumble after compilers is to miss something quite essential about how LLMs differ from compilers or high-level languages. The disengagement with the process of physically programming something quite specific will take down with it the ability, over time, to formulate useful prompts and competently review the output.
> I would contend you got the knowledge by typing the code yourself, that there's no other way to get it, and that if you stop typing the code yourself--and the slogging that entails--you'll lose the ability to prompt LLMs effectively.
For sure, and I've tried picking up a new language with the use of LLMs, but the concepts just don't stick because I don't actually do the work. That's why I do try to limit my use of LLMs to fields I'm already closely familiar with, and also keep its output contained to actually reviewable chunks. Or things that are just tedious, like OCR and large text transformations.
> the concepts just don't stick because I don't actually do the work
Yes! And this is the part of the de-skilling puzzle that is completely unaddressed by AI boosters.
Maybe LLMs are a force multiplier, but there still has to be some force to multiply, and I don't think a lot of folks ask the question of how that force is actually cultivated. This nebulous, airy-fairy notion that humans add "architecture" or "taste" doesn't tell the story of how, concretely, they came to have it. It seems to me there is no escaping that it came from typing the code.
Like you, I have a much better grasp of code and API surfaces I've physically typed than things LLMs have emitted and I have reviewed, in a conceptual sense, but which I could not have typed myself then, nor can type now.
I think you are underestimating how fast this stuff is moving. Calling AI "just autocomplete" for system architecture is outdated.
If you were to ask it (especially with Claude's newer model, Fable 5) to draft an architecture design diagram, it would do a better job than any junior dev ever could. Sure, giving context is still necessary, but you can just speak in plain, normal English and it will understand. AI is getting scary good at system design.
> Calling AI "just autocomplete" for system architecture is outdated.
I didn't say that, though.
> it would do a better job than any junior dev ever could. Sure, giving context is still necessary, but you can just speak in plain, normal English and it will understand. AI is getting scary good at system design.
Sure. But that's not my point. Don't you still want those juniors to be capable of creating those diagrams? So they can understand them? If the juniors don't get what the AI is outputting, why are you even creating the diagrams in the first place, right?
That's what this topic is about. Just like how calculators helped us do complex tasks, we still needed to understand math. Same with AI, you still need to be understand the topics you ask it to do.
I think that most of people misunderstand what calculator changed. Calculator didn't replace people doing math, calculator replaced mathematical tables, slide rulers and other already existing devices.
And regarding making people dumber ... Math teachers who saw this change said that there was a clear shift – students started to think less critically. With slide rulers and tables you had to think about answers – significant figures etc – with calculators you don't.
You still have to think about significant figures with calculators I used. Most will produce an irrational number result with as many insignificant digits that will fit on the display.
Only with final answer, if you have to round etc. With slide rulers and tables you had to think about it with every step, where to put your decimal separator etc.
One could argue that those who think calculators didn't make themselves dumb cannot be the judge of that. Cannot be the claimant, defendant and judge all at the same time.
I would say that calculators definitely reduced my more simplistic mathematics skills but did open up some more advanced stuff. I wouldn't say it was a gain or a loss, just depends on what your needs are.
> In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more.
While I appreciate you laying it out so plainly, I disagree. A novel is a bunch of words and I don't care if they were written by one person, five, an AI, or infinite monkeys on typewriters. What's valuable in a novel (or a poem) is in the words.
Wow, I could not disagree with this more. What’s valuable about a novel is the social relationships it fosters, both in relation to the author, and also all the others that have read it. I read a book to better understand what other people are thinking, how they see the world — both directly from the author, and indirectly, in discovering what other readers may have found valuable.
I can maybe understand finding value in a machine-written novel if others also read it and enjoyed it, but having an LLM spit out a novel and reading it in isolation, that would be a complete waste of time to me.
For me in that case, I'd value the story but not the writing. I want to know that a human has had the experience of writing what I am having the experience reading.
If I find out after the fact that an AI wrote it, the writing becomes bland, like a magic trick exposed.
It seems to me that you don't like reading (which is fine). Some people enjoy reading words strung together in a certain way. The value comes from the simple relationship between the text and the reader, not on some kind of social connection.
It seems to be that you don't appreciate the relationship between the author and the reader and the author and the text, which is fine, but is a bit tragic.
Not that I doubt that one day people will simply gather around the AI infinite story creation bot. They just won't know what they're missing. :(
I once read Ben Franklin's autobiography. I have yet to experience any great blossoming of our relationship.
More seriously, the novel is a unidirectional medium of communication. Not a relationship. That said, it is meant to convey a perspective. I don't care if it's a human, a robot, or a little green man's perspective. It just has to be an interesting, useful, or enlightening perspective that I didn't already share.
Right now LLM's don't really have unique perspectives, that may change. So I withhold judgement. As of today, I wouldn't read an LLM's novel. In 10 years, who knows?
> I once read Ben Franklin's autobiography. I have yet to experience any great blossoming of our relationship.
I've also read it, and got to know the man that much better. It especially added a great deal of texture to the experience of visiting his home site, his church, and his grave.
I'd argue that humans rarely have unique perspectives, being highly influenced by their environment and experience. And have we not seen AI coming up with novel math?
I don't think unique perspectives are necessarily the compelling part. I think the words are just a medium for thoughts. AI only produces the former, not the latter.
That's my relationship with Ben Franklin. It's not that I read what he wrote; it's that I know what he thought.
> I think the words are just a medium for thoughts. AI only produces the former, not the latter.
I think this is exactly what I was trying to say above when I talked about perspective Reading is a form of telepathy, and there's just nothing INSIDE of most AI writing today. When that changes, I'll take it more seriously. I do suspect that it will very soon though.
As you say, AI's are already coming up with novel math and I've seen some glimmers of purely mechanical brilliance in the writing already.
When you read a book, you hypothesize a writer and have some kind of inner relationship with the hypothetical writer. But it's not a real relationship. It's in the words, not outside them.
Ah I always felt like novels were a partnership in a way. He is putting down words and I am using that to create a world in my mind. 2 people reading the same novel are creating 2 different worlds.
I've always been fond of calling it telepathy. That's really what it is. Further, it can even let us see into the minds of the long dead. Humanities superpower is that it is the only race which can preserve knowledge beyond a single lifespan.
With LLM's we can also now preserve actual thought patterns and skills beyond a single lifespan. I can't wait to see what the future brings.
Reading is beautiful because of language's inability to fully capture emotion and reality. Every author tries to tell a story in the way they think captures their perspective best. Every author and book you read is a different perspective on the world that shapes your own.
Reading writing from an LLM loses all that. It's just words, there is no expression. It's literally the most boring parts of reading.
You do care though. You may not think you care but you absolutely do. If somebody tells you that in the Hood canal west of Seattle, there is a population of endangered tree octopuses, who are unique among octopuses in that they climb up to land and lay their eggs under fallen tree trunks in old growth forests, and that they are now endangered because road construction is cutting their access to the old growth forests, so they cannot lay more eggs.
You would care about that story, until you found out that this story is a lie, an old April’s fools joke that escaped confinement. The words are the same, but your reactions to the exact words have changed with new information about the source.
When we read personal stories it affects our emotions as we empathize with the author, or otherwise share the feelings that the author is trying to convey. When we find out there is no such author, our empathy and our notion of shared feelings vanishes with the new information even though the words stay the same.
> You do care though. You may not think you care but you absolutely do.
This is something I have a lot of trouble explaining and generally don't try to because I've never actually studied this or anything. So I can only go from my 45 years experiencing of experiencing art along with others.
Of course if you are just putting on music to work to, this isn't going to matter much if at all, but...
Generally people do really seem to care about the person behind something they are experiencing. The simplest example I can give is one of those extremely well shot photos that very few people have taken from a sitting position of their feet dangling off a massive building. I would have a very hard time believing anyone claiming that such a photo wouldn't give them very different feelings if they knew it was a real person v not. Again, this is the simplest example I can think of but I think it goes much deeper with all sorts of art, ie, most people to some degree are attempting experiencing art through the person who created it whether they "know" it or not. This is evident when presented with something they don't like and say something like, "Who would make this?"
Your example hinges on whether a bunch of words is true, not on how they came to be written.
> When we read personal stories it affects our emotions as we empathize with the author, or otherwise share the feelings that the author is trying to convey. When we find out there is no such author, our empathy and our notion of shared feelings vanishes with the new information even though the words stay the same.
This isn't true for me. If I read an incredibly moving poem and later learned that it was written by someone casting the I Ching and picking words out of a hat, it would not affect how I felt about the poem.
Generally speaking - I agree with you. Source bias is a real logical fallacy, and failing to evaluate the content itself, regardless of the source, is a problematic view of the world.
In the same way that I don't need the lumber in my house "hand sawed" for it to achieve my goal of creating a habitable space.
---
But more broadly, I do think there's space to at least question the use and role of AI.
Because while content can (and should) be addressed directly, there's a valid meta-conversation about the intent of producing content, and the results producing that content might have.
What goal does producing this content achieve?
What is the role of this content in society?
Is this content, on this scale, an appropriate thing to be making?
These are MUCH harder questions - often because we've shifted from concrete (content) to abstract (value judgements).
To go back to my housing analogy: We're no longer evaluating the benefits of hand-sawed vs power sawed timber. We're discussing whether our housing is built in the right spots, if we're building enough of it, and are we allocating it in the right ways.
This is irrelevant because we’re not using the source to judge the strength of an argument. Logical fallacies have only to do with the strength of a logical argument, nothing else.
To illustrate this, if I sell you some energy and you ask whether the energy comes from burning children or from a solar farm, I can’t say that it’s a logical fallacy for you to care because all energy powers your home just the same. You don’t owe any consideration to the content (energy) at all, because energy is not an argument that is subject to the source fallacy, to treat it as such is a fundamental category error.
Even to take your tack and say “We should consider the energy generated from burning children apart from its source, because we don’t want to fall prey to the source fallacy. However, the societal effects should also be taken into consideration…” shouldn’t be countenanced.
There's a road in California that is the last hand-built road made in the state. Nothing special about it, in particular (aside from the terrifying, exposed drop), otherwise.
But since I know the human history of the road, it _feels_ different when I'm on it than other machine-built roads.
It's the same in a hand-made house. Knowing the human labor gives the house a different vibe.
If we look at 10 paintings and one was painted by a human master artist, it becomes, to me, more impressive than the AI works, even if it isn't the award-winner.
These sentiments are incredibly subjective, of course. Some people simply feel no difference between a hand-made brick and a machine-made brick other than the latter is likely cheaper and of higher material quality.
But for those of us looking for the indescribable _soul_ of the work, we fail to find it in those produced by machine.
I just visited Lowell National Park and watched the mechanical looms in action. The cloth they produced was blandly soulless, like all the cloth we use and wear and discard. The loom itself, on the other hand, was a hand-built mechanical work of art, and felt amazingly _human_ by compassion.
This isn't something we tend to value in the US, though. The closest we get is people hanging their kids' childhood art on the walls and buying custom art at great expense to increase social standing. And a few support-your-indy-artist types.
>it _feels_ different when I'm on it than other machine-built roads.
Well, because roads are hard to build and take a lot of energy, it's going to be way narrower, curvier, and hillier. The guy shoveling out rocks all day would have gladly switched the shovel for a bulldozer.
But all of these value statements about value statements are kind of ignoring the Moloch problem in the corner of the room. What becomes the purpose of humans in a world of automation? At our current rate, it's being exterminated under the boots of corporations that swallow up all the value in the universe. The conversation of "How can things be made better for all humankind" seems to have disappeared after the greed is good conversation won.
I care because it’s not personal, it’s not informed by lived experience, etc. It has no meaning. AI “novels” are just facsimiles of other people’s writings assembled in a way that appears new, but ultimately it’s just an approximation of a person writing a book. Their’s no author intent or context of any kind to consider. It removes core pillars of experiencing literature.
I don’t want a mathematical approximation of writing informed by feelings, knowledge, experiences, etc. anymore than I would want to see an “AI band” perform just because the music is supposedly great. There’s no personality, there’s nothing personal period.
while I understand this argument - I think it consistently falls down when compared to the types of content that are generally purchased and consumed.
I'd argue that most folks get plenty of value from the content itself, entirely separate from the intent, context, or even existence of the author.
I don't care about the motivations of the director to enjoy an action movie. I don't care about the life history of the author to enjoy a good fiction.
I think there is (and should be) space for content where people care, but I'd suggest it's the fringes (ex - majority of music is production grade pop, not meaningful songs, majority of books sales are erotic fiction, etc).
> I don't care about the life history of the author to enjoy a good fiction.
But the fiction is a product of the author's lived experiences. If the author had lived a different life, they would have written a different book. Or none at all. Without life experience, where would stories even come from? Why would they matter at all?
> types of content that are generally purchased and consumed.
This is the entire issue though. We’ve boiled creation down to “consumable content” like we are all in a boardroom talking market strategy. I am reading a book to enrich my life. Yes I enjoy popcorn entertainment and “low brow“ stuff, not everything needs to be Citizen Kane, but what does it say about us if we’re willing to just sit around “consuming” LLM content which is just facsimiles of actual creation by real people? What is the point when we have more “content to consume” than ever before? It’s just saturating us with impersonal stuff lazily achieved by scraping the real thing and prompting until it outputs something acceptable. Why is the person even making it? The answer unfortunately is almost always “a quick buck,” so I’m not sold.
What is the point of reading a middling fantasy book that a person didn’t even create when there are already likely countless fantasy books in existence/being written right now?
While I subscribe to an absurdist viewpoint rather than the nihilists, attempting to take an objective view on what one should consume is a difficult and most likely completely incorrect task.
The entire talk of should we mass product LLM crap generally side tracks the discussion of 'how can we create a world where everyone can consume mindless crap instead of endless toil'.
> I don't care about the motivations of the director to enjoy an action movie. I don't care about the life history of the author to enjoy a good fiction.
...and you don't need to know those things, that's insane. But you very likely are, even (not so) subconsciously, questioning some of these things.
LLMs are inherently backward looking; their training corpus consists of existing works and their ability to extrapolate is ultimately constrained by that corpus. However powerful their proponents claim them to be, they aren't powerful enough to predict the evolution of human society and culture as a whole.
Changes to that corpus that reflect the real world are going to come from incorporating future works created by humans. (LLM generated training data can reinforce fidelity to the current corpus but don't alter it.) The future still belongs to human creativity.
Completely untrue. Stephen King has a loyal following. These same people aren't going to pick up a generic scary book written by a machine. This goes for music as well.
> What's valuable in a novel (or a poem) is in the words.
Even if the words are a lie? Misleading? False?
I'm not even talking about LLMs. What if it's propaganda designed to influence your thinking, possibly against your own interests; are those still valuable words that you'd cherish reading?
My point is that the source matters, intent matters, and authenticity matters. To me, anyway.
Good news! I have so many words for sale. Humans spend so much time trying to put them together in 'meaningful' ways. I take all the hassle out of it! Dirt cheap! Billions of words! How many would you like?
> If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share. I want to support that
I'd extend on this as well: the process of creating changes you. In a technical sense, where you approach a problem and the way you solve that problem informs you. Both your problem solving skills, creative skills, but also even understanding how a compromise works.
This is why I have minimal compunction about an experienced engineer using AI-assisted coding ("hey claude, define this data class") versus finding AI-art to be repugnant.
The act of creating an artistic work is both an expression, but also the act of ideating and then executing on that changes you. Experience, emotion, and other more intangible concepts.
I use AI to make songs and pictures for my friends. I barely even listen to music that I didn’t have a hand in making. It’s my favorite genre by far. My kids too. Personalized studio quality songs all day, every day. I suddenly have the power and time to make things I enjoy. I think there’s going to be a hollowing out that’s going to happen where we might not have another Michael Jackson or Taylor Swift or famous musicians because why bother when you can just listen to your own custom tailored tunes. All that matters to me is that I like the song. I don’t care if someone built a custom keyboard or did it on a Stradivarius violin.
> I suddenly have the power and time to make things I enjoy.
You don't make. You ask the ai to make a song about some vague concept and then clap like a seal as it delivers you un-challenging and bias-confirming slop.
> I think there’s going to be a hollowing out ... All that matters to me is that I like the song.
Everyone consumes, and nobody makes everything. But you're describing a future in which nobody makes anything and we all just suckle directly on the soft serve machine.
I think for many people, maybe even most people, the story and the people behind the music is every bit as important as the music itself. I don't see how an AI can capture this.
you should really consider trying things that don't exactly fit your ideals. Listen to music you don't think you'll like, or watch a movie that you've never heard of. You find threads that you do like and learn to understand the parts you don't. Discomfort and friction is how we grow as people, it's how we gain perspective, and how our interests evolve.
Otherwise, if you're going to keep consuming the exact slop content that you're looking for, you're going to stagnate as a person.
I mostly agree with you but, how do you square the use of other types of automation that are not AI? e.g computer animation vs hand-drawn, CAD software for engineering and architecture… Are those lesser art forms or we’re just used to them?
I don't have enough experience in those fields to really firm up a judgement from my perspective.
The act of creation not just creates, it informs you. For example, when placing a wall with CAD for architecture, maybe you realize that some walls on an non-level roof have visually unappealing or difficult wall/roof joints. And then you consider that for future designs? Or you pause and think further about how a person interacts with that room. You can see how the act of creation delivered meaningful feedback to you, as a person.
IDK that's just an example I could come up with, you should really ask an architect or ID guy
> But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you.
Except calculators have been a problem for decades, it's why they're not allowed in school when you're still learning. Without doing the math yourself and internalizing how it works, you won't develop the number sense to tell if the result makes sense (broken calculator, typo, wrong equation, etc).
I still remember my physics teacher using one of the student's test answers as an example of how he should have known it was wrong and gone back over it (I think it was a pendulum on an elevator, his result had negative gravity (so gravity going upwards)).
I see an AI as an advanced calculator, but I see even a simple calculator as a poison to education if not strongly controlled.
Number sense is such a nebulous concept, but it captures what I've seen people struggle with. A general sixth sense about where an equation should or shouldn't go, a feeling that random numbers don't appear quite random enough and might have a pattern, or even recognizing that having a pattern signifies some relationship that otherwise has no evidence.
We had exams where the calculator was allowed, because we were tested on our ability to type in the right calculations. The calculator wouldn't save you.
Even in the 80s HPs would do derivatives (and simple integrals).
But, open book tests are the worst, nothing could save you!
I still remember giving Mathematica a relational equation of the atomic radius expectation values for it to integrate by parts and collect terms... at the time it failed to find the right integrating factor and gave gibberish. Probably works now.
If you can't work in order of magnitude calculations and estimations you're not going to last long in physics. Many useful models give good answers only within a certain range of inputs, while at the same time you often can tell where in a dynamic space you are by external observations. For example, estimating Martian wind speed within a factor of 3x from the size of the sand dunes, can be done having only seen pictures and with only the knowledge of a few dimensionless parameters.
That's also how you can tell if your calculator is "lying" to you (or you typed wrong). I guess I have a few similar tools in spaces where I'm more experienced to see when "hallucinations" and gibberish are being generated by LLMs. Of course having it "check sources" and evaluate its own solutions sometimes also works, if those are reliable, but you're on the hairy edge at that point.
> We have people here who use LLMs to raise their children, to manage relationships, to design products.
My first thoughts were around trying to understand why these people would do that. What I see a lot around me, is people being afraid to fail, as it's some inherently dumb and bad thing, not realizing that sometimes failing is what makes you learn enough so you don't fail later, or builds resilience in other ways that later will be useful, for others or for yourself. Avoiding the path of harsh failure will put you down the path of mediocrity.
It's passing the buck, isn't it, if I ask AI and AI tells me something dumb, I can blame AI and absolve myself of responsibility.
At the same time, the troubling allure is that the machine has ingested a million books and has better knowledge than me (e.g. in things like JavaScript or Japanese), so why not trust it about how to raise kids.
There's a difference between capability and mastery. If you have a calculator, you are capable of doing basic arithmetic (probably). If you can do fairly complex arithmetic in your head, that's mastery. If all you need is capability, it's fair to say that acquiring mastery might be wasted effort. However, if you want to go on to do advanced math, physics, etc., then mastery of basic arithmetic, calculus, algebra, etc. are necessary stepping stones. If you have to go back to calculators, math programs, textbooks, etc. every time for basic things while trying to formulate and manipulate equations, you're going to have a very hard time getting along.
Sure, rely on AI for things that you merely need as capabilities, but recognize that doing so prevents you from developing mastery required to progress to other things. If you don't want to progress to those things, then it's fine. If you do, then you should rethink your approach.
I'm acutely aware that even 6-12 months off the keyboard is brutal to come back from, it's a real issue now because there are pockets of work where the agents can drive the whole thing. If you just autopilot there? The comprehension and skill debt becomes really hard to pay down in as little as a week or two for the really high complexity stuff.
I try hard to keep a serious feature branch going on every project where I'm driving it in emacs or vscode or whatever. `gptel` is great for this because it can tool call for the docs, be a super search engine like Google used to be but you're keeping your reflexes up, keeping the comprehension debt under control.
For me this is almost exactly like being in an EM role: it's so easy to lose track of the details, you have all these other demands on your time, there's pressure to grow headcount, and the costs of letting the skills slip is all but invisible for a while. But before you know it you're rubber stamping stuff you don't really grasp, you're not able to help people who are having trouble in the codebase, it's all bad. And depending on why you got into this business, it can become pretty joyless to be totally out of the loop.
Now every engineer has to navigate this. It's going to be interesting.
I don't relate with this at all, are you saying that founders who execute through employees are not creating anything by "themselves", they have no "unique contribution"?
Most of us just want to create great products and hopefully get paid for them. And at the end, if I create a great product through this method, I feel equally happy, who wants to grind for a decade over some manual software creation process ?
*some of us. This is one of the differentiators (can’t find a better word right now). Some became software developers because they enjoy(ed) the process. Some became software developers because that was the way to make products that made money. Neither is more correct or genuine than the other, but neither can also claim to be a significant majority.
TBH, the calculator example isn’t an encouraging one. I use a calculator (these days N PP, or spotlight). But by the time I do that I already have an approximate ide of why the answer is, so if it’s way off, or the sign is wrong, I see it immediately.
Same with GPS, though many folks have no idea how to navigate on their own anymore so often don’t notice when they’ve accidentally had a typo or chosen the wrong Springfield.
So I find using an LLM can be useful when I already have an idea of that the problem is.
At their request I reviewed a colleague’s plan for a project. I looked up some of the baseline parameters so I could understand the plan, than I made some comments w/questions about things I didn’t quite understand.
A couple of minutes late for our morning mtg I came in while my colleague was saying that I had found a bunch of things that Claude had not.
This is why they are bad news for new grads who haven’t yet learned the and pitfalls of their fields. And this was a senior person who just outsourced the work to the LLM.
For most people, GPS did not improve sense of direction, spellchecking did not help to write without making mistake, deepl did not help to be better in foreign languages. But replacing a bicycle by a motorcycle forces to acquire new skills without losing any, and we can find many example of symbiosis between "The man and the machine" (Lindbergh wrote a book named "WE"). AI could be something like that, after all it is human knowledge reachable in a conversational and contextual manner.
So AI can be used to learn: "tell me what's wrong in my code, or if it can be improved". I also tend to think that the more we code, the more we give AI valuable piece of knowledge to learn from, the best code it can produce, the less the produced code seems alien. It can be a win/win, all depend on the mindset. I like to code and even if I am skeptical about many aspects of AI I can share the workload with a robot, as an exercise or if the time or budget is constrained.
> I'm not good at lifting weights or running, so my thinking is the only thing I have.
Even if you were, the best you could hope for is a career in entertainment ("sports"). The necessity for bodily strength was abolished in the industrial revolution of the 1800s and 1900s. Now the maximalists predictably say we have another "industrial revolution" that makes our intellect redundant. I don't believe that but if it were true, the question what defines the value of a human would become even more pressing, with less leeway as to what the answer might be. H. G. Wells needed The Time Machine to find a world like that.
Still, if enough people believe that LLMs do make our intellect redundant, it will trigger a cultural crisis similar to the one at the previous turn of the century when men struggled with their defining attribute (physical strength) being taken away by machines. The reverse centaur is over 100 years old, even if that name wasn't used then.
Having learned math in an accelerated program that banned calculators, my experience has been that they do substantially diminish mathematical learning. They are useful for things like building an intuition about the behavior of functions... but ask a student to manually graph and compare the behavior of some fairly trivial functions ( ln x, 1/x, sqrt x, x^3 ) over the interval [0, 1] and you'll pretty quickly see the difference between kids who are trained to think for themselves and kids who aren't.
The ability to engage with math "critically" is fundamental to being able to use it constructively/creatively.
I think we're seeing the impact of LLMs more broadly in the drastic decline in incoming freshman academic competency levels, and will continue to observe people becoming measurably, meaningfully stupider until we clamp down on / outright ban LLM usage in education.
"What's your unique contribution to the world" is a good question, and I've been thinking about it a lot. One framing I very much like is thinking about the priceable versus unpriceable contributions of a person. As an example, your software engineering ability is priceable (it's your labor). Your value to your parents isn't priceable. The problem is, right now the default assumed value of the latter is 0. Our society only tries to put a value on priceable facets of a person. There's reasons for that, wisdom of the crowds and whatnot, but of course it also means that if an AI can do that thing, economics drives society to adopt AI over employing people.
It's possible to set that latter value to be nonzero. You can't use free markets to set it because they necessarily cannot see what price to set, so you kinda have to guess, but IMO, it isn't zero, and I'd hazard to say it's positive.
Silly nitpick but the free market absolutely can, and has in the past, put a monetary (positive)value on your person.
We have of course abolished slavery since then.
Which is a weird way of saying I don't think we necessarily should put a monetary value to people. Instead we need to realize that money isn't everything in life. So much of life is unpriceable, and that's as it should be. Attempting to price the unpriceable necessarily makes a mockery of the unpriceable thing.
Yes, but also the prompts you give after, and the iterations you continue to do. Art is rarely something you build once and forget about. A lot of that going on in AI no doubt, but the builders who polish their work and strive for perfection are still the builders they were before AI.
You mention the novel writer and yes that person who's great at writing novels will create great art. But the ones that are 80% of the way their, but maybe can't nail the ending, or segue into different plot-lines, can now create great art and have AI get them there. Subconsciously, they will get better at those things by using AI and seeing the result.
I'm always reminded of that I, Robot scene where the scientist says "you must ask the right questions." And that is true now more than ever.
>But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you. On the flip side, if you use an LLM do most of your thinking, what's left? We have people here who use LLMs to raise their children, to manage relationships, to design products. So what's your unique contribution to this world - is it the prompt you once wrote? You're standing in front of a token-generating machine, pulling a lever, sometimes receiving gifts. Is that your edge, your unique experience, your purpose in life?
Cognitive offloading seems to be a feature of humans, we do it with spouses and people we work with.
I dont see why offloading some other stuff to an LLM is going to dehumanise someone.
When people interact with spouses and coworkers, it's still human-to-human interaction. There is a person on the other end. Not quite the same when your head doctor, DJ, gym coach, and nutritionist are the same sycophant machine who's eager to let you know how absolutely right you are.
> On the flip side, if you use an LLM do most of your thinking, what's left?
If your answer is "not much", you gotta step up so that the answer is closer to "more than ever was".
You dug the holes in the ground with a shovel your entire life. Suddenly somebody gifts you a digger, nearly for free. If your reaction to this is "what's left for me now? anyone could just pull levers, no strength or skill needed!" not, "I could dig so much more and deeper holes with this digger!" then you need to chill, reflect, then fight your fear and perverse laziness and learn how to become the best person to pull the levers.
Most people only do the simplest of math on a day to day basis. I’d bet that if you asked most adults to do something even remotely non-trivial (addition, subtraction, multiplication) a lot would have trouble without a calculator and/or make a large number of mistakes. That’s probably fine because it’s unlikely most adults are all of a sudden going to have to do non-trivial math without a calculator. That probably isn’t the case for people who are having agents do effectively everything for them in their lives
> Many LLM maximalists say they use the tech to learn new things, but to what effect?
"Many book maximalists say they read books to learn new things, but to what effect?"
First we had verbal transmission, then books, then the Internet, now LLMs. And they all kind of do the same thing.
For me, the value of an LLM is it can take my imprecise query, scour its memory AND the Internet, and then return an answer (with citations if you ask) quicker than I can look for it "manually".
> So what's your unique contribution to this world - is it the prompt you once wrote
Why not? You're implying writing good prompts doesn't require effort or thought, which is false to me.
> Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?
To write better prompt is a good reason to learn new things. It's so obvious that more knowledge and experience you have, the better results you get from LLMs.
The calculator argument is interesting because I am for sure slower at arithmetic by hand than when I was in gradeschool able to rattle off times tables and long division and what not, due to calculator use. I know plenty of other people who can’t easily do basic math, like multiplying something by 1.5x as a recent example in my life. Their gradeschool self probably would also run laps around them.
> Many LLM maximalists say they use the tech to learn new things, but to what effect?
I would also argue that they use LLMs to learn "about" new things, but not really the thing itself. I, myself, learned a lot about complex SQL queries just recently using LLMs. But I really don't know how to write complex queries myself.
People often confuse reading about a thing with knowing the thing.
My take is that you learn math before learning how to use a calculator. And can understand what it does and reproduce it (slower) by hand if needed, but everyone is still using it for stuff we don't need to think about.
If you use AI exactly like this I find no issue, but I fear most people are using the calculator without knowing the math and that's the real issue to me.
Taking the context of raising children, the ‘unique contribution’ of a lot of parents is traumatising their kids. If it’s helping us make better decisions, I don’t see how that could be worse. I think that translates to all the non creative things you spoke about.
> If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share
But the market doesnt reward that. Different consumers different markets, but (using a salad analogy) overall consumers dont care how a salad is made, only whether it tastes good and is cheap
>We have people here who use LLMs to raise their children, to manage relationships
Anyone who's spent time in professions in or adjacent to social work would say that a lot of their clients would frankly be better off outsourcing these decisions to LLMs.
> I'm not good at lifting weights or running, so my thinking is the only thing I have
I'm not saying this to be snarky, but maybe it's time to work on lifting weights and running. The worth of a novel is, of course, subjective, but most people would judge it based on the entertainment value it provides and not how much human effort went into it. Hobbies like fitness (which don't have an output that's meant to be consumed) seem like a safer harbor in this new era.
I was going to say, neither of those forms of exercise are expressions of a natural talent... they're things you simply do (at whatever levels you can currently manage) to stay healthy. They're good for your mental state, too! (And I could stand to do both a bit more regularly...)
The same was probably said when machines started making furniture, cloth, harvesting wheat, etc. A desk made custom by a human to your specs is cool, beautiful, and life giving for both involved. And will cost $10-$50k. Maybe with the AIs and robots doing the grunt work, humans could get back to building things for each other. That would be great but I'm not sure what percentage of people are interested in that though.
> In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share.
In my mind, it's pretty simple too and not dissimilar to what you say: we've got cognition, LLMs don't. When a human writes a novel, it doesn't look like a novel because the human has been trained by reading every single novel ever written in every single human language out there.
The human doesn't imitate the act of writing a novel: the human just writes a novel.
> But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you. On the flip side, if you use an LLM do most of your thinking, what's left?
Really not much. The IQ of an oyster more or less. But the LLM is not thinking, so it's not possible to "offload your thinking".
But tbh those who believe these LLMs are thinking machines and that it's normal to offload "thinking" to those non-thinking machines probably don't have very deep thoughts anyway.
So nothing of value is lost.
P.S: I happily not one but three AI subscriptions and use LLMs daily to code. Which is probably why I'm such a critic of AI. They're both useful and pathetically bad. They really write crappy code.
> In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share.
.... What?
You open up with the calculator argument. Great. Then state "on the flip side". Ok.
And then declare it very simple in your mind when there is _only_ the flip side.
The whole issue is the not-simple gray area and where each of us believes the line between empowered human and brainless idiot is drawn.
> Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?
We had some contractors at the house doing some work. The contractors had been doing this work for a long time. I could tell they were good at what they did because I had some experience with that type of work when I was younger and they were way better than me.
We ran into an unusual situation (old house, old building techniques, overlapping old renovations) that they had never seen before. The contractors pulled up ChatGPT and started chatting with it. Eventually they came up with a solution with some good materials and techniques to make it work without turning it into a bigger tear out and complete rebuild project, and they got quick links to confirm it matched code.
This worked because they had the experience to judge the accuracy of the answers. They still checked the claims against the description of the materials we needed to order. After they did the work they had the experience to do it again, and it didn’t take them any guessing, trial and error, or searching through forums or Facebook groups (which are very popular with contractors) to get advice.
This is the right way to benefit from LLMs in learning something: Get the information then put it into practice so you learn it.
I worry when I see someone claim to learn something from an LLM but then the next turn is having the LLM do the work. It’s like skimming a math textbook but then skipping the exercises and never doing any quizzes. Every student in school learns that you have to practice to really understand something because reading the words is not enough. You have to work through it to internalize it.
LLMs can be very powerful tools in this way. They can also trick people into thinking they’re learning when they’re not. I think everyone has to go through that learning process about how to learn all over again like we all did with calculators in elementary school. There will be a lot of people who get trapped into being LLM promoters, whose output ceiling is limited by what LLMs can do as limited by their ability to recognize what’s right and wrong.
By now, almost everyone can think of someone they know who is happy to turn their brain off and press enter in Claude until it says the task is complete. I worry how this is going to intersect with the Reddit generation who grew up consuming doomer content about how jobs are just “bullshit jobs” and how your goal should be to do as little as possible at work, because there will come a time where those who spent their careers offloading their thinking to LLMs without doing the hard work of learning will not have much compelling reason to be hired over any other intern who can prompt Claude.
On the long term view I can see a lot of the LLM prompters getting replaced by LLMs doing the prompting. On social media like Reddit you can already see a lot of people gloating about how they just copy from Jira into Claude, push a PR, and then have Claude handle code review from their peers while they do nothing. Not hard to see that someone like that isn’t doing much that couldn’t be done by going one level deeper with Claude and letting it automate that person away. You have to put in some work to stay valuable.
Similar to your contractor example, I like to think of the useful way to use LLMs as doing "grep -C" (show lines surrounding the match).
You provide some context you are looking to expand on and let the LLM autocomplete provide you with relevant terms and concepts that have historically come up in that context and may or may not be actually useful. Then you do due diligence to learn about those related concepts, determine their applicability and distill it all down into a solution.
I find it helpful to investigate "I did it with an LLM" by imagining that (perhaps without the user's knowledge) the other end was another human, and asking whether that ought to change the answer.
This quickly engages a lot of social circuitry we've already developed when it comes to questions of what constitutes the user truly learning and having skills, what is meritorious, and who deserves most of the credit.
For example:
* If the spell-checker for your paper was secretly another human, that wouldn't become cheating. (Ditto for a calculator.)
* If you prompt "write an introductory paragraph" for a paper and it was actually a human ghostwriter, you'd obviously be cheating.
* If an executive prompts "write a congratulatory e-mail for me" to an LLM, it's not worse than asking a human secretary to do it.
* If you say "I learned kung-fu" while hiring a [robot/human] to kick the spine out of your opponent, you're a liar either way.
Eh, not convinced. Certainly what works for you works for you.
But the idea that “outsourcing” to experts is somehow detrimental is one of the modern tropes I can’t stand. And I’m not sure I believe there’s a material difference between e.g. reading research about raising children versus asking an LLM that is grounded in (and cites) that research.
Maybe I’m a cynic, but I don’t think most of us have a unique contribution to the world. Emphasis on unique.
So why not leverage all resources in the effort to maximize happiness for ourselves and those around us?
Focusing just on AI wrt our work (let’s say software engineering for now) and letting go the other items. I see a like when this comes up the story is if the AI is doing the arch/impl then what are you even doing anymore? That is honestly not where my value add is, many people can do those two things plenty well, it’s basically commoditized at this point (at a certain experience level). What I bring is all the soft skills and intuition to glue it together and ship and maintain it.
This just sounds like the black and white fallacy ad nauseum. Like every sentence is constructed to frame the entire issue around a false binary. Like you assume that if "maximalists" use LLMs more than the average joe, they're not "thinking." Black and white. Either-or. Maximalists are hobbyists. They usually read up on the newest models, try out different ones, maybe even build pcs specifically to run local models. You honestly think they're not thinking as much as you? Also there is absolutely no reason to value "being unique" for its own sake. Unique people aren't walking around trying to avoid tech to max out their uniqueness potential. That's just neuroticism. It does seem to me as though a lot of people have this misplaced faith in humanism and it just feels like copium at this point. Worse yet, it honestly just seems like a way to dunk on those they perceive as conformists. So basically just a high school mall goth. You are not in a position to lecture other people about their purpose. Your brain is a pattern recognition machine just like everyone else. You've grown fond of some patterns you picked up on that emerged during the enlightenment and you seem to be just superficially regurgitating them, not unlike an LLM. Calculators made us dumber in the sense that we no longer have to perform mathematical operations on the fly. That gives us room to do something else. What other people do is up to them. That's what freedom is. You assume the maximalist is just sitting there prompting all day. If they are, whatever. That's their hobby. It’s not much different than playing video games all day. Either way, Im pretty sure their lives are filled with sadness and joy just like you and everyone else.
Just live your life and don't support the AI machine. Nobody is forcing you, your life quality might get much better and you live your life as an intelligent person.
On the other side, people pushing the intelligence is for losers narrative are already too stupid to create anything that benefits humanity or the planet, so might as well ignore them.
How to build a circular LCD clock, step 1: buy a circular LCD that looks like a clock.
I'm not kidding, that's the extent of the build. They simply connect that display to a computer over HDMI. The only hacking here is browser-based JS for clock animations.
> Because people are going to realize you don’t need to pay someone $400k per year to hand out advice about moderate exercise and which antibiotic is appropriate for a sneeze-cough with yellow mucus.
Who is going to realize that?
The same forces that prevent you from walking into a pharmacy and asking for antibiotics based on what you found on WebMD will prevent you from doing it with a ChatGPT printout in hand. Lawyers and doctors are the best-known examples of industries that are in control of who gets admitted to practice the profession.
It's basically a formality now to get a prescription for what you read on WebMD. Every Insurance has telemedicine, you just call, read the symptom list and get the prescription. Some even let you just email. There's a doctor or at least person with subscribing ability in the loop, but they are barely doing more than rubber stamping hundreds of requests per day.
Idk man I was in the EU recently as a US citizen. One 15min call about my sore throat and dude gave me not just Tylenol and antibiotics (not too wild) but also oral steroids which was beyond overkill for what I had - very rare moment where I didn’t do what the doctor ordered. I was very surprised to see that prescribed. They didn’t even recommend nexium or anything to protect my stomach.
If you’re youngish and healthy a short course of an oral steroid is very low risk and doesn’t require a PPI. And it’s indicated for severe throat pain.
1) not that young and 2) definitely not “severe throat pain.” Drip + mild sore throat that went a few too many days with discolored phlegm. Clearly bacterial infection, clearly needed antibiotics, but definitely not severe
Discolored phlegm doesn’t mean bacterial infection. Viral infections, and allergic sinusitis can cause it too.
Bacterial sinus infections are absurdly over treated [1]. It can take weeks to recover from a viral infection.
Not saying that you are a wuss, but my guess is, the doctor probably over-indexed on the throat pain, assumed you were a bit of wuss for calling the doctor for a sore throat, and decided to give you something to make you feel better quickly.
Either way unless you were 65+ a short course of oral steroids is very safe (and even then it’s only very mildly dangerous).
“Without antibiotics, rhinosinusitis resolved in 46% of patients after one week and in 64% of patients after 14 days.
Antibiotics can shorten time to resolution but in only five to 11 more people per 100 compared with placebo or no treatment.
Despite this, approximately 86% of U.S. ambulatory visits for acute rhinosinusitis result in oral antibiotic prescriptions.1 In Europe, antibiotic prescription rates for acute rhinosinusitis in primary care range from 72% to 92%”
For all these completely anonymous, AI-generated investigative pieces that are hitting the front page of HN every week, I'd love to see the prompts. Because I suspect they say more about what the proprietor of the site is trying to achieve than the article itself.
I'm not trying to gatekeep or belittle the author, but there's something funny about this genre of articles that assert with utmost conviction what it takes to be successful in business. Of course, we all have opinions, but this piece doesn't say "here's what I think". It says, more or less, "here's how you succeed".
There are two things that are undoubtedly beneficial. The first one is a degree of unreasonable optimism or arrogance that pushes some people to try while everyone else is sitting in comfy armchairs, explaining why the idea is boring or can't work. The second is a financial safety net, so that you can try and try again.
As for everything else, it's reading tea leaves. There are folks who believe you need to be like Steve Jobs. There are people think you need to be like Elon Musk, or Bill Gates, or Dario Amodei, or Sergey Brin, or Warren Buffett. Good luck reconciling that.
> I'm not trying to gatekeep or belittle the author, but there's something funny about this genre of articles that assert with utmost conviction what it takes to be successful in business. Of course, we all have opinions, but this piece doesn't say "here's what I think". It says, more or less, "here's how you succeed".
A difficult lesson I’ve struggled to come to terms with is that it’s socially advantageous to appear confident that you’re right than it is to actually be right.
I am someone completely unable to fake appearances, and would never want to.
But I can vouch that being completely convinced of a solution path to an ambitious problem, is a shockingly helpful condition to get long term, long shot help, with no strings.
I have never been in a situation where I needed, wanted or felt I deserved help before. And I would have had no idea how to ask for it. But dire need and absolute belief (not faked) in what I am doing has resulted in so much help over so much time (far more than planned of course), I am humbled. And also very happy that it worked out.
In addition to what others said, home-etched PCBs are just much lower quality. We're no longer living in a world where every component has leads on a 2.54 mm grid, so tolerances matter a lot more. Also, SMD means you really need vias and a solder mask.
The problem isn't that people outsource it, it's that they need to outsource to China. That's an abject failure of domestic PCB manufacturers who simply don't want to deal with retail customers.
Exactly. Even for this small board, the IC has 0.65mm spacing of the LGA pads. It would be pretty difficult to assemble it without solder mask.
Almost everyone who has the experience of etching a board, or even using a router to automate making one, and also has the experience of ordering a PCB, never makes one by hand again.
Their main selling point is that "most projects die in the setup: a screen, buttons, power and sensors to wire up before you can even begin". With their product, you get plug-and-play blocks for everything. But that idea has been tried many times before. It's Aduino "shields", Beaglebone "capes", an entire ecosystem of Raspberry Pi accessories, etc.
Maybe it'll take off this time around, especially if they can make it cute enough, but at nearly $200 for the base device, I think they're gonna face an uphill battle. I still wish them well.
No, it starkly contrasts the RPi ecosystem because it has an "all in one" main unit, which works without shields, cables, extra cases, etc. It's approximately as self-contained as a mobile phone. But on top of that it offers GPIO pins.
The text is also pretty clearly AI-generated. I guess there's now a market for "I asked an LLM so that you don't have to", but the funny thing is that it's such a wall of text that no one upvoting it will have actually read it. So it's vibe-writing, vibe-coding, and vibe-reading. Full end-to-end synergy.
Hi, author here. Sorry, you felt that way. I did put lot of effort in communicating my understanding, and used Claude only for review and visualizations.
That's 100+ pages of in-depth technical writing in a matter of days. Amazing, really amazing. Also, not something that a human can do.
And FWIW, Pangram classifies vast swathes of "your" article as entirely AI-generated. Yes, I know the tool is not perfect, but between your superhuman productivity, and all the subjective tells, and the output from that tool, it goes onto my mental "AI slop" pile.
What I can't understand is why people can't just own it that they used a chatbot and that it was more than just "for review".
Short publication intervals. That's the heuristic you're going with? I have dozens of such unpublished long form articles on fundamental topics, just waiting for me to carve some time to polish. I expect that on a sabbatical I could finish and post them in quick succession. It's an unreliable indicator to determine AI-writing.
Author here, when I share something on my blog, i take full responsibility on the writing and the concepts being explained. This couldn't have been generated with just LLMs. If it was that easy, i wouldn't have bothered to work on it. I am sorry you felt that this was just slop, but i really thought people would enjoy reading it.
I have my rough drafts ready for all the topics which i want to explain. And I actually spent majority of my week working on the networking article, it's a lot.
How were you able to pick this up? Not challenging your assertion. Just really curious. Can you point to some clues? I read it (with my own eyes). I can't see actual evidence the text itself is artificial or at least, that it is not human-curated.
This is not a remark about AI, but there's something funny about mathematics in that every novel result is broadly perceived as a big deal.
We attach basically zero value to writing a new program that hasn't existed before, or a piece of text that hasn't existed before. It's boring, or even a net negative, unless you can show that the result benefits the world in some way. We'd find it weird if OpenAI put out a release saying that an LLM authored an interesting blog post.
For mathematics, I think it's really a matter of two things. First, the generation of proof was so severely resource-constrained on the human end that they could actually afford to celebrate every contribution - akin to how software engineering would look like if you had just 200 active SWEs in the entire world. But compounding that, mathematics is basically the only scientific discipline that rejected any notion of utility. It would be fundamentally wrong for you to ask what's the value of solving the Erdős–Hajnal conjecture; the value is that it's solved.
> rejected any notion of utility. It would be fundamentally wrong for you to ask what's the value of solving the Erdős–Hajnal conjecture; the value is that it's solved.
I disagree. Mathematicians care about the utility of a result. It is just that they regard mathematical understanding as a valid type of utility, and that can be arbitrarily far removed from practical utility. But a proof that doesn't help anyone understand anything interesting is not valued. I could go out and define some pointless construction and create proofs about it immediately. It would only matter if I connect it to some other subject of interest within math.
I would argue that mathematical understanding is valuable for extrinsic reasons, but it is true that by the time you're a math grad student, you're usually willing to pursue it for no external purpose.
It seems in mathematics that the utility of a problem is directly correlated with how difficult it is to solve, for some odd reason. If I defined some pointless construction and it turned out to be very difficult to prove, it would automatically over time become considered a "high utility" mathematics problem (again, for some odd reason).
Mathematics is largely just smart people working on pointless puzzles, and only by coincidence do these puzzles turn out to have practical applications (it cannot be predicted). Or I guess all the obviously practical problems in mathematics have already been solved -- we're now in a world where math is rarely the limiting factor for human progress (like it was, say, pre-calculus; was FFT the last significant unblock from math?).
It's such a waste of the best human minds. Or maybe the best human minds are actually doing something else, maybe we only notice the handful of Terence Taos, not the hundreds of people of equal brilliance who realized pure math is pointless and decided to pursue physics, rocketry, or quantitative finance.
> If I defined some pointless construction and it turned out to be very difficult to prove, it would absolutely and automatically over time be considered a "high utility" problem (again, for some odd reason).
Yes and no.
No: There are lots of very hard open problems which are judged to be of little value by mathematicians and hence garner little attention.
Yes: If a conjecture resists proof for a long time, this can indicate that we still have a substantial gap in our understanding. We project utility into an eventual closure of this gap, not into the statement of the concrete conjecture at hand. The gain in understanding is what we actually work for. It just turns out that chasing specific results, even if they are mostly dead ends on their own, is useful for orientation.
The (by now solved) problem by Fermat (for all integers a ≥ 1, b ≥ 1, c ≥ 1, n ≥ 3, the equation aⁿ + bⁿ = cⁿ does not hold) and the (still open) Collatz conjecture are perhaps good illustrations of this situation.
Your comment is somewhat emotionally charged, but I choose to respond to the overall point as a mathematician. I think it could be true that utility is correlated with difficulty but it is certainly not defined by it.
In pure mathematics, we reason about a world of abstract objects which are considered interesting ab initio. It may be because they arise directly or often from extremely basic operations, they are connected to many other interesting objects, or that they present special and surprising properties. The importance is basically, there is some surprising, interesting, phenomena which occurs in our world which we don't understand and which we seek to understand. Like science but in the non-physical world.
I think if you create a simple to describe system/construction with a property which is extremely difficult to prove. You are creating an object in our world which is basic but have properties which we don't understand (because we can't prove this property). So indeed I believe it would be an interesting thing to study and be of value. I don't see any problems/issues with this. I don't think the only valuable pursuit of humans is to improve the welfare of other humans. I think understanding the world is also valuable.
Most fields, in the aggregate, produce a lot of pointless work, but if you judge mathematics by its best examples, as judged by the field itself, and also by the outside intellectual community, it is a coherent body of work (& brilliantly creative). It is not pointless puzzles at all. William Thurston's geometrization theorem, Klein's erlangen program, Witten's work in physics-inspired mathematics, Langlands program, the Grothendieck school of Algebraic Geometry are deep and abiding intellectual achievements of true understanding. If you don't understand the meaning behind this work, it speaks to your ignorance, not to their significance. The "obvious practical problems" are not solved. Fluid Dynamics is wide open. Non-perturbative quantum theories are wide open. Heck, there are open mathematical problems in General Relativity. Dynamical systems are very poorly understood. Go read a book or something.
There was no useful math after 1964? What? Or do you not count the entire field of computer science as math (arguably FFT belongs to computer science too)?
>mathematics is basically the only scientific discipline that rejected any notion of utility
I think this might depend on the department, but I was at a pure math department last year, and struggling with my Linear Algebra textbook (written by the professor, incidentally, who was not a great communicator).
I consulted the machines, and learned, to my great delight, that linear algebra is used in like 20 different fields in the real world. It's "perhaps the most applied branch of mathematics in existence".
I complained in the group chat, that our didactic materials, specifically tasked with providing motivation and concrete examples, did not contain a single application, of this most richly applied field.
I was promptly pilloried, and shunned.
(Apparently that particular department was the wrong one, to ask a question like that!)
I’m a physicist, so I’m biased, but my experience of pure maths was about the same. We had to do it, but at no point was any utility actually demonstrated - that was left to the physics professors. It was all just “look at this thing I can do with these symbols” without any actual tangible relationship to anything.
Then again, I remember how we were taught calculus at high school - we were taught how to mechanistically integrate and derive everything under the sun. At no point did anyone think to explain that we were measuring the areas under curves, or their rates of change - it was all just “memorise this operation”. Again it was left to the physics teachers to explain why this was useful, and what we were actually doing.
Poor teaching, if you ask me, and it more often than not left me retrospectively wondering if said mathematicians had actually understood any of what they did, or if they just had little blind symbol manipulation Turing machines in their heads.
I remember squinting at the definitions in my calculus textbook for a full 10 minutes before realizing... oh my God, if this had been a 5 second animation, I would have understood it instantly.
I remember being rather upset! "If it's so obvious and intuitive, why didn't they just say so?"
I want to say, "well, unfortunately you cannot put an animation in a textbook." But that's actually false!
Even as a child I made flip book animations. And there was plenty of room here in the margins for several of those :)
Teaching only the practical side risks not teaching the subject with the appropriate theoretical depth and the ability to generalize it to other applications. Courses for purely applied fields utilize calculus to solve the current problem and then move on without teaching the finer points. Basing a calculus course on physics alone might be preferable in high school, but would be of disservice to students in university.
>At no point did anyone think to explain that we were measuring the areas under curves, or their rates of change
In my experience you get taught the definition of a derivative of a function at a point is equal to the instantaneous rate of change and that integrals are defined as a Reimann Sum, the sum of the area under the curve. Everything in the class comes from building on top of those definitions.
That you think this way (and if like me, it makes you excited!) I think it's because it has clicked for you.
For many that light bulb above their head doesn't flash on, hence they get to dislike the subject or forget it after they are done with their studies. I was lucky enough to appreciate math that much to redo it in my free time after high-school and make it click for me.
No, my calculus class in HS very literally started with finding the area under a curve “manually” and introduced integration as a generalization of that. I’m not surprised to hear that calculus is sometimes taught very poorly, but it’s not universal.
The title is a slight misnomer in my opinion. The author did make heavy use of spaced repetition of course, but deemed it neither necessary nor sufficient for the result he described.
The actual active ingredient is curiosity to the degree of obsession. Actually wanting to understand what's going on.
You can do that with pure math just as much as with applied.
But it may be harder for many people for such a desire to arise if there is no obvious connection to real-world matters. (I certainly had such thoughts, sometimes! What is this for? What's the point?)
> I complained in the group chat, that our didactic materials, specifically tasked with providing motivation and concrete examples, did not contain a single application, of this most richly applied field.
> I was promptly pilloried, and shunned.
Heh. In my day I may have participated in the pillorying.
I do think that there is value/merit in professors mentioning real world applications, where they exist.
What they're sensitive about are the theorems where there aren't real world applications. They don't want to (and shouldn't) justify them.
So even when there are real world applications, the posture is "Who knows if someone is making good use of this in the world somewhere? I don't care. It's not why we learn or teach this!"
Knowledge for its own sake is great, but it's worth noting that many "useless" fields of mathematics turned out to be very practical in the long run.
Number theory was long thought to have no practical application, but now it's the backbone of cryptography. Boolean algebra was developed in the 19th century (George Boole died in 1864), decades before it was used to build computers.
Those "useless" theorems being proved today may turn out to unlock a world-changing technology centuries from now. When the breakthrough comes we'll be grateful for the people who laid the foundations.
For a lot of math departments, that is exactly why they teach this. Education is rooted in application. We have entire careers that depend on certain aspects of mathematics, so most companies gatekeep that career by a degree. The degree requires the class. The student taking the class may not even be old enough to drink alcohol yet, and they can't possibly be expected to know of all the applications. Knowing and not telling them is doing them a disservice.
> For a lot of math departments, that is exactly why they teach this.
Depends on the course. That's why some departments have separate calculus courses for math majors - because otherwise the whole class will be full of non-math majors (engineers, etc) and focusing on their needs does a disservice to the students in their own department.
> The degree requires the class. The student taking the class may not even be old enough to drink alcohol yet, and they can't possibly be expected to know of all the applications.
If I'm a CS major, and the degree is requiring a class outside of the CS department, you shouldn't expect the professor of the class to know why the CS department is requiring it. It's on the CS department and its faculty to explain it.
The ultimate goal of university education is to raise researchers, who are the people that investigate the knowledge frontier of their field and then advancing it. To do that they have to understand a large part of the existing field so they can communicate with their peers, avoid investigating things that have already been throroughly explored, and draw useful connections to other fields.
Even in more applied fields it can take decades before advances become practically relevant. Restricting teaching to topics that have immediate practical relevance would therefore do students a huge disservice and prevent them from approaching the knowledge frontier of the field.
CS is special in that it is about the science, and not the application. They'll teach you mostly theory. That's why some universities/countries have a separate degree called "SW Engineering".
Other engineering departments are not as fussy about the distinction.
I think for many people (myself included) understanding mathematics is rooted in application because it helps bridge the divide between intuition and rote memorization. Without the application, IMO instructors are doing a disservice to their students and pedagogy of mathematics itself. They’re intentionally ignoring a significant fraction of the class, unless they’re teaching some esoteric grad level pure math.
Yeah this was my position. I don't mean "Sally has six apples" necessarily, but my brain needs some concrete handles to grab onto so it can actually manipulate this stuff intuitively.
See also: Using spaced repetition to see through a piece of mathematics
Where the author describes his mind eventually turning the mathematical objects (symbols) into some kind of mental objects he was able to actually manipulate "directly". As a result of this facility, his mind spontaneously produced many new insights into the matter.
That level of fluency obviously requires a great deal of work, but in my experience it can be approximated and accelerated with analogies. (e.g. see Feynman's teaching.)
I love teaching kids and young adults calculus by socratic method. They get so mad when they figure out you were teaching them math, but they often admit it was pretty fun. Only had the chance to teach like that a few times but it's dynamite when it happens.
I did this when I taught my third grader calculus on the train, when she asked a question about the train accelerating faster sometimes than other times. She loved it, but I was just taking advantage of children's natural curiosity.
Do you have some examples that the adult could instigate, rather than waiting for the child to express curiosity?
I've used filling a tank, balloon, or bucket (rate of flow, can be subdivided to teach limits, and use weird shapes for teaching area under curve and interpolation en route to integrals), or the classic throwing a ball back and forth and trying to describe the shape, the distance it flies, peak speed vs peak height, figuring out how hard you are "actually" throwing instantaneously. Honestly as soon as you start thinking about bulk substances moving around (gravel piles! fuel tanks!) it's easier to find examples than you'd ever reckon. Rate of change is everywhere.
Seems like I start by asking "how do we know how much this tank holds?", or "how fast does this line go up on the side of the tank?" and curiosity goes from there usually.
I thought linear algebra was pretty much the poster child of applied mathematics - the entire field was invented to represent computations in a regularized form to feed into computers. Well not really, but much like Boolean algebra or the Fourier Transform, it was pretty much a curiosity until computers came along.
Many important advances in linear algebra happened in WWI to solve optimization problems for logistical and industrial planning. A lot of these applications boil down to high-dimensional systems of linear equations, which back then were solved by hand. Efficient algorithms translate to reduced labor costs.
despite being theoretical i would have greatly benefitted in learning linear algebra if i had seen even one or two not-obvious applications, like galois fields for reid solomon erasure coding.
Even when this really isn't the case, the math is preferably reduced to a linear problem whenever possible since dealing with the original complexity is usually so much more painful.
As a friend of mine who also happens to be a math professor once said: mathematicians are like sculptors who marvel about the beauty of their creation, and are kind of disgusted when a physicist comes nearby and says “that's a cool hammer you got there, may I borrow it?”.
Typical pure-math linear algebra course has to cover so much material that there's really no time for applications! That's why applied math is typically separate track.
Biologists celebrate the discovery of new species of fruit fly hidden deep in the Amazon rainforest. Astronomers celebrate the discovery of new giant rocks located zillions of light years away. Neither of these things is immediately “useful” to the world, although they may turn out to be enormously beneficial in ways we can’t immediately predict. To me, these fields also feel central to the human experience—discovering new types of life, or learning more about our place in the universe. I don’t think a mathematical proof is any different.
> This is not a remark about AI, but there's something funny about mathematics in that every novel result is broadly perceived as a big deal.
This isn't true using the level of originality you're implying with your software examples.
Technically speaking, many novel mathematics proofs are written all the time (quite a few textbook exercises are actually technically novel problems that have never been posed before they were written in a textbook!) that get absolutely no fanfare. Overwhelmingly though they are not very original or difficult and really just required a fairly routine combination of different pre-existing techniques, even if technically speaking that combination didn't exist before. Those textbook problems are hence easy and therefore not given much public attention even if they are technically novel problems.
Indeed over the course of developing a new mathematical result, many many novel results are glossed over to the extent that even their proofs are left out ("as an exercise for the reader") because they are fairly trivial.
This is true for the overwhelming majority of new software as well. A new CRUD program may, technically speaking, be novel, but it's almost certainly just a routine combination of different pre-existing things.
Mathematics open problems that are actually named are generally problems that have resisted the low hanging fruit of the most obvious combinations of pre-existing problems. When those are solved they are a big deal precisely because they usually require some novelty!
Similarly in software, if someone were to create a new kind of database that solves a variety of new classes of problems that current databases fail to solve that would be a big deal! Truly novel software is also perceived as a big deal. Software that is, technically speaking new, but doesn't actually stray far from a fairly obvious remix of pre-existing techniques, isn't really celebrated.
In both software and mathematics, the intuitive benchmark is if other practitioners in the field look at the result and would say "Wow! How did you do that?" Professional software developers generally don't look at, e.g. a new blogging platform, and boggle at "Wow! How did they make that?!!"
I'm not a mathematician, but I don't think that's true..? It's just that some problems are considered "hard" or known to have been "open" for a long time or that involve some clever/pioneering new technique. There's tons of math papers out there that are in some technical sense a novel contribution but in practice just languish without much attention except maybe from like two other people working in the same subfield.
This feels mistaken; we develop abstract objects i.e. graphs based on real-world utility or whatever. As we try to improve our understanding of graphs, we value proofs that help us do so, or help other fields of mathematics. We assign 0 value to random proofs about stuff no one cares about... This conjecture had value, simply because some people found it interesting. It is not really different from music, in a sense.
> It would be fundamentally wrong for you to ask what's the value of solving the Erdős–Hajnal conjecture; the value is that it's solved.
No, the value is that Erdos's name is attached to it.
Lots of mathematicians prove things they don't publish, or their manuscripts get rejected - not because of a flaw in the proof but because no one cares about the theorem they proved.
And I'm sure it'll be the case with LLM models performing proofs. It'll be notable only when the theorem is a known one that people have had difficulty proving.
It’s far from a perfect analogy but I would imagine that people were pretty hyped about the novelty of the first legitimately useful compiled programs where they didn’t have to allocate their own registers. I wonder how long it took for that novelty to wear off?
Or in other words I’d argue novelty is contextual and that these kinds of discoveries’ novelty will eventually wear off too but for right now it’s pretty cool that the “math discovery compiler” works well enough to do this (again imperfect analogy).
Math is something humans invented and is a model, nothing else. There is no logic per se, but a model that works quite well for us.
I studied Math and CS as a very highly gifted and quickly found out, there is no beauty of Mathematical Logic, only humans approval of what they deem most accurate.
A good example is set theory. Cantor was not openly welcomed after he introduced his "theory" to others. In fact, he was received quite some pushback and hostility - this doesn't sound like someone received love the mathematical logic's way.
In fact, the story of Cantor is really a tragic one. He left math for quite some time, due to the pushback.
Only later humans accepted his theory and found it useful. Well, well, what is Mathematical Logic and what not is after all just broad consensus by humans.
And if you go deeper, you will hear more of these stories. Math is anything else but logic. Proofs are religious things, often so complicated, they are simply accepted as "approved by a committee". Many profs cannot really explain simple proofs, they refer to the textbook.
This doesn't sound like romance nor easily reproducible logic.
Consensus may give a hint to what is or isn't reality. But consensus—even expert consensus—does not determine reality. Experts can be wrong. Most of the experts, even, can be wrong simultaneously.
Philosophy is the exercise of testing ideas for oneself in the laboratory of one's own mind.
When I test the idea that math is discovered in my own mind, from my own perspective, with my own experience and education brought to bear, I find it unconvincing.
When you test the same idea in the laboratory of your mind, with your experience and your education applied, and get a different result, that is interesting. Your result is relevant information to me. If nothing else, it's a good prompt/trigger for me to revisit my earlier conclusion and see if it still holds.
But your disagreement—or indeed, the disagreement of a majority of trained mathematicians—does not constitute an automatic reason for me to conclusively determine that you/they are right and I am wrong.
I still have my own examination of the concept, with my own supporting and detracting arguments. And the result of my examination continues to be that math being invented is the significantly more persuasive view.
> We attach basically zero value to writing a new program
What does it mean "new"? And, was it a difficult or trivial accomplishment?
A solution to a well known open math problem is both new and non-trivial- you know that many, very smart, very well trained human experts have dedicated time to the problem and haven't been able to solve it, despite good incentives.
The reason novelty matters for mathematics is that they strictly deduplicate all claims. If someone claim they proved something that we already knew was solved, than that wouldn't be considered novelty. Novelty and deduplication is the combo here. This is not true for blog posts.
> there's something funny about mathematics in that every novel result is broadly perceived as a big deal.
Is this true? Or is it just that mathematics is an isolated enough field that only the results that are a big deal get broadcast widely to the public.
I know little of the inner workings of the field of mathematics, but my naive assumption would be that there's probably lots of novel but boring results being discovered/proven all the time and we don't hear about them because no-one outside of the person doing the work and a handful of their colleagues is really that interested in it. Likely a lot aren't published in any way, because they're just stepping stones towards the goal of the actual area/paper/whatever being worked on.
There is a tension between applied and theoretical mathematics, and it's as old as the whole science itself. Mathematics arose to solve practical problems (land surveying and division, as well as trade) and recognizing the underlying principles make it possible to abstract that knowledge. That might lead to centuries of ivory tower activity and what could be regarded as a purely artistic pursuit until somebody figures out how to apply a theory to a new practical problem. Or relations to another theory are discovered, and suddenly there is a new approach to previously intractable questions. A good example is be number theory, which is the foundation of modern cryptography.
The difference is discovering or proving a universal truth that will go into the corpus of human knowledge forever versus some app to shuttle money around or help people count how long they’re sleeping. It has gravitas unlike some nifty super performant text editor.
> We attach basically zero value to writing a new program that hasn't existed before
We don't? People write new programs that go on to be successful software companies that make millions of dollars! Basic CRUD apps make money for their creators in their niche! There's so much money in software that it's taking over the world. The market is different, you're not getting worldwide household recognition for every little fart or sneeze of programming you output, but how can you say that we attach zero value to new programs when the history of computers is insanely valuable companies making new software and selling it. Windows, Oracle, mongoDB, etc.
> It would be fundamentally wrong for you to ask what's the value of solving the Erdős–Hajnal conjecture..
I'm not sure about this, TBH I ask myself this quite frequently. In a world where machines are routinely solving very high end math problems every day, producing more proofs than humans would ever really be able to absorb or fully understand.... would that be a good thing? Would that in itself be valueable? It feels like that is a probable future, but I'm not sure that would actually be something we want. I think there's probably more than "value is that it's solved"
Isn't it immediately obvious that solving something that humans have been unable to do for decades or more is the most tangible proof of ASI, or at the very least pretty good AGI?
There’s only so many people with the necessary skills to solve this. And you need these humans to choose to spend their time solving this, and not something else.
>There’s only so many people with the necessary skills to solve this. And you need these humans to choose to spend their time solving this, and not something else.
Sure, but that doesn't mean a lot of very skilled people hadn't attempted and failed to solve this.
A lot of mathematics often takes 100+ years to find a practical use because we have developed it so much that we have use all the easy maths. Things like CS or SWE are so new that you can still find stuff today that can be used tomorrow. Things like computation and cryptography was all discovered like 100 years before we had a practical use for it. Its an example of late stage scientific discipline. Things like physics, chemistry and biology will get here as well eventually.
It's newsworthy because it's a milestone. It was something no human was able to do (despite trying very hard), but a machine did. Humans have written lots of interesting blog posts.
The idea that mathematics has rejected any notion of utility is absurd. It's not like topics get picked at random. Conjectures like this are interesting because they are a test of our understanding. The problem sounds easy, but apparently was quite hard.
there is no "software" that a lot of people want, yet nobody managed to create yet because they failed too due to it was being hard to implement (excluding AGI/ASI which is not really software)
As a person who has a number of relatively niche hobbies, I assure you that this is not true. There's a ton of simple things that can be build and will make an immediate difference in the lives of thousands. Watch the workflow any musician, videographer, machinist, etc - they're full of small, weird inefficiencies that AI hasn't really solved for them.
It's just that you can't build a billion-dollar company around it. No one could go to a VC and say "we're going to be the Uber of focus stacking and dust removal for microscopy" or "we're the Uber of aligning the beats in two audio tracks".
> there is no "software" that a lot of people want, yet nobody managed to create yet because they failed too due to it was being hard to implement (excluding AGI/ASI which is not really software)
What!? I can think of about a billion examples... but for one, I'm still waiting for a good enough CFD/FEM coupled system to model paraglider dynamics across collapse/recovery. And I expect to be waiting quite a while.
5 minutes of wikipedia search would give you plenty examples of complicated software engineering problems that would have a big impact on everyone's life.
In math, the utility lies in the proof itself. A novel proof of a hard problem usually comes with new insights and abstractions that help solve even more mathematical problems.
To go with your analogy, mathematicians care more about the source code of the program than about the result of the program. But I'm afraid that we will see things change with the increase of vibecoded proof slop. A black box proof is not as useful, even if it is correct.
Mathematics is what everything else is built upon. I'm no mathematician but a very good friend of mine is: teacher at a big uni, researcher. Pure math.
His entire life he's had --and still has-- to deal with comments like the one you just made, implying that the only value is solving pointless conjecture (if it wasn't pointless, according to your logic, then the value wouldn't be that it is solved).
"every novel result is broadly perceived as a big deal" is not at all true. AI companies hype any novel result as proof that AI is good for mathematics, but professional mathematicians write tens of thousands of papers every year, and for 99.99% of them, nobody cares or writes it up. Mathematicians certainly don't go around saying each and every novel proof in their papers are a big deal. Do you have any evidence supporting your statement that it is "broadly perceived" (by whom?) as a big deal?
Sorry but I completely disagree with your statement that "every novel result is broadly perceived as a big deal". Most results certainly are not consider this way (even though the average result has difficulties that are much higher than novel computer program you may have in mind -- no offense)
I mean, OpenAI delayed the public release of GPT-2 back in 2019 because it seemed capable of authoring interesting blog posts (that also happened to be untrue). It was a pretty big deal the first time Transformer models were capable of generating that kind of output--no one found it weird. We've just grown to take it for granted that large Transformer models are this capable.
The same cycle is happening now for a harder frontier. And proofs represent a pretty good benchmark for model capabilities, so a new model proving a result that a previous model didn't is generally notable in the same way that a model scoring higher on a benchmark is.
I'm sure we'll take it for granted in the not-too-distant future.
We generally do give a lot of credit to programs that do something novel. The first gets a lot of credit. But if its just another CRUD app, nobody cares.
Its the same with proofs. First time someone proves something gets a lot of credit. The second proof for the same theorem gets a lot less buzz.
But even then, math proofs mostly get buzz when its something famous or at least important. Proving a random lemma usually doesn't get much buzz.
So I suppose the value is that something like this gets used as a primitive to solve something that actually has impact. Ah, mathematics, never change!
I don't like this "just" framing. It's like saying that crushing orphans was always a thing and Microsoft Orphan Crusher 2000 is just making it easier.
It's a qualitative change. It used to cost time and effort to produce slop, which gave genuine content a fighting chance. Now, the cost is basically zero, so there's no limit to how much trash you can produce.
But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you. On the flip side, if you use an LLM do most of your thinking, what's left? We have people here who use LLMs to raise their children, to manage relationships, to design products. So what's your unique contribution to this world - is it the prompt you once wrote? You're standing in front of a token-generating machine, pulling a lever, sometimes receiving gifts. Is that your edge, your unique experience, your purpose in life?
Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?
In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share. I want to support that. And I want to be a human who can write novels, the old-fashioned way. I'm not good at lifting weights or running, so my thinking is the only thing I have.