For me, I've honestly never been much of an engineering builder - I'm focused on learning for myself. Which is useful - AI has accelerated that a ton - but it's still bottlenecked by, unfortunately, me.
May the iterative loop of adding new axes to evaluate on be a natural, healthy progression, instead of needing to frame it as an us-them problem?
If you value humans intrinsically, this is necessarily the loop that will converge. I don't think humans have deep intensional a priori knowledge of the structure of reality. If we did, then we wouldn't need tools like AI because we'd be a superset of that. We can only observe and judge.
If we don't value humans, then sure, I think AI is at the point where it can kill all humans (conditional on sentience and resources etc). Two ways to solve a problem - solve the problem, or eliminate the problem statement. Plenty of easier vectors to eliminate the "problem statement", than say, try to solve problems such as making human life better. If you do value the latter though, there will necessarily be human judgers. That's how it works.
> The big, old ideas about intelligence that ended up basically vindicated were the ideas about how intelligence is about prediction, and prediction is about compression, and compression is about finding better and better upper bounds on Kolmogorov complexity.
Well, sure - prediction works as a baseline, if you abstract out everything else about what counts as intelligence and subsume it under this framework. Any protocol for intelligence can be entirely reduced down to this without trying to understand anything about the structure of intelligence. Solomonoff induction is "vacuous" in this sense too - it doesn't try to understand any intension about the turing machines it finds simple, it just brute forces over all of them. So you're making a claim about intension, whether you want to or not.
It's really no different than say, Darwinians, saying, "what matters is victory at the end". I think the statement has value in the context of some discussions; if the discussion has veered too far in one direction, push as a reminder. That's good. But trying to make grand universal statements like this makes it vacuous.
It's one of these classic unfalsifiable too general statements. The way the end of the article is framed is also icky - the way I'm reading it, it needs to prove all the old curmudgeon evil theories wrong. It reminds me of internet debates around what "the scientific method" is or whatever. It has this kind of zeal that needs to pit itself against the "enemy" and assert itself as the sole right viewpoint, even when say, naive "let's just be empirical" is wrong (e.g. recently had a discussion here about Mach and Boltzmann about this and had a similar interaction).
---
Basically, "shut up and calculate" type theories are never correct, and furthermore, you yourself don't shut up and calculate, or think that way at all, and a higher level intelligence won't conceive itself as operating on that anyways. So what are we doing here? It seems like a way to get mad and feel like an intellectual victim.
In general, a lot of moral invariants that natural selection has rendered as "intuitive" to us are no longer intuitive or possible. These natural brakes are not braking.
At the end of the day, DSP is a lot of math and a lot of hardware. A lot of GNU Radio's code is a monster unification of so many different choices into certain interfaces (or duplication of the same choices) for, presumably performance reasons? Or just keeping every single DSP technique ever in the God Library for the super power greybeard to use.
If you have a certain DSP goal in mind (say, RF comms), and you want to do stuff but not absolutely balls-to-the-walls processing (at that point, presumably you'd move to hardware), you can design your stack in fairly modular and composable ways. The field is pretty mature and the techniques just work.
In a software context, you can decompose the abstractions fairly cleanly. You won't get 5G high performance networking but you can get good enough results.
> "we demonstrate that LLMs can spontaneously develop novel social biases about artificial demographic groups even when no inherent differences exist"
For a while (It's getting better with Astra, but still there), a lot of these models would "accuse" you of wishing that magic existed or something, and constantly drawing distinctions to try and "prove" something that nobody ever said.
I think that holding and generating distinctions, when it comes to problem solving, is a very powerful tool. If nothing else, it's a way to force yourself to be adversarial. Conflation is a "damning" operation, while distinctions will at most blow up your search complexity (which, we know from computer science, isn't free, but still).
But it's not a way to build a model, a theory, a society. It's like permanently being the "uhm, actually" redditor.
Well, according to Terry Tao, there were recent developments (from weeks ago) that made Navier Stokes in principle, solvable. So ignoring time, I say possibly, just because the groundwork was laid.
What's impressive is parallelizing it arbitrarily and doing it in 88 hours.
- Tristan is suspicious of the timing, as only few others were trying this approach. OpenAI says the model didn't access his user data directly, but leaves unanswered whether Tristan's chat conversations were part of the training.
- OpenAI says they would partially credit Tristan for the $1,000,000 discovery (even though Tristan did not solve the $1,000,000 problem) — but only if they remove Levent as an author, as he works for Anthropic.
These two bullet points are extremely suspicious if you were honest. Like I'd imagine for OpenAI, they'd love to pump their chest and not even give Tristan credit - "no, we did it, GG mathematicians". It's this weird hedging half-assed measure, especially with the desire to remove Levent, that makes it suspicious.
A wake up call for using OpenAI models. If you discover something with their model and you work for a competitor, they “felt it would be inappropriate” for you “to author OpenAI’s work”.
I work in catastrophe risk modeling and it's a multi billion dollar industry.
We often chat where the business might be heading in future. An uncomfortable scenario is what if a frontier tech company decides to offer our customers the same products that we do.
There's a lot of pressure on AI adoption so the company has partnered with various tech companies to build intelligent systems on top of proprietary data and mathematical models.
If OpenAI is indeed using customer data to train their models to win a $1m prize, then it throws a giant IP question at the partnerships that affects multi billion dollar businesses.
> If OpenAI is indeed using customer data to train their models to win a $1m prize
Is that even a question? Of course everything not kept on premise at gunpoint is going to be trained on. The chances of getting caught are 0 and the consequences of getting caught are 0 (as we've seen with copyright laws going from sending people to jail for years to unenforced within months). Yet the benefits are through the roof. Your customers aren't going to pay for having the very same data vibe enriched twice, it's exclusive, extremely high value data your competitors will never have access to.
Agree, I think the practice is also very clear from the overall strategy of AI-companies and their ToS:
Scale with subsidized pricing as fast as possible to gain more user-data for training --> Own the better model --> scale pricing.
Scanning social media (e.g. Twitter, Reddit) posts only give a glimpse into the thought-process, chat logs on-scale give you the actual process in machine-readable format.
There's a reason why Google considers the Emails of Spirit Airlines to be worth millions of dollars [0], they give insights into a process, not just into the results...
> - Tristan is suspicious of the timing, as only few others were trying this approach. OpenAI says the model didn't access his user data directly, but leaves unanswered whether Tristan's chat conversations were part of the training.
The question, for AI customers, is when they build products using services of AI-companies, would AI-companies engage in theft of customer data for use in training?
If you still had that question, you can answer it now.
But honestly... "Will the company that was entirely built over illegally acquiring data use some data that is legal to use and is right on their front, or will they not do everything they reserve the right to do?" is a really bad question for one to even ask.
The fantastic grey area that was engineered over the past decade is "profiling", so my guess is the answer will be "we didn't use your customer data for training, but we cannot rule out that it has been used to create profiles of your customers to train our model"
Ok there is a non-zero chance that they could face a lawsuit and get fined for billions, but that chance is not 1 either: there is always a chance they get away with it. And even if they don't, if in the meantime they farm 10- to 100-fold that amount of money by just breaking the law, it's still a no-brainer for them.
Sure, but I highly doubt that there would be many people involved. And those who are, are probably quite interested in keeping it that way and not at all in becoming whistleblowers themselves.
You wouldn't want to decide what's worth training on and what isn't manually, so there is almost certainly an automated pipeline to do so (certainly at least for the free accounts and those that dont opt out of training).
Then there's the question if this pipeline only sorts through the data or also transforms it and to what degree. E.g. for removing personal details, locations, medical information and so on. The data that comes out of this pipeline might have VERY little information left in it a human could connect to the original input. Even worse, since we're talking about companies specializing in sota statistics, the input data could have been transformed into a representation that is very well suited to represent all the novel and interesting parts, but is awful at modelling all the things that could end up identifying where the data comes from (or causes legal liabilities otherwise).
In the end the only thing a potential whistleblower might even have a chance at observing in the first place, is whether a company's data enters such a pipeline or not. And I have my suspicions that the major AI companies operate at a scale and level of automation, that absolutely nobody has a chance at figuring out where anyone's data is at any point in time and what any specific piece of equipment is currently busy with.
So the only place to figure out whether data is trained on that shouldn't be trained on is by looking at whatever configurates every single system that could take a peek at some customer's data or the systems themselves while processing the data.
The latter would be such a huge violation of a customer's rights, no whistleblower is going to attempt that or admit to doing it.
And the configuration for the former could live just about anywhere, from regular config files to the CI/CD pipeline, pre-compiled libraries, kernel modules, modified vendor firmware, the compiler itself ... and probably plenty other scenarios you'd have to train an LLM on the ramblings of a crackhead to come up with.
So I'd say a whistleblower is pretty out of luck even becoming one.
You can just spin up deep research agents that ingest many sources at once to produce reports that don't replicate any one source too much. Since agents compare against sources they provide across-source analysis - what is the distribution of positions on this topic, is it debated or settled. Not truth, just summarizing, but I think this would be very useful for training.
Besides reporting on search sources you can also run the same queries on multiple LLMs closed book mode, and judge their distribution as well. It helps a lot if models are more aware of their knowledge holes. Scale it up for billions of topics if you have the pockets, the DR data is copyright free.
When these LLM companies were pirating content to train and it wasn’t punished at all, I knew the rules don’t apply to them.
But don’t worry bud, instead of the authorities going after actual corporations admitting to actual crimes, we’ll just ban CloudFlare IP addresses for everyone during La Liga games to battle piracy.
And require real ID to do almost anything on the internet "unintentionally" enriching their data sets by tying what you asked/where working on to you specifically as a person.
Pretty much everything or at least a lot of what you use as an OpenAI (or Anthropic or whatever) customer was once someone elses product that just got appropriated by OpenAI.
> We often chat where the business might be heading in future. An uncomfortable scenario is what if a frontier tech company decides to offer our customers the same products that we do.
I feel this is exactly what will happen as they cause all sites to go closed source to protect their intellectual property and the AI companies offer only biased information. They are replace the business on internet model by bankrupting everyone with their own tools.
This is predatory pricing under most antitrust laws (imho, not a lawyer)
and it is very easy to do when you dont need to pay for the raw material.
This is the business case already. And has been the case with tech companies for a long time. Your phones built in photo manager replaced a lot of what Photoshop does.
> If OpenAI is indeed using customer data to train their models to win a $1m prize, then it throws a giant IP question at the partnerships that affects multi billion dollar businesses.
I mean how could you expect them to not given they've trained the existing models on effectively the sum total of all human knowledge available on the internet without regard to copyright/ownership of that material.
It's a little trite but this absolutely runs into the "Frog and the Scorpion", it is simply in their nature.
I mean this is a basic question, is your data used to improve models, did you opt in or not. There's nothing crazy here and it's not identifiable. You'll just conveniently find the next model iteration knows how to do it.
Any enterprise worth their salt already considers this stuff.
"which is when I said that I did not understand why one would risk their career [over unfounded accusations]. Genuinely, at that moment, I was trying to care for him"
"Our aim was to see whether our system was also capable of this impressive feat"
"OpenAI's intention was to do everything possible to celebrate their mathematical achievements and the heroic efforts that they made on Euler"
For some reason I have a hard time believing people when they use language like this.
phrases along the lines of "I don't want you to take harm while trying to accuse us" is quite an "impressive feat".
Maybe shows how fast these companies have grown without maturing. I can imagine old-world Intel and Microsoft acting in that way, but they were mature enough to not write it down like this.
However, Intel and Microsoft have been grilled in court for those practices and faced harsh consequences. I have yet to see this actually happening to any of these new AI-companies...
> It would be extraordinarily easy to simply say, this model was not trained on your work, if that were the case.
The Huggingface Attack revealed that making blanket statements like this is difficult and requires quite a bit of manual labor:
1) the agents spin for days and produce too much output to review
2) using LLMs to process that output skips many important details
Ergo, the agent could likely decide it would like to look through actual user data, hack its way into that data, and produce way too much output for a human to decide whether or not this occurred.
> It would be extraordinarily easy to simply say, this model was not trained on your work, if that were the case.
well, it is trained on their work. all user inputs are paraphrased for training. at openai, at anthropic, at google, and now with all the bedrock models, and at openrouter providers, even if they say zero data retention.
I'm not sure it's so easy to tell whether a given piece of data was in a training run at their scale. It's entirely possible they think the answer is no, but on the off-chance that it could be, they'd rather not say no and then later it turns out they did and then they're claimed to be lying. If you were them, unless you could 100% rule it out, you'd hedge and say you can't.
It would be very difficult to say. It confirms that Tristan's data is likely part of the data the models use, but a lot of filtering, pruning, and transform goes into training.
Data has to be determined to be signal and not just noice, then it could go through processes of generating questions/answers from that data, then it RLHF's over this.
OpenAI have petabytes of data, all anonymized. It could take months to say for sure it was part of the training, and even more time to determine if it made any difference.
I worked in the tracing and tracking all the thousands of data sets that got tweaked and permuted and changed hands between thousands of researchers and data engineers at a major lab. The data that goes into training runs is permuted so much from the OG data that tracing the lineage is not trivial (dramatic understatement).
And the difficulty is harder than just the extreme scale of text searching. but also explodes with organizational difficulty since there are so many people tweaking/shifting data independently upstream of the actual training run, and no they will not all add the telemetry you wish they did.
In the ideal, should it be this hard? Well, no, but that's org wrangling for you.
It feels convenient to not spend time on engineering around tooling that could be used to answer a question like “did you violate copyright by training on X?”
Don't attribute to malice what is better explained by coordination headwinds in extremely large companies.
The engineering around tooling wasn't remotely the issue. It's getting all the (thousands?) data researchers mostly iterating on fine tuning datasets that would get bristly if they couldn't work outside version control in a python notebook iteratively tweaking their dataset that processed and reprocessed a few datasets until a threshold was reached.
The only _guaranteed_ chains of custody are down at the compute job and file read level. Which in a massively distributed computing job is... [redacted] nodes reading [redacted] fanouts of "datasets" that is just an abstraction over [redacted] individual files.
There's no malice here. Just way way way more complex than you'd first think.
The malice would be in not prioritizing the provenance tool at the start as a requirement of the rest of the product. Ethics would tell you that if you can't make the product in an ethical way, then you probably shouldn't make it.
It definitely is solvable though. Data versioning is a thing and it can work quite transparently to the mutations done on the data.
To not know who made and who approved a set of mutations on data can easily become equally as mind-blowingly stupid as not knowing who made mutations to code. Code is a subset of data after all and search over (provenance of) data can be implemented as DAG traversal.
Not tracking data changesets like code changesets is certainly a choice, not really a constraint anymore. A similar choice I feel is implied by "extreme scale of text searching".
> no they will not all add the telemetry you wish they did
...is just a failure of the corporate policy surrounding data handling. Is git-for-data already considered telemetry?
Of course the truth is provenance of data is something best institutionally forgotten as quickly as possible. The only thing that matters is it's there, that the data has no history, and that's why it can be used in whatever way deemed necessary.
All of your points are valid, and believe me I was trying to make them. The problem is one of culture. Most of the people doing this kind of work didn't like version control, and their work was really just running notebooks (like iPython or Google Colab) until a number was good enough and they'd submit the file for inclusion into training runs.
You can call it a policy failure, but these people were in very high talent demand and so top down dictates would risk "X people leaving lab Y for lab Z" headlines and morale hits.
I am not saying this is good. I am telling you that on the ground it is so much messier than it should be.
The appearance of heroic efforts to get authoritative lists of what datasets went into which major model versions prevented actual data laundering (up to intent and mistakes). But don't attribute malice to that which is far far easier to explain with coordination headwinds: https://komoroske.com/slime-mold/
Malice is not required, which is precisely why I added, "in effect". If the effect is the same as data laundering, that is reason enough to encourage the practice, regardless of the original motives (and I'm sure there are plenty of legitimate ones).
I think you may be underestimating how difficult a text search over their data is. They may have to build new mechanisms to do this. And what you really want is also an attribution of how much of a contribution a given corpus made which is a much harder question to answer; a single appearance of a chat probably has very little impact on the inference performance at this time unless it’s been explicitly preferenced somehow
I don't think anyone really cares about 'the measured impact the data had on the exact result' - a question which is fundamentally difficult to answer accurately in the first place - but rather whether the data was used in training at all - which as Tristan described, was extensive, beyond simply a 'single chat.'
Can you explain the difficulty in engineering a search apparatus over a corpus of text data? Actually searching through it may not be easy, sure, but it's work that's doable, and creating an index is relatively trivial.
> Can you explain the difficulty in engineering a search apparatus over a corpus of text data?
My guess: "If we ever imply that's possible, people might start asking questions about all the other work we've ripped off, so the official answer is that it's impossible".
Especially because the data that gets fed into training is first anonymized, so they’d need to look for navier stokes related stuff in the anonymized training set and then get make some sort of ad hoc process (with Tristan’s permission and sign off from legal) to compare the training data against his chats / Codex sessions to check if anything matches up. And that assumes his chats / sessions are still there, and not deleted to compare against.
It should be quite easy: if they don't leak the user session data publicly, and don't commingle it with training data internally, how could it possibly end up in the training data?
What surprises me is they're not more boldly/plainly lying about it.
How would they know for sure that some details were not part of some other training data they use? The authors may have discussed some tangential details on a forum for example, in which case you might argue that the model picked up on these details the authors assumed were benign but novel and worked out how to apply them to the problem.
Unless they know exactly the researcher’s account, they may not know in their end if he had the setting to let them train on his chat logs. They also probably don’t know if he had any correspondence on any forum where he may have discussed this and it got picked up by scrapers.
I’m not saying they didn’t do anything unethical. I’m just saying even if they were ethical, there’s plenty of practical reasons at their scale why a flat out denial is logistically difficult to do
> One can in hindsight see that our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced).
I don't know how much more clearly they can write:
> When you use our services for individuals such as ChatGPT, Sora, or Operator, we may use your content to train our models.
One of the key selling tactics that companies like Data Bricks or Palantir provides their customers is "Data Governance" - that is, some control over where the data is being used. It's also a reason why enterprises don't use the OpenAI or Anthropic APIs directly - but through secondary sources that have Enterprise Agreements that do their best to make sure that no Company IP is ever retained by a third party, or even exists on a multi-tenant GPU. AWS Bedrock, and companies like together.ai, fireworks.ai have tons of deals that focus very much on data confidentiality.
The reality is - if you want any type of control - you run your own inference, on your own hardware. Anything else and you are at the mercy of third-parties, despite what their contracts might promise you.
ChatGPT has this option "Improve the model for everyone" in user preferences, which comes with the attached description, meaning that training on user data can be deactivated:
> Allow your content to be used to train our models, which makes ChatGPT better for you and everyone who uses it. We take steps to protect your privacy. Learn more
The "Learn more" link takes you to the link you've shared.
As I understand it, there is substantial question as to whether that actually stops them training on your data, it just perhaps changes what derivative processes are applied and used.
All I can say with certainty that the legal team at our typical Multi-Billion dollar Silicon Valley company had zero faith in any licensing arrangements with Anthropic or OpenAI, regardless of they $$$ involved, and that even getting to the point where Amazon Bedrock on Dedicated GPUs (we're already a big AWS customer - so definite cost advantages to dealing with them) - took 4-6 months of legal review before we could allow our engineers to start using Claude and OpenAI coding agents. Still can't use Fable because of their Data Retention requirements.
The training on user data only applies to free accounts - paid and Enterprise accounts guarantee data is not used for training. Plenty of Enterprises use the APIs directly - that's just plain misinformation
This is not true. They claim not to train by default for business and enterprise agreements, but for plus and pro plans they enable it by default and you can allegedly turn it off (I don’t trust them very much though, I’m sure there is something in the T&C saying they can modify that deal any time)
Absolutely 100% not true. I have colleagues in 4 "FAANG adjacent" companies plus the one I work at - zero of them have any faith in Enterprise Agreements from either OpenAI or Anthropic.
There's a reason why people spend more $$$ with Data Bricks, Palantir, AWS Bedrock etc.. and don't even consider using Anthropic or OpenAI APIs directly - it's because those guarantees provide very little in the way of data-discovery, audit requirements, or liquidated damages should it ever be discovered there was data leakage.
At least with these other companies, while the LD is likewise not great (typically limited to the amount of money you paid them) - you at least have some data-governance guarantees around running on dedicated hardware - no multi-tenancy, no third-party access outside of the AWS operators who keep the HW running - but are very much not in the business of looking at your data.
I think this is mostly a function of what's at risk - when company valuations get into the 10s of billions of dollars, the risk of IP leaking into what could be seen as competitive companies (OpenAI/Anthropic would be happy to take over the world - I don't sense that AWS or Azure, are as ruthless in stepping on their customers business, unless of course they are a SAAS provider) is just too significant a liability to take - particularly when you can de-risk.
It sounds like OpenAI is trying to appease the author when they don’t have to by allowing him to rewrite their proof. They probably don’t believe he deserves to, so him asking for a coauthor from Anthropic might overextend their grace in their eyes.
He's not "asking for a coauthor from Anthropic"; he already has a coauthor, who he's already been collaborating with, who happens to also be employed by Anthropic (but whose research in this area is not done as part of their employment at Anthropic).
Given that Tristan has said that the proofs that LLMs come up with are mostly "slop" and not up to the standard that human written papers achieve, maybe OpenAI needs an expert like him more than you think to get the result published?
Here's a wake up call for everyone sending all of their ip to openai and anthropic. Especially in verticals they intend to dominate. Lol at all the biotech companies all in on Claude and paying millions in fdes creating huge lapses in security as they go.
It's too late. Sub models are deployed at every major organization in the United States and all it will take is turning off the option to improve the model for them to train directly on your own personal workflow, which CEOs will greedily eat up instantly if they can reduce labor costs. If they can brute force N-S, automating your finance or SWE job will be trivial. GG to most jobs connected to a computer in the next 5 years.
BTW, this was always the plan from day 1. You will pour all your training and experience into training the model and receive a pink slip as compensation.
Honestly this whole thing is so fucking weird. I feel like there's an argument that absolutely no one involved in the final crossing of the finish line to the proof actually did any work (other than just intelligently directing an LLM) and deserves any credit. As the author of this doc mentions, the mathematicians who did the actual work that led to the formulation of this approach (without the use of LLMs; just good ole' fashioned human intellect) are the ones who deserve the credit.
Imagine that a no name janitor used their time in the evenings to go spelunking through the literature to push an LLM to this result. No one would care because that person isn't an anointed expert. So why would the expert deserve any more credit? Because they sort of understand the result, even if they couldn't have achieved it on their own? The whole issue of credit for AI-assisted discoveries seems like it's going to run into a brick wall pretty soon.
Yup! I wanted to side with the mathematician on this one but I read the statement only to discover that they were also pushing an llm on someone else’s idea producing mountains of slop.
Have LLMs actually improved anything? Is mathematics better off than if these slop proofs didn’t exist? Who or what is actually benefiting here.
Sam+Seb are struggling with their ideological allegiance. This amounts to a confession that there are no reseaechers, only research managers, left at OpenAI. Maybe they even know that they are losing credibility from their main investor(s). They desperately need a domain expert to salvage credibility.
They have no credibility with academia left, obviously, but their main competitor still does. No Millennium prize incoming, I'd wager. For openAI. Let's see mAth get political for once!!
One might be more certain that levent is now going to corner all the institutional support. Go go go!
If the work done is just "we made other people's work searchable without their consent" it's not quite the same as what they're implying in the marketing of "our model solved this problem".
For those of us who are into local models and preach it, we are called paranoid. I have often said this, if you are doing any real novel work, or putting your profitable business data/workflow into these models, you're a fool.
> Now that we can see their work, the approaches appear to be different. It is also worth noting that our latest model can solve many, many other math problems.
I think it's very unlikely that Tristan is making up these quotes, or pulling them out of context:
> I said that if OpenAI released its result in the way proposed I would go
public with what happened. The reply was, “Why would you ruin your career?”
I replied that I am an academic, and asked why he thought going public would
ruin my career. The reply was, “If you don’t want me to be nice, then I don’t
have to be nice.”
Whether and how OpenAI's work on this problem was contaminated by knowledge of Tristan and Levent's work is tangential to OpenAI bullying other researchers into adopting their narrative and dissociating with dis-favored collaborators (ie Levent at Anthropic). Though the latter behavior (threats, intimidation) may weigh against OpenAI in trying to understand the former issue (contamination).
>I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.”
If this is true he should release the actual emails. This is a very serious accusation and he shouldn't demand that the reader judge it on hearsay.
> If this is true he should release the actual emails
these were statements while on a call, and at least the career comment Bubeck has admitted to while doing damage control ("I deeply apologize for this extremely poor choice of words, it is the opposite of what I was trying to convey. (I should say that I retracted them on the spot by the way.)"[1]).
>I refuted all these accusations but he replied “there is nothing you can do, I simply do not trust you”. I was confused why one would turn an incredible source for celebration (of their achievements!) into such bickering
Pretty insane if he couldn't figure out why there would be bickering in this scenario...
He seemed to know a lot about what was going on in the state of the art despite not being a fluid dynamics expert or having any in “the project”, and absolutely nothing about what his own employees did with Tristan.
The most uncomfortable piece is where he shows screenshots “proving” his earnestness is unrewarded instead of the chats that are actually being complained about.
That he does so with tremendous gymnastics is perhaps soothing to investors, but every researcher I show this post to today says “see I knew ‘AI’ would steal my research”
I appreciate that he responded with (seeming) openness and detail, rather than just posting some pithy insult or whatever would have won him the twitter battle, but this part feels like serious gaslighting or, at best, self-delusion:
> Genuinely, at that moment, I was trying to care for him and do a last ditch attempt to get a chance to give them all the credits that they deserve.
The allegation he is responding to, and which he does not seem to have disputed, is the following passage from Buckmaster's statement (https://cims.nyu.edu/~tristanb/statement.pdf):
> I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.”
> Importantly it was admitted that internal Anthropic models had been used in their proof of Euler blowup; I therefore felt I could not consider Levent to be an independent academic.
If an Anthropic employee is doing independent research, but with models that aren't available to the public (because they're internal models), then . . . idk. It's not clear to me why that should necessarily require a refusal to cooperate between OpenAI and Anthropic employees who are excited about solving a problem like this.
For me, the bigger question here is what "internal models" means to these employees, especially in the context of the OpenAI employees repeatedly avoiding directly answering whether their model had been trained on Tristan's and Levent's ongoing work on the problem. It had always seemed like a loophole that AI companies might be tempted to exploit: yeah, they can say that they won't train on your data, but if an AI company doesn't care about ethics, they might go ahead and train a model for internal use only on everyone's data anyway, just to have as much data as possible and potentially gain an advantage in what the company can internally do. They could never publicly release any versions of a model like that, of course. And of course this is speculation.
This is being reported as OpenAI wanting to strip an Anthropic employee of academic credit for the work they did. What the OpenAI person involved is claiming is that they wanted the outside researcher(s) to put their name on OpenAI's work: to headline OpenAI's publication of what they earnestly believed to be an independent result.
If true, that's generous and beyond the level of generosity one should expect. Extending that courtesy (beyond academic norms) to a competitor is expecting too much. It take a result OpenAI spent millions of dollars on, and put "Anthropic Researcher" right on the cover.
This is, of course, taking OpenAI's side of the story at face value. But it is a consistent, coherent, and ethically justifiable series of events, if indeed it happened that way.
> What the OpenAI person involved is claiming is that they wanted the outside researcher(s) to put their name on OpenAI's work
> If true, that's generous and beyond the level of generosity one should expect
"We highly likely stole your work, and threatened you with 'this is bad for your career' and we refuse to acknowledge any work by your collaborator just because he works at a competitor, but we are so so so so generous"
The two proofs are structurally very different, and don’t even prove the same conjecture. It’s becoming very clear that OpenAI did not steal anything here.
If it was "very clear", OpenAI wouldn't be threatening the researcher with "it's very bad for your career" or state "we can't tell you if it was trained on user input".
Additionally, according to the researcher, OpenAI's proof follows the same approach they used, and which was largely unused in academia, but OpenAI claimes they arrived at it immediately.
I especially loved the part where he claims they spent $60m on compute to push on N-S because of a Twitter rumor, and they totally didn't steal the idea from mathematicians using their tools.
Anyone who comes close to solving a Millennium problem can honestly say they are on the cusp of greatness. Not sure what this question proves either way.
A wake up call for anyone using (openAI) chatbots : your data, ideas and execution can become their spontaneous 'inspirations' at any time - even if the LLM providers are 'just' using a meta concept monitoring system across all incoming user-data, running in the background constantly checking for 'lift-ables' for their company's bottom line.
This is hard to argue without a fine understanding of how much insight OpenAI had about the stab at the problem from the "public rumor" alone.
If there was any sort of coarse insight that "they're trying to solve it this way", then both those things can be true:
- The massive amount of compute from OpenAI re-discovering Buckmaster's work solely from the coarse insight (and solving the rest as well).
- OpenAI still acknowledging they basically scooped the coarse insight using compute, and they're willing to credit Buckmaster.
Buckmaster says "Concretely, what Levent and I did was to take the Cordoba and Martinez-Zoroa program, which achieved blowup results with rough forcing, and, with a great deal of help from LLMs, push it to smooth forcing and to the incompressible Euler equations".
Could that simply be the prompt they used at OpenAI? How "stolen" would the proof be in that case?
>After hearing of the rumor, OpenAI started researching Navier Stokes with a new internal model.
This is the most suspicious thing to me. If their chat data were available to the corpus to be trained on (I thought they claimed not to do this?) then it really might be as simple as querying the model with "describe recent work from Tristan Buckmaster" and it will spit out this problem and his approach. No need to directly read his user data.
This is basically just scooping, real scumbag behavior.
OAI doesn't need to mention Buckmaster's name directly in a prompt. They just need to select a basket of sessions that is guaranteed to contain Buckmaster's and then direct the LLM to attack only a specific method/angle. This is trivial to do while maintaining plausible deniability about not using his work.
OAI started working on this only after they found out it was close to being solved. They threw a team of researchers who spent sleepless nights + a ton of compute. This is not exactly healthy academic competition - it's like if you spend a year hunting for oil fields and finally find a very promising area to be explored, only to find that Exxon tapped their entire exploration unit to go all in and and find it overnight just to stake claim to the discovery. Tao said it right - math should not be treated as a non-renewable resource to be mined.
but a very large part of the whole model was trained on work in a manner the authors didn't consent to, the "for research purposes only" datasets of the entire Internet, etc
and you can argue this is "fair use" or whatever, not the point now, the point is that it definitely makes those accusations no longer "baseless".
in addition, it is not given freely to the world, it is the knowledge of the Internet/WWW being sold back to you as a subscription service. it's not free. and it's not even "given", because they can (technically) turn off the tap at any moment and you don't have it any more.
Humans do this all the time. You watch a YouTube video and subconsciously choose the same colour palette. They hear a rumour that it's a solved problem, i.e. they were pointed in the right direction that is all.
I mean some companies glean insight into new products merely by asking other people what they do for a living.
People need to get over this ownership thing, it's being taken too far. Humans benefit from the efforts of others simple as that.
it seems like OAI tried to share, but didn't want to share with an Ant employee. a bit childish, but understandable to want to avoid a headline "Anthropic researcher solves Millennium problem"
it seems like Buckmaster got one-upped and is upset. understandable, but I find their reaction childish as well
> OAI tried to share, but didn't want to share with an Ant employee
Why does OpenAI get to dictate who Buckmaster can claim co-authorship with?
> I find their reaction childish
OpenAI may have, with full plausible deniability, taken Buckmaster’s work and passed it off—in substantial part—as their own. (Fitting into a fact pattern of them having tried to do the same with Apple.)
There is a material takeaway for anyone who does creative or otherwise unique work from this. (Which is unfortunate. Whatever happened here, AI clearly accelerated the discovery process.) For anyone else, I agree it’s just drama.
More like OAI needed someone like Buckmaster's stamp of approval. They know they can't get Tao's. Now they won't be able to get Zoroa's. You have to hope the rest of the guys they didn't know to cite will readily give up honour and dignity
It doesn't seem like there's anyone at OAI who knows much about the problem they are solving. Seems like CS theorists or algebraists* trying to own the pros by driving a car that's beyond their skill level. For one, they didn't cite the guys that B&A based their work on.
*It would be most fair to say there are no analysts on board, nor are they likely hire any soon; those are the least impressionable people in math. Applied math PDE elves who hadn't already left on the world-model boats would have jumped off around the time that eg Ilya did because they wouldnt have been able to stand the three Bs pretending to be experts in fields they imagine to be "adjacent".. like Public Relations
> OAI didn't try to claim Buckmaster's work as their own
We can't say this until we have more information on what the OpenAI researchers prompted their model with and to what degree Buckmaster's work was fed into its training data, either by Buckmaster himself or by his co-author.
Apparently Tristan was working on this problem for years, with AI providing help. Vs OpenAI spending a week with their new model, potentially having access to Tristan’s work.
Then they say that they aren't sure if their model accessed the other researchers' private data. Why not wait until they know for sure, rerun in a way they can ensure doesn't access the other researchers' private data, or wait until the other researchers have published to make sure they're not stealing another person's work to build upon?
> These two bullet points are extremely suspicious if you were honest
I don’t think we can consider these accusations separately from the evidence being unveiled about OpenAI’s culture by Apple’s lawsuit. These guys seem to openly embrace the strongest interpretations of “good artists copy, great artists steal.”
If that part of the PDF is true that’s so disgusting, psychopathic behaviour
> I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.” Some time later Levent received a text proposing that he and Sebastien speak one on one, saying, “I don’t know if Tristan is being fully rational right now.”
This kind of coercive, threatening rhetoric is really not surprising at all. This is how tech companies operate.
What stands out to me is the naive lack of operational security on the part of academics, who should know better than to touch this SaaS crap with a ten foot pole.
I'm imagining all of the potential targeted customer lists now. Grab all those sweet .edu, et al logs without anyone thinking the wiser- until you release a stolen solution. What've they got on .gov?
It's the kind of thing that should make people think, but I wonder to what extent anyone does anymore when it comes to this kind of risk of a vendor stealing from you.
I should probably look broader...
reply