Selling AGI like AG1: Will Consumers Push Back Against Proprietary Blends of Herbs and of Data?
The race to produce premiere AI products with high price tags might change the standards around data disclosure.
AG1 is a popular “greens supplement”. It’s also controversial for its price, proprietary blend, and frequency of appearance in sponsored segments in podcasts. Notably, founder and longevity experimentalist (with many thoughts on AI!) Bryan Johnson has taken aim at it. Various critiques against this kind of product highlight the issues with selling “upper tier” supplements that don’t tell you exactly what they’re made of. The proprietary nature of something like AG1 makes it fundamentally less useful for very serious athletes or longevity enthusiasts, who care about dosing and interactions. Instead, it targets a particular band of consumer who cares enough to spend a big chunk of change on supplements and might place stock in claims from influencers that “it makes me feel better”, but don’t care enough to want to titrate their specific doses.
Well, OpenAI (an organization that has written quite a bit about AGI, not to be confused with AG1) is now selling a “Pro” subscription product priced at $200 per month (a similar order of magnitude to AG1). Software-focused AI company Devin is offering a $500/mo team product. It wouldn’t be a surprise if Anthropic follows suit with a similar product; Google, Microsoft, and others may follow, and overall more companies will likely start offering AI services that use a “you must talk to a salesperson” model.
In both cases, consumer preferences may provide a correcting force against proprietary blends. In the case of Generative AI products, I think there’s a serious chance that consumer reaction to highly priced proprietary blend products may create demand for data transparency, which will in turn help to alleviate some of the other issues downstream of mysterious datasets (verifying consent, clarifying the role of copyright, working out new models for IP, etc.).
Interestingly, to get a bit more personal than a typical post, a unique perspective I bring to bear here is that due to a variety of characteristics, some perhaps shameful (time spent in California, stereotypical simultaneous interest in configuring both code editors and exercise programs, etc.), I am a member of the target audience for both $80/mo AG1 and $200/mo AGI. I’m seriously considering trying OpenAI’s product (and competing offerings) — I do see this as part of my job, as it informs the HCI-heavy parts of my research.
But — putting only my “target consumer” hat on and taking my “AI policy” hat off, there’s a serious problem with the marketing pitch I’m currently receiving. The higher the price rises, the more I care about the nitty gritty details of what I’m paying for. In the case of supplements, I’d like to know if I’m getting an efficacious dose, if there might be interactions with other products, and so on. In the case of AI (again, putting aside the societal issues downstream of mystery data), if I think I can get some value out of bumping my subscription up by $180, I want to be able to reason about why the new product might be better, beyond word of mouth and promises that there’s “more intelligence packed into these weights.”
I expect going forward that LLMs will play some role in both the production of research code and of teaching materials. So I need to make a very real judgment call about spending more on compute vs. paying more participants vs. buying other tools. This decision will affect both me and my lab (should I get my grad students more cloud credits or an OpenAI Pro subscription?). While I might try out various models myself and read benchmark results paper, I’d also want to know — how many “good research codebases” and “top tier pedagogical” materials were involved in the pre and post-training of the model I’m about to dump out my wallet for? If I know, for instance, that OpenAI donated their own internal codebase (or created some parallel code of equivalent quality), my confidence will go up. It’s not just about data inclusion — I might also want to know that the codebases I wrote as a first-year grad student are not leaving heavy footprints on the model weights.
Another factor here is that input transparency and output quality are two sides of the same coin. “Vibes based evals” work in part because users are intuiting some aspects of the training data. Or rather, because there will always be some binding between data choices and outputs, learning that the data is very high quality and hearing that the outputs are very high quality are two views into the same underlying set of facts. Of course, the outputs should dominate user decision making (I can have the best training data in the world but screw up my training process, in which case you shouldn’t use my AI product) but as evaluations become even more niche and harder to interpret, the data side could dominate consumer decision making (i.e., it might become more compelling marketing to tell me about the quality of inputs rather than explaining some inscrutably impressive output).
Going back to the topic of supplements, we might expect health literacy initiatives and science communication to reduce the relative market share of proprietary blend products (though a really good blend could still succeed off word of mouth). Similarly, I expect “AI literacy” efforts (allegedly, a goal of both academic research and the industry labs) to lower the market share of proprietary blend AI products.
The point of this post isn’t to rag on the pro subscription or AI companies — like I said, I very well might buy it, and I generally think very positively of the products put out by OpenAI, Anthropic, Google, etc.! Rather, I just wanted to share an extended argument that’s rather hopeful — I think this recent move will actually create pressure in the long term towards documented models. It might be the case that because of all the effort to avoid sharing pre-training details thus far, the first wave of transparency might focus more on “post-training” transparency (i.e., a company tells us about the contractor labor that went into the $200/mo product, but can’t yet provide descriptive stats on the pre-training data), but in the long-run I think picky high-end consumers will be a force towards transparency.
I’d love to see more discussion on this front — is the view presented here overly optimistic? Are we going to spend thousands of dollars per month for mystery powder, and are we going to happy about it?
Finally, as I’m writing (some) of this during coffee time at NeurIPS, I wanted to flag some relevant papers!
Consent in Crisis, https://arxiv.org/abs/2407.14933
A Systematic Review of NeurIPS Dataset Management Practices, https://arxiv.org/abs/2411.00266
LLM Dataset Inference: Did you train on my dataset?, https://arxiv.org/abs/2406.06443
Image from ChatGPT web interface: share link.