AI: “Show me the Money”

...'Extractive Data' war intensifies

As Large Language Model AI gets even more voracious for Data to train on, the tug of war is intensifying for compensation to the creators of all the stuff being scraped off the internet. The industry calls it ‘Extractive Data’, appropriately enough, as I’ve outlined and illustrated before: 

This is an issue I’ve been writing about these accelerating tussles for a while now, and it is all increasingly coming to a head this year. This data is at the heart of the initial Foundation LLM AI model ‘Training’ data, and the subsequent critical “Reinforcement Learning ‘Inference’ loops” that make these models do their ‘emergent’ magic, and become far more capable over time. As the WSJ reports on the latest ‘Outcry’:

“A col­lec­tive cry is break­ing out as au­thors, artists and in­ter­net pub­lish­ers re­al­ize that the gen­er­ative AI phe­nomenon sweep­ing the globe is built partly on the back of their work.”

“The emerg­ing aware­ness has set up a war be­tween the forces be­hind the in­puts and the out­puts of these new ar­ti­fi­cial-in­tel­li­gence tools, over whether and how con­tent orig­i­na­tors should be com­pen­sated. The dis­putes threaten to throw sand into the gears of the AI boom just as it seems poised to rev­o­lu­tion­ize the global econ­omy.”

Goldman Sachs is estimating $7 trillion plus in additional GDP growth over the next decade, as I’ve outlined before. The Firm (my former professional alma mater) is also emphasizing the underlying growth of the online content ‘Creator’ Economy growing from $250 billion this year to almost $500 billion by 2027. 

The companies behind the Foundation LLM AI models know there are a combination of negotiations and lawsuits ahead, and are getting ready:

“Ar­ti­fi­cial-in­tel­li­gence com­pa­nies in­clud­ing Ope­nAI, its backer Mi­crosoft, and Google built gen­er­a­tive-AI sys­tems such as Chat­GPT by scrap­ing oceans of in­for­ma­tion from the in­ter­net and feed­ing it into train­ing al­go­rithms that teach the sys­tems to im­i­tate hu­man speech. The com­pa­nies gen­er­ally say their data use with­out com­pen­sa­tion is per­mit­ted, but they have left the door open to dis­cussing the is­sue with con­tent cre­ators.”

“Ope­nAI hasn’t re­vealed much about the data used to train its lat­est lan­guage model, GPT-4, cit­ing com­pet­i­tive con­cerns. Its prior re­search pa­pers show that ear­lier ver­sions of its GPT model were trained in part on Eng­lish-lan­guage Wiki­pedia pages and data scooped by a non­profit called Com­mon Crawl. It also trained its soft­ware us­ing an Ope­nAI-com­piled cor­pus of cer­tain Red­dit posts that re­ceived a user score, or “karma,” of at least three.”

The originators of the underlying content for these and other sites are also ramping up:

“Ear­lier in July, thou­sands of au­thors in­clud­ing Mar­garet At­wood and James Pat­ter­son signed an open let­ter de­mand­ing that top AI com­pa­nies ob­tain per­mis­sion and pay writ­ers for the use of their works to train gen­er­a­tive-AI mod­els. Co­me­dian Sarah Sil­ver­man and other au­thors also filed law­suits against Ope­nAI and Face­book-par­ent Meta Plat­forms for al­legedly train­ing their AI mod­els on il­le­gal copies of their books that were cap­tured and left on the in­ter­net.”

“News pub­lish­ers have called the un­li­censed use of their con­tent a copy­right vi­o­la­tion. Some—in­clud­ing Wall Street Jour­nal par­ent News Corp, Dot­dash Mered­ith owner IAC and pub­lish­ers of the New Yorker, Rolling Stone and Politico—have dis­cussed with tech com­pa­nies ex­plor­ing ways they might be paid for the use of their con­tent in AI train­ing, ac­cord­ing to peo­ple fa­mil­iar with the mat­ter.”

As are the biggest online sites that have long been the source material for the LLM AI models for years already:

“Red­dit, the so­cial-dis­cus­sion and news-ag­gre­ga­tion site, has be­gun charg­ing for some ac­cess to its con­tent. Elon Musk has blamed AI com­pa­nies scrap­ing “vast amounts of data” for a re­cent de­ci­sion by X, then called Twit­ter, to limit the num­ber of tweets some users could view. And strik­ing ac­tors and writ­ers have cited con­cerns that Hol­ly­wood stu­dios could use AI to copy their like­nesses or elim­i­nate their jobs.”

Some deals are already being cut:

“The As­so­ci­ated Press and Ope­nAI an­nounced a deal this month for the tech com­pany to li­cense sto­ries in the AP ar­chive.”

And a lot more will come behind these presumably. At the same time, the LLM AI Data scrapers are setting up their base positions and clarifying their starting positions:

“Ope­nAI and Google have both said they train their AI mod­els on “pub­licly avail­able” in­for­ma­tion, a phrase that ex­perts say en­com­passes a spec­trum of con­tent, in­clud­ing from pay­walled and pi­rated sites. Ope­nAI also said in a state­ment that it re­spects the rights of cre­ators and au­thors, and that many cre­ative pro­fes­sion­als use Chat­GPT.“

“Tech com­pa­nies have pointed to the le­gal doc­trine of fair use, which per­mits the use of copy­right ma­te­r­ial with­out per­mis­sion un­der some cir­cum­stances, in­clud­ing if the end prod­uct is suf­fi­ciently dif­fer­ent from the orig­i­nal work. AI pro­po­nents say free ac­cess to in­for­ma­tion is vi­tal for tech­nol­ogy that learns sim­i­larly to peo­ple and that has huge po­ten­tial up­sides for how we work and live.”

But the other side is setting up their base positions as well:

“Other pro­posed class-ac­tion law­suits were filed sep­a­rately against Ope­nAI, Mi­crosoft and Google on be­half of in­ter­net users al­leg­ing that the com­pa­nies’ scrap­ing of web­sites to train their AI mod­els vi­o­lated the users’ pri­vacy rights and copy­rights.”

As I said a few weeks ago in a post titled ‘AI: Data and Copyrights’, the next area of contention is likely going to be the vast array of video content in services like Google YouTube, Meta, and TikTok. And then some:

“Soon Google’s YouTube videos, and other video sources could be a new source of Data for Foundation LLM AI models to train on and provide exponentially better AI results to users worldwide.”

“And how the Data all the models are using right now are but ‘a tip of the tip of the iceberg’.”

“But there’s one obvious issue that companies large and small are going to have to navigate before it’s clear sailing in AI land. And that of course, is the issue of IP and copyrights.”

“Since all the Data items in Box 4 in the AI Tech Stack chart  are the raw source material to train hundreds and soon thousands of LLM AI models worldwide, and much of that data is material that has IP and copyrights under a different digital world, the question of “Fair Use” and potential copyrights of course comes up. Get ready for some epic tussles.”

The good news this time is that all the stakeholders have seen these Tech models evolve with the PC and the Internet waves before, and know that a lot of this can be negotiated and not just litigated. There is serious money here over time, likely enough for most of the parties.

But there are also a lot of uncertain technology development, and outcomes to be clarified ahead. Certainly a while before all the stakeholders are going to have a clearer view of what’s ahead in terms of the Money flows to be ultimately seen and shown over the next few years. Stay tuned.