AI: OpenAI levels up AI Coding. RTZ #610

...the focus for Developers intensifies

I’ve long underlined the point that this AI Tech Wave in these early days is all about the attention and coding habits of software Developers of all stripes. That even though LLM AI can seemingly ‘do the job’ of developers, there’s opportunity for AI to augment developers of all types to be well, better developers.

OpenAI is laser focused on this opportunity, as the Information outlines in “OpenAI Targets AGI with System That Thinks Like a Pro Engineer”:

“OpenAI’s artificial intelligence for writing software code has fueled a multibillion-dollar revenue business, with human programmers using the company’s ChatGPT as a coding helper.”

“Now the company is working on AI to help senior software engineers handle more complex programming tasks, a key step in the company’s attempt to develop artificial general intelligence that outperforms people at most economically valuable work, according to three people who spoke to OpenAI leaders about the product.”

“The effort is part of CEO Sam Altman’s goal to automate software engineering and release a powerful new product that handles coding problems involving multiple steps, aiming to replicate the work of an experienced programmer, according to a person who heard him speak about the project. Such a product is commonly known as an agent, a type of system Altman recently said could “join the workforce” as soon as this year.”

This is a coding agent above coding agents thus far:

“An advanced coding agent, combined with AI that can solve math problems and run experiments, could one day automate the work of AI researchers and engineers at a large scale. The agent could boost an internal tool OpenAI has already developed to help its AI researchers work faster, including by generating code for their experiments related to AI models, according to an employee. The product is popular internally, this person said.”

“Supercharging AI research with a coding agent could lead to the type of “superintelligent tools” Altman has said “could massively accelerate scientific discovery and innovation well beyond what we are capable of doing on our own.”

OpenAI apparently has had this percolating for awhile, and now closer to release:

“OpenAI has been preparing to test an early version of the advanced coding agent with a select number of potential customers as it tries to make the product useful and desirable, one of these people said. This process is known as finding product-market fit.”

“This type of technology could also reshape the enterprise software industry if customers can use it to replicate workplace apps for a fraction of the cost.”

This product becomes an important new arrow in OpenAI’s quiver of AI products:

“OpenAI will also need new products to help it reach a goal of generating $100 billion in revenue in 2029, up from around $4 billion in 2024. Later this week, the company is releasing a new product, Operator, that will automate complex tasks that people typically do in a Web browser.”

And it’s a field wide and deep with competitors:

“OpenAI faces increased competition in AI-powered coding—namely from Anthropic. OpenAI last fall doubled down on improving its AI for coding after Anthropic’s rival software exceeded OpenAI’s by some measures. A slew of smaller startups, including Cognition and Anysphere, are also developing advanced coding assistants that aid engineers in writing and debugging code, help engineers understand how their company’s existing code works, or even build entire apps independently.”

“Developers of automated coding tools cater to different customers, from people with no coding abilities to those who have some experience to those who are highly experienced.”

The market has many layers:

“ChatGPT and other chatbots are especially useful for the first two groups, as they can produce code in many programming languages and for many types of applications, depending on what customers ask them to do. Improving the general coding abilities of the underlying AI models will help OpenAI reach its goal of more than tripling the number of people who use ChatGPT to 1 billion daily active users by the end of the year, up from 300 million weekly active users as of a few months ago. Altman has articulated that goal privately, said a person who heard him speak about it.”

“Other AI products like GitHub Copilot or Anysphere’s Cursor, which have been embraced by a wide array of junior and mid-level software engineers, suggest lines of code to programmers while they type.”

OpenAI is of course focused at the upper levels of software developers:

“The new coding agent OpenAI is developing, by contrast, is geared toward senior software engineers. It would likely connect to their code repositories so it could handle complex tasks like code refactoring, the process of simplifying code or making it more understandable to human programmers so they can modify it more easily or prevent the introduction of glitches into the code base. It could also help identify and reduce duplicate code across a code base.”

“Another possible task the product might handle is changing an application’s code to make it easier to handle how the app keeps track of and updates data or to add features. That could make it easier to switch to new, cheaper data storage systems without changing how the application works. The product might also be able to rewrite the code to integrate a new personalization feature that suggests products or content to users based on their behavior.”

“Level 6 Engineer”

“The goal of OpenAI’s advanced AI coding assistant, according to someone who spoke to senior OpenAI leaders about it, is to replicate a Level 6 engineer, sometimes known as an L6 or senior staff engineer, who can take broad guidance from managers and design new software apps or features, or develop the systems that power them.”

And the product of course augments the company’s AI reasoning and agentic offerings in the pipeline on the way to AGI:

“That changed after the release of o1 and the recent launch of ChatGPT Pro, a $200-a-month subscription version of OpenAI’s chatbot that allows its reasoning models to spend more computational resources working through harder problems, including those in software engineering, Pani said.”

“Now, he said, it’s more common for senior engineers to use such technology to get ideas for how to tackle complicated problems.”

“One challenge of automating the work of a senior engineer is that AI systems don’t have many examples of how such a person thinks and the numerous steps they take to solve complex problems. That’s fueled a boom in the hiring of software engineers as contractors to continually teach models how to do such work.”

And other big tech companies are also focused on this big opportunity for developers:

“In the last year, code generation has emerged as one of the most promising applications of large language models. In August, Amazon CEO Andy Jassy said the company’s coding assistant had saved “4,500 developer-years of work,” and in October, Google CEO Sundar Pichai said AI was already generating over a quarter of new code at the company.”

“These AI coding capabilities have sparked debate about the future of software engineering, especially junior roles that AI might handle. In October, Anthropic co-founder Daniela Amodei told The Information that, although AI coding can’t fully replace engineers yet and needs “some coaching,” the Claude chatbot has boosted developers’ productivity to the point where Anthropic might alter its hiring plans.”

A key variable for software coding agents is the data, as the Information highlights in a separate piece on the subject:

“OpenAI’s development of an “agent” to automate the work of a senior software engineer—which my colleagues scooped yesterday—will raise the stakes in an already competitive market for AI coding tools. But OpenAI still has to overcome a major data problem before such a product can become widely available.”

“While the internet is full of coding examples for training models, there is a dearth of information on how senior engineers turn a broad idea from managers—say, rewriting how an app collects user data—into a completed project, according to people we’re talking to.”

And that data will have to be constructed both with human and synthetic means.

This is one of several areas that need to be accelerated for accelerated progress in this area.

Then there is the broader debate on whether these software developer tool products are zero sum, or market expanding for developers. I’m in the latter camp. It’ll may take a bit longer to see how really capable these new AI software engineers turn out to be. But the race to build them has more than started. Stay tuned.

(NOTE: The discussions here are for information purposes only, and not meant as investment advice at any time. Thanks for joining us here)