Scripted tests can’t keep up with AI-written code. Datadog is betting agents can.

Nearly a year into the agentic coding explosion, many teams now use AI to write software as their de facto approach. As a result, Anthropic’s engineers are shipping eight times as much code as before, and some engineers are shipping thousands of PRs per month.

Management teams expecting their engineering capacity to explode will find that AI-written code doesn’t necessarily compress the software development lifecycle (SDLC) on its own.

The critical bottleneck in software creation has shifted from writing code to a later SDLC stage. After coding agents spit out new code, it still needs testing, and traditionally that meant humans writing the tests. That work includes deciding which user flows need testing, scripting tests for them, and rejiggering them when those flows change.

We’ve made it easy to write code and, in the process, increased the burden on humans to test new code. If we want to slash software development timelines, something has to change.

The observability and monitoring giant Datadog bought Propolis earlier this year, adding agent-written, goal- and output-oriented test writing to its platform. The approach lets agents explore applications, follow user flows on their own, and generate tests from plain-language prompts.

Add Datadog’s observability heft, and you get a system that can test code while providing the logs and traces it needs to vet system performance without the fragility hardcoded testing brings.

REGISTER NOW FOR THIS WEBINAR

Which best describes where you are with AI-generated or automated testing? *

Please select We're actively evaluating tools and expect to make a decision in the next six months We already use AI to create or maintain tests and want to expand how we use it Test coverage is falling behind our release pace, but we haven't started looking at solutions I'm researching the topic for a future project at my organization I'm just getting started with this topic

Email *

First Name *

Last Name *

Job Role *

Please select AI Researcher / Research Scientist Architect Business Development/Marketing/Sales Community Manager / Developer Advocate Data Analyst / Business Intelligence Analyst Data Engineer Data Scientist Developer / Software Engineer DevOps Engineer Educator / Instructor Enthusiast/Hobbyist Founder / Entrepreneur / Investor / VC IT management, including CIO/CISO/CTO/CDO Machine Learning Engineer MLOps Engineer / Infrastructure Engineer Product Manager Security / Privacy Professional Statistician / Quantitative Analyst Student SysAdmin/Operations/SRE Technical Writer UX / UI Designer / Data Visualization Specialist Job Level *

Please select C-Level Founder/Owner VP/Director Manager/Supervisor Mid Level or Senior Individual Contributor Entry Level or Junior Individual Contributor Freelancer/Contractor Educator (Teacher, Instructor, Professor) Student/Intern Other N/A Industry *

Please select Advertising/Marketing Aerospace/Aviation Agriculture Automotive Biotech/Pharmaceutical Business Services (accounting, consulting, etc.) Computers/Information Technology Construction Education Facilities/Service Industry Finance/Financial Services (banking, insurance, etc.) Government Healthcare Human Resources Legal Life sciences (biotech, pharmaceuticals, etc.) Manufacturing Media Non-profit Real Estate Retail/Consumer Goods Telecommunications Transportation/Logistics Travel/Hospitality/Entertainment Utility/Energy Organization *

Country *

Please select United States Canada United Kingdom Afghanistan Albania Algeria Andorra Angola Antigua and Barbuda Argentina Armenia Australia Austria Azerbaijan Bahamas Bahrain Bangladesh Barbados Belarus Belgium Belize Benin Bhutan Bolivia Bosnia and Herzegovina Botswana Brazil Brunei Bulgaria Burkina Faso Burundi Cabo Verde Cambodia Cameroon Central African Republic Chad Chile China Colombia Comoros Congo Congo (Democratic Republic) Costa Rica Cote d’Ivoire Croatia Cuba Cyprus Czech Republic Denmark Djibouti Dominica Dominican Republic Ecuador Egypt El Salvador Equatorial Guinea Eritrea Estonia Eswatini Ethiopia Fiji Finland France Gabon Gambia Georgia Germany Ghana Greece Grenada Guatemala Guinea Guinea-Bissau Guyana Haiti Honduras Hungary Iceland India Indonesia Iran Iraq Ireland Israel Italy Jamaica Japan Jordan Kazakhstan Kenya Kiribati Korea (North) Korea (South) Kuwait Kyrgyzstan Laos Latvia Lebanon Lesotho Liberia Libya Liechtenstein Lithuania Luxembourg Madagascar Malawi Malaysia Maldives Mali Malta Marshall Islands Mauritania Mauritius Mexico Micronesia Moldova Monaco Mongolia Montenegro Morocco Mozambique Myanmar Namibia Nauru Nepal Netherlands New Zealand Nicaragua Niger Nigeria North Macedonia Norway Oman Pakistan Palau Palestine Panama Papua New Guinea Paraguay Peru Philippines Poland Portugal Qatar Romania Russia Rwanda Saint Kitts and Nevis Saint Lucia Saint Vincent and the Grenadines Samoa San Marino Sao Tome and Principe Saudi Arabia Senegal Serbia Seychelles Sierra Leone Singapore Slovakia Slovenia Solomon Islands Somalia South Africa South Sudan Spain Sri Lanka Sudan Suriname Sweden Switzerland Syria Taiwan Tajikistan Tanzania Thailand Timor-Leste Togo Tonga Trinidad and Tobago Tunisia Turkey Turkmenistan Tuvalu Uganda Ukraine United Arab Emirates Uruguay Uzbekistan Vanuatu Vatican City Venezuela Vietnam Yemen Zambia Zimbabwe State *

Phone Number

Register By registering, you consent to The New Stack’s Privacy Policy, Terms of Use and to receiving email communication from The New Stack and our event partner. You may opt out at any time.

You have successfully registered for the webinar.

The New Stack will sit down with Ruxanda Lueck and Matthew White of Datadog on Wednesday, November 4, at 9 a.m. Pacific / 12 p.m. Eastern.

Lueck is a group product manager working on goal-based testing of AI features. White co-founded and served as CTO of Propolis. These are precisely the people to help us understand the current state of the art and what comes next.

We’ll ask how much test-writing teams can offload today and whether agentic testing has reached its own Claude Code moment. We’ll also ask whether plain-language testing instructions expand the number of staffers who can help test new code, and how long it will be before agents can read agent-written code and handle testing without humans lifting a finger.

Agent-written code has created tensions across the SDLC that no single tool can settle on its own. If testing can speed up code writing, though, teams clear one more obstacle between an idea and working software in production.

You’ll learn:

How agents can test user flows that no one thought of

How to write plain-language testing prompts

How goal-based testing persists even when user flows change

How to test AI features that return probabilistic, instead of deterministic, outputs

Where scripted tests still have their place

The post Scripted tests can’t keep up with AI-written code. Datadog is betting agents can. appeared first on The New Stack.