OpenAI’s Codex Can Now Learn From Watching You Work

OpenAI added Record and Replay to its Codex Mac app, letting users demonstrate a task once and turn it into a reusable AI skill.

By Daniel Mercer Edited by Maria Konash Published:

OpenAI has added a feature called Record and Replay to its Codex app for macOS, letting users automate a task by demonstrating it once rather than describing it in a prompt.

The user performs a workflow while Codex watches, capturing the actions and on-screen content. Codex then drafts a reusable skill, stored as an editable file, that lays out when to use the workflow, which inputs change, the steps to follow and how to confirm success.

The feature shipped on June 18 in Codex app version 26.616. To run it again, the user starts a new thread, asks Codex to use the skill and supplies the values that differ, such as a file to upload or a date range.

The approach is OpenAI’s take on a long-studied idea, programming by demonstration, now powered by a reasoning model. Unlike traditional automation tools such as UiPath, which replay fixed pixel coordinates and break when an interface changes, Codex generates a natural-language description of intent that it can adapt when file names, dates or layouts shift. On replay it can mix Computer Use, browser actions and installed plugins to finish the job.

OpenAI suggests it for repetitive, preference-heavy tasks that are easier to show than to write out, like filing an expense, creating a correctly configured ticket, publishing a video or pulling a recurring report.

The feature signals Codex’s move beyond coding into general office automation. Originally a software engineering agent, Codex has expanded into broad computer use, and OpenAI increasingly pitches it as a digital coworker. Record and Replay requires Computer Use to be enabled and is available to paid ChatGPT tiers, including Plus at $20 a month and Pro at $200 a month, along with Business, Enterprise and Edu. The Codex app itself is free to download on Mac. The skills it produces build on an open agent skills standard, the same SKILL.md format used to package reusable workflows.

Why It’s a Shift

Record and Replay lowers a real barrier to AI automation: the need to describe a process precisely. By learning from a demonstration, it can capture the tacit knowledge in everyday operational tasks that rarely get automated because scripting them is not worth the effort.

That pushes Codex squarely into the territory of robotic process automation, a market long held by enterprise tools built around screen recording. The difference OpenAI is betting on is reasoning. A recorded skill is meant to flex with changing inputs rather than fail the moment a button moves, though whether that proves more reliable in practice remains to be seen.

Limits and Open Questions

The rollout comes with clear constraints. It is macOS only and, at launch, unavailable in the European Economic Area, the United Kingdom and Switzerland, leaving global teams unable to deploy it uniformly. OpenAI has not explained the exclusions, though the EU AI Act’s transparency rules for agentic systems take effect on August 2.

The company’s own advice to avoid secrets and sensitive data hints at a genuine risk that a recording could capture credentials or personal information. Costs can also stack up, since a single replay may combine model calls, Computer Use, browser actions and plugins. For teams, the open question is whether replayed skills hold up as the apps they touch keep changing.

AI & Machine Learning, News
Exit mobile version