How to Compare Claude Code and Codex on a Motion Brief Fairly

Two attractive videos are not a fair comparison of two coding agents. One may have received better assets, a more detailed prompt, an extra repair round, or access to tools the other did not have. A side-by-side screenshot hides those differences.

Motion work makes this especially easy to get wrong. A model can produce an impressive first frame while missing timing requirements. Another can produce a plain but editable project that survives a local revision. Which one is better depends on the job you intended to delegate.

For a Claude Code vs. Codex comparison on motion graphics, start with the same acceptance task and assets, then inspect exact copy, timing, controls, and the repair effort needed for a usable result. The worked case follows a Codex output through playback, a seek-control failure, and a repair. Claude Code could not authenticate for the paired run; a model-to-model comparison remains open.

Choose a Task That Reveals the Capability You Need

“Make a beautiful product video” leaves too much room for interpretation. A better test contains a real communication requirement and a bounded technical deliverable. For example, create a six-second editable offer scene with exact pricing text, deterministic seeking, and a shared reading interval.

The scene can be visually simple while still testing important behavior. Exact copy checks factual preservation. A stable hold checks temporal reasoning. A seek control checks whether the animation is driven by time rather than a sequence that only works when played from the beginning. A local revision checks whether the source remains controllable.

Do not overload the first test with every possible feature. If you include narration, music, captions, chart data, several aspect ratios, and a complex interface, failures become difficult to attribute. Use a small representative task first, then add a second task for capabilities the first cannot reveal.

Decide whether you are comparing code generation or full agent workflows. A code-only test can hold tools constant by disabling them and asking for a self-contained document. A workflow test should allow each agent to inspect assets, run the project, and repair errors under comparable permissions. These are different experiments and should not be merged into one ranking.

For a team adopting a tool, the workflow test is often more relevant. The useful result is not merely source code; it is a correct, editable artifact produced through a practical process. But a smaller code-only probe can expose requirements and environment problems before you spend time on a larger comparison.

Freeze the Brief and the Assets

Use the same brief verbatim for both runs. Save it before invoking either agent. If you discover an ambiguity after seeing the first result, revise the brief for both and treat the new pair as another test rather than giving only one agent the clarification.

For the local probe accompanying this article, the brief requested a self-contained HTML document at 1280 by 720. The fictional scene was titled Still Studio and had to display the exact statements “US$29 per month” and “US$348 billed annually.” Both lines were to enter from zero to 0.8 seconds, remain stable through 4.8 seconds, and exit by six seconds.

The brief also requested visible Play/Pause and seek controls, an initial paused time of 2.4 seconds, system sans-serif typography, no remote assets, and a fictional-study label. These constraints were chosen to make the output inspectable and avoid differences caused by external images or fonts.

The text and billing relationship are important. A model that changes the annual amount has failed a factual requirement even if its typography is more attractive. A model that shows the condition only at the end has failed a timing requirement even if the correct words exist in the source.

If you use real product assets, provide identical copies and preserve their identities. Do not let one agent use a newer screenshot or a different logo. If an asset cannot be redistributed, describe the access requirement and keep the comparison within the authorized environment.

Record the Environment That Actually Ran

A tool’s name does not identify the complete system. Record the CLI version, model reported by the run, relevant settings, available tools, permissions, and whether the session was fresh or continued. A default model can change over time, so the reported model matters more than an assumption based on the product label.

Separate account access from model performance. A failed login, a rate limit, or a network interruption can affect the practical workflow, but it does not reveal the quality of a video the model never produced. Keep those failures in the record without converting them into a creative score.

The Codex run used ChatGPT authentication through the OpenAI provider. Its log identifies CLI version 0.163.0 and model gpt-6.1-sol; the output is a standalone HTML document.

Authentication and configuration need explicit attention because they can select different access paths. Consult the tool’s own documentation and preserve the route actually used; do not expose tokens or private account files in a benchmark package. Claude Code authentication documentation

Preserve Raw Outputs Before Repairing Them

Save the first response exactly as received, along with the brief and run status. If you need to remove Markdown fences to open an HTML document, record that transformation. Do not silently repair syntax and then call the result the original first draft.

Store failed requests separately from completed artifacts so the test record distinguishes setup failures from output defects.

For a code-only comparison, open the returned document without editing it. Check whether the required words appear, whether the scene loads without remote dependencies, and whether the controls are present. Then play the full sequence and seek directly to its entrance, hold, and exit.

The source also contains a range input and a time-driven render function. Follow-up behavioral testing found that those controls were present but not all functional. Actual playback advanced from the initial state past 3.1 seconds, reached six seconds, and stopped. Seeking exposed a separate defect: the displayed time stayed at the old value instead of moving to the requested one.

Evaluate Meaning Before Style

Start with nonnegotiable requirements. Are the exact facts present? Are they related correctly? Can the viewer read the complete statement during the required interval? Does the output use the supplied assets without inventing interface behavior?

Then inspect temporal behavior. Seek directly to the middle, the start, the entrance boundary, the exit boundary, and the end. A deterministic animation should produce the expected state regardless of the path taken to that time. If the state depends on having played earlier frames, the source may be unsuitable for reliable frame rendering.

Only after those checks should you compare visual quality. Consider hierarchy, spacing, typography, motion coherence, and how well the composition serves the message. Keep the judgment anchored to the brief. A deliberately restrained offer scene should not lose merely because another version adds more spectacle.

Use a small rubric with separately reported dimensions rather than one mysterious total. Factual accuracy, timing, editability, visual communication, and delivery reliability can disagree. A single score hides the tradeoff a team actually needs to understand.

What the Follow-up Test Actually Found

The seek failure was reproducible after playback reached six seconds. Keyboard Home did not move the output to zero. Sending input events for 0.8, 4.8, and 5.4 seconds also left the output at six seconds. This matters because an editor needs to inspect a particular state without replaying the entire sequence. A visible timeline is not enough.

The cause was the event order. The input handler called pause() before reading the requested slider value. In this implementation, pausing also rendered the old state back into the control. The subsequent read therefore retrieved the old value. The animation math could be correct while the control used to inspect it was broken.

The author repaired a separate copy by reading the requested value before pausing and applying it afterward; the original model output is preserved.

// Preserve the requested value before pause() renders the old state.
const requestedTime = Number(seek.value);
pause();
state.time = Math.max(0, Math.min(motion.duration, requestedTime));
render();

The repaired copy responded to keyboard Home and End with zero and six seconds. At 0.8 and 4.8 seconds, both pricing lines had full opacity and no translation. At 5.4 seconds, both had opacity 0.5 and an upward translation of 18 pixels.

Directly editing the contenteditable lines to US$39 per month and US$468 billed annually left both visible at 2.4 seconds. A receiving test should next save, reload, and render those changes.

Reading the displayed time is the quickest way to separate a playback defect from a seek defect. Here the original advances normally when played, then refuses a later request for a specific state. The repair changes the event-handler order, leaving the entrance and exit curves intact. In a paired agent test, this becomes a precise repair task: fix seeking, preserve the six-second duration and both pricing lines, then check Home, End, and one intermediate time.

Make the Same Local Revision

The first draft is only half the task. Give both successful runs the same revision: for example, change the monthly equivalent and annual charge, preserve the layout and timing, and keep the two statements visible together. This tests whether the agent can revise a bounded part without unnecessarily rewriting the rest.

Preserve the original source and compare the revision against it. Check what changed in text, geometry, timing, controls, and asset references. A visually similar output can conceal a broken seek control or a shifted ending. A large source diff can also be acceptable if the resulting behavior is correct, but it deserves inspection.

Set the allowed repair budget in advance. If one agent gets three increasingly specific prompts and the other gets one, report that difference instead of presenting the final pair as equal first attempts. Assistance is part of the cost of reaching a usable result.

For a full workflow test, let the agents run and inspect their own artifacts under comparable permissions. Record the tools they actually use and any human intervention required. An agent that notices and fixes a loading error is doing useful work that a screenshot-only comparison would miss.

Measure Cost Without Making Up Precision

Elapsed time is easy to capture and easy to misinterpret. The Codex request here took about 179 seconds, including startup and model-discovery timeouts reported in the log. That is not a clean measurement of generation speed. Reporting it as “Codex takes three minutes to animate” would overstate what was measured.

For a more useful comparison, separate setup, request time, repair time, render time, and human review time where the tooling permits. The total time to a usable artifact can matter more than response latency. A fast first draft that requires several careful repairs may cost more effort than a slower correct one.

Do not infer dollar cost from elapsed time or a token count without the actual billing context. Subscription usage, API billing, cached input, and model settings can differ. If you cannot obtain a trustworthy cost figure, report the observable usage information and leave cost unmeasured.

Include failures in the operational story, but classify them. Authentication setup is different from a model producing invalid code. A render dependency failure is different from an unreadable design. The distinction makes the comparison useful for choosing both a tool and the supporting workflow.

Repeat representative tasks before drawing a broad conclusion. A single small scene can reveal a failure mode. It cannot establish that one agent is generally better at product launches, technical explainers, charts, localization, and all future motion work.

Keep Human Assistance Visible

A fair comparison should record the work the operator performs between requests. Installing a missing package, locating an asset, correcting a filename, or explaining an error can materially improve the final result. If that help is invisible, the comparison attributes the human’s contribution to the agent.

Decide which assistance belongs to the test. In a realistic workflow, an operator may reasonably answer a product question or approve access to a file. In a controlled code-generation probe, any manual source repair changes the artifact being evaluated. Neither approach is inherently wrong; the problem is presenting one as the other.

Preserve a short intervention log. It can say that the operator supplied the same screenshot to both systems, clarified a billing phrase for both, or repaired a local server before opening either result. If only one system needed a repair, record that too. The extra work may be relevant to the team’s decision even when the final visuals look equally good.

Separate operator learning from agent performance across repeated runs. After testing the first system, the operator may understand the task better and provide more useful guidance to the second. Alternating run order or freezing all inputs before starting helps reduce that advantage. A small local study cannot eliminate every source of variation, but it can make them visible.

This transparency is especially valuable for motion work because visual polish invites quick judgment. A result that required careful manual timing adjustments should not be described as an untouched one-shot generation. Readers can still admire it while understanding how it was made.

Publish a Result Others Can Inspect

A credible comparison includes the brief, asset set or acquisition instructions, environment record, raw outputs, revision prompts, render settings, and observed checks. Readers should be able to understand where your conclusion comes from without trusting an attractive comparison image.

Keep the result at the level of the requirement: exact copy passed at the initial state, playback reached the end, and seeking failed before repair. A later paired run can use the same brief and checks to compare how the agents diagnose and revise the artifact.

Inspect the original output, independent repair, and behavior records.