Amazon Calls AI Token Counts Deeply Flawed, What Its Engineering Memo Means for How Engineers Are Judged

Amazon's engineering leaders say metrics such as token usage and adoption of AI tools are deeply flawed ways to gauge AI use. The guidance sits in an engineering leadership memo posted on About Amazon, which was circulated internally in September and posted to the company's website on Tuesday, October 27, so the memo itself dates from September. The leaders write that they believe it is best to measure, and be measured by, long-term outcomes. For Metaintro readers who write code for a living, or want to, this guide covers what the memo asks, how AI use will be weighed, what changed after Amazon's leaderboard episode, and what Amazon's own interview pages ask candidates to prepare.

What Does Amazon's Engineering Memo Actually Say?

The memo goes with a revised role guideline for software development engineers (SDEs) and their managers. It was introduced on About Amazon by Terk Terkowitz, who is listed as VP/Distinguished Engineer, Worldwide Amazon Stores, and the guideline was developed after engineering leadership, including Distinguished and Senior Principal Engineers, from across Amazon recently came together to reaffirm what it means to be an SDE at Amazon.

Amazon's most prolific engineers have demonstrated that neither the developer role nor their output is defined by the use of AI, and Terkowitz's introduction to the memo describes the current period as the era of the Renaissance Developer, blending AI tools with human judgment and domain expertise to build entirely new things.

Amazon has historically tried to avoid having the SDE role guidelines be explicit about technology, and the authors say any definition we write now will likely look antiquated by this time next year. Instead, the guideline focuses on what is not changing, namely impact on our customers, our businesses, and our teams.

That does not make AI optional. The memo is direct on this point, if you haven't already, you should take the time to learn and apply AI into your daily work as soon as possible, and it adds that we all need to embrace this change and take the time to learn, or we risk being left behind.

Why Does Amazon Call Token Counts and Adoption Rates Deeply Flawed?

Tokenmaxxing is the practice in which developers use AI as much as possible with little regard for output. Some developers give useless tasks to AI agents just to inflate their token consumption and look productive. Tokens are the units of data that are processed and generated by AI models.

Real outcomes can be time delayed, difficult to measure, and hard to isolate from other trends, so in some cases Amazon used input metrics instead, like token usage or adoption of AI tools, or throughput metrics like code changes and reviews. The leaders then write that we are all aware of how deeply flawed these metrics are, primarily because they are disconnected from what matters.

Used as targets, these metrics could raise costs, since tokenmaxxing is at odds with frugality, and add complexity, since measuring LOC is at odds with invent and simplify. Amazon's published Frugality principle reads Accomplish more with less, and it adds that there are no extra points for growing headcount, budget size, or fixed expense.

We know that using these metrics could, if used as a target instead of a measure, drive negative outcomes, Amazon's engineering leaders wrote to SDEs and SDMs, according to the About Amazon memo post.

Will Amazon Engineers Still Be Judged on Their AI Use?

Partly. Amazon stopped short of telling staffers to ignore those metrics altogether. The memo's own line is that using these input metrics, with care and judgement, as you think about your career and your organization's performance is appropriate.

The leaders also write that these input metrics correlate well with the outputs that matter and expect they will continue to do so for some time. Teams that have embraced AI the most are called frontier teams, and the memo warns that teams and individuals who are not adopting AI or increasing output metrics are falling behind.

Leaders are asked to watch both ends of the curve. The memo recommends paying special attention to the 10th percentile (finding those teams who aren't embracing AI effectively and understanding why) and the 90th percentile (finding those teams who are heavily using AI and understanding whether they're seeing delivery increases to match) of AI adoption. In other words, low use triggers a question, and so does very high use that does not show up in delivery.

Outside the memo, AWS vice president and distinguished engineer Marc Brooker said Amazon has looked at metrics such as feature velocity to measure how AI is improving engineer productivity, counting how many launches are we doing, how many successful changes are we making in production. Amazon hasn't instituted caps on employees' token usage, an Amazon spokesperson said.

What Does a Finding, Not a Failing Mean for Engineers?

The memo speaks directly to engineers whose work AI does not help. It reads, If your work does not benefit from current AI tooling, that is a finding, not a failing. The memo then tells engineers to document why, share it with your community of peers, bring it to your manager, and work to find solutions.

It also reframes skipping AI as a skill when it is explained well. The leaders write that an engineer who can explain where AI does not yet help their work is demonstrating the judgement we are asking for.

What Does the Memo Ask Amazon's Technical Leaders to Do With AI?

The memo sets specific expectations for technical and organizational leaders. Part of a leader's job is deeply understanding these technologies yourself, working with them hands-on rather than relying on what you hear.

It sets a plain test for what counts as understanding. Sufficient understanding means you have used these tools on your production work, can describe a task where they helped and a task where they did not, and can explain that judgement to your team. The memo is plain with leaders, we expect you to be taking ownership of connecting the inputs and outputs in your team or organization.

The leaders single out one principle as the one that pays off. The leaders having the most impact are those doubling down on learn and be curious, and the published Learn and Be Curious principle reads Leaders are never done learning and always seek to improve themselves. The memo adds that teams who followed their curiosity, appropriately balanced with deliver results and insist on the highest standards, saw the most positive impact from AI adoption.

Why Is Amazon Asking Engineers to Get Closer to Customers?

The memo links AI gains to how well an engineer knows the product. Amazon found that SDEs and SDMs who are closer to their products and businesses are able to take the most advantage of AI and agentic software development.

It also names a problem. As Amazon has grown, we've seen some of our ICs and SDMs become less connected to their customers and business, and the leaders write that we want to reverse this trend. The memo recommends that individual contributors of all levels become deeply connected to your customers, and your team or org's product direction, with the scope of that ownership growing with seniority.

This matches the first of Amazon's published principles, Customer Obsession, which reads Leaders start with the customer and work backwards. Another principle, Dive Deep, asks leaders to stay connected to the details, audit frequently, and are skeptical when metrics and anecdote differ.

What Happened to Amazon's AI Leaderboard in May?

The memo follows an earlier step on AI usage. In May, Amazon shut down an internal leaderboard after discovering employees were tokenmaxxing to climb the ranks. The ranking was called Kirorank, and employees had assigned AI agents to unnecessary tasks to improve their ranking through token consumption, in a Financial Times account carried by heise online.

Amazon senior vice president Dave Treadwell reportedly told staff Please don't use AI just for the sake of using AI, and the company reportedly said the leaderboard was created by a group of employees who wanted to drive awareness for how AI can accelerate work.

Usage targets were part of the backdrop. Amazon has set targets that 80 percent of the company's developers should use AI weekly, the Financial Times reported. In place of token consumption, Amazon is now using a different metric called normalised deployments, which measures how regularly developers use AI for meaningful code, in the same FT account.

What we don't want to see is wasted effort where we are going and spending those tokens solving problems that don't matter, Amazon Web Services vice president Marc Brooker said, according to the CNBC report on the memo.

Why Are Other Companies Pulling Back From Tokenmaxxing Too?

Amazon is far from the first major tech company to look to curb AI token spend. While token prices are falling, the cost of completing a task is rising as AI firms shift from flat subscriptions to usage-based pricing. Some companies have set limits on employees' usage of tokens to slow rising bills, and others have switched to cheaper, open-source models for certain tasks.

In March 2026, Nvidia chief executive Jensen Huang warned that he would be deeply alarmed if a software developer with a salary of $500,000 did not spend at least $250,000 on tokens each year. In May, Uber disclosed that it had spent its entire 2026 AI budget by the end of April, and its chief operating officer called the spending harder to justify.

Meta reportedly stopped a similar leaderboard created by employees called Claudeonomics in April.

How Should Software Engineers Prepare for an Amazon Interview Now?

The memo sets out how Amazon's engineering leaders want current staff measured, while job candidates face a separate set of tests. For an SDE II role, the online assessment gives you 90 minutes to complete two technical questions followed by 20 minutes of Systems Design scenarios and an 8-minute multiple choice Work Style Survey related to our Leadership Principles. If that goes well, the loop will include four 55-minute interviews, and the final step is an Interview Outcome (within 5 business days).

The coding bar is specific. Candidates should expect to be asked to write syntactically correct code with no pseudo code, and the most important thing a SDE does at Amazon is write scalable, robust, and well-tested code. Each interviewer will typically ask two or three behavioral-based questions tied to the Leadership Principles, and the advice is to use the STAR method to frame your responses.

For students and recent graduates, the full-time online assessment lists a Coding assessment (average time: 70 minutes), a Work Simulation (average time: 60 minutes), a Workstyles Assessment (average time: 15 minutes) and a Feedback Survey (average time: 5 minutes). Depending on the country the role is in, the OA will be in one or two parts, and you should set aside up to two hours of uninterrupted time for the OA. During the Coding Assessment, candidates may use publicly accessible online resources such as the Java Developer Kit, but not sites that require a login or are private repositories, and your browser usage is logged. The rule is to avoid copying and pasting in the assessment. We expect this to be your own code.

Behavioral answers should be backed by numbers, since your answers should include metrics or data where applicable, so build your answers to Amazon interview questions with the STAR method around results.

What This Means for Software Engineers, and What Should You Do Next?

The memo puts the weight on what you ship rather than how much AI you use, while still saying that teams and individuals not adopting AI are falling behind. AI is still expected, since the leaders are certain that whatever future best practices are, AI will be at the center of software engineering practice. It also counts a clear explanation of where AI did not help as the judgement we are asking for.

That matters beyond Amazon because Amazon believes sharing its vision for the SDE role of the future can serve as a model for its customers. Separately, Amazon let go fewer than 1,000 white-collar workers earlier in October, mostly from its retail unit, and chief executive Andy Jassy has predicted that AI will change the scope and speed of day-to-day work.

• Keep a short AI log. For each project, note where a tool saved time and where it did not. That mirrors the test the memo sets for leaders.
• Show outcomes, not usage. Lead with launches and successful changes in production, and put your software engineer resume in the same terms.
• Get closer to the customer. The memo recommends that engineers become deeply connected to your customers, so learn the product problem behind your tickets.

Lacey Kaelani, Metaintro's CEO, puts it this way: Using more AI was never the skill. Knowing when it helps, when it does not, and being able to say why is what employers are now paying for.

Related Articles

• Google Is Piloting AI Fluency Tests for Software Engineer Candidates
• Companies Now Track Employees' AI Usage in Performance Reviews
• Laid Off by Amazon? Some Former Workers Are Being Asked Back for AI and Cloud Roles
• AI Coding Tools May Soon Cost More Than Human Developers
• Software Engineers Are Becoming AI Agent Managers, How to Stay Ahead of the Shift
• Managing AI Agents Is Becoming Its Own Job and Its Own Burnout Risk
• Inside Anthropic's Brutal Hiring Process That Bans AI in Interviews
• What Jobs Former Software Engineers Move Into and How to Make the Switch

People Also Asked

Q: What is tokenmaxxing?

A: Tokenmaxxing is when developers use AI as much as possible with little regard for output, sometimes to inflate their token consumption and look productive, and Amazon's memo says tokenmaxxing is at odds with frugality.

Q: Does Amazon still track how much AI its engineers use?

A: Yes. The memo calls token usage and adoption rates flawed but says using them with care and judgement, as you think about your career and your organization's performance is appropriate, and an Amazon spokesperson said the company hasn't instituted caps on employees' token usage.

Q: Will Amazon engineers be penalized for not using AI on a task?

A: The memo does not mention penalties. It says if your work does not benefit from current AI tooling, that is a finding, not a failing, and asks engineers to document why and bring it to their manager.

Future-proof your career? Metaintro brings together 50M jobs in one place. Create a free profile and start your search today.