How to work with AI, October 11. Five tips: let the agent wait, restart a long chat

Today's tips come from people who work with agents every day and count the cost. The Every team wrote three times this week about how it built its agent and where it went wrong. There is also a short trick from Simon Willison and the experience of Block.

I added my opinion to each tip. It shows how the tip works in car sales and in a talk with a person who is choosing a car.

Let AI wait for an event and tell you when it happens#

Simon Willison is a developer and the author of a blog about AI. On October 10 he was waiting for a new Python version to appear in a GitHub repository. He did not refresh the page himself. He gave ChatGPT one task and showed it in his post. Here it is.

Clone https://github.com/actions/python-versions and git pull once an hour until they add the stable 3.15 - then tell me about it

The version appeared, and Willison learned about it from the assistant. The post gives no other details.

In simple words. Do not check ten times a day yourself. Name the condition for the AI and ask it to write to you when the condition is met.

My opinion. A showroom is full of waiting. The customer waits for a car in the right color, and the salesperson waits for the bank. The sales manager waits for new terms of a factory program. Usually somebody remembers a week later.

I would give the task like this. "Check the page with the program terms once a day. When the rate or the end date changes, write to me what changed." It works where the assistant can run scheduled tasks. After that, a person calls the customer, not a robot.

Source Simon Willison's Weblog

Start a new chat with a short recap when the conversation gets long#

Paridhi Agarwal, an engineer at Every, wrote on October 8 about how the team built one shared agent for the whole company. One of the lessons is about long conversations. The longer the thread with the agent, the more memory problems it has and the more each answer costs.

The team began to start a fresh session earlier, with a short recap of the old one. In a rerun, total costs fell 39%. No exchange cost more than before.

In simple words. A long chat is a heavy backpack that the AI carries in every answer. Press it into a few lines and start clean.

My opinion. A salesperson runs one deal in one chat for a week. It holds the customer's messages, three finance calculations and a trade-in appraisal. By Friday, the assistant mixes up the first price and the last one.

I would ask like this. "Press this conversation into ten lines. Who the customer is, which car, what we promised, what is left to do".

I would paste these ten lines into a new chat and continue there. The same recap goes into the customer card in the CRM. No record of every contact, no sales department, and nothing to write a recap from.

Source Every

Ask the agent to read and explain first, and do not let it change anything#

Laura Entis of Every described on October 7 how the same team cut its agent costs by more than 80% across 11 frequent tasks. Paridhi Agarwal did the work. The article has a four-step workflow that anyone can repeat.

Here are the steps. The agent reads the system and explains how it works, with no changes. Then it proposes what to remove. Then the team measures one change on a safe copy. Then the agent looks for the rest and waits for approval.

Every prompt has the same pattern. Read first, change nothing until approved, and separate verified findings from untested ideas.

In simple words. Let the AI show that it understood the task. It starts to edit after your "yes".

My opinion. A sales manager asks the assistant to "improve" the reply templates for inquiries. The assistant rewrites all twenty, and nobody knows which one worked before.

I would write it another way. "Read our reply templates. Explain what each one promises the customer. Change nothing. Write apart what you verified in the text and what you assume".

I would allow a change in one template and compare customer replies for a week. AI prepares, and a person decides and answers for the result.

Source Every

Rehearse a talk with an AI copy of a person, but do not tune your text to its score#

Mike Taylor is head of evals at Every, which means he checks the quality of AI answers. On October 9 he described his experiment. He built a file from one year of his manager's messages that showed how she makes decisions. Then he told the model to improve an email draft until the "copy" rated it 10 out of 10.

After 73 rounds, the score never went above 9. The email became formal, and his proposal disappeared from it. The model added promises that he had not made. Taylor counts this as a loss.

The second experiment used a copy of a future interview guest. Out of 12 answers from the copy, 2 matched the real person, 4 matched in part and 6 missed. His conclusion is this. The copy can tell what the person would object to, but not when an objection should not matter.

In simple words. AI in the role of a customer is good for practice. It must not write for you to win the "customer's score".

My opinion. Before a meeting with a hard customer, I would paste their messages into a chat and ask this. "You are this customer. Name five objections to our trade-in offer, from the strongest to the weakest." That is ten minutes of preparation instead of improvising.

I would not take the second step. Do not ask the model to "improve the offer until the customer agrees". It will add a discount and dates that you did not promise. AI collects the objections, and the salesperson writes the offer.

Source Every

Give the plan to the strong model and routine to the cheap one, and keep the decision#

Bradley Axen is Head of AI Capabilities at Block, the company behind Square and Cash App. In an interview with Anthropic from October 8, he described how Block runs large tasks. The strongest model makes the plan and hands out the work. Cheaper models make the edits, up to dozens at a time. A human engineer steers and makes the hard decisions.

In one migration, Block merged about 1,000 code changes this way, and Axen reports no drop in quality. He is direct about cost. Using the most expensive model to fix a typo is a bad use of resources.

Systems, not trust, close the risky steps. Two humans must approve a production deploy. Cheap fixed rules do the easy checks, with no model.

In simple words. Call the expensive model to think and the cheap one to do. Keep the last word for yourself.

My opinion. A sales manager prepares a mailing to customers who bought a car three years ago. I would give the plan to the strong model. "Split the database into groups by ownership time and brand. Propose what to write to each group." A cheaper model can write two hundred messages from this plan.

I would close the sending with a rule. No message with a price leaves without a "yes" from the salesperson who works with the customer. On Friday I would look at the list of messages that nobody sent. AI does not replace the manager, it shows who did not do the work.

Source Claude

What to watch#

These are new releases and talks from the last two days. I have not reviewed them yet, so the notes follow the titles.

• OpenAI. How Oracle uses Codex to help business users get answers, a customer story
• AI Engineer. The lethal trifecta is already on your laptops, a talk by Michael Patterson of Coder on agent risks
• AI Engineer. AI agents don't read your policy docs, they hit your APIs, a talk by Gravitee on access
• AI Engineer. Latency is a budget, humanlike is the goal, a talk by Jesse Hall of LiveKit on voice agents
• AI Engineer. From context to memory, a talk by Anders Swanson of Oracle on agent memory
• AI Engineer. Why your company needs a context graph and how to build it, a talk by Gil Feig of Merge