Skip to content

Correcting an AI email summarizer once, for good

Part 06 of 6

On this page

An AI email summarizer writes a short account of each conversation, so you can decide about it before you open it. What it hands you is a reading. A reading can be faithful to every word in a thread and still be wrong about what the thread means. At 9:40 on a Friday those two failures look identical on the screen. One of them is a sentence you say once, and it holds from then on. The other one can’t be said at all.

The fifth part of this series ended on the second kind. It’s the harder one, and the more interesting one. It left Ellen looking at a wrong line, with none of the tools this series has spent five parts building.

Twenty seconds, and nothing in the list

Priya’s reply to the fee letter lands at 9:40, and it’s long. Ellen reads the summary instead of the message, the way she’s read everything since day three. Four lines. She agrees with three of them. The fourth one tells her the new rate is accepted and there’s a question about billing.

So she does what the third part of this series taught her. The first move is to name what’s wrong in terms of the thing on the screen, rather than in terms of her reaction to it. So: the summary says a question is settled that isn’t settled, rather than this is off.

Then the list. Seven sentences, newest at the top. The first question is whether each one could have fired here. All seven are about a message Point writes. Which version of her name goes at the bottom, whether her cell number is on it, where the date sits relative to the explaining. Each one reaches a message she sends, and Friday’s four lines are an account of somebody else’s mail. So all seven fall on the first question, together, in about the time it takes to read them.

Twenty seconds, seven lines, no candidates. Part three gave an empty search a meaning. If nothing in the list could have fired, then nothing was obeyed and something was misread out of the thread, so the repair belongs in the draft and the list stays as it is. That reading was right where part three made it. Here it’s wrong, and the difference is the hinge of this part, so it’s worth being exact about. In part three the list governed the thing on the screen, and it had simply not touched this bit of it. On Friday the list governs a different job from the one on the screen. Ellen searched a real list for the wrong act.

Five parts of this series are about mail Point writes for her. This is mail Point read for her. Two jobs, and two records behind them. Part three said as much in passing, when it warned that taking a summary complaint to the memory panel is a wasted trip. What’s worth correcting in a summary, how to word it, and how to tell two weeks later whether it stuck is worked through where it belongs. That guide keeps it.

That guide sorts corrections into two piles. Preferences become rules. Repairs stay repairs. Friday produced a third thing, and it’s worth the rest of this page, because it’s the reading half of the boundary the first part of this series drew before any of the teaching started.

There is no folder of what you read

Part one listed three things available before you’ve taught anything at all. Your own sent mail. The thread in front of you. The correspondence with this particular person, which is what settles the register.

Cross to the reading side and two of the three survive. The thread is there, entire. Seven years of Priya is there. The sent folder belongs to the writing side, and it belongs there for a plain reason. A sent folder records how Ellen writes. Reading leaves nothing behind, so the record of how Ellen reads lives in her head. She’s been through something like a quarter of a million messages since she started the firm, and what she took out of each one stayed with her.

Two things follow. The second one is what the last part owed you.

The frequency effect stops at the border. On the writing side, routine mail comes back close and rare mail comes back like somebody else, because there are hundreds of specimens of the first and almost none of the second. Summaries are even. They read the weekly status report and the fee reply to the same general standard, because a general standard is the whole of what’s available until you say something. That’s a different failure shape, and it’s worth expecting the right one. Volume alone leaves a summarizer where it started. One sentence from you moves it.

And this is what “a different memory behind it” amounts to. The memory behind a draft is the list of sentences plus years of your own writing, and that writing half is large and silent and does most of the work. The memory behind a summary is the list of sentences plus the thread it’s looking at. One of those two arrives half full. The other arrives with exactly what you’ve put in it. Both of them sit in the one panel you already know.

The edit that buys nothing

Part two gave a piece of advice that was honest at the time, and it stays on the writing side. When a boundary is hard to state, leave the rule unwritten and keep making the edit by hand. Four seconds, correct every time, and the list stays as short as it was.

That works on a draft for a reason easy to miss. The manual edit produces the thing you were trying to produce. The object of the exercise was the email, and the email is now right, so the list can stay out of it.

Correct a summary by hand and what you hold is a corrected summary. It goes nowhere. You’re its only reader, and you read it before you touched it. Whatever it cost you was spent in the two seconds before the correction existed: the wrong first impression, the thread you left alone because it looked handled, the four minutes you gave to something that could have waited. Retyping the line afterward leaves all three where they are.

So the reading side offers two options. Keep the sentence, and the next message of that kind arrives in the new shape. Skip it, and you pay the same small tax every week, always on the morning you have the least patience for stopping to deal with it. That’s the whole argument for correcting once, and it’s arithmetic rather than a claim about how clever the software is. The arithmetic is about where the value of a summary sits, and the value sits entirely in front of the correction.

Reporting and concluding are two jobs

Three quite different things get called a bad summary. Friday is the one people sort wrongly.

The shape is wrong. Faithful, and three times longer than the decision needs. Or arranged as a story, so the date that governs the week sits in the fourth sentence. That’s a preference, it’s stable, and a standing rule is exactly what it’s for.

A fact is wrong. A date treated as agreed when the message it came from was only floating it. A figure that belongs to a different engagement. Nothing to store. No sentence in the list makes a number correct, which is the same finding part three reached about drafts, and it has the same shape here.

Every fact is right and the conclusion is wrong. This is Friday, and on the screen it looks exactly like the second one.

Go back to the message. Priya’s reply holds a good deal about the increase itself, a question about which rate the January work falls under, a mention of the engagement letter, and one line near the bottom about how she’s thinking it over. Every sentence in it stops short of accepting anything. Accepted appears nowhere in the reply. It’s a conclusion, and the evidence for it is the silence where an objection would have been.

That names the mechanism, and the mechanism is the summary working. A summary does two jobs at once. It reports what was said. It resolves what that adds up to. The second job is the whole point. An account that only reported would run as long as the thread, and four lines stand in for eleven messages precisely because somebody decided what they came to. A resolution is a judgment, though, and the judgments most likely to be wrong are the ones drawn from what’s absent rather than from what’s there. A missing objection reads as agreement. It reads that way to any general standard, because in most mail that’s what it is.

What Ellen has, and nothing else does, is seven years of Priya. Priya raises a thing by asking about the mechanism next to it, rather than head-on the first time. That’s a fact about a person, learned from a hundred threads, and it sits in Ellen’s head rather than in any of them. Part one put the equivalent on the writing side at three phrases out of ninety words. On Friday it’s the difference between a client with a billing question and a client working out what it would cost her to leave.

The rule is not about Priya

The correction that occurs first is the one about Priya. It’s worth writing out, because it’s so nearly right.

Priya raises objections sideways. When her mail asks about a mechanism, tell me what she’s actually asking about.

Two things are wrong with it, and the second one is fatal. It fails part two’s five-second test, because two people reading a summary would argue about whether it had been followed, which makes it a wish rather than a rule. And it has the problem part five named for anything learned on rare mail. Ellen sends a fee letter about once every three years. A sentence written on that occasion waits untested until the next one comes around, and by then the letter that provoked it is three years back and she’s forgotten what she was reacting to. In between, it’s a line in the list that every search she runs has to read through.

Part five’s move on a rare message was to say the departure in the instruction and let it die with the message. That’s right for a departure. Friday is something else, because what went wrong is going to go wrong again. So the move changes: leave the rare part alone, and find the part of Friday that happens often.

The rare part is Priya and the fee. The part that happens weekly is this. Ellen asked for something, and a reply came back that neither agreed nor refused. That happens all over her week. It’s quotes, proposals, requests for documents, dates offered and left unconfirmed, a scope change put to a client on a Tuesday. In every one of those replies a summary has to decide whether the question is settled now, and it’s the same decision every time.

So the sentence is about the class rather than about the letter:

When I have asked for something and the reply doesn’t say yes or no, tell me what it did say about it. Don’t tell me it’s settled.

It names a finished state instead of a direction. Two people could read a summary and agree on whether it obeyed. The scope is in the sentence, and it covers one thing rather than two. And the property that makes it worth twelve seconds is frequency. It fires several times a week, so if the wording turns out to be one word too wide, Ellen finds that out on Tuesday rather than in the spring.

That’s the general move, and it’s the most useful thing on this page. When a rare message provokes you, look inside it for the thing that happens weekly, and write the rule about that. A rare message is a good place to notice something and a bad place to test it.

Notice what the sentence asks for. It asks for a stopping point: report the words, and leave the question open. Knowing Priya stays Ellen’s job. Everything else in this series has been about teaching a habit. This one teaches a limit, which turns out to be the same act pointed the other way, and it’s the one kind of instruction that reaches the third pile at all. The judgment stays yours. You can say where you want it handed back.

Two records, one panel

Point runs on the Gmail or Microsoft 365 mailbox you already have. Your address, your history and your sent folder stay where they are. The summarizing is finished before you sit down. Every conversation in the feed carries a plain account of what’s in it, so the deciding starts before the reading does. What a summary has to carry to be worth acting on is the subject of a plain summary of every email.

A summary that’s faithful and still wrong for you gets corrected on the spot, in the feed, rather than on a settings screen somewhere else. The four lines you were given sit next to the box where you write, so you’re objecting to the thing itself rather than to your recollection of it thirty seconds later. What you type is one sentence in your own language. Then that sentence stops being about Friday. It goes on the record as a standing rule, and the next message of that kind from that sender is read again from scratch with the rule in force.

The surface is different from the drafting one, and so is the memory behind it. The panel is the same one. Six months in, Ellen’s list holds sentences about how she writes and sentences about how she reads, in one place, newest at the top. The ones she typed are marked as hers. Every one of them is editable or removable in the same few seconds it took to make.

That mixture makes the lookup in part three cheaper rather than harder, which surprises people until they run one. A line about sign-offs belongs to a message she sends, so an account of somebody else’s message clears it on sight. A line about what a summary may conclude clears the same way under a reply she’s sending. The two halves eliminate each other on the first question, before any thought about what either of them does. Two separate panels would mean choosing which one to open, and the morning something is wrong is the morning you’d choose the wrong one.

The rest of the boundaries are the ones this series has been drawing all along. A rule travels with the work wherever Point is acting for you, so the blast radius of a summary rule is every summary it fits, and not only the sender who prompted it. Corrections work forward. Friday’s four lines stay what they were, and the reading they cost is already spent. Undo reaches what Point did inside the accounts you control, and a message sitting on somebody else’s server sits outside those, which undoing what Point did is exact about.

One thing stays exactly where it is, and it’s worth saying because people reach for it. The autonomy dial is held per kind of work, and it runs from suggest-only through review to fully handled. Drafting starts at review. The words are prepared there, and the sending waits. Correcting a summary leaves the dial alone, in either direction. The dial governs how much Point finishes on your behalf, and a summary isn’t something finished on your behalf: it’s the reading you would have done, handed to you before you do it. The benefits page has the rest of what’s already done by the time you open the inbox.

The message that goes nowhere near any of this

Ellen types her sentence. It takes about as long as the search that came back empty did. Then she opens the thread, which is what the summary was for.

She reads Priya’s last paragraph in Priya’s own words, and she knows what it means in four seconds. That part is hers, and it stays hers. It’s the same thing part one found sitting under an ordinary ninety-word reply, and it has held since: Ellen is the one who knows what her client is telling her.

At 9:52 she picks up the phone, which is where the three rehearsed sentences from Thursday were always going. The series about teaching software to write and read your mail ends on the one message of the year that goes nowhere near it. That’s the boundary working correctly, on the message where it matters most.

Eight lines in the list, then. It was eight in November, and seven after the pruning in part four. The new one is a different kind of sentence from any of the others. It marks a place where she wants something to stop short of a judgment and hand it back to her. Which is what all eight are, read properly. Six months of teaching, and what the list holds is the border between her voice and her thinking, written down one sentence at a time on the mornings she happened to notice.

Six parts, and this is the last of them. What’s left is your own Monday, and one question to put to it: which correction have you made twice? On the writing side it’s easy to spot, because it’s the sign-off you keep retyping. On the reading side it’s quieter. A summary you fixed in your own head left nothing behind, so the second time you fix it feels exactly like the first.

Ready for a calmer inbox?

Join the private beta

We're onboarding a few teams at a time. Leave your email, confirm it once, and we'll send an invitation the moment a place opens.

By joining you agree to our privacy policy.

Private beta

What you're joining

It runs on the mail you have

Point sits on top of Gmail or Outlook. Your address, your history and your contacts stay exactly as they are, so there is nothing to migrate.

You set how much Point does

Out of the box everything waits for your review, replies included. You hand over only what you trust, one kind of work at a time.

Join the private beta

We're onboarding a few teams at a time. Leave your email, confirm it once, and we'll send an invitation the moment a place opens.

By joining you agree to our privacy policy.