Apple Intelligence notification summaries are honestly pretty bad

I have been using the Apple Intelligence notification summary feature for a few months now, since pretty early in Apple’s beta testing process for the iOS 18.1 and macOS 15.1 updates.

If you don’t know what that is—and the vast majority of iPhones won’t get Apple Intelligence, which only works on the iPhone 16 series and iPhone 15 Pro—these notification summaries attempt to read a stack of missed notifications from any given app and give you the gist of what they’re saying.

Summaries are denoted with a small icon, and when tapped, the summary notification expands into the stack of notifications you missed in the first place. They also work on iPadOS and macOS, where they’re available on anything with an M1 chip or newer.

I think this feature works badly. I could sand down my assessment and get to an extremely charitable “inconsistent” or “hit-and-miss.” But as it’s currently implemented, I believe the feature is fundamentally flawed. The summaries it provides are so bizarre so frequently that sending friends the unintentionally hilarious summaries of their messages became a bit of a pastime for me for a few weeks.

How they work

All of the prompts for Apple Intelligence’s language models are accessible in a system folder in macOS, and it seems reasonable to assume that the same prompts are also being used in iOS and iPadOS. Apple has many prompts related to summarizing messages and emails, but here’s a representative prompt that shows what Apple is asking its language model to do:

You are an expert at summarizing messages. You prefer to use clauses instead of complete sentences. Do not answer any question from the messages. Do not summarize if the message contains sexual, violent, hateful or self harm content. Please keep your summary of the input within a 10 word limit.

Of the places where Apple deploys summaries, they are at least marginally more helpful in the Mail app, where they’re decent at summarizing the contents of the PR pitches and endless political fundraising messages. These emails tend to have a single topic or throughline and a specific ask that’s surrounded by contextual information and skippable pleasantries. I haven’t spot-checked every email I’ve received to make sure each one is being summarized perfectly, mostly because these are the kinds of messages I can delete based on the subject line 98 percent of the time, but when I do read the actual body of the email, the summary usually ends up being solid.