arch/x86/hyperv/mmu.c | 4 ++-- arch/x86/include/asm/tlb.h | 19 +++++++++++++++- arch/x86/include/asm/tlbflush.h | 6 +++-- arch/x86/kernel/smpboot.c | 1 + arch/x86/mm/tlb.c | 39 +++++++++++++++++++++++---------- include/asm-generic/tlb.h | 17 ++++++++++++++ mm/mmu_gather.c | 15 +++++++++++++ 7 files changed, 84 insertions(+), 17 deletions(-)
Hi all,
When page table operations require synchronization with software/lockless
walkers, they call tlb_remove_table_sync_{one,rcu}() after flushing the
TLB (tlb->freed_tables or tlb->unshared_tables).
On architectures where the TLB flush already sends IPIs to all target CPUs,
the subsequent sync IPI broadcast is redundant. This is not only costly on
large systems where it disrupts all CPUs even for single-process page table
operations, but has also been reported to hurt RT workloads[1].
This series introduces tlb_table_flush_implies_ipi_broadcast() to check if
the prior TLB flush already provided the necessary synchronization. When
true, the sync calls can early-return.
A few cases rely on this synchronization:
1) hugetlb PMD unshare[2]: The problem is not the freeing but the reuse
of the PMD table for other purposes in the last remaining user after
unsharing.
2) khugepaged collapse[3]: Ensure no concurrent GUP-fast before collapsing
and (possibly) freeing the page table / re-depositing it.
Two-step plan as David suggested[4]:
Step 1 (this series): Skip redundant sync when we're 100% certain the TLB
flush sent IPIs. INVLPGB is excluded because when supported, we cannot
guarantee IPIs were sent, keeping it clean and simple.
Step 2 (future work): Send targeted IPIs only to CPUs actually doing
software/lockless page table walks, benefiting all architectures.
Regarding Step 2, it obviously only applies to setups where Step 1 does not
apply: like x86 with INVLPGB or arm64. Step 2 work is ongoing; early
attempts showed ~3% GUP-fast overhead. Reducing the overhead requires more
work and tuning; it will be submitted separately once ready.
On a 64-core Intel x86 server, the CAL interrupt count in
/proc/interrupts dropped from 646,316 to 785 when collapsing a 20 GiB
range with this series applied.
David Hildenbrand did the initial implementation. I built on his work and
relied on off-list discussions to push it further - thanks a lot David!
[1] https://lore.kernel.org/linux-mm/1b27a3fa-359a-43d0-bdeb-c31341749367@kernel.org/
[2] https://lore.kernel.org/linux-mm/6a364356-5fea-4a6c-b959-ba3b22ce9c88@kernel.org/
[3] https://lore.kernel.org/linux-mm/2cb4503d-3a3f-4f6c-8038-7b3d1c74b3c2@kernel.org/
[4] https://lore.kernel.org/linux-mm/bbfdf226-4660-4949-b17b-0d209ee4ef8c@kernel.org/
v9 -> v10:
- Rename the x86 flush_tlb_info bit from freed_tables to wake_lazy_cpus,
to make the lazy-TLB behavior explicit (per Dave, thanks!)
- https://lore.kernel.org/linux-mm/20260420030851.6735-1-lance.yang@linux.dev/
v8 -> v9:
- Rebase on mm-new; re-tested, no code changes.
- https://lore.kernel.org/linux-mm/20260324085238.44477-1-lance.yang@linux.dev/
v7 -> v8:
- Pick up Acked-by tags from David, thanks!
- Add CAL interrupt numbers to the cover letter (per Andrew, thanks!)
- Rewrite the [2/2] changelog and reword the comment (per David, thanks!)
- https://lore.kernel.org/linux-mm/20260309020711.20831-1-lance.yang@linux.dev/
v6 -> v7:
- Simplify init logic and eliminate duplicated X86_FEATURE_INVLPGB checks
(per Dave, thanks!)
- Remove flush_tlb_multi_implies_ipi_broadcast property because no PV
backend sets it today.
- https://lore.kernel.org/linux-mm/20260304021046.18550-1-lance.yang@linux.dev/
v5 -> v6:
- Use static_branch to eliminate the branch overhead (per Peter, thanks!)
- https://lore.kernel.org/linux-mm/20260302063048.9479-1-lance.yang@linux.dev/
v4 -> v5:
- Drop per-CPU tracking (active_lockless_pt_walk_mm) from this series;
defer to Step 2 as it adds ~3% GUP-fast overhead
- Keep pv_ops property false for PV backends like KVM: preempted vCPUs
cannot be assumed safe (per Sean, thanks!)
https://lore.kernel.org/linux-mm/aaCP95l-m8ISXF78@google.com/
- https://lore.kernel.org/linux-mm/20260202074557.16544-1-lance.yang@linux.dev/
v3 -> v4:
- Rework based on David's two-step direction and per-CPU idea:
1) Targeted IPIs: per-CPU variable when entering/leaving lockless page
table walk; tlb_remove_table_sync_mm() IPIs only those CPUs.
2) On x86, pv_mmu_ops property set at init to skip the extra sync when
flush_tlb_multi() already sends IPIs.
https://lore.kernel.org/linux-mm/bbfdf226-4660-4949-b17b-0d209ee4ef8c@kernel.org/
- https://lore.kernel.org/linux-mm/20260106120303.38124-1-lance.yang@linux.dev/
v2 -> v3:
- Complete rewrite: use dynamic IPI tracking instead of static checks
(per Dave Hansen, thanks!)
- Track IPIs via mmu_gather: native_flush_tlb_multi() sets flag when
actually sending IPIs
- Motivation for skipping redundant IPIs explained by David:
https://lore.kernel.org/linux-mm/1b27a3fa-359a-43d0-bdeb-c31341749367@kernel.org/
- https://lore.kernel.org/linux-mm/20251229145245.85452-1-lance.yang@linux.dev/
v1 -> v2:
- Fix cover letter encoding to resolve send-email issues. Apologies for
any email flood caused by the failed send attempts :(
RFC -> v1:
- Use a callback function in pv_mmu_ops instead of comparing function
pointers (per David)
- Embed the check directly in tlb_remove_table_sync_one() instead of
requiring every caller to check explicitly (per David)
- Move tlb_table_flush_implies_ipi_broadcast() outside of
CONFIG_MMU_GATHER_RCU_TABLE_FREE to fix build error on architectures
that don't enable this config.
https://lore.kernel.org/oe-kbuild-all/202512142156.cShiu6PU-lkp@intel.com/
- https://lore.kernel.org/linux-mm/20251213080038.10917-1-lance.yang@linux.dev/
Lance Yang (2):
mm/mmu_gather: prepare to skip redundant sync IPIs
x86/tlb: skip redundant sync IPIs for native TLB flush
arch/x86/hyperv/mmu.c | 4 ++--
arch/x86/include/asm/tlb.h | 19 +++++++++++++++-
arch/x86/include/asm/tlbflush.h | 6 +++--
arch/x86/kernel/smpboot.c | 1 +
arch/x86/mm/tlb.c | 39 +++++++++++++++++++++++----------
include/asm-generic/tlb.h | 17 ++++++++++++++
mm/mmu_gather.c | 15 +++++++++++++
7 files changed, 84 insertions(+), 17 deletions(-)
--
2.49.0
On Fri, 24 Apr 2026 14:25:26 +0800 Lance Yang <lance.yang@linux.dev> wrote:
> When page table operations require synchronization with software/lockless
> walkers, they call tlb_remove_table_sync_{one,rcu}() after flushing the
> TLB (tlb->freed_tables or tlb->unshared_tables).
>
> On architectures where the TLB flush already sends IPIs to all target CPUs,
> the subsequent sync IPI broadcast is redundant. This is not only costly on
> large systems where it disrupts all CPUs even for single-process page table
> operations, but has also been reported to hurt RT workloads[1].
>
> This series introduces tlb_table_flush_implies_ipi_broadcast() to check if
> the prior TLB flush already provided the necessary synchronization. When
> true, the sync calls can early-return.
>
> A few cases rely on this synchronization:
>
> 1) hugetlb PMD unshare[2]: The problem is not the freeing but the reuse
> of the PMD table for other purposes in the last remaining user after
> unsharing.
>
> 2) khugepaged collapse[3]: Ensure no concurrent GUP-fast before collapsing
> and (possibly) freeing the page table / re-depositing it.
Sashiko questions:
https://sashiko.dev/#/patchset/20260424062528.71951-1-lance.yang@linux.dev
(I never know if my Sashiko emails are welcome/useful. Maybe Sashiko
said the same stuff about v9 and it's all wrong. But better safe than
sorry!)
On 04-24 06:30, Andrew Morton wrote:
> On Fri, 24 Apr 2026 14:25:26 +0800 Lance Yang <lance.yang@linux.dev> wrote:
>
> > When page table operations require synchronization with software/lockless
> > walkers, they call tlb_remove_table_sync_{one,rcu}() after flushing the
> > TLB (tlb->freed_tables or tlb->unshared_tables).
> >
> > On architectures where the TLB flush already sends IPIs to all target CPUs,
> > the subsequent sync IPI broadcast is redundant. This is not only costly on
> > large systems where it disrupts all CPUs even for single-process page table
> > operations, but has also been reported to hurt RT workloads[1].
> >
> > This series introduces tlb_table_flush_implies_ipi_broadcast() to check if
> > the prior TLB flush already provided the necessary synchronization. When
> > true, the sync calls can early-return.
> >
> > A few cases rely on this synchronization:
> >
> > 1) hugetlb PMD unshare[2]: The problem is not the freeing but the reuse
> > of the PMD table for other purposes in the last remaining user after
> > unsharing.
> >
> > 2) khugepaged collapse[3]: Ensure no concurrent GUP-fast before collapsing
> > and (possibly) freeing the page table / re-depositing it.
>
> Sashiko questions:
> https://sashiko.dev/#/patchset/20260424062528.71951-1-lance.yang@linux.dev
>
> (I never know if my Sashiko emails are welcome/useful. Maybe Sashiko
> said the same stuff about v9 and it's all wrong. But better safe than
> sorry!)
These emails are helpful; but, I do not believe you should have to
manually follow up with a link to every new patch series.
Perhaps Sashiko could automatically send a summary email in response to
the cover letter, or provide a link once the reviews are complete. For
the kexec ML, we opted-in with Roman to receive automated emails from
sashiko.
+Cc: Roman.
>
>
On Fri, 24 Apr 2026 13:37:04 +0000 Pasha Tatashin <pasha.tatashin@soleen.com> wrote:
> On 04-24 06:30, Andrew Morton wrote:
> > On Fri, 24 Apr 2026 14:25:26 +0800 Lance Yang <lance.yang@linux.dev> wrote:
> >
> > > When page table operations require synchronization with software/lockless
> > > walkers, they call tlb_remove_table_sync_{one,rcu}() after flushing the
> > > TLB (tlb->freed_tables or tlb->unshared_tables).
> > >
> > > On architectures where the TLB flush already sends IPIs to all target CPUs,
> > > the subsequent sync IPI broadcast is redundant. This is not only costly on
> > > large systems where it disrupts all CPUs even for single-process page table
> > > operations, but has also been reported to hurt RT workloads[1].
> > >
> > > This series introduces tlb_table_flush_implies_ipi_broadcast() to check if
> > > the prior TLB flush already provided the necessary synchronization. When
> > > true, the sync calls can early-return.
> > >
> > > A few cases rely on this synchronization:
> > >
> > > 1) hugetlb PMD unshare[2]: The problem is not the freeing but the reuse
> > > of the PMD table for other purposes in the last remaining user after
> > > unsharing.
> > >
> > > 2) khugepaged collapse[3]: Ensure no concurrent GUP-fast before collapsing
> > > and (possibly) freeing the page table / re-depositing it.
> >
> > Sashiko questions:
> > https://sashiko.dev/#/patchset/20260424062528.71951-1-lance.yang@linux.dev
> >
> > (I never know if my Sashiko emails are welcome/useful. Maybe Sashiko
> > said the same stuff about v9 and it's all wrong. But better safe than
> > sorry!)
>
> These emails are helpful; but, I do not believe you should have to
> manually follow up with a link to every new patch series.
>
> Perhaps Sashiko could automatically send a summary email in response to
> the cover letter, or provide a link once the reviews are complete. For
> the kexec ML, we opted-in with Roman to receive automated emails from
> sashiko.
Yep. I'd be OK with an automatic reply-to-all. Maybe some won't like
that.
An alternative I've discussed with Roman is an automated
reply-to-author with a cc to a dedicated list (we could use mm-commits
for now).
Preferences?
(I'd suggest automated-reply-to-all, see who complains, then figure out
why they can't figure out mail filtering ;))
On 4/24/26 16:15, Andrew Morton wrote: > On Fri, 24 Apr 2026 13:37:04 +0000 Pasha Tatashin <pasha.tatashin@soleen.com> wrote: > >> On 04-24 06:30, Andrew Morton wrote: >>> >>> >>> Sashiko questions: >>> https://sashiko.dev/#/patchset/20260424062528.71951-1-lance.yang@linux.dev >>> >>> (I never know if my Sashiko emails are welcome/useful. Maybe Sashiko >>> said the same stuff about v9 and it's all wrong. But better safe than >>> sorry!) >> >> These emails are helpful; but, I do not believe you should have to >> manually follow up with a link to every new patch series. >> >> Perhaps Sashiko could automatically send a summary email in response to >> the cover letter, or provide a link once the reviews are complete. For >> the kexec ML, we opted-in with Roman to receive automated emails from >> sashiko. > > Yep. I'd be OK with an automatic reply-to-all. Maybe some won't like > that. > > An alternative I've discussed with Roman is an automated > reply-to-author with a cc to a dedicated list (we could use mm-commits > for now). > > Preferences? Reply-to-author would likely be better as a first step. -- Cheers, David
On Fri, 24 Apr 2026 16:20:55 +0200 "David Hildenbrand (Arm)" <david@kernel.org> wrote: > On 4/24/26 16:15, Andrew Morton wrote: > > On Fri, 24 Apr 2026 13:37:04 +0000 Pasha Tatashin <pasha.tatashin@soleen.com> wrote: > > > >> On 04-24 06:30, Andrew Morton wrote: > >>> > >>> > >>> Sashiko questions: > >>> https://sashiko.dev/#/patchset/20260424062528.71951-1-lance.yang@linux.dev > >>> > >>> (I never know if my Sashiko emails are welcome/useful. Maybe Sashiko > >>> said the same stuff about v9 and it's all wrong. But better safe than > >>> sorry!) > >> > >> These emails are helpful; but, I do not believe you should have to > >> manually follow up with a link to every new patch series. > >> > >> Perhaps Sashiko could automatically send a summary email in response to > >> the cover letter, or provide a link once the reviews are complete. For > >> the kexec ML, we opted-in with Roman to receive automated emails from > >> sashiko. > > > > Yep. I'd be OK with an automatic reply-to-all. Maybe some won't like > > that. > > > > An alternative I've discussed with Roman is an automated > > reply-to-author with a cc to a dedicated list (we could use mm-commits > > for now). > > > > Preferences? > > Reply-to-author would likely be better as a first step. Why do you think so? Pasha, is it too early to determine how reply-to-all is working out for kexec?
On Fri, 24 Apr 2026 07:31:45 -0700 Andrew Morton <akpm@linux-foundation.org> wrote: > On Fri, 24 Apr 2026 16:20:55 +0200 "David Hildenbrand (Arm)" <david@kernel.org> wrote: > > > On 4/24/26 16:15, Andrew Morton wrote: > > > On Fri, 24 Apr 2026 13:37:04 +0000 Pasha Tatashin <pasha.tatashin@soleen.com> wrote: > > > > > >> On 04-24 06:30, Andrew Morton wrote: > > >>> > > >>> > > >>> Sashiko questions: > > >>> https://sashiko.dev/#/patchset/20260424062528.71951-1-lance.yang@linux.dev > > >>> > > >>> (I never know if my Sashiko emails are welcome/useful. Maybe Sashiko > > >>> said the same stuff about v9 and it's all wrong. But better safe than > > >>> sorry!) > > >> > > >> These emails are helpful; but, I do not believe you should have to > > >> manually follow up with a link to every new patch series. > > >> > > >> Perhaps Sashiko could automatically send a summary email in response to > > >> the cover letter, or provide a link once the reviews are complete. For > > >> the kexec ML, we opted-in with Roman to receive automated emails from > > >> sashiko. > > > > > > Yep. I'd be OK with an automatic reply-to-all. Maybe some won't like > > > that. > > > > > > An alternative I've discussed with Roman is an automated > > > reply-to-author with a cc to a dedicated list (we could use mm-commits > > > for now). > > > > > > Preferences? > > > > Reply-to-author would likely be better as a first step. > > Why do you think so? > > Pasha, is it too early to determine how reply-to-all is working out for > kexec? DAMON is also opted in to Sashiko's email service. It is replied to only the author and damon@lists.linux.dev. I understand this is the default form of Sashiko's email service, and can flexibly tuned. So I cannot say how reply-to-all will work for mm. But I can say the default setup email service was a huge improvement for my life. I made tools for making my Sashiko handling easier and was quite convinced with that. But the Sashiko's email service made my life much better. Of course, it may differntly applied to others. My personality is not necessarily same to others. Thanks, SJ
On 4/24/26 16:31, Andrew Morton wrote: > On Fri, 24 Apr 2026 16:20:55 +0200 "David Hildenbrand (Arm)" <david@kernel.org> wrote: > >> On 4/24/26 16:15, Andrew Morton wrote: >>> >>> >>> Yep. I'd be OK with an automatic reply-to-all. Maybe some won't like >>> that. >>> >>> An alternative I've discussed with Roman is an automated >>> reply-to-author with a cc to a dedicated list (we could use mm-commits >>> for now). >>> >>> Preferences? >> >> Reply-to-author would likely be better as a first step. > > Why do you think so? The most important part for me is that authors are aware of the reports. Sending them as a mail forces people to publicly reply to the feedback, and at this point in time, I am not convinced that that is the right approach. -- Cheers, David
On Fri, Apr 24, 2026 at 11:36 AM David Hildenbrand (Arm) <david@kernel.org> wrote: > > On 4/24/26 16:31, Andrew Morton wrote: > > On Fri, 24 Apr 2026 16:20:55 +0200 "David Hildenbrand (Arm)" <david@kernel.org> wrote: > > > >> On 4/24/26 16:15, Andrew Morton wrote: > >>> > >>> > >>> Yep. I'd be OK with an automatic reply-to-all. Maybe some won't like > >>> that. > >>> > >>> An alternative I've discussed with Roman is an automated > >>> reply-to-author with a cc to a dedicated list (we could use mm-commits > >>> for now). > >>> > >>> Preferences? > >> > >> Reply-to-author would likely be better as a first step. > > > > Why do you think so? > > The most important part for me is that authors are aware of the reports. > > Sending them as a mail forces people to publicly reply to the feedback, and at > this point in time, I am not convinced that that is the right approach. But I imagine it's useful for reviewers to see Sashiko's feedback as well (without having to go look on the website). It's possible that Sashiko is right but the author isn't convinced, so getting more eyes on the feedback would help. If Sashiko is wrong, it's still useful for more people to see it and point it out, instead of Sashiko privately misguiding the author, especially that reviewers are probably more likely to tell if Sashiko is right or wrong. I understand the fear of too much noise, but I think it's easy to ignore Sashiko if we want to. It's also a good exercise to spell out why Sashiko is wrong (e.g. improve changelogs/comments to document assumptions more obviously). We can always tune it down later if we think it's too much, right?
On 4/24/26 20:50, Yosry Ahmed wrote: > On Fri, Apr 24, 2026 at 11:36 AM David Hildenbrand (Arm) > <david@kernel.org> wrote: >> >> On 4/24/26 16:31, Andrew Morton wrote: >>> >>> >>> Why do you think so? >> >> The most important part for me is that authors are aware of the reports. >> >> Sending them as a mail forces people to publicly reply to the feedback, and at >> this point in time, I am not convinced that that is the right approach. > > But I imagine it's useful for reviewers to see Sashiko's feedback as > well (without having to go look on the website). I read a lot of them, yes. Maintainers know how to find it. I even think maintainers should briefly go over it before applying to spot if anything in there is still left unanswered. But that doesn't need a mailing list posting. I enjoy if contributors are aware of the reports and use that input in a reasonable, like Zi just did [1]. And if they are unsure, they usually ask to double-check. [1] https://lore.kernel.org/r/15191F7F-0D10-4907-B963-DA4EA0E36EB6@nvidia.com > It's possible that > Sashiko is right but the author isn't convinced, so getting more eyes > on the feedback would help. Forcing contributors to reply to everything. I don't like that, in particular not as long as there is no way for contributors to run it early in private. In most cases, contributors just do the reasonable thing: incorporate the feedback in a new version. -- Cheers, David
On Fri, Apr 24, 2026 at 12:09 PM David Hildenbrand (Arm) <david@kernel.org> wrote: > > On 4/24/26 20:50, Yosry Ahmed wrote: > > On Fri, Apr 24, 2026 at 11:36 AM David Hildenbrand (Arm) > > <david@kernel.org> wrote: > >> > >> On 4/24/26 16:31, Andrew Morton wrote: > >>> > >>> > >>> Why do you think so? > >> > >> The most important part for me is that authors are aware of the reports. > >> > >> Sending them as a mail forces people to publicly reply to the feedback, and at > >> this point in time, I am not convinced that that is the right approach. > > > > But I imagine it's useful for reviewers to see Sashiko's feedback as > > well (without having to go look on the website). > > I read a lot of them, yes. Maintainers know how to find it. > > I even think maintainers should briefly go over it before applying to spot if > anything in there is still left unanswered. But that doesn't need a mailing list > posting. > > I enjoy if contributors are aware of the reports and use that input in a > reasonable, like Zi just did [1]. And if they are unsure, they usually ask to > double-check. > > [1] https://lore.kernel.org/r/15191F7F-0D10-4907-B963-DA4EA0E36EB6@nvidia.com > > > It's possible that > > Sashiko is right but the author isn't convinced, so getting more eyes > > on the feedback would help. > > Forcing contributors to reply to everything. I don't like that, in particular > not as long as there is no way for contributors to run it early in private. > > In most cases, contributors just do the reasonable thing: incorporate the > feedback in a new version. The usefulness of the mailing list posting is that it makes it easier to respond and discuss the review. Yes, what Zi did is great, but it would be nice if contributors/reviewers didn't need to manually quote Sashiko. That being said, I understand the concerns and the pressure to respond to everything as you mention below. Maybe at some point this will become an easier decision to make as reviews become more refined.
On 4/24/26 21:18, Yosry Ahmed wrote: > On Fri, Apr 24, 2026 at 12:09 PM David Hildenbrand (Arm) > <david@kernel.org> wrote: >> >> On 4/24/26 20:50, Yosry Ahmed wrote: >>> On Fri, Apr 24, 2026 at 11:36 AM David Hildenbrand (Arm) >>> <david@kernel.org> wrote: >>> >>> But I imagine it's useful for reviewers to see Sashiko's feedback as >>> well (without having to go look on the website). >> >> I read a lot of them, yes. Maintainers know how to find it. >> >> I even think maintainers should briefly go over it before applying to spot if >> anything in there is still left unanswered. But that doesn't need a mailing list >> posting. >> >> I enjoy if contributors are aware of the reports and use that input in a >> reasonable, like Zi just did [1]. And if they are unsure, they usually ask to >> double-check. >> >> [1] https://lore.kernel.org/r/15191F7F-0D10-4907-B963-DA4EA0E36EB6@nvidia.com >> >>> It's possible that >>> Sashiko is right but the author isn't convinced, so getting more eyes >>> on the feedback would help. >> >> Forcing contributors to reply to everything. I don't like that, in particular >> not as long as there is no way for contributors to run it early in private. >> >> In most cases, contributors just do the reasonable thing: incorporate the >> feedback in a new version. > > The usefulness of the mailing list posting is that it makes it easier > to respond and discuss the review. Yes, what Zi did is great, but it > would be nice if contributors/reviewers didn't need to manually quote > Sashiko. I think the most important part is the contributor side. A private posting is sufficient for that. So if possible, I'd start with that and see how it goes. > > That being said, I understand the concerns and the pressure to respond > to everything as you mention below. Maybe at some point this will > become an easier decision to make as reviews become more refined. There is another thing to consider here: impact on other reviewers, in particular, new contributors, when they spot excessive review already posted to the mailing list. And the first thing being them reading that. -- Cheers, David
On Sat, 25 Apr 2026 07:17:18 +0200 "David Hildenbrand (Arm)" <david@kernel.org> wrote: > > The usefulness of the mailing list posting is that it makes it easier > > to respond and discuss the review. Yes, what Zi did is great, but it > > would be nice if contributors/reviewers didn't need to manually quote > > Sashiko. > > I think the most important part is the contributor side. A private posting is > sufficient for that. Thinking through how is this going to work out. - contributor sends patchset - akpm looks at the Sashiko scan and thinks "huh, we might be seeing a new version soon". - Now what? Maybe contributor doesn't like AI, maybe contributor thinks it was all nonsense. But I don't know this! So I send the email "I see that Sashiko said stuff - can we expect a new version?". Which is no improvement over what's happening now. What I would like to have is some reasonably reliable and prompt means by which we all learn contributor's views on the Sashiko scan. One way to bring this about might be to set suitable reply-to headers in the Sashiko->contributor email, along with a few words asking contributor to let everyone know the status. But whatever - let's not overthink this. To start somewhere, let's send that private email, spend a few weeks evaluating then perhaps make adjustments.
On 4/25/26 13:36, Andrew Morton wrote: > On Sat, 25 Apr 2026 07:17:18 +0200 "David Hildenbrand (Arm)" <david@kernel.org> wrote: > >>> The usefulness of the mailing list posting is that it makes it easier >>> to respond and discuss the review. Yes, what Zi did is great, but it >>> would be nice if contributors/reviewers didn't need to manually quote >>> Sashiko. >> >> I think the most important part is the contributor side. A private posting is >> sufficient for that. > > Thinking through how is this going to work out. > > - contributor sends patchset > > - akpm looks at the Sashiko scan and thinks "huh, we might be seeing > a new version soon". > > - Now what? Maybe contributor doesn't like AI, maybe contributor > thinks it was all nonsense. But I don't know this! Right. In the common case, people will just read the output and process it in a reasonable way, like some people do today even without any notification. But there will be cases where people ignore it, I am sure. > > So I send the email "I see that Sashiko said stuff - can we expect > a new version?". Which is no improvement over what's happening now. That only happens in the scenarios where there is no further feedback from other reviewers and there won't be a new revision. That's why I am saying that the AI review on the last upstream posting before merging is the most important one from a maintainer perspective. I agree that when we are about to merge something and AI review was not addressed (no feedback from contributor), that it's a problem. > > What I would like to have is some reasonably reliable and prompt means > by which we all learn contributor's views on the Sashiko scan. Right. At the same time I don't want full AI review posted to the mailing list immediately to each and every revision of a patch set. As first step, I would be fine with having AI review send a review status in case it went into a problem to everyone as reply to each patch set, just stating whether it failed to apply, or if something was found etc. Paired with a link like you would send. > > One way to bring this about might be to set suitable reply-to headers > in the Sashiko->contributor email, along with a few words asking > contributor to let everyone know the status. Yes, that's an alternative if we were to send the full AI review to contributors. Which might make sense in addition to the AI review status (above). > > But whatever - let's not overthink this. To start somewhere, let's > send that private email, spend a few weeks evaluating then perhaps make > adjustments. Right, let's do this step by step. -- Cheers, David
On Fri, 24 Apr 2026 12:18:49 -0700 Yosry Ahmed <yosry@kernel.org> wrote: > On Fri, Apr 24, 2026 at 12:09 PM David Hildenbrand (Arm) > <david@kernel.org> wrote: > > > > On 4/24/26 20:50, Yosry Ahmed wrote: > > > On Fri, Apr 24, 2026 at 11:36 AM David Hildenbrand (Arm) > > > <david@kernel.org> wrote: > > >> > > >> On 4/24/26 16:31, Andrew Morton wrote: > > >>> > > >>> > > >>> Why do you think so? > > >> > > >> The most important part for me is that authors are aware of the reports. > > >> > > >> Sending them as a mail forces people to publicly reply to the feedback, and at > > >> this point in time, I am not convinced that that is the right approach. > > > > > > But I imagine it's useful for reviewers to see Sashiko's feedback as > > > well (without having to go look on the website). > > > > I read a lot of them, yes. Maintainers know how to find it. > > > > I even think maintainers should briefly go over it before applying to spot if > > anything in there is still left unanswered. But that doesn't need a mailing list > > posting. > > > > I enjoy if contributors are aware of the reports and use that input in a > > reasonable, like Zi just did [1]. And if they are unsure, they usually ask to > > double-check. > > > > [1] https://lore.kernel.org/r/15191F7F-0D10-4907-B963-DA4EA0E36EB6@nvidia.com > > > > > It's possible that > > > Sashiko is right but the author isn't convinced, so getting more eyes > > > on the feedback would help. > > > > Forcing contributors to reply to everything. I don't like that, in particular > > not as long as there is no way for contributors to run it early in private. > > > > In most cases, contributors just do the reasonable thing: incorporate the > > feedback in a new version. > > The usefulness of the mailing list posting is that it makes it easier > to respond and discuss the review. Yes, what Zi did is great, but it > would be nice if contributors/reviewers didn't need to manually quote > Sashiko. I agree. I had to implement hkml-Sashiko integration [1] to avoid the manual works. People could use such existign tools or develop their own. But I can say Sashiko's direct mailing service has improved my dasy much more than the hkml feature. > > That being said, I understand the concerns and the pressure to respond > to everything as you mention below. Maybe at some point this will > become an easier decision to make as reviews become more refined. I agree here, too. Apparently [2] my tooling-motivated Sashiko reply forwarding didn't convince all. My honest feeling was that it is not only unconvincing but might making someone slightly annoyed. I personally feel no problem at Sashiko review is publicly replied to my patches, but I understand my personality is not necessarily same to others. If some change can make someone happy while also making someone sad, I'm up to reducing sadness. [1] https://github.com/sjp38/hackermail/blob/master/USAGE.md#forwarding-sashikodev-statuscomments-to-mailing-list [2] https://lore.kernel.org/20260331045245.67438-1-sj@kernel.org/ Thanks, SJ
On Fri, Apr 24, 2026 at 09:09:21PM +0200, David Hildenbrand (Arm) wrote: > Forcing contributors to reply to everything. I don't like that, in particular > not as long as there is no way for contributors to run it early in private. > > In most cases, contributors just do the reasonable thing: incorporate the > feedback in a new version. Yes, so I understand that you can run it locally, but then you get to pay for the tokens (and sashiko burns tokens like dry grass). I also understand why Google/LF doesn't open this up, its a money pit no doubt. OTOH, I see this causing more reposts of series than we would have had previously -- and we all love more versions of series. I'm not sure I see a solution, other than to read less email -- its not like I can even begin to read all email I get anyway.
On 4/24/26 21:17, Peter Zijlstra wrote: > On Fri, Apr 24, 2026 at 09:09:21PM +0200, David Hildenbrand (Arm) wrote: > >> Forcing contributors to reply to everything. I don't like that, in particular >> not as long as there is no way for contributors to run it early in private. >> >> In most cases, contributors just do the reasonable thing: incorporate the >> feedback in a new version. > > Yes, so I understand that you can run it locally, but then you get to > pay for the tokens (and sashiko burns tokens like dry grass). I also > understand why Google/LF doesn't open this up, its a money pit no doubt. Indeed. > > OTOH, I see this causing more reposts of series than we would have had > previously -- and we all love more versions of series. Fortunately, reposts are easy to review, at least for me, when tags are properly incorporated and the version diff properly describes the changes. > > I'm not sure I see a solution, other than to read less email -- its not > like I can even begin to read all email I get anyway. Sad but true. I consider the last AI review before something gets merged the most important one. -- Cheers, David
On Fri, 24 Apr 2026 11:50:03 -0700 Yosry Ahmed <yosry@kernel.org> wrote: > > >>> Preferences? > > >> > > >> Reply-to-author would likely be better as a first step. > > > > > > Why do you think so? > > > > The most important part for me is that authors are aware of the reports. > > > > Sending them as a mail forces people to publicly reply to the feedback, and at > > this point in time, I am not convinced that that is the right approach. > > But I imagine it's useful for reviewers to see Sashiko's feedback as > well (without having to go look on the website). It's possible that > Sashiko is right but the author isn't convinced, so getting more eyes > on the feedback would help. If Sashiko is wrong, it's still useful for > more people to see it and point it out, instead of Sashiko privately > misguiding the author, especially that reviewers are probably more > likely to tell if Sashiko is right or wrong. Also, Sashiko does like to find unrelated bugs in surrounding code. I saw two of these today. But whatever, one step at a time. My main interest at present is to stop having to send lame emails linking to the Sashiko reports!
On Fri, Apr 24, 2026 at 11:50:03AM -0700, Yosry Ahmed wrote: > But I imagine it's useful for reviewers to see Sashiko's feedback as > well (without having to go look on the website). That's why I have a script; if I press 'S' in mutt on an 0/n email, said script goes and does webrequest to sashiko and inserts the review comments into my local maildir as properly threaded replies to the series at hand. No looking at website needed.
On 24 Apr 2026, at 15:01, Peter Zijlstra wrote: > On Fri, Apr 24, 2026 at 11:50:03AM -0700, Yosry Ahmed wrote: > >> But I imagine it's useful for reviewers to see Sashiko's feedback as >> well (without having to go look on the website). > > That's why I have a script; if I press 'S' in mutt on an 0/n email, said > script goes and does webrequest to sashiko and inserts the review > comments into my local maildir as properly threaded replies to the > series at hand. > > No looking at website needed. Do you mind sharing that script? It looks like a great work flow. Thanks. Best Regards, Yan, Zi
On Fri, Apr 24, 2026 at 03:12:34PM -0400, Zi Yan wrote:
> On 24 Apr 2026, at 15:01, Peter Zijlstra wrote:
>
> > On Fri, Apr 24, 2026 at 11:50:03AM -0700, Yosry Ahmed wrote:
> >
> >> But I imagine it's useful for reviewers to see Sashiko's feedback as
> >> well (without having to go look on the website).
> >
> > That's why I have a script; if I press 'S' in mutt on an 0/n email, said
> > script goes and does webrequest to sashiko and inserts the review
> > comments into my local maildir as properly threaded replies to the
> > series at hand.
> >
> > No looking at website needed.
>
> Do you mind sharing that script? It looks like a great work flow. Thanks.
Sure, it is in a 'works-for-me' state and probably should get a once
over before you run it locally to make sure it fits your environment (at
the very least you should probably change my email address :-)
This is cobbled toghether from a script I got from Thomas and staring at
the JSON with generous hints from Gemini (because I can't write Python
at all).
.muttrc has:
macro index S "<pipe-entry>grep -i \"\^Message-ID:\" | cut -d: -f2- | tr -d \" <>\" | ~/bin/sashiko.py<enter>"
sashiko.py:
---
#!/usr/bin/env python3
import sys
import mailbox
import requests
from argparse import ArgumentParser
from email.message import EmailMessage
from email.utils import formatdate
from datetime import datetime
def getsash(url, msgid):
surl = f'{url.rstrip("/")}/api/patch'
try:
session = requests.Session()
session.headers.update({'User-Agent': 'sash2txt_0.2'})
resp = session.get(surl, params={'id': msgid}, timeout=30)
if resp.status_code == 404:
print(f"Error: Patch ID {msgid} not found (404).", file=sys.stderr)
return
resp.raise_for_status()
data = resp.json()
except requests.RequestException as ex:
print(f"Request failed: {ex}", file=sys.stderr)
return
patches = data.get('patches', [])
if not patches:
print(f"No patches found for (msgid).", file=sys.stderr)
return
msgids = {}
subjects = {}
for p in patches:
msgids[p['id']] = p['message_id']
subjects[p['id']] = p['subject']
reviews = data.get('reviews', [])
if not reviews:
print(f"No reviews found for {msgid}.", file=sys.stderr)
return
mdir = mailbox.Maildir('~/Maildir/');
for i, r in enumerate(reviews):
inline = r.get('inline_review', '') or ''
if not inline:
continue
author_name = r.get('author_name', 'Sashiko Reviewer')
author_email = r.get('author_email', 'noreply@sashiko.dev')
msgid = msgids[r['patch_id']]
subject = subjects[r['patch_id']]
# Create the Email object
msg = EmailMessage()
msg['Subject'] = f"Re: {subject}"
msg['From'] = f"{author_name} <{author_email}>"
msg['To'] = "peterz@infradead.org"
msg['Date'] = formatdate(localtime=True)
# The critical threading headers
msg['Message-ID'] = f"<review-{i}-{msgid}>"
msg['References'] = f"<{msgid}>"
msg['In-Reply-To'] = f"<{msgid}>"
msg.set_content(inline)
mdir.add(msg)
mdir.flush()
if __name__ == '__main__':
url = 'https://sashiko.dev/'
# Check if there's data in stdin (piped input)
if not sys.stdin.isatty():
for line in sys.stdin:
msgid = line.strip()
if msgid:
getsash(url, msgid)
else:
# Fallback to arguments if no pipe is detected
parser = ArgumentParser(description='Sashiko retriever to mbox')
parser.add_argument('msgid', nargs='?', help='Message id of the patch series')
args = parser.parse_args()
if args.msgid:
getsash(url, args.msgid)
else:
print("Usage: echo <msgid> | ./sashiko.py OR ./sashiko.py <msgid>", file=sys.stderr)
On Fri, Apr 24, 2026 at 09:22:51PM +0200, Peter Zijlstra wrote:
> mdir = mailbox.Maildir('~/Maildir/');
>
> for i, r in enumerate(reviews):
> inline = r.get('inline_review', '') or ''
> if not inline:
> continue
>
> author_name = r.get('author_name', 'Sashiko Reviewer')
> author_email = r.get('author_email', 'noreply@sashiko.dev')
>
> msgid = msgids[r['patch_id']]
> subject = subjects[r['patch_id']]
>
> # Create the Email object
> msg = EmailMessage()
> msg['Subject'] = f"Re: {subject}"
> msg['From'] = f"{author_name} <{author_email}>"
> msg['To'] = "peterz@infradead.org"
> msg['Date'] = formatdate(localtime=True)
>
> # The critical threading headers
> msg['Message-ID'] = f"<review-{i}-{msgid}>"
> msg['References'] = f"<{msgid}>"
> msg['In-Reply-To'] = f"<{msgid}>"
>
> msg.set_content(inline)
>
> mdir.add(msg)
>
> mdir.flush()
So Ideally I would have 'Reply-to' header set to the original sender and
added 'Cc' like the original email. However, AFAICT the JSON does not
contain this information, and while I could use the mdir object to find
the original message in my Inbox, this is incredibly slow.
I have a TODO to use python-notmuch to do this, but haven't gotten
around to doing this yet.
With that 'fixed' I could actually reply to these messages and it would
all 'just' work. For now I copy/paste when needed.
Peter Zijlstra <peterz@infradead.org> writes:
> On Fri, Apr 24, 2026 at 09:22:51PM +0200, Peter Zijlstra wrote:
>
>> mdir = mailbox.Maildir('~/Maildir/');
>>
>> for i, r in enumerate(reviews):
>> inline = r.get('inline_review', '') or ''
>> if not inline:
>> continue
>>
>> author_name = r.get('author_name', 'Sashiko Reviewer')
>> author_email = r.get('author_email', 'noreply@sashiko.dev')
>>
>> msgid = msgids[r['patch_id']]
>> subject = subjects[r['patch_id']]
>>
>> # Create the Email object
>> msg = EmailMessage()
>> msg['Subject'] = f"Re: {subject}"
>> msg['From'] = f"{author_name} <{author_email}>"
>> msg['To'] = "peterz@infradead.org"
>> msg['Date'] = formatdate(localtime=True)
>>
>> # The critical threading headers
>> msg['Message-ID'] = f"<review-{i}-{msgid}>"
>> msg['References'] = f"<{msgid}>"
>> msg['In-Reply-To'] = f"<{msgid}>"
>>
>> msg.set_content(inline)
>>
>> mdir.add(msg)
>>
>> mdir.flush()
>
> So Ideally I would have 'Reply-to' header set to the original sender and
> added 'Cc' like the original email. However, AFAICT the JSON does not
> contain this information, and while I could use the mdir object to find
> the original message in my Inbox, this is incredibly slow.
>
> I have a TODO to use python-notmuch to do this, but haven't gotten
> around to doing this yet.
>
> With that 'fixed' I could actually reply to these messages and it would
> all 'just' work. For now I copy/paste when needed.
I can add an api to return a json with the review and all
meta-information by the original msgid, will it be useful?
On Fri, Apr 24, 2026 at 08:03:14PM +0000, Roman Gushchin wrote:
> Peter Zijlstra <peterz@infradead.org> writes:
>
> > On Fri, Apr 24, 2026 at 09:22:51PM +0200, Peter Zijlstra wrote:
> >
> >> mdir = mailbox.Maildir('~/Maildir/');
> >>
> >> for i, r in enumerate(reviews):
> >> inline = r.get('inline_review', '') or ''
> >> if not inline:
> >> continue
> >>
> >> author_name = r.get('author_name', 'Sashiko Reviewer')
> >> author_email = r.get('author_email', 'noreply@sashiko.dev')
> >>
> >> msgid = msgids[r['patch_id']]
> >> subject = subjects[r['patch_id']]
> >>
> >> # Create the Email object
> >> msg = EmailMessage()
> >> msg['Subject'] = f"Re: {subject}"
> >> msg['From'] = f"{author_name} <{author_email}>"
> >> msg['To'] = "peterz@infradead.org"
> >> msg['Date'] = formatdate(localtime=True)
> >>
> >> # The critical threading headers
> >> msg['Message-ID'] = f"<review-{i}-{msgid}>"
> >> msg['References'] = f"<{msgid}>"
> >> msg['In-Reply-To'] = f"<{msgid}>"
> >>
> >> msg.set_content(inline)
> >>
> >> mdir.add(msg)
> >>
> >> mdir.flush()
> >
> > So Ideally I would have 'Reply-to' header set to the original sender and
> > added 'Cc' like the original email. However, AFAICT the JSON does not
> > contain this information, and while I could use the mdir object to find
> > the original message in my Inbox, this is incredibly slow.
> >
> > I have a TODO to use python-notmuch to do this, but haven't gotten
> > around to doing this yet.
> >
> > With that 'fixed' I could actually reply to these messages and it would
> > all 'just' work. For now I copy/paste when needed.
>
> I can add an api to return a json with the review and all
> meta-information by the original msgid, will it be useful?
Oh yes, that would be more convenient. Thanks!
On Fri, Apr 24, 2026 at 10:11:59PM +0200, Peter Zijlstra wrote:
> On Fri, Apr 24, 2026 at 08:03:14PM +0000, Roman Gushchin wrote:
> > Peter Zijlstra <peterz@infradead.org> writes:
> >
> > > On Fri, Apr 24, 2026 at 09:22:51PM +0200, Peter Zijlstra wrote:
> > >
> > >> mdir = mailbox.Maildir('~/Maildir/');
> > >>
> > >> for i, r in enumerate(reviews):
> > >> inline = r.get('inline_review', '') or ''
> > >> if not inline:
> > >> continue
> > >>
> > >> author_name = r.get('author_name', 'Sashiko Reviewer')
> > >> author_email = r.get('author_email', 'noreply@sashiko.dev')
> > >>
> > >> msgid = msgids[r['patch_id']]
> > >> subject = subjects[r['patch_id']]
> > >>
> > >> # Create the Email object
> > >> msg = EmailMessage()
> > >> msg['Subject'] = f"Re: {subject}"
> > >> msg['From'] = f"{author_name} <{author_email}>"
> > >> msg['To'] = "peterz@infradead.org"
> > >> msg['Date'] = formatdate(localtime=True)
> > >>
> > >> # The critical threading headers
> > >> msg['Message-ID'] = f"<review-{i}-{msgid}>"
> > >> msg['References'] = f"<{msgid}>"
> > >> msg['In-Reply-To'] = f"<{msgid}>"
> > >>
> > >> msg.set_content(inline)
> > >>
> > >> mdir.add(msg)
> > >>
> > >> mdir.flush()
> > >
> > > So Ideally I would have 'Reply-to' header set to the original sender and
> > > added 'Cc' like the original email. However, AFAICT the JSON does not
> > > contain this information, and while I could use the mdir object to find
> > > the original message in my Inbox, this is incredibly slow.
> > >
> > > I have a TODO to use python-notmuch to do this, but haven't gotten
> > > around to doing this yet.
> > >
> > > With that 'fixed' I could actually reply to these messages and it would
> > > all 'just' work. For now I copy/paste when needed.
> >
> > I can add an api to return a json with the review and all
> > meta-information by the original msgid, will it be useful?
>
> Oh yes, that would be more convenient. Thanks!
Roman, did you find time to do this? If not, I'll patiently wait :-)
On Fri, Apr 24, 2026 at 12:12 PM Zi Yan <ziy@nvidia.com> wrote: > > On 24 Apr 2026, at 15:01, Peter Zijlstra wrote: > > > On Fri, Apr 24, 2026 at 11:50:03AM -0700, Yosry Ahmed wrote: > > > >> But I imagine it's useful for reviewers to see Sashiko's feedback as > >> well (without having to go look on the website). > > > > That's why I have a script; if I press 'S' in mutt on an 0/n email, said > > script goes and does webrequest to sashiko and inserts the review > > comments into my local maildir as properly threaded replies to the > > series at hand. > > > > No looking at website needed. > > Do you mind sharing that script? It looks like a great work flow. Thanks. Yeah that sounds great :)
On Fri, 24 Apr 2026 12:15:30 -0700 Yosry Ahmed <yosry@kernel.org> wrote: > On Fri, Apr 24, 2026 at 12:12 PM Zi Yan <ziy@nvidia.com> wrote: > > > > On 24 Apr 2026, at 15:01, Peter Zijlstra wrote: > > > > > On Fri, Apr 24, 2026 at 11:50:03AM -0700, Yosry Ahmed wrote: > > > > > >> But I imagine it's useful for reviewers to see Sashiko's feedback as > > >> well (without having to go look on the website). > > > > > > That's why I have a script; if I press 'S' in mutt on an 0/n email, said > > > script goes and does webrequest to sashiko and inserts the review > > > comments into my local maildir as properly threaded replies to the > > > series at hand. > > > > > > No looking at website needed. > > > > Do you mind sharing that script? It looks like a great work flow. Thanks. > > Yeah that sounds great :) FWIW my workflow was also similar to PeterZ [1], except I use hkml's Sashiko integration. The hkml integration requires two key strokes to get the Sashiko review, so I think Peter's script is superior. It provides a draft [2] for sharing full Sashiko review that I can further writeup my comment and directly send, though. It is helpful for asking the author their opinion about Sashiko's review, with a fine pointer to the question. So I initially started opening web browser to check Sashiko review. After that I used hkml to read the review on terminal and copy-pasted the output to ask question to authors with quote of Sashiko review. Finally I was able to just forward Sashiko review together with my questions to authors. Each step of the changes made my life much easier. DAMON is opted in to the Sashiko's mail reply service from the beginning, and recently Sashiko started the mail delivery. Now I don't need to use the hkml integration, and I feel my life has been even much easier than before. [1] https://github.com/sjp38/hackermail/blob/master/USAGE.md#sashikodev [2] https://github.com/sjp38/hackermail/blob/master/USAGE.md#forwarding-sashikodev-statuscomments-to-mailing-list Thanks, SJ
On 04-24 07:31, Andrew Morton wrote: > On Fri, 24 Apr 2026 16:20:55 +0200 "David Hildenbrand (Arm)" <david@kernel.org> wrote: > > > On 4/24/26 16:15, Andrew Morton wrote: > > > On Fri, 24 Apr 2026 13:37:04 +0000 Pasha Tatashin <pasha.tatashin@soleen.com> wrote: > > > > > >> On 04-24 06:30, Andrew Morton wrote: > > >>> > > >>> > > >>> Sashiko questions: > > >>> https://sashiko.dev/#/patchset/20260424062528.71951-1-lance.yang@linux.dev > > >>> > > >>> (I never know if my Sashiko emails are welcome/useful. Maybe Sashiko > > >>> said the same stuff about v9 and it's all wrong. But better safe than > > >>> sorry!) > > >> > > >> These emails are helpful; but, I do not believe you should have to > > >> manually follow up with a link to every new patch series. > > >> > > >> Perhaps Sashiko could automatically send a summary email in response to > > >> the cover letter, or provide a link once the reviews are complete. For > > >> the kexec ML, we opted-in with Roman to receive automated emails from > > >> sashiko. > > > > > > Yep. I'd be OK with an automatic reply-to-all. Maybe some won't like > > > that. > > > > > > An alternative I've discussed with Roman is an automated > > > reply-to-author with a cc to a dedicated list (we could use mm-commits > > > for now). > > > > > > Preferences? > > > > Reply-to-author would likely be better as a first step. > > Why do you think so? > > Pasha, is it too early to determine how reply-to-all is working out for > kexec? It is too early, we have opted-in only at the end of last week. Pasha
© 2016 - 2026 Red Hat, Inc.