docs: fault-injection: document cache-filter feature for failslab

[PATCH] docs: fault-injection: document cache-filter feature for failslab

Posted by Breno Leitao 2 months, 3 weeks ago

The failslab fault injection mechanism has an undocumented capability
that provides significant utility in testing and debugging. This feature,
introduced in commit 4c13dd3b48fcb ("failslab: add ability to filter slab
caches"), allows for targeted error injection into specific slab caches.

However, it was inadvertently left undocumented at the time of its
implementation.

Add documentation for the cache-filter feature in the failslab mode
description. Also, providing a practical example demonstrating how to
use cache-filter to inject failures specifically when allocating socket
buffers (skbs).

Signed-off-by: Breno Leitao <leitao@debian.org>
---
 .../fault-injection/fault-injection.rst       | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/Documentation/fault-injection/fault-injection.rst b/Documentation/fault-injection/fault-injection.rst
index 70380a2a01b4..07c24710bd21 100644
--- a/Documentation/fault-injection/fault-injection.rst
+++ b/Documentation/fault-injection/fault-injection.rst
@@ -141,6 +141,14 @@ configuration of fault-injection capabilities.
 	default is 'Y', setting it to 'N' will also inject failures into
 	highmem/user allocations (__GFP_HIGHMEM allocations).
 
+- /sys/kernel/debug/failslab/cache-filter
+	Format: { 'Y' | 'N' }
+
+        default is 'N', setting it to 'Y' will only inject failures when
+        objects are requests from certain caches.
+
+        Select the cache by writing '1' to /sys/kernel/slab/<cache>/failslab:
+
 - /sys/kernel/debug/failslab/ignore-gfp-wait:
 - /sys/kernel/debug/fail_page_alloc/ignore-gfp-wait:
 
@@ -459,6 +467,18 @@ Application Examples
     losetup -d $DEVICE
     rm testfile.img
 
+------------------------------------------------------------------------------
+
+- Inject only skbuff allocation failures ::
+
+    # mark skbuff_head_cache as faulty
+    echo 1 > /sys/kernel/slab/skbuff_head_cache/failslab
+    # Turn on cache filter (off by default)
+    echo 1 > /sys/kernel/debug/failslab/cache-filter
+    # Turn on fault injection
+    echo 1 > /sys/kernel/debug/failslab/times
+    echo 1 > /sys/kernel/debug/failslab/probability
+
 
 Tool to run command with failslab or fail_page_alloc
 ----------------------------------------------------
-- 
2.43.0

Re: [PATCH] docs: fault-injection: document cache-filter feature for failslab

Posted by Jonathan Corbet 2 months, 1 week ago

Breno Leitao <leitao@debian.org> writes:

> The failslab fault injection mechanism has an undocumented capability
> that provides significant utility in testing and debugging. This feature,
> introduced in commit 4c13dd3b48fcb ("failslab: add ability to filter slab
> caches"), allows for targeted error injection into specific slab caches.
>
> However, it was inadvertently left undocumented at the time of its
> implementation.
>
> Add documentation for the cache-filter feature in the failslab mode
> description. Also, providing a practical example demonstrating how to
> use cache-filter to inject failures specifically when allocating socket
> buffers (skbs).
>
> Signed-off-by: Breno Leitao <leitao@debian.org>
> ---
>  .../fault-injection/fault-injection.rst       | 20 +++++++++++++++++++
>  1 file changed, 20 insertions(+)

I've applied this, thanks.

It seems to me that the fault-injection docs should really move under
dev-tools; does anybody object to that?

Thanks,

jon

Re: [PATCH] docs: fault-injection: document cache-filter feature for failslab

Posted by Breno Leitao 2 months, 1 week ago

On Wed, Aug 07, 2024 at 12:54:31PM -0600, Jonathan Corbet wrote:
> Breno Leitao <leitao@debian.org> writes:
> 
> > The failslab fault injection mechanism has an undocumented capability
> > that provides significant utility in testing and debugging. This feature,
> > introduced in commit 4c13dd3b48fcb ("failslab: add ability to filter slab
> > caches"), allows for targeted error injection into specific slab caches.
> >
> > However, it was inadvertently left undocumented at the time of its
> > implementation.
> >
> > Add documentation for the cache-filter feature in the failslab mode
> > description. Also, providing a practical example demonstrating how to
> > use cache-filter to inject failures specifically when allocating socket
> > buffers (skbs).
> >
> > Signed-off-by: Breno Leitao <leitao@debian.org>
> > ---
> >  .../fault-injection/fault-injection.rst       | 20 +++++++++++++++++++
> >  1 file changed, 20 insertions(+)
> 
> I've applied this, thanks.
> 
> It seems to me that the fault-injection docs should really move under
> dev-tools; does anybody object to that?

I don't object at all. This doc should be inside dev-tools.

Thanks Jonathan,
--breno

Re: [PATCH] docs: fault-injection: document cache-filter feature for failslab

Posted by Akinobu Mita 2 months, 1 week ago

2024年8月8日(木) 16:41 Breno Leitao <leitao@debian.org>:
>
> On Wed, Aug 07, 2024 at 12:54:31PM -0600, Jonathan Corbet wrote:
> > Breno Leitao <leitao@debian.org> writes:
> >
> > > The failslab fault injection mechanism has an undocumented capability
> > > that provides significant utility in testing and debugging. This feature,
> > > introduced in commit 4c13dd3b48fcb ("failslab: add ability to filter slab
> > > caches"), allows for targeted error injection into specific slab caches.
> > >
> > > However, it was inadvertently left undocumented at the time of its
> > > implementation.
> > >
> > > Add documentation for the cache-filter feature in the failslab mode
> > > description. Also, providing a practical example demonstrating how to
> > > use cache-filter to inject failures specifically when allocating socket
> > > buffers (skbs).
> > >
> > > Signed-off-by: Breno Leitao <leitao@debian.org>
> > > ---
> > >  .../fault-injection/fault-injection.rst       | 20 +++++++++++++++++++
> > >  1 file changed, 20 insertions(+)
> >
> > I've applied this, thanks.
> >
> > It seems to me that the fault-injection docs should really move under
> > dev-tools; does anybody object to that?
>
> I don't object at all. This doc should be inside dev-tools.

I also agree with moving the document.

Re: [PATCH] docs: fault-injection: document cache-filter feature for failslab

Posted by Akinobu Mita 2 months, 3 weeks ago

2024年7月26日(金) 21:09 Breno Leitao <leitao@debian.org>:
>
> The failslab fault injection mechanism has an undocumented capability
> that provides significant utility in testing and debugging. This feature,
> introduced in commit 4c13dd3b48fcb ("failslab: add ability to filter slab
> caches"), allows for targeted error injection into specific slab caches.
>
> However, it was inadvertently left undocumented at the time of its
> implementation.
>
> Add documentation for the cache-filter feature in the failslab mode
> description. Also, providing a practical example demonstrating how to
> use cache-filter to inject failures specifically when allocating socket
> buffers (skbs).
>
> Signed-off-by: Breno Leitao <leitao@debian.org>
> ---
>  .../fault-injection/fault-injection.rst       | 20 +++++++++++++++++++
>  1 file changed, 20 insertions(+)
>
> diff --git a/Documentation/fault-injection/fault-injection.rst b/Documentation/fault-injection/fault-injection.rst
> index 70380a2a01b4..07c24710bd21 100644
> --- a/Documentation/fault-injection/fault-injection.rst
> +++ b/Documentation/fault-injection/fault-injection.rst
> @@ -141,6 +141,14 @@ configuration of fault-injection capabilities.
>         default is 'Y', setting it to 'N' will also inject failures into
>         highmem/user allocations (__GFP_HIGHMEM allocations).
>
> +- /sys/kernel/debug/failslab/cache-filter
> +       Format: { 'Y' | 'N' }
> +
> +        default is 'N', setting it to 'Y' will only inject failures when
> +        objects are requests from certain caches.
> +
> +        Select the cache by writing '1' to /sys/kernel/slab/<cache>/failslab:
> +
>  - /sys/kernel/debug/failslab/ignore-gfp-wait:
>  - /sys/kernel/debug/fail_page_alloc/ignore-gfp-wait:
>
> @@ -459,6 +467,18 @@ Application Examples
>      losetup -d $DEVICE
>      rm testfile.img
>
> +------------------------------------------------------------------------------
> +
> +- Inject only skbuff allocation failures ::
> +
> +    # mark skbuff_head_cache as faulty
> +    echo 1 > /sys/kernel/slab/skbuff_head_cache/failslab
> +    # Turn on cache filter (off by default)
> +    echo 1 > /sys/kernel/debug/failslab/cache-filter
> +    # Turn on fault injection
> +    echo 1 > /sys/kernel/debug/failslab/times
> +    echo 1 > /sys/kernel/debug/failslab/probability
> +
>
>  Tool to run command with failslab or fail_page_alloc
>  ----------------------------------------------------

Looks good.

Reviewed-by: Akinobu Mita <akinobu.mita@gmail.com>