[v1] x86/sgx: Add code to inject hwpoison into SGX memory

[PATCH] x86/sgx: Add code to inject hwpoison into SGX memory

Posted by Thomas Tai 3 years, 6 months ago

Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
address validation), add a similar code in hwpoison_inject function to
check if the address is located in SGX Memory. The error will then be
handled by the arch_memory_failure function in the SGX driver.

Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
---
 Documentation/mm/hwpoison.rst | 44 +++++++++++++++++++++++++++++++++++
 mm/hwpoison-inject.c          |  3 +++
 2 files changed, 47 insertions(+)

diff --git a/Documentation/mm/hwpoison.rst b/Documentation/mm/hwpoison.rst
index b9d5253c1305..8a542aca4744 100644
--- a/Documentation/mm/hwpoison.rst
+++ b/Documentation/mm/hwpoison.rst
@@ -162,6 +162,50 @@ Testing
 
   Some portable hwpoison test programs in mce-test, see below.
 
+* Special notes for injection into SGX enclaves
+
+  1) Determine physical address of enclave page
+
+	dmesg | grep "sgx: EPC"
+
+	sgx: EPC section 0x8000c00000-0x807f7fffff
+	sgx: EPC section 0x10000c00000-0x1007fffffff
+
+  2) Convert the EPC address to page frame number.
+
+	For 4K page size, the page frame number for 0x8000c00000 is
+	0x8000c00000 / 0x1000 = 0x8000c00.
+
+  3) Trace memory_failure
+
+	echo nop > /sys/kernel/tracing/current_tracer
+	echo *memory_failure > /sys/kernel/tracing/set_ftrace_filter
+	echo function > /sys/kernel/tracing/current_tracer
+
+  4) Inject a memory error
+
+	modprobe hwpoison-inject
+	echo "0x8000c00" > /sys/kernel/debug/hwpoison/corrupt-pfn
+
+  5) Check the trace output
+
+	cat /sys/kernel/tracing/trace
+
+	# tracer: function
+	#
+	# entries-in-buffer/entries-written: 2/2   #P:128
+	#
+	#                            _-----=> irqs-off
+	#                           / _----=> need-resched
+	#                          | / _---=> hardirq/softirq
+	#                          || / _--=> preempt-depth
+	#                          ||| / _-=> migrate-disable
+	#                          |||| /     delay
+	#       TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
+	#          | |         |   |||||     |         |
+	        bash-12167   [002] .....   113.136808: memory_failure<-simple_attr_write
+	        bash-12167   [002] .....   113.136810: arch_memory_failure<-memory_failure
+
 References
 ==========
 
diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c
index 65e242b5a432..8134dc983699 100644
--- a/mm/hwpoison-inject.c
+++ b/mm/hwpoison-inject.c
@@ -21,6 +21,9 @@ static int hwpoison_inject(void *data, u64 val)
 	if (!capable(CAP_SYS_ADMIN))
 		return -EPERM;
 
+	if (arch_is_platform_page(pfn << PAGE_SHIFT))
+		goto inject;
+
 	if (!pfn_valid(pfn))
 		return -ENXIO;
 
-- 
2.31.1

Re: [PATCH] x86/sgx: Add code to inject hwpoison into SGX memory

Posted by Jarkko Sakkinen 3 years, 6 months ago

On Tue, Sep 27, 2022 at 02:34:11PM -0400, Thomas Tai wrote:
> Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
> address validation), add a similar code in hwpoison_inject function to
> check if the address is located in SGX Memory. The error will then be
> handled by the arch_memory_failure function in the SGX driver.
> 
> Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
> ---
>  Documentation/mm/hwpoison.rst | 44 +++++++++++++++++++++++++++++++++++
>  mm/hwpoison-inject.c          |  3 +++
>  2 files changed, 47 insertions(+)
> 
> diff --git a/Documentation/mm/hwpoison.rst b/Documentation/mm/hwpoison.rst
> index b9d5253c1305..8a542aca4744 100644
> --- a/Documentation/mm/hwpoison.rst
> +++ b/Documentation/mm/hwpoison.rst
> @@ -162,6 +162,50 @@ Testing
>  
>    Some portable hwpoison test programs in mce-test, see below.
>  
> +* Special notes for injection into SGX enclaves
> +
> +  1) Determine physical address of enclave page
> +
> +	dmesg | grep "sgx: EPC"
> +
> +	sgx: EPC section 0x8000c00000-0x807f7fffff
> +	sgx: EPC section 0x10000c00000-0x1007fffffff
> +
> +  2) Convert the EPC address to page frame number.
> +
> +	For 4K page size, the page frame number for 0x8000c00000 is
> +	0x8000c00000 / 0x1000 = 0x8000c00.
> +
> +  3) Trace memory_failure
> +
> +	echo nop > /sys/kernel/tracing/current_tracer
> +	echo *memory_failure > /sys/kernel/tracing/set_ftrace_filter
> +	echo function > /sys/kernel/tracing/current_tracer
> +
> +  4) Inject a memory error
> +
> +	modprobe hwpoison-inject
> +	echo "0x8000c00" > /sys/kernel/debug/hwpoison/corrupt-pfn
> +
> +  5) Check the trace output
> +
> +	cat /sys/kernel/tracing/trace
> +
> +	# tracer: function
> +	#
> +	# entries-in-buffer/entries-written: 2/2   #P:128
> +	#
> +	#                            _-----=> irqs-off
> +	#                           / _----=> need-resched
> +	#                          | / _---=> hardirq/softirq
> +	#                          || / _--=> preempt-depth
> +	#                          ||| / _-=> migrate-disable
> +	#                          |||| /     delay
> +	#       TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
> +	#          | |         |   |||||     |         |
> +	        bash-12167   [002] .....   113.136808: memory_failure<-simple_attr_write
> +	        bash-12167   [002] .....   113.136810: arch_memory_failure<-memory_failure
> +
>  References
>  ==========
>  
> diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c
> index 65e242b5a432..8134dc983699 100644
> --- a/mm/hwpoison-inject.c
> +++ b/mm/hwpoison-inject.c
> @@ -21,6 +21,9 @@ static int hwpoison_inject(void *data, u64 val)
>  	if (!capable(CAP_SYS_ADMIN))
>  		return -EPERM;
>  
> +	if (arch_is_platform_page(pfn << PAGE_SHIFT))
> +		goto inject;
> +
>  	if (!pfn_valid(pfn))
>  		return -ENXIO;
>  
> -- 
> 2.31.1
> 

Hey, this is really useful addition. Thanks for doing this.

Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>

BR, Jarkko

RE: [PATCH] x86/sgx: Add code to inject hwpoison into SGX memory

Posted by Thomas Tai 3 years, 6 months ago

> -----Original Message-----
> From: Jarkko Sakkinen <jarkko@kernel.org>
> Sent: September 30, 2022 5:44 PM
> To: Thomas Tai <thomas.tai@oracle.com>
> Cc: tony.luck@intel.com; dave.hansen@linux.intel.com;
> reinette.chatre@intel.co; naoya.horiguchi@nec.com; linmiaohe@huawei.com;
> akpm@linux-foundation.org; linux-mm@kvack.org; linux-
> kernel@vger.kernel.org
> Subject: Re: [PATCH] x86/sgx: Add code to inject hwpoison into SGX memory
> 
> On Tue, Sep 27, 2022 at 02:34:11PM -0400, Thomas Tai wrote:
> > Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
> > address validation), add a similar code in hwpoison_inject function to
> > check if the address is located in SGX Memory. The error will then be
> > handled by the arch_memory_failure function in the SGX driver.
> >
> > Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
> > ---
> >  Documentation/mm/hwpoison.rst | 44
> +++++++++++++++++++++++++++++++++++
> >  mm/hwpoison-inject.c          |  3 +++
> >  2 files changed, 47 insertions(+)
> >
> > diff --git a/Documentation/mm/hwpoison.rst
> b/Documentation/mm/hwpoison.rst
> > index b9d5253c1305..8a542aca4744 100644
> > --- a/Documentation/mm/hwpoison.rst
> > +++ b/Documentation/mm/hwpoison.rst
> > @@ -162,6 +162,50 @@ Testing
> >
> >    Some portable hwpoison test programs in mce-test, see below.
> >
> > +* Special notes for injection into SGX enclaves
> > +
> > +  1) Determine physical address of enclave page
> > +
> > +	dmesg | grep "sgx: EPC"
> > +
> > +	sgx: EPC section 0x8000c00000-0x807f7fffff
> > +	sgx: EPC section 0x10000c00000-0x1007fffffff
> > +
> > +  2) Convert the EPC address to page frame number.
> > +
> > +	For 4K page size, the page frame number for 0x8000c00000 is
> > +	0x8000c00000 / 0x1000 = 0x8000c00.
> > +
> > +  3) Trace memory_failure
> > +
> > +	echo nop > /sys/kernel/tracing/current_tracer
> > +	echo *memory_failure > /sys/kernel/tracing/set_ftrace_filter
> > +	echo function > /sys/kernel/tracing/current_tracer
> > +
> > +  4) Inject a memory error
> > +
> > +	modprobe hwpoison-inject
> > +	echo "0x8000c00" > /sys/kernel/debug/hwpoison/corrupt-pfn
> > +
> > +  5) Check the trace output
> > +
> > +	cat /sys/kernel/tracing/trace
> > +
> > +	# tracer: function
> > +	#
> > +	# entries-in-buffer/entries-written: 2/2   #P:128
> > +	#
> > +	#                            _-----=> irqs-off
> > +	#                           / _----=> need-resched
> > +	#                          | / _---=> hardirq/softirq
> > +	#                          || / _--=> preempt-depth
> > +	#                          ||| / _-=> migrate-disable
> > +	#                          |||| /     delay
> > +	#       TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
> > +	#          | |         |   |||||     |         |
> > +	        bash-12167   [002] .....   113.136808: memory_failure<-
> simple_attr_write
> > +	        bash-12167   [002] .....   113.136810: arch_memory_failure<-
> memory_failure
> > +
> >  References
> >  ==========
> >
> > diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c
> > index 65e242b5a432..8134dc983699 100644
> > --- a/mm/hwpoison-inject.c
> > +++ b/mm/hwpoison-inject.c
> > @@ -21,6 +21,9 @@ static int hwpoison_inject(void *data, u64 val)
> >  	if (!capable(CAP_SYS_ADMIN))
> >  		return -EPERM;
> >
> > +	if (arch_is_platform_page(pfn << PAGE_SHIFT))
> > +		goto inject;
> > +
> >  	if (!pfn_valid(pfn))
> >  		return -ENXIO;
> >
> > --
> > 2.31.1
> >
> 
> Hey, this is really useful addition. Thanks for doing this.
> 
> Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>

Thanks, Jarkko, for your review.

Thomas

> 
> BR, Jarkko

Re: [PATCH] x86/sgx: Add code to inject hwpoison into SGX memory

Posted by Miaohe Lin 3 years, 6 months ago

On 2022/9/28 2:34, Thomas Tai wrote:
> Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
> address validation), add a similar code in hwpoison_inject function to
> check if the address is located in SGX Memory. The error will then be
> handled by the arch_memory_failure function in the SGX driver.
> 
> Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
> ---
>  Documentation/mm/hwpoison.rst | 44 +++++++++++++++++++++++++++++++++++
>  mm/hwpoison-inject.c          |  3 +++
>  2 files changed, 47 insertions(+)
> 
> diff --git a/Documentation/mm/hwpoison.rst b/Documentation/mm/hwpoison.rst
> index b9d5253c1305..8a542aca4744 100644
> --- a/Documentation/mm/hwpoison.rst
> +++ b/Documentation/mm/hwpoison.rst
> @@ -162,6 +162,50 @@ Testing
>  
>    Some portable hwpoison test programs in mce-test, see below.
>  
> +* Special notes for injection into SGX enclaves
> +
> +  1) Determine physical address of enclave page
> +
> +	dmesg | grep "sgx: EPC"
> +
> +	sgx: EPC section 0x8000c00000-0x807f7fffff
> +	sgx: EPC section 0x10000c00000-0x1007fffffff
> +
> +  2) Convert the EPC address to page frame number.
> +
> +	For 4K page size, the page frame number for 0x8000c00000 is
> +	0x8000c00000 / 0x1000 = 0x8000c00.
> +
> +  3) Trace memory_failure
> +
> +	echo nop > /sys/kernel/tracing/current_tracer
> +	echo *memory_failure > /sys/kernel/tracing/set_ftrace_filter
> +	echo function > /sys/kernel/tracing/current_tracer
> +
> +  4) Inject a memory error
> +
> +	modprobe hwpoison-inject
> +	echo "0x8000c00" > /sys/kernel/debug/hwpoison/corrupt-pfn
> +
> +  5) Check the trace output
> +
> +	cat /sys/kernel/tracing/trace
> +
> +	# tracer: function
> +	#
> +	# entries-in-buffer/entries-written: 2/2   #P:128
> +	#
> +	#                            _-----=> irqs-off
> +	#                           / _----=> need-resched
> +	#                          | / _---=> hardirq/softirq
> +	#                          || / _--=> preempt-depth
> +	#                          ||| / _-=> migrate-disable
> +	#                          |||| /     delay
> +	#       TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
> +	#          | |         |   |||||     |         |
> +	        bash-12167   [002] .....   113.136808: memory_failure<-simple_attr_write
> +	        bash-12167   [002] .....   113.136810: arch_memory_failure<-memory_failure
> +
>  References
>  ==========
>  
> diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c
> index 65e242b5a432..8134dc983699 100644
> --- a/mm/hwpoison-inject.c
> +++ b/mm/hwpoison-inject.c
> @@ -21,6 +21,9 @@ static int hwpoison_inject(void *data, u64 val)
>  	if (!capable(CAP_SYS_ADMIN))
>  		return -EPERM;
>  
> +	if (arch_is_platform_page(pfn << PAGE_SHIFT))

Maybe it's better to add a comment above. Anyway, this patch looks good to me. Thanks.

Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>


Thanks,
Miaohe Lin

RE: [PATCH] x86/sgx: Add code to inject hwpoison into SGX memory

Posted by Thomas Tai 3 years, 6 months ago

> -----Original Message-----
> From: Miaohe Lin <linmiaohe@huawei.com>
> Sent: September 28, 2022 4:00 AM
> To: Thomas Tai <thomas.tai@oracle.com>
> Cc: tony.luck@intel.com; dave.hansen@linux.intel.com; jarkko@kernel.org;
> reinette.chatre@intel.co; naoya.horiguchi@nec.com; akpm@linux-
> foundation.org; linux-mm@kvack.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH] x86/sgx: Add code to inject hwpoison into SGX memory
> 
> On 2022/9/28 2:34, Thomas Tai wrote:
> > Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
> > address validation), add a similar code in hwpoison_inject function to
> > check if the address is located in SGX Memory. The error will then be
> > handled by the arch_memory_failure function in the SGX driver.
> >
> > Signed-off-by: Thomas Tai <thomas.tai@oracle.com>
> > ---
> >  Documentation/mm/hwpoison.rst | 44
> +++++++++++++++++++++++++++++++++++
> >  mm/hwpoison-inject.c          |  3 +++
> >  2 files changed, 47 insertions(+)
> >
> > diff --git a/Documentation/mm/hwpoison.rst
> b/Documentation/mm/hwpoison.rst
> > index b9d5253c1305..8a542aca4744 100644
> > --- a/Documentation/mm/hwpoison.rst
> > +++ b/Documentation/mm/hwpoison.rst
> > @@ -162,6 +162,50 @@ Testing
> >
> >    Some portable hwpoison test programs in mce-test, see below.
> >
> > +* Special notes for injection into SGX enclaves
> > +
> > +  1) Determine physical address of enclave page
> > +
> > +	dmesg | grep "sgx: EPC"
> > +
> > +	sgx: EPC section 0x8000c00000-0x807f7fffff
> > +	sgx: EPC section 0x10000c00000-0x1007fffffff
> > +
> > +  2) Convert the EPC address to page frame number.
> > +
> > +	For 4K page size, the page frame number for 0x8000c00000 is
> > +	0x8000c00000 / 0x1000 = 0x8000c00.
> > +
> > +  3) Trace memory_failure
> > +
> > +	echo nop > /sys/kernel/tracing/current_tracer
> > +	echo *memory_failure > /sys/kernel/tracing/set_ftrace_filter
> > +	echo function > /sys/kernel/tracing/current_tracer
> > +
> > +  4) Inject a memory error
> > +
> > +	modprobe hwpoison-inject
> > +	echo "0x8000c00" > /sys/kernel/debug/hwpoison/corrupt-pfn
> > +
> > +  5) Check the trace output
> > +
> > +	cat /sys/kernel/tracing/trace
> > +
> > +	# tracer: function
> > +	#
> > +	# entries-in-buffer/entries-written: 2/2   #P:128
> > +	#
> > +	#                            _-----=> irqs-off
> > +	#                           / _----=> need-resched
> > +	#                          | / _---=> hardirq/softirq
> > +	#                          || / _--=> preempt-depth
> > +	#                          ||| / _-=> migrate-disable
> > +	#                          |||| /     delay
> > +	#       TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
> > +	#          | |         |   |||||     |         |
> > +	        bash-12167   [002] .....   113.136808: memory_failure<-
> simple_attr_write
> > +	        bash-12167   [002] .....   113.136810: arch_memory_failure<-
> memory_failure
> > +
> >  References
> >  ==========
> >
> > diff --git a/mm/hwpoison-inject.c b/mm/hwpoison-inject.c
> > index 65e242b5a432..8134dc983699 100644
> > --- a/mm/hwpoison-inject.c
> > +++ b/mm/hwpoison-inject.c
> > @@ -21,6 +21,9 @@ static int hwpoison_inject(void *data, u64 val)
> >  	if (!capable(CAP_SYS_ADMIN))
> >  		return -EPERM;
> >
> > +	if (arch_is_platform_page(pfn << PAGE_SHIFT))
> 
> Maybe it's better to add a comment above. Anyway, this patch looks good to
> me. Thanks.

Thanks, Miaohe, for your suggestion. I will add a comment and resend the patch.

Thanks,
Thomas

> 
> Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
> 
> 
> Thanks,
> Miaohe Lin

RE: [PATCH] x86/sgx: Add code to inject hwpoison into SGX memory

Posted by Luck, Tony 3 years, 6 months ago

> Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
> address validation), add a similar code in hwpoison_inject function to
> check if the address is located in SGX Memory. The error will then be
> handled by the arch_memory_failure function in the SGX driver.

Looks good to me.

[I wish that Linux hadn't called this "hwpoison_inject()" because there's nothing
hardware related about this injection. But you are just an innocent consumer of
that poor naming choice]

Reviewed-by: Tony Luck <tony.luck@intel.com>

-Tony

RE: [PATCH] x86/sgx: Add code to inject hwpoison into SGX memory

Posted by Thomas Tai 3 years, 6 months ago

> -----Original Message-----
> From: Luck, Tony <tony.luck@intel.com>
> Sent: September 27, 2022 4:35 PM
> To: Thomas Tai <thomas.tai@oracle.com>; dave.hansen@linux.intel.com;
> jarkko@kernel.org; reinette.chatre@intel.co; naoya.horiguchi@nec.com;
> linmiaohe@huawei.com; akpm@linux-foundation.org; linux-mm@kvack.org;
> linux-kernel@vger.kernel.org; Thomas Tai <thomas.tai@oracle.com>
> Subject: RE: [PATCH] x86/sgx: Add code to inject hwpoison into SGX memory
> 
> > Inspired by commit c6acb1e7bf46 (x86/sgx: Add hook to error injection
> > address validation), add a similar code in hwpoison_inject function to
> > check if the address is located in SGX Memory. The error will then be
> > handled by the arch_memory_failure function in the SGX driver.
> 
> Looks good to me.
> 
> [I wish that Linux hadn't called this "hwpoison_inject()" because there's nothing
> hardware related about this injection. But you are just an innocent consumer of
> that poor naming choice]
> 
> Reviewed-by: Tony Luck <tony.luck@intel.com>

Thanks, Tony, for your review.

Cheers,
Thomas

> 
> -Tony