From patchwork Tue Apr 28 14:03:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Naveen N. Rao" X-Patchwork-Id: 1278512 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49BNlG0VKFz9sT1 for ; Wed, 29 Apr 2020 00:07:54 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 49BNlF5xsfzDqvx for ; Wed, 29 Apr 2020 00:07:53 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=naveen.n.rao@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 49BNgg1PgmzDq83 for ; Wed, 29 Apr 2020 00:04:46 +1000 (AEST) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 03SDVfw9060825; Tue, 28 Apr 2020 10:04:16 -0400 Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0b-001b2d01.pphosted.com with ESMTP id 30pjxuxnue-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 28 Apr 2020 10:04:15 -0400 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.16.0.27/8.16.0.27) with SMTP id 03SE2hSQ019522; Tue, 28 Apr 2020 14:04:14 GMT Received: from b06cxnps4074.portsmouth.uk.ibm.com (d06relay11.portsmouth.uk.ibm.com [9.149.109.196]) by ppma04fra.de.ibm.com with ESMTP id 30mcu58wjd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 28 Apr 2020 14:04:13 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 03SE4BGH37421278 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 28 Apr 2020 14:04:11 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6C55411C04C; Tue, 28 Apr 2020 14:04:11 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 12DD411C04A; Tue, 28 Apr 2020 14:04:10 +0000 (GMT) Received: from naverao1-tp.ibmuc.com (unknown [9.85.73.214]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 28 Apr 2020 14:04:09 +0000 (GMT) From: "Naveen N. Rao" To: Subject: [PATCH v2 1/2] powerpc/ftrace: Simplify error checking when patching instructions Date: Tue, 28 Apr 2020 19:33:58 +0530 Message-Id: <46df7406ed1a426800de687324262e344fd19434.1588082133.git.naveen.n.rao@linux.vnet.ibm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.676 definitions=2020-04-28_09:2020-04-28, 2020-04-28 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 phishscore=0 adultscore=0 malwarescore=0 impostorscore=0 mlxlogscore=999 bulkscore=0 lowpriorityscore=0 mlxscore=0 suspectscore=1 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2004280102 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Steven Rostedt Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Introduce a macro PATCH_INSN_OR_GOTO() to simplify instruction patching, and to make the error messages more uniform and useful: - print an error message that includes the original return value - print the function name and line numbers, so that the offending location is clear - goto a label which always return -EPERM, which ftrace_bug() expects for proper error handling Also eliminate use of patch_branch() since most such uses already call create_branch() for error checking before patching. Instead, use the return value from create_branch() with PATCH_INSN_OR_GOTO(). Signed-off-by: Naveen N. Rao --- arch/powerpc/kernel/trace/ftrace.c | 96 +++++++++++++++++++----------- 1 file changed, 60 insertions(+), 36 deletions(-) diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c index 7ea0ca044b65..63edbd48af42 100644 --- a/arch/powerpc/kernel/trace/ftrace.c +++ b/arch/powerpc/kernel/trace/ftrace.c @@ -31,6 +31,17 @@ #ifdef CONFIG_DYNAMIC_FTRACE +#define PATCH_INSN_OR_GOTO(addr, instr, label) \ +do { \ + int rc = patch_instruction((unsigned int *)(addr), instr); \ + if (rc) { \ + pr_err("%s:%d Error patching instruction at 0x%pK (%pS): %d\n", \ + __func__, __LINE__, \ + (void *)(addr), (void *)(addr), rc); \ + goto label; \ + } \ +} while (0) + /* * We generally only have a single long_branch tramp and at most 2 or 3 plt * tramps generated. But, we don't use the plt tramps currently. We also allot @@ -78,10 +89,12 @@ ftrace_modify_code(unsigned long ip, unsigned int old, unsigned int new) } /* replace the text with the new text */ - if (patch_instruction((unsigned int *)ip, new)) - return -EPERM; + PATCH_INSN_OR_GOTO(ip, new, patch_err); return 0; + +patch_err: + return -EPERM; } /* @@ -204,12 +217,12 @@ __ftrace_make_nop(struct module *mod, } #endif /* CONFIG_MPROFILE_KERNEL */ - if (patch_instruction((unsigned int *)ip, pop)) { - pr_err("Patching NOP failed.\n"); - return -EPERM; - } + PATCH_INSN_OR_GOTO(ip, pop, patch_err); return 0; + +patch_err: + return -EPERM; } #else /* !PPC64 */ @@ -276,10 +289,12 @@ __ftrace_make_nop(struct module *mod, op = PPC_INST_NOP; - if (patch_instruction((unsigned int *)ip, op)) - return -EPERM; + PATCH_INSN_OR_GOTO(ip, op, patch_err); return 0; + +patch_err: + return -EPERM; } #endif /* PPC64 */ #endif /* CONFIG_MODULES */ @@ -322,7 +337,7 @@ static int add_ftrace_tramp(unsigned long tramp) */ static int setup_mcount_compiler_tramp(unsigned long tramp) { - int i, op; + unsigned int i, op; unsigned long ptr; static unsigned long ftrace_plt_tramps[NUM_FTRACE_TRAMPS]; @@ -366,16 +381,14 @@ static int setup_mcount_compiler_tramp(unsigned long tramp) #else ptr = ppc_global_function_entry((void *)ftrace_caller); #endif - if (!create_branch((void *)tramp, ptr, 0)) { + op = create_branch((void *)tramp, ptr, 0); + if (!op) { pr_debug("%ps is not reachable from existing mcount tramp\n", (void *)ptr); return -1; } - if (patch_branch((unsigned int *)tramp, ptr, 0)) { - pr_debug("REL24 out of range!\n"); - return -1; - } + PATCH_INSN_OR_GOTO(tramp, op, patch_err); if (add_ftrace_tramp(tramp)) { pr_debug("No tramp locations left\n"); @@ -383,6 +396,9 @@ static int setup_mcount_compiler_tramp(unsigned long tramp) } return 0; + +patch_err: + return -EPERM; } static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr) @@ -416,12 +432,12 @@ static int __ftrace_make_nop_kernel(struct dyn_ftrace *rec, unsigned long addr) } } - if (patch_instruction((unsigned int *)ip, PPC_INST_NOP)) { - pr_err("Patching NOP failed.\n"); - return -EPERM; - } + PATCH_INSN_OR_GOTO(ip, PPC_INST_NOP, patch_err); return 0; + +patch_err: + return -EPERM; } int ftrace_make_nop(struct module *mod, @@ -557,17 +573,18 @@ __ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) } /* Ensure branch is within 24 bits */ - if (!create_branch(ip, tramp, BRANCH_SET_LINK)) { + op[0] = create_branch(ip, tramp, BRANCH_SET_LINK); + if (!op[0]) { pr_err("Branch out of range\n"); return -EINVAL; } - if (patch_branch(ip, tramp, BRANCH_SET_LINK)) { - pr_err("REL24 out of range!\n"); - return -EINVAL; - } + PATCH_INSN_OR_GOTO(ip, op[0], patch_err); return 0; + +patch_err: + return -EPERM; } #else /* !CONFIG_PPC64: */ @@ -603,10 +620,12 @@ __ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) pr_devel("write to %lx\n", rec->ip); - if (patch_instruction((unsigned int *)ip, op)) - return -EPERM; + PATCH_INSN_OR_GOTO(ip, op, patch_err); return 0; + +patch_err: + return -EPERM; } #endif /* CONFIG_PPC64 */ #endif /* CONFIG_MODULES */ @@ -650,12 +669,18 @@ static int __ftrace_make_call_kernel(struct dyn_ftrace *rec, unsigned long addr) return -EINVAL; } - if (patch_branch(ip, tramp, BRANCH_SET_LINK)) { - pr_err("Error patching branch to ftrace tramp!\n"); + op = create_branch(ip, tramp, BRANCH_SET_LINK); + if (!op) { + pr_err("Branch out of range\n"); return -EINVAL; } + PATCH_INSN_OR_GOTO(ip, op, patch_err); + return 0; + +patch_err: + return -EPERM; } int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) @@ -748,10 +773,8 @@ __ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, /* The new target may be within range */ if (test_24bit_addr(ip, addr)) { /* within range */ - if (patch_branch((unsigned int *)ip, addr, BRANCH_SET_LINK)) { - pr_err("REL24 out of range!\n"); - return -EINVAL; - } + op = create_branch((unsigned int *)ip, addr, BRANCH_SET_LINK); + PATCH_INSN_OR_GOTO(ip, op, patch_err); return 0; } @@ -776,17 +799,18 @@ __ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, } /* Ensure branch is within 24 bits */ - if (!create_branch((unsigned int *)ip, tramp, BRANCH_SET_LINK)) { + op = create_branch((unsigned int *)ip, tramp, BRANCH_SET_LINK); + if (!op) { pr_err("Branch out of range\n"); return -EINVAL; } - if (patch_branch((unsigned int *)ip, tramp, BRANCH_SET_LINK)) { - pr_err("REL24 out of range!\n"); - return -EINVAL; - } + PATCH_INSN_OR_GOTO(ip, op, patch_err); return 0; + +patch_err: + return -EPERM; } #endif From patchwork Tue Apr 28 14:03:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Naveen N. Rao" X-Patchwork-Id: 1278517 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49BNz63lntz9sSy for ; Wed, 29 Apr 2020 00:18:10 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 49BNz53XwJzDqxB for ; Wed, 29 Apr 2020 00:18:09 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=naveen.n.rao@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 49BNgm3cMjzDqDC for ; Wed, 29 Apr 2020 00:04:52 +1000 (AEST) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 03SE2EGW091450; Tue, 28 Apr 2020 10:04:21 -0400 Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0a-001b2d01.pphosted.com with ESMTP id 30mhc15rkx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 28 Apr 2020 10:04:18 -0400 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.16.0.27/8.16.0.27) with SMTP id 03SE2g4q019519; Tue, 28 Apr 2020 14:04:15 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma04fra.de.ibm.com with ESMTP id 30mcu58wjf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 28 Apr 2020 14:04:15 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 03SE4DRx54132986 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 28 Apr 2020 14:04:13 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5703D11C05B; Tue, 28 Apr 2020 14:04:13 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DF53511C058; Tue, 28 Apr 2020 14:04:11 +0000 (GMT) Received: from naverao1-tp.ibmuc.com (unknown [9.85.73.214]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 28 Apr 2020 14:04:11 +0000 (GMT) From: "Naveen N. Rao" To: Subject: [PATCH v2 2/2] powerpc/kprobes: Check return value of patch_instruction() Date: Tue, 28 Apr 2020 19:33:59 +0530 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.676 definitions=2020-04-28_09:2020-04-28, 2020-04-28 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=3 impostorscore=0 priorityscore=1501 adultscore=0 lowpriorityscore=0 spamscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2004280107 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Steven Rostedt Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" patch_instruction() can fail in some scenarios. Add appropriate error checking so that such failures are caught and logged, and suitable error code is returned. Fixes: d07df82c43be8 ("powerpc/kprobes: Move kprobes over to patch_instruction()") Fixes: f3eca95638931 ("powerpc/kprobes/optprobes: Use patch_instruction()") Signed-off-by: Naveen N. Rao --- arch/powerpc/kernel/kprobes.c | 13 +++- arch/powerpc/kernel/optprobes.c | 109 +++++++++++++++++++++++--------- 2 files changed, 89 insertions(+), 33 deletions(-) diff --git a/arch/powerpc/kernel/kprobes.c b/arch/powerpc/kernel/kprobes.c index 81efb605113e..9eb1cc05ddd5 100644 --- a/arch/powerpc/kernel/kprobes.c +++ b/arch/powerpc/kernel/kprobes.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -138,13 +139,21 @@ NOKPROBE_SYMBOL(arch_prepare_kprobe); void arch_arm_kprobe(struct kprobe *p) { - patch_instruction(p->addr, BREAKPOINT_INSTRUCTION); + int rc = patch_instruction(p->addr, BREAKPOINT_INSTRUCTION); + + if (WARN_ON(rc)) + pr_err("Failed to patch trap at 0x%pK: %d\n", + (void *)p->addr, rc); } NOKPROBE_SYMBOL(arch_arm_kprobe); void arch_disarm_kprobe(struct kprobe *p) { - patch_instruction(p->addr, p->opcode); + int rc = patch_instruction(p->addr, p->opcode); + + if (WARN_ON(rc)) + pr_err("Failed to remove trap at 0x%pK: %d\n", + (void *)p->addr, rc); } NOKPROBE_SYMBOL(arch_disarm_kprobe); diff --git a/arch/powerpc/kernel/optprobes.c b/arch/powerpc/kernel/optprobes.c index 024f7aad1952..20897600db2e 100644 --- a/arch/powerpc/kernel/optprobes.c +++ b/arch/powerpc/kernel/optprobes.c @@ -139,52 +139,80 @@ void arch_remove_optimized_kprobe(struct optimized_kprobe *op) } } +#define PATCH_INSN_OR_GOTO(addr, instr, label) \ +do { \ + int rc = patch_instruction((unsigned int *)(addr), instr); \ + if (rc) { \ + pr_err("%s:%d Error patching instruction at 0x%pK (%pS): %d\n", \ + __func__, __LINE__, \ + (void *)(addr), (void *)(addr), rc); \ + goto label; \ + } \ +} while (0) + /* * emulate_step() requires insn to be emulated as * second parameter. Load register 'r4' with the * instruction. */ -void patch_imm32_load_insns(unsigned int val, kprobe_opcode_t *addr) +static int patch_imm32_load_insns(unsigned int val, kprobe_opcode_t *addr) { /* addis r4,0,(insn)@h */ - patch_instruction(addr, PPC_INST_ADDIS | ___PPC_RT(4) | - ((val >> 16) & 0xffff)); + PATCH_INSN_OR_GOTO(addr, PPC_INST_ADDIS | ___PPC_RT(4) | + ((val >> 16) & 0xffff), + patch_err); addr++; /* ori r4,r4,(insn)@l */ - patch_instruction(addr, PPC_INST_ORI | ___PPC_RA(4) | - ___PPC_RS(4) | (val & 0xffff)); + PATCH_INSN_OR_GOTO(addr, PPC_INST_ORI | ___PPC_RA(4) | + ___PPC_RS(4) | (val & 0xffff), + patch_err); + + return 0; + +patch_err: + return -EFAULT; } /* * Generate instructions to load provided immediate 64-bit value * to register 'r3' and patch these instructions at 'addr'. */ -void patch_imm64_load_insns(unsigned long val, kprobe_opcode_t *addr) +static int patch_imm64_load_insns(unsigned long val, kprobe_opcode_t *addr) { /* lis r3,(op)@highest */ - patch_instruction(addr, PPC_INST_ADDIS | ___PPC_RT(3) | - ((val >> 48) & 0xffff)); + PATCH_INSN_OR_GOTO(addr, PPC_INST_ADDIS | ___PPC_RT(3) | + ((val >> 48) & 0xffff), + patch_err); addr++; /* ori r3,r3,(op)@higher */ - patch_instruction(addr, PPC_INST_ORI | ___PPC_RA(3) | - ___PPC_RS(3) | ((val >> 32) & 0xffff)); + PATCH_INSN_OR_GOTO(addr, PPC_INST_ORI | ___PPC_RA(3) | + ___PPC_RS(3) | ((val >> 32) & 0xffff), + patch_err); addr++; /* rldicr r3,r3,32,31 */ - patch_instruction(addr, PPC_INST_RLDICR | ___PPC_RA(3) | - ___PPC_RS(3) | __PPC_SH64(32) | __PPC_ME64(31)); + PATCH_INSN_OR_GOTO(addr, PPC_INST_RLDICR | ___PPC_RA(3) | + ___PPC_RS(3) | __PPC_SH64(32) | __PPC_ME64(31), + patch_err); addr++; /* oris r3,r3,(op)@h */ - patch_instruction(addr, PPC_INST_ORIS | ___PPC_RA(3) | - ___PPC_RS(3) | ((val >> 16) & 0xffff)); + PATCH_INSN_OR_GOTO(addr, PPC_INST_ORIS | ___PPC_RA(3) | + ___PPC_RS(3) | ((val >> 16) & 0xffff), + patch_err); addr++; /* ori r3,r3,(op)@l */ - patch_instruction(addr, PPC_INST_ORI | ___PPC_RA(3) | - ___PPC_RS(3) | (val & 0xffff)); + PATCH_INSN_OR_GOTO(addr, PPC_INST_ORI | ___PPC_RA(3) | + ___PPC_RS(3) | (val & 0xffff), + patch_err); + + return 0; + +patch_err: + return -EFAULT; } int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *p) @@ -216,30 +244,33 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *p) * be within 32MB on either side of the current instruction. */ b_offset = (unsigned long)buff - (unsigned long)p->addr; - if (!is_offset_in_branch_range(b_offset)) + if (!is_offset_in_branch_range(b_offset)) { + rc = -ERANGE; goto error; + } /* Check if the return address is also within 32MB range */ b_offset = (unsigned long)(buff + TMPL_RET_IDX) - (unsigned long)nip; - if (!is_offset_in_branch_range(b_offset)) + if (!is_offset_in_branch_range(b_offset)) { + rc = -ERANGE; goto error; + } /* Setup template */ /* We can optimize this via patch_instruction_window later */ size = (TMPL_END_IDX * sizeof(kprobe_opcode_t)) / sizeof(int); pr_devel("Copying template to %p, size %lu\n", buff, size); - for (i = 0; i < size; i++) { - rc = patch_instruction(buff + i, *(optprobe_template_entry + i)); - if (rc < 0) - goto error; - } + for (i = 0; i < size; i++) + PATCH_INSN_OR_GOTO(buff + i, *(optprobe_template_entry + i), + patch_err); /* * Fixup the template with instructions to: * 1. load the address of the actual probepoint */ - patch_imm64_load_insns((unsigned long)op, buff + TMPL_OP_IDX); + if (patch_imm64_load_insns((unsigned long)op, buff + TMPL_OP_IDX)) + goto patch_err; /* * 2. branch to optimized_callback() and emulate_step() @@ -248,6 +279,7 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *p) emulate_step_addr = (kprobe_opcode_t *)ppc_kallsyms_lookup_name("emulate_step"); if (!op_callback_addr || !emulate_step_addr) { WARN(1, "Unable to lookup optimized_callback()/emulate_step()\n"); + rc = -ERANGE; goto error; } @@ -259,21 +291,29 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *p) (unsigned long)emulate_step_addr, BRANCH_SET_LINK); - if (!branch_op_callback || !branch_emulate_step) + if (!branch_op_callback || !branch_emulate_step) { + rc = -ERANGE; goto error; + } + + PATCH_INSN_OR_GOTO(buff + TMPL_CALL_HDLR_IDX, branch_op_callback, + patch_err); - patch_instruction(buff + TMPL_CALL_HDLR_IDX, branch_op_callback); - patch_instruction(buff + TMPL_EMULATE_IDX, branch_emulate_step); + PATCH_INSN_OR_GOTO(buff + TMPL_EMULATE_IDX, branch_emulate_step, + patch_err); /* * 3. load instruction to be emulated into relevant register, and */ - patch_imm32_load_insns(*p->ainsn.insn, buff + TMPL_INSN_IDX); + if (patch_imm32_load_insns(*p->ainsn.insn, buff + TMPL_INSN_IDX)) + goto patch_err; /* * 4. branch back from trampoline */ - patch_branch(buff + TMPL_RET_IDX, (unsigned long)nip, 0); + PATCH_INSN_OR_GOTO(buff + TMPL_RET_IDX, + create_branch(buff + TMPL_RET_IDX, (unsigned long)nip, 0), + patch_err); flush_icache_range((unsigned long)buff, (unsigned long)(&buff[TMPL_END_IDX])); @@ -282,9 +322,11 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *p) return 0; +patch_err: + rc = -EFAULT; error: free_ppc_optinsn_slot(buff, 0); - return -ERANGE; + return rc; } @@ -307,6 +349,7 @@ void arch_optimize_kprobes(struct list_head *oplist) { struct optimized_kprobe *op; struct optimized_kprobe *tmp; + int rc; list_for_each_entry_safe(op, tmp, oplist, list) { /* @@ -315,9 +358,13 @@ void arch_optimize_kprobes(struct list_head *oplist) */ memcpy(op->optinsn.copied_insn, op->kp.addr, RELATIVEJUMP_SIZE); - patch_instruction(op->kp.addr, + rc = patch_instruction(op->kp.addr, create_branch((unsigned int *)op->kp.addr, (unsigned long)op->optinsn.insn, 0)); + if (rc) + pr_err("%s:%d: Error patching instruction at 0x%pK: %d\n", + __func__, __LINE__, + (void *)(op->kp.addr), rc); list_del_init(&op->list); } }