From patchwork Sun Apr 24 09:40:13 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Holger Freyther X-Patchwork-Id: 614131 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.osmocom.org (lists.osmocom.org [IPv6:2a01:4f8:191:444b::2:7]) by ozlabs.org (Postfix) with ESMTP id 3qt48p0Sgjz9t3t for ; Sun, 24 Apr 2016 19:40:24 +1000 (AEST) Received: from lists.osmocom.org (lists.osmocom.org [144.76.43.76]) by lists.osmocom.org (Postfix) with ESMTP id D9B0A1392C; Sun, 24 Apr 2016 09:40:20 +0000 (UTC) X-Original-To: openbsc@lists.osmocom.org Delivered-To: openbsc@lists.osmocom.org Received: from gandharva.secretlabs.de (gandharva.secretlabs.de [5.9.72.18]) by lists.osmocom.org (Postfix) with ESMTP id F02B41391E for ; Sun, 24 Apr 2016 09:40:19 +0000 (UTC) Received: from 217-197-81-37.in-berlin.de.de (unknown [217.197.81.37]) by gandharva.secretlabs.de (Postfix) with ESMTPSA id 1509E77C4D for ; Sun, 24 Apr 2016 09:40:19 +0000 (UTC) From: Holger Hans Peter Freyther To: openbsc@lists.osmocom.org Subject: [PATCH] milenage/aes: Address undefined behavior on bitshift Date: Sun, 24 Apr 2016 11:40:13 +0200 Message-Id: <1461490813-5032-1-git-send-email-holger@freyther.de> X-Mailer: git-send-email 2.6.3 X-BeenThere: openbsc@lists.osmocom.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Development of OpenBSC, OsmoBSC, OsmoNITB, OsmoCSCN" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openbsc-bounces@lists.osmocom.org Sender: "OpenBSC" From: Holger Hans Peter Freyther Extend the u8 to u32 before going to shift it. Fixes: milenage/aes-internal.c:799:4: runtime error: left shift of 128 by 24 places cannot be represented in type 'int' #0 0x7f84e9fe86a2 in rijndaelKeySetupEnc (/home/builder/jenkins/workspace/Osmocom_Sanitizer/source/libosmocore/src/gsm/.libs/libosmogsm.so.5+0xfa6a2) #1 0x7f84e9febad8 in aes_encrypt_init (/home/builder/jenkins/workspace/Osmocom_Sanitizer/source/libosmocore/src/gsm/.libs/libosmogsm.so.5+0xfdad8) #2 0x7f84e9fe7d14 in aes_128_encrypt_block (/home/builder/jenkins/workspace/Osmocom_Sanitizer/source/libosmocore/src/gsm/.libs/libosmogsm.so.5+0xf9d14) #3 0x7f84e9febe7d in milenage_f1 (/home/builder/jenkins/workspace/Osmocom_Sanitizer/source/libosmocore/src/gsm/.libs/libosmogsm.so.5+0xfde7d) #4 0x7f84e9fee2ce in milenage_generate (/home/builder/jenkins/workspace/Osmocom_Sanitizer/source/libosmocore/src/gsm/.libs/libosmogsm.so.5+0x1002ce) #5 0x7f84e9fe76d7 in milenage_gen_vec (/home/builder/jenkins/workspace/Osmocom_Sanitizer/source/libosmocore/src/gsm/.libs/libosmogsm.so.5+0xf96d7) #6 0x7f84e9fe6c08 in osmo_auth_gen_vec (/home/builder/jenkins/workspace/Osmocom_Sanitizer/source/libosmocore/src/gsm/.libs/libosmogsm.so.5+0xf8c08) #7 0x401441 in main (/home/builder/jenkins/workspace/Osmocom_Sanitizer/source/libosmocore/tests/auth/.libs/lt-milenage_test+0x401441) #8 0x7f84e8e33a3f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x20a3f) #9 0x400e58 in _start (/home/builder/jenkins/workspace/Osmocom_Sanitizer/source/libosmocore/tests/auth/.libs/lt-milenage_test+0x400e58) --- src/gsm/milenage/aes_i.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/gsm/milenage/aes_i.h b/src/gsm/milenage/aes_i.h index c831757..5d89abc 100644 --- a/src/gsm/milenage/aes_i.h +++ b/src/gsm/milenage/aes_i.h @@ -66,7 +66,7 @@ extern const u8 rcons[10]; #else /* AES_SMALL_TABLES */ -#define RCON(i) (rcons[(i)] << 24) +#define RCON(i) ((u32)rcons[(i)] << 24) static inline u32 rotr(u32 val, int bits) {