From patchwork Fri Apr 10 20:28:20 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eric Blake X-Patchwork-Id: 460219 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 307251402F5 for ; Sat, 11 Apr 2015 06:29:00 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=fail reason="verification failed; unprotected key" header.d=comcast.net header.i=@comcast.net header.b=RdHdAz3Y; dkim-adsp=none (unprotected policy); dkim-atps=neutral Received: from localhost ([::1]:40984 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YgfXt-0000k7-P9 for incoming@patchwork.ozlabs.org; Fri, 10 Apr 2015 16:28:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53092) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YgfXX-0000CF-9q for qemu-devel@nongnu.org; Fri, 10 Apr 2015 16:28:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YgfXR-0000EK-FD for qemu-devel@nongnu.org; Fri, 10 Apr 2015 16:28:35 -0400 Received: from resqmta-po-04v.sys.comcast.net ([96.114.154.163]:52256) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YgfXR-0000DC-8H for qemu-devel@nongnu.org; Fri, 10 Apr 2015 16:28:29 -0400 Received: from resomta-po-04v.sys.comcast.net ([96.114.154.228]) by resqmta-po-04v.sys.comcast.net with comcast id ELUG1q0034vw8ds01LUSAa; Fri, 10 Apr 2015 20:28:26 +0000 Received: from red.redhat.com ([24.10.254.122]) by resomta-po-04v.sys.comcast.net with comcast id ELUM1q00N2fD5rL01LURp7; Fri, 10 Apr 2015 20:28:26 +0000 From: Eric Blake To: qemu-devel@nongnu.org Date: Fri, 10 Apr 2015 14:28:20 -0600 Message-Id: <1428697701-31743-1-git-send-email-eblake@redhat.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1428206887-7921-1-git-send-email-eblake@redhat.com> References: <1428206887-7921-1-git-send-email-eblake@redhat.com> MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20140121; t=1428697706; bh=GqZqCIMr0zEqCLqBHBB4M5vptPtKiPOypJZmoQaYisg=; h=Received:Received:From:To:Subject:Date:Message-Id:MIME-Version: Content-Type; b=RdHdAz3YUPLYfy5bSEf6nn7uRWxf98VxexBSlFs2B00n4AsYA2WVhpYVj1Jog9f29 Qarxwihwy8rvQJ1qu8FvPVJ21vjskTjgKmS4zav3KvoG7ZTac3vbpizMJNy1ARFhwQ zpw9tNGhYU1togF+NLmJQ3I0pNmUVZa+0Ic3DqgjHto1IivM4kVVgbUgZk08MH7L3H NtFxXzPJoFVwK7eFBOhYIMq98OAVHusiWE75xil5LFJg4R3F/vGupqYzEnGnzupR87 +SEvpygtUog7DAIqEaFqeuDuUr0dnuqRKeQb9V3RvS+gZ5ACEnP3RCpORsQbYu9LBt 2MYql8im4FF4w== X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 96.114.154.163 Cc: kwolf@redhat.com, berto@igalia.com, armbru@redhat.com Subject: [Qemu-devel] [PATCH v6 37/36] qapi: Support (subset of) \u escapes in strings X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The handling of \ inside QAPI strings was less than ideal, and really only worked JSON's \/, \\, \", and our extension of \' (an obvious extension, when you realize we use '' instead of "" for strings). For other things, like '\n', it resulted in a literal 'n' instead of a newline. Of course, at the moment, we really have no use for escaped characters, as QAPI has to map to C identifiers, and we currently support ASCII only for that. But down the road, we may add support for default values for string parameters to a command or struct; if that happens, it would be nice to correctly support all JSON escape sequences, such as \n or \uXXXX. This gets us closer, by supporting Unicode escapes in the ASCII range. Since JSON does not require \OCTAL or \xXX escapes, I did not add it here, but it would be an easy addition if we desired it. Signed-off-by: Eric Blake --- v6: new patch --- scripts/qapi.py | 33 +++++++++++++++++++++++++++++++- tests/Makefile | 1 + tests/qapi-schema/escape-too-big.err | 1 + tests/qapi-schema/escape-too-big.exit | 1 + tests/qapi-schema/escape-too-big.json | 3 +++ tests/qapi-schema/escape-too-big.out | 0 tests/qapi-schema/escape-too-short.err | 1 + tests/qapi-schema/escape-too-short.exit | 1 + tests/qapi-schema/escape-too-short.json | 3 +++ tests/qapi-schema/escape-too-short.out | 0 tests/qapi-schema/ident-with-escape.err | 1 - tests/qapi-schema/ident-with-escape.exit | 2 +- tests/qapi-schema/ident-with-escape.json | 2 +- tests/qapi-schema/ident-with-escape.out | 3 +++ tests/qapi-schema/unicode-str.err | 1 + tests/qapi-schema/unicode-str.exit | 1 + tests/qapi-schema/unicode-str.json | 2 ++ tests/qapi-schema/unicode-str.out | 0 18 files changed, 52 insertions(+), 4 deletions(-) create mode 100644 tests/qapi-schema/escape-too-big.err create mode 100644 tests/qapi-schema/escape-too-big.exit create mode 100644 tests/qapi-schema/escape-too-big.json create mode 100644 tests/qapi-schema/escape-too-big.out create mode 100644 tests/qapi-schema/escape-too-short.err create mode 100644 tests/qapi-schema/escape-too-short.exit create mode 100644 tests/qapi-schema/escape-too-short.json create mode 100644 tests/qapi-schema/escape-too-short.out create mode 100644 tests/qapi-schema/unicode-str.err create mode 100644 tests/qapi-schema/unicode-str.exit create mode 100644 tests/qapi-schema/unicode-str.json create mode 100644 tests/qapi-schema/unicode-str.out diff --git a/tests/qapi-schema/unicode-str.out b/tests/qapi-schema/unicode-str.out new file mode 100644 index 0000000..e69de29 diff --git a/scripts/qapi.py b/scripts/qapi.py index 60ed34a..853f9a3 100644 --- a/scripts/qapi.py +++ b/scripts/qapi.py @@ -173,7 +173,38 @@ class QAPISchema: raise QAPISchemaError(self, 'Missing terminating "\'"') if esc: - string += ch + if ch == 'b': + string += '\b' + elif ch == 'f': + string += '\f' + elif ch == 'n': + string += '\n' + elif ch == 'r': + string += '\r' + elif ch == 't': + string += '\t' + elif ch == 'u': + value = 0 + for x in range(0, 4): + ch = self.src[self.cursor] + self.cursor += 1 + if ch not in "0123456789abcdefABCDEF": + raise QAPISchemaError(self, + '\\u escape needs 4 ' + 'hex digits') + value = (value << 4) + int(ch, 16) + # If Python 2 and 3 didn't disagree so much on + # how to handle Unicode, then we could allow + # Unicode string defaults. But most of QAPI is + # ASCII-only, so we aren't losing much for now. + if value > 0x7f: + raise QAPISchemaError(self, + 'For now, \\u escape ' + 'only supports values ' + 'up to \\u007f') + string += chr(value) + else: + string += ch esc = False elif ch == "\\": esc = True diff --git a/tests/Makefile b/tests/Makefile index f37cd01..0cd114f 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -212,6 +212,7 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \ enum-clash-member.json enum-max-member.json enum-union-clash.json \ enum-bad-name.json funny-char.json indented-expr.json \ missing-type.json bad-ident.json ident-with-escape.json \ + escape-too-short.json escape-too-big.json unicode-str.json \ double-type.json bad-base.json bad-type-bool.json bad-type-int.json \ bad-type-dict.json double-data.json unknown-expr-key.json \ redefined-type.json redefined-command.json redefined-builtin.json \ diff --git a/tests/qapi-schema/escape-too-big.err b/tests/qapi-schema/escape-too-big.err new file mode 100644 index 0000000..7f3976e --- /dev/null +++ b/tests/qapi-schema/escape-too-big.err @@ -0,0 +1 @@ +tests/qapi-schema/escape-too-big.json:3:14: For now, \u escape only supports values up to \u007f diff --git a/tests/qapi-schema/escape-too-big.exit b/tests/qapi-schema/escape-too-big.exit new file mode 100644 index 0000000..d00491f --- /dev/null +++ b/tests/qapi-schema/escape-too-big.exit @@ -0,0 +1 @@ +1 diff --git a/tests/qapi-schema/escape-too-big.json b/tests/qapi-schema/escape-too-big.json new file mode 100644 index 0000000..62bcecd --- /dev/null +++ b/tests/qapi-schema/escape-too-big.json @@ -0,0 +1,3 @@ +# we don't support full Unicode strings, yet +# { 'command': 'é' } +{ 'command': '\u00e9' } diff --git a/tests/qapi-schema/escape-too-big.out b/tests/qapi-schema/escape-too-big.out new file mode 100644 index 0000000..e69de29 diff --git a/tests/qapi-schema/escape-too-short.err b/tests/qapi-schema/escape-too-short.err new file mode 100644 index 0000000..934de59 --- /dev/null +++ b/tests/qapi-schema/escape-too-short.err @@ -0,0 +1 @@ +tests/qapi-schema/escape-too-short.json:3:14: \u escape needs 4 hex digits diff --git a/tests/qapi-schema/escape-too-short.exit b/tests/qapi-schema/escape-too-short.exit new file mode 100644 index 0000000..d00491f --- /dev/null +++ b/tests/qapi-schema/escape-too-short.exit @@ -0,0 +1 @@ +1 diff --git a/tests/qapi-schema/escape-too-short.json b/tests/qapi-schema/escape-too-short.json new file mode 100644 index 0000000..6cb1dec --- /dev/null +++ b/tests/qapi-schema/escape-too-short.json @@ -0,0 +1,3 @@ +# the \u escape requires 4 hex digits +# { 'command': 'a' } +{ 'command': '\u61' } diff --git a/tests/qapi-schema/escape-too-short.out b/tests/qapi-schema/escape-too-short.out new file mode 100644 index 0000000..e69de29 diff --git a/tests/qapi-schema/ident-with-escape.err b/tests/qapi-schema/ident-with-escape.err index f7d1c55..e69de29 100644 --- a/tests/qapi-schema/ident-with-escape.err +++ b/tests/qapi-schema/ident-with-escape.err @@ -1 +0,0 @@ -tests/qapi-schema/ident-with-escape.json:3: Expression is missing metatype diff --git a/tests/qapi-schema/ident-with-escape.exit b/tests/qapi-schema/ident-with-escape.exit index d00491f..573541a 100644 --- a/tests/qapi-schema/ident-with-escape.exit +++ b/tests/qapi-schema/ident-with-escape.exit @@ -1 +1 @@ -1 +0 diff --git a/tests/qapi-schema/ident-with-escape.json b/tests/qapi-schema/ident-with-escape.json index cfb2050..5661750 100644 --- a/tests/qapi-schema/ident-with-escape.json +++ b/tests/qapi-schema/ident-with-escape.json @@ -1,4 +1,4 @@ -# FIXME: we should allow escape sequences in strings, if they map back to ASCII +# we allow escape sequences in strings, if they map back to ASCII # { 'command': 'fooA', 'data': { 'bar1': 'str' } } { 'c\u006fmmand': '\u0066\u006f\u006FA', 'd\u0061ta': { '\u0062\u0061\u00721': '\u0073\u0074\u0072' } } diff --git a/tests/qapi-schema/ident-with-escape.out b/tests/qapi-schema/ident-with-escape.out index e69de29..4028430 100644 --- a/tests/qapi-schema/ident-with-escape.out +++ b/tests/qapi-schema/ident-with-escape.out @@ -0,0 +1,3 @@ +[OrderedDict([('command', 'fooA'), ('data', OrderedDict([('bar1', 'str')]))])] +[] +[] diff --git a/tests/qapi-schema/unicode-str.err b/tests/qapi-schema/unicode-str.err new file mode 100644 index 0000000..f621cd6 --- /dev/null +++ b/tests/qapi-schema/unicode-str.err @@ -0,0 +1 @@ +tests/qapi-schema/unicode-str.json:2: 'command' uses invalid name 'é' diff --git a/tests/qapi-schema/unicode-str.exit b/tests/qapi-schema/unicode-str.exit new file mode 100644 index 0000000..d00491f --- /dev/null +++ b/tests/qapi-schema/unicode-str.exit @@ -0,0 +1 @@ +1 diff --git a/tests/qapi-schema/unicode-str.json b/tests/qapi-schema/unicode-str.json new file mode 100644 index 0000000..5253a1b --- /dev/null +++ b/tests/qapi-schema/unicode-str.json @@ -0,0 +1,2 @@ +# we don't support full Unicode strings, yet +{ 'command': 'é' }