Refactoring of idl_parser (#4948)

* Refactoring of numbers parser

More accurate parse of float and double.
Hexadecimal floats.
Check "out-of-range" of uint64 fields.
Check correctness of default values and metadata.

* Remove locale-independent code strtod/strtof from PR #4948.

* small optimization

* Add is_(ascii) functions

* is_ascii cleanup

* Fix format conversation

* Refine number parser

* Make code compatible with Android build

* Remove unnecessary suppression of warning C4127

docs/source/Grammar.md

@@ -49,11 +49,26 @@ file_extension_decl = `file_extension` string\_constant `;`
 
 file_identifier_decl = `file_identifier` string\_constant `;`
 
-integer\_constant = `-?[0-9]+` | `true` | `false`
-
-float\_constant = `-?[0-9]+.[0-9]+((e|E)(+|-)?[0-9]+)?`
-
 string\_constant = `\".*?\"`
 
 ident = `[a-zA-Z_][a-zA-Z0-9_]*`
 
+`[:digit:]` = `[0-9]`
+
+`[:xdigit:]` = `[0-9a-fA-F]`
+
+dec\_integer\_constant = `[-+]?[:digit:]+`
+
+hex\_integer\_constant = `[-+]?0[xX][:xdigit:]+`
+
+integer\_constant = dec\_integer\_constant | hex\_integer\_constant
+
+dec\_float\_constant = `[-+]?(([.][:digit:]+)|([:digit:]+[.][:digit:]*)|([:digit:]+))([eE][-+]?[:digit:]+)?`
+
+hex\_float\_constant = `[-+]?0[xX](([.][:xdigit:]+)|([:xdigit:]+[.][:xdigit:]*)|([:xdigit:]+))([pP][-+]?[:digit:]+)`
+
+special\_float\_constant = `[-+]?(nan|inf|infinity)`
+
+float\_constant = decimal\_float\_constant | hexadecimal\_float\_constant | special\_float\_constant
+
+boolean\_constant = `(true|false)` | (integer\_constant ? `true` : `false`)

docs/source/Schemas.md

@@ -385,6 +385,31 @@ When parsing JSON, it recognizes the following escape codes in strings:
 It also generates these escape codes back again when generating JSON from a
 binary representation.
 
+When parsing numbers, the parser is more flexible than JSON.
+A format of numeric literals is more close to the C/C++.
+According to the [grammar](@ref flatbuffers_grammar), it accepts the following 
+numerical literals:
+
+-   An integer literal can have any number of leading zero `0` digits.
+    Unlike C/C++, the parser ignores a leading zero, not interpreting it as the 
+    beginning of the octal number.
+    The numbers `[081, -00094]` are equal to `[81, -94]`  decimal integers.
+-   The parser accepts unsigned and signed hexadecimal integer numbers.
+    For example: `[0x123, +0x45, -0x67]` are equal to `[291, 69, -103]` decimals.
+-   The format of float-point numbers is fully compatible with C/C++ format.
+    If a modern C++ compiler is used the parser accepts hexadecimal and special 
+    float-point literals as well:
+    `[-1.0, 2., .3e0, 3.e4, 0x21.34p-5, -inf, nan]`.
+    The exponent suffix of hexadecimal float-point number is mandatory.
+    
+    Extended float-point support was tested with:
+    - x64 Windows: `MSVC2015` and higher.
+    - x64 Linux: `LLVM 6.0`, `GCC 4.9` and higher.
+
+-   For compatibility with a JSON lint tool all numeric literals of scalar 
+    fields can be wrapped to quoted string:
+    `"1", "2.0", "0x48A", "0x0C.0Ep-1", "-inf", "true"`.
+
 ## Guidelines
 
 ### Efficiency