Correct calculation of vector element size (#5831)

Number of elements on the stack shouldn't affect the calculation
of ElemWidth(). Variable 'start' needs to be subtracted from the
loop variable 'i' to make indexing zero-based.

There is an additional unit test to pack nested vectors. Size of
the packed buffer *without* this fix is 798 and only 664 bytes
*with* the fix.
This commit is contained in:
Dmitriy Kovalev
2020-03-26 08:54:52 -07:00
committed by GitHub
parent 6b271b7ecb
commit c37c989ed6
2 changed files with 13 additions and 1 deletions

View File

@@ -1522,7 +1522,8 @@ class Builder FLATBUFFERS_FINAL_CLASS {
Type vector_type = FBT_KEY;
// Check bit widths and types for all elements.
for (size_t i = start; i < stack_.size(); i += step) {
auto elem_width = stack_[i].ElemWidth(buf_.size(), i + prefix_elems);
auto elem_width =
stack_[i].ElemWidth(buf_.size(), i - start + prefix_elems);
bit_width = (std::max)(bit_width, elem_width);
if (typed) {
if (i == start) {