-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Open
Labels
BugBug in learning semantics, critical by defaultBug in learning semantics, critical by default
Description
Describe the bug
Hello VowpalWabbit team!
As part of our ongoing audit of this project, we've identified a critical integer overflow vulnerability in the sync_queries()
function that could lead to a buffer overflow. This vulnerability occurs when processing user-controlled data in the all-reduce operation, specifically when the all_reduce_type is set to SOCKET.
Mechanism of Vulnerability
- The
sync_queries()
function calculatestotal_sum
by summing upsizes[i]
values without proper overflow checks. - The
sizes
array values originate from user-controlledunflushed_bytes_count
data. - An attacker could manipulate the input to cause
total_sum
to overflow, resulting in a small wrapped value. - This wrapped value is then used to allocate a buffer via
calloc_or_throw()
. - Subsequent operations attempt to read more data than the allocated buffer can hold, leading to a buffer overflow.
Relevant Code Snippets
kernel_svm.cc
- sync_queries()
size_t* sizes = VW::details::calloc_or_throw<size_t>(all.runtime_state.all_reduce->total);
sizes[all.runtime_state.all_reduce->node] = b->unflushed_bytes_count();
VW::details::all_reduce<size_t, add_size_t>(all, sizes, all.runtime_state.all_reduce->total);
size_t prev_sum = 0, total_sum = 0;
for (size_t i = 0; i < all.runtime_state.all_reduce->total; i++)
{
if (i <= (all.runtime_state.all_reduce->node - 1)) { prev_sum += sizes[i]; }
total_sum += sizes[i]; // Integer overflow possible here
}
if (total_sum > 0)
{
queries = VW::details::calloc_or_throw<char>(total_sum); // Buffer allocation with overflowed size
// ... subsequent operations using the under-allocated buffer
}
Required Conditions
all.runtime_config.selected_all_reduce_type
must be set to SOCKET- Attacker must be able to control the
unflushed_bytes_count
values
Suggested Action
The code should be modified to:
- Add overflow checks when calculating
total_sum
- Implement upper bounds validation for the allocation size
- Consider using a safer integer type or implementing proper bounds checking
- Add validation to ensure the allocated buffer size is sufficient for the actual data size
Example fix:
size_t total_sum = 0;
for (size_t i = 0; i < all.runtime_state.all_reduce->total; i++)
{
if (i <= (all.runtime_state.all_reduce->node - 1)) { prev_sum += sizes[i]; }
if (total_sum > SIZE_MAX - sizes[i]) {
THROW("Integer overflow detected in total_sum calculation");
}
total_sum += sizes[i];
}
How to reproduce
Version
9.10.0
OS
Linux
Language
C
Additional context
No response
Metadata
Metadata
Assignees
Labels
BugBug in learning semantics, critical by defaultBug in learning semantics, critical by default