Vyper's sqrt doesn't define rounding behavior

Vyper's sqrt() builtin uses the babylonian method to calculate square roots of decimals. Unfortunately, improper handling of the oscillating final states may lead to sqrt incorrectly returning rounded up results.

the fix is tracked in https://github.com/vyperlang/vyper/pull/4486

Vulnerability Details

Vyper injects the following code to handle calculation of decimal sqrt. x is the input provided by user.

assert x >= 0.0
z: decimal = 0.0

if x == 0.0:
    z = 0.0
else:
    z = x / 2.0 + 0.5
    y: decimal = x

    for i: uint256 in range(256):
        if z == y:
            break
        y = z
        z = (x / z + z) / 2.0

Notably, the terminal condition of the algorithm is either z_cur == z_prev, or the algorithm runs for 256 rounds.

However, for certain inputs, z might actually oscillate between N and N + epsilon, where N ** 2 <= x < (N + epsilon) ** 2. This means that the current behavior does not define whether it will round up or down to the nearest epsilon.

The example snippet here returns 0.9999999999, the rounded up result for sqrt(0.9999999998). This is due to the oscillation ending in N + epsilon instead of N.

@external
def test():
    d: decimal = 0.9999999998
    r: decimal = sqrt(d)    #this will be 0.9999999999

Note that sqrt() diverges from isqrt() here -- isqrt() consistently rounds down, so it is not subject to the same issue.

Impact Details

Since sqrt() can be used for determining boundary conditions, rounding down is preferred. However, since sqrt() is used very rarely in the wild, this advisory has been assigned an impact of low.