Python is not known to be a very efficient language to execute. In addition, looping is a very time-consuming operation in any language. If any simple single-step operation takes 1 unit of time, repeating the operation tens of thousands of times will eventually increase the time spent tens of thousands of times.

while and for are two keywords commonly used in Python to implement loops, there is a real difference in their efficiency. For example, the following test code.

import timeit

def while_loop(n=100_000_000):
    i = 0
    s = 0
    while i < n:
        s += i
        i += 1
    return s

def for_loop(n=100_000_000):
    s = 0
    for i in range(n):
        s += i
    return s

def main():
    print('while loop\t\t', timeit.timeit(while_loop, number=1))
    print('for loop\t\t', timeit.timeit(for_loop, number=1))

if __name__ == '__main__':
    main()

# => while loop 4.718853999860585
# => for loop 3.211570399813354

This is a simple summation operation that computes the sum of all natural numbers from 1 to n. You can see that the for loop is 1.5 seconds faster than the while loop.

The difference is mainly due to the different mechanics of the two.

In each loop, while actually performs two more steps than for: a bounds check and the self-incrementing of variable i. That is, for each loop, while performs two more steps than for. That is, for each loop, while does a bounds check (while i < n) and a self-increment calculation (i += 1). Both of these operations are explicitly pure Python code.

The for loop does not require a bounds check and self-increment operation, and adds no explicit Python code (pure Python code is less efficient than the underlying C code). When the number of loops is large enough, a significant efficiency gap emerges.

Two more functions can be added to add unnecessary bounds checking and self-incrementing to the for loop.

import timeit

def while_loop(n=100_000_000):
    i = 0
    s = 0
    while i < n:
        s += i
        i += 1
    return s

def for_loop(n=100_000_000):
    s = 0
    for i in range(n):
        s += i
    return s

def for_loop_with_inc(n=100_000_000):
    s = 0
    for i in range(n):
        s += i
        i += 1
    return s

def for_loop_with_test(n=100_000_000):
    s = 0
    for i in range(n):
        if i < n:
            pass
        s += i
    return s

def main():
    print('while loop\t\t', timeit.timeit(while_loop, number=1))
    print('for loop\t\t', timeit.timeit(for_loop, number=1))
    print('for loop with increment\t\t',
          timeit.timeit(for_loop_with_inc, number=1))
    print('for loop with test\t\t', timeit.timeit(for_loop_with_test, number=1))

if __name__ == '__main__':
    main()

# => while loop 4.718853999860585
# => for loop 3.211570399813354
# => for loop with increment 4.602369500091299
# => for loop with test 4.18337869993411

As you can see, the addition of the bounds check and self-increment operations does significantly affect the efficiency of the for loop.

As mentioned earlier, Python’s underlying interpreter and built-in functions are implemented in C. C is much more efficient than Python.

For the above sum of equal variables operation, Python’s built-in sum function can be used to obtain a much more efficient execution than a for or while loop.

import timeit

def while_loop(n=100_000_000):
    i = 0
    s = 0
    while i < n:
        s += i
        i += 1
    return s

def for_loop(n=100_000_000):
    s = 0
    for i in range(n):
        s += i
    return s

def sum_range(n=100_000_000):
    return sum(range(n))

def main():
    print('while loop\t\t', timeit.timeit(while_loop, number=1))
    print('for loop\t\t', timeit.timeit(for_loop, number=1))
    print('sum range\t\t', timeit.timeit(sum_range, number=1))

if __name__ == '__main__':
    main()

# => while loop 4.718853999860585
# => for loop 3.211570399813354
# => sum range 0.8658821999561042

As you can see, using the built-in sum function instead of a loop increases the efficiency of code execution exponentially.

The sum operation of the built-in function is actually a loop, but it is implemented in C, whereas the sum operation in the for loop is implemented in pure Python code s += i. C > Python.

Expand your thinking a bit. As a child, you’ve heard the story of the childhood Gaussian who cleverly calculated the sum of 1 to 100. 1…100 equals (1 + 100) * 50. This same calculation can be applied to the summation operation above.

import timeit

def while_loop(n=100_000_000):
    i = 0
    s = 0
    while i < n:
        s += i
        i += 1
    return s

def for_loop(n=100_000_000):
    s = 0
    for i in range(n):
        s += i
    return s

def sum_range(n=100_000_000):
    return sum(range(n))

def math_sum(n=100_000_000):
    return (n * (n - 1)) // 2

def main():
    print('while loop\t\t', timeit.timeit(while_loop, number=1))
    print('for loop\t\t', timeit.timeit(for_loop, number=1))
    print('sum range\t\t', timeit.timeit(sum_range, number=1))
    print('math sum\t\t', timeit.timeit(math_sum, number=1))

if __name__ == '__main__':
    main()

# => while loop 4.718853999860585
# => for loop 3.211570399813354
# => sum range 0.8658821999561042
# => math sum 2.400018274784088e-06

The final execution time of math sum is about 2.4e-6, which is a million times shorter. The idea here is that since loops are inefficient, a piece of code has to be executed hundreds of millions of times over.

So we just don’t need to loop, and use mathematical formulas to turn hundreds of millions of loops into a single step operation. The efficiency is naturally enhanced to an unprecedented degree.

Final conclusion (a bit riddler).

The fastest way to implement loops – is to not use them

For Python, use built-in functions whenever possible to minimize the pure Python code in the loop.

Of course, built-in functions aren’t the fastest in some cases. When creating lists, for example, it’s the literal writing that’s faster.

Source: https://www.starky.ltd/2021/11/23/the-fastest-way-to-loop-in-python https://mp.weixin.qq.com/s/lJcl-4Xwb9XYEZNG9kuimg