User Tools

Site Tools


Sidebar

Public
Internal
public:it:python

This is an old revision of the document!


Python

Logging

Chunking

adding = set() #of many things
i = iter(adding)
while True:
  chunk = list(islice(i, 10))
  if not chunk:
    break
  else:
    print(",".join(chunk))

Memory Usage with decorators

memory_profiler

Install:

python3 -m pip install memory_profiler

Set the function decorator @profile on any function you want to test.

@profile
def my_func(arg1):
  # do stuff

Run your code like this:

python3 -m memory_profiler helloworld.py

You get output like this:

Line #    Mem usage  Increment   Line Contents
==============================================
     3                           @profile
     4      5.97 MB    0.00 MB   def my_func():
     5     13.61 MB    7.64 MB       a = [1] * (10 ** 6)
     6    166.20 MB  152.59 MB       b = [2] * (2 * 10 ** 7)
     7     13.61 MB -152.59 MB       del b
     8     13.61 MB    0.00 MB       return a

Timing functions with decorators

import time, functools

def timer(func):
  """Print runtime of decorated function"""
  @functools.wraps(func)
  def wrapper_timer(*args, **kwargs):
    start_time = time.perf_counter()
    value = func(*args, *kwargs)
    end_time = time.perf_counter()
    run_time = end_time - start_time
    print(f"Finished {func.__name__!r} in {run_time:.4f} secs")
    return value
  return wrapper_timer


@timer
def dostuff():
   # does stuff
public/it/python.1633987600.txt.gz · Last modified: 2021/10/11 16:26 by phil