ActiveState Code

Recipe 286222: Memory usage.

This recipe provides a number of functions to get the memory usage of a Python application on Linux.

Python 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
import os

_proc_status = '/proc/%d/status' % os.getpid()

_scale = {'kB': 1024.0, 'mB': 1024.0*1024.0,
          'KB': 1024.0, 'MB': 1024.0*1024.0}

def _VmB(VmKey):
    '''Private.
    '''
    global _proc_status, _scale
     # get pseudo file  /proc/<pid>/status
    try:
        t = open(_proc_status)
        v = t.read()
        t.close()
    except:
        return 0.0  # non-Linux?
     # get VmKey line e.g. 'VmRSS:  9999  kB\n ...'
    i = v.index(VmKey)
    v = v[i:].split(None, 3)  # whitespace
    if len(v) < 3:
        return 0.0  # invalid format?
     # convert Vm value to bytes
    return float(v[1]) * _scale[v[2]]


def memory(since=0.0):
    '''Return memory usage in bytes.
    '''
    return _VmB('VmSize:') - since


def resident(since=0.0):
    '''Return resident memory usage in bytes.
    '''
    return _VmB('VmRSS:') - since


def stacksize(since=0.0):
    '''Return stack size in bytes.
    '''
    return _VmB('VmStk:') - since

Discussion

To find the memory usage in a particular section of code these functions are typically used as follows

<pre> m0 = memory() ... m1 = memory(m0) </pre>

Getting and parsing the /proc//status file is not the most efficient way to determine the memory usage.

This recipe has been tested on RedHat Linux 8.0 with Python 2.3.2.

Comments

  1. 1. At 3:40 a.m. on 18 oct 2004, sa li said:

    RE: memory usage. it's good idea! thank you.

    but it can only get the information for current process. in my Application, i want only to get the memory usage for children process(os.popen("children_process") or os.exec("children_process")).

    how can i do?

  2. 2. At 3:14 p.m. on 7 nov 2004, Jean Brouwers (the author) said:

    memory usage of child processes. You can get the memory usage of any process by using the proper _process_status string: simply replace os.getpid() with the pid of the process. To get the pid of a spawned process, use the Popen3 class from the standard popen2 module.

    For more details, see recipe 6.6 "Capturing the Output and Error Streams from a Unix Shell Command" by Brent Burley on page 227 of the Python Cookbook by O'Reilly, check the documentation of the popen2 module and source code in file .../lib/python.../popen2.py of your Python installation.

  3. 3. At 11:39 a.m. on 17 nov 2004, sa li said:

    i think that the "proper_process_status" can be used only in Linux, it can't be runned in Unix (i.e.Solaris). because the file /proc/pid/sttus is a binary-file.

    perhaps the function "resource.getrusage(resource.RUSAGE_CHILDREN)" is a choice too. but I don't konw how use it. :(

    import resoure os.popen("command")

    >>> resource.getrusage(resource.RUSAGE_CHILDREN)
(0.004999, 0.0089979999999999991, 0, 0, 0, 0, 444, 22, 0, 0, 0, 0, 0, 0, 40, 3)
[2:4] is the memory usage, but "0"!, why!?
can you help me?

Sign in to comment