Problem: Complex long-running Python systems that gobble up virtual memory.
Solution 1: Find all your cycles and remove them manually.
Solution 2: Plug in a garbage collector.
The Boehm-Demers-Weiser garbage collector from Xerox:
Two approaches are described. The first makes almost no changes to Python; but uses the GC version of malloc and free. [reference counting is left intact] The second approach removes the reference-counting code, and requires patches to about 10 of the source files. This is a more dangerous option, but should perform better.
If you're on linux, version 4.10 of the gc library may not build; a manual include of 'sigcontext.h' will do the trick. Near the top of "gc/os_dep.c":
#if 0 # define __KERNEL__ # include <asm/signal.h> # undef __KERNEL__ #endif # include <sigcontext.h> # endif # if !defined(OS2) && !defined(PCR) && !defined(AMIGA) && !defined(MACOS) # include <sys/types.h> # endif # include <stdio.h> # include <signal.h>
Stick this in Include/mymalloc.h just before the definition of PyMem_NEW:
#include "/usr/src/gc/gc.h" #define malloc(n) GC_malloc(n) #define calloc(m,n) GC_malloc((m)*(n)) #define realloc(o,n) GC_realloc(o,n) #define free(n) GC_free(n)
The explicit include path is important! Don't use "-I/usr/src/gc",
because the gc library also uses a 'config.h' file, which you don't
want to include by mistake.
Now, run 'make', but include the gc library in the definition of the symbol 'LIBS':
make LIBS="-lieee -ldl /usr/src/gc/gc.a" ^^^^^^^^^^^ [these are taken from the LIBS definition in a normal Makefile yours may be different] $ mv python gc-python
That should do it!
This is a patched version of Python-1.5.1 that removes reference counting altogether. Start with a virgin copy of 1.5.1.
$ cd Python-1.5.1 $ patch --strip=1 < ../python-gc-no-refcnt.patch $ ./configure --with-libs="/usr/src/gc/gc.a" $ make OPT="-O -DPy_USE_GC"