9 vote(s)

Add comment

Number of comments: 26

Problem with closing file descriptors, Graham Ashton, 2004/05/08
Nicely documented recipe. But I can't see how opening file descriptors like this would correctly handle stdin, stdout and stderr:

# Redirect the standard file descriptors to /dev/null.
os.open("/dev/null", os.O_RDONLY)    # standard input (0)
os.open("/dev/null", os.O_RDWR)      # standard output (1)
os.open("/dev/null", os.O_RDWR)      # standard error (2)

Obviously, you don't really want to close the existing ones, but I once saw a good trick in Python Standard Library (Lundh) for doing a similar thing:

class NullDevice:
    def write(self, s):
        pass
sys.stdin.close()
sys.stdout = NullDevice()
sys.stderr = NullDevice()

I've used that quite a bit, with some success.
Add comment

How it works, Chad J. Schroeder, 2004/05/12

In general, when creating a daemon, you  want to close all file
descriptors inherited  from the parent process.  createDaemon()
closes all file descriptors from 0 to maxfd.  This isn't always
necessary, but it's good practice.

Next, three calls to os.open() are made.  This function returns,
when  successful, the lowest file descriptor not currently open
for  the process.   Since  the  standard  fd's  (0, 1, 2)  were
previously closed,  they're now recreated in the daemon process
with an association to /dev/null rather than the actual standard
I/O streams.

Now, anytime a reference is made to the standard I/O streams in
the daemon process, it's redirected to /dev/null.

As visual proof, try the following.  Modify the os.open() calls
to:

os.open("/testlog", os.O_CREAT|os.O_APPEND|os.O_RDONLY) # stdin 
os.open("/testlog", os.O_CREAT|os.O_APPEND|os.O_RDWR)   # stdout
os.open("/testlog", os.O_CREAT|os.O_APPEND|os.O_RDWR)   # stderr

And add the following to the end of the file:

...

# won't be created and no errors will be reported.
open("createDaemon.log", "w").write("rc: %s; pid: %d; ppid: %d; pgrp: %d\n"%\
   (retCode, os.getpid(), os.getppid(), os.getpgrp()))

sys.stdout.write("test stdout\n")
sys.stdout.flush()
sys.stderr.write("test stderr\n")
sys.stderr.flush()

...

The output contained in /testlog verifies the standard I/O streams,
stdout and stderr, are redirected.

Hope this helps.

Add comment

Problem with closing file descriptors, Graham Ashton, 2004/05/08
Nicely documented recipe. But I can't see how opening file descriptors like this would correctly handle stdin, stdout and stderr:

# Redirect the standard file descriptors to /dev/null.
os.open("/dev/null", os.O_RDONLY)    # standard input (0)
os.open("/dev/null", os.O_RDWR)      # standard output (1)
os.open("/dev/null", os.O_RDWR)      # standard error (2)

Obviously, you don't really want to close the existing ones, but I once saw a good trick in Python Standard Library (Lundh) for doing a similar thing:

class NullDevice:
    def write(self, s):
        pass
sys.stdin.close()
sys.stdout = NullDevice()
sys.stderr = NullDevice()

I've used that quite a bit, with some success.
Add comment

Chad J. Schroeder, 2004/05/12

Add comment

How is this better than "&"?, Doug DeCoudras, 2004/06/28
Hi, I'm newish to Python and I'm wondering how this approach to writing a daemon is better than running a Python script in the background (run from a Linux or UNIX command line) as follows? : $ python myPython.py Hi, I'm newish to Python and I'm wondering how this approach to writing a daemon is better than running a Python script in the background (run from a Linux or UNIX command line) as follows? : $ python myPython.py
Add comment

Depends On Use, tuco Leone, 2005/10/08

"How is this better than [background task]?"

For long-running processes that are not tied to a terminal, for example.
Add comment

Why O_RDWR for stdout and stderr, Blair Zajac, 2004/06/30
Good read. One question. Why do you reopen fd's 1 and 2 using O_RDWR instead of O_WRONLY?
Add comment

RE: Why O_RDWR for stdout and stderr, Chad J. Schroeder, 2004/07/06

I guess it's a matter of habit and how I've seen it done in the past.

When programming a daemon in C, I tend to open/create a file descriptor to "/dev/null" (or any file)
[fd = open(DEVNULL, O_RDWR)] with the RDWR flag and then duplicate the standard descriptors:

dup2(fd, STDIN_FILENO);
dup2(fd, STDOUT_FILENO);
dup2(fd, STDERR_FILENO);

I just carried the idea/style to the pythonized daemon code.

Add comment

The Daemon Process Is Using Most of the CPU, Brad Touesnard, 2004/07/13
First off, this is some great advice for writting daemon applications.

Problem:

I am running a daemon application using your code to convert the process to a daemon. It works great, but shortly after running, it starts to occupy almost all the CPU as if the process is in spinlock. However, the process should be blocking (rather than spinning) as it is waiting to read from a named pipe. Any idea why the process is taking up most of the CPU and how to stop it from doing so? Here's the code after I call the "createDaemon()" function:

# Write process id to a file
fp = open(pid_file, 'w')
fp.write(str(os.getpid()))
fp.close()

fpipe = open(pipe_path, 'r')

while 1:
  log_line = fpipe.readline()

  input_list = log_line.split('\t')

  if len(input_list) < 4:
    continue

  file_path = input_list[4]

  slash_pos1 = file_path.find('/', 1)
  slash_pos2 = file_path.find('/', slash_pos1 + 1)

  homedir_path = file_path[:slash_pos2]
  logs_path = homedir_path + '/logs'

  if debug_on:
    print 'Logging to ' + logs_path

  if os.path.exists(logs_path):
    fp = open(logs_path + '/xferlog', 'a')
    fcntl.lockf(fp, fcntl.LOCK_EX)
    fp.write(log_line)
    fcntl.lockf(fp, fcntl.LOCK_UN)
    fp.close()

fpipe.close()

Add comment

RE: The Daemon Process Is Using Most of the CPU, Chad J. Schroeder, 2004/07/16

My two cents -

It looks like you're creating a busy while loop.  The fpipe.readline()
returns immediately when there is nothing to read, it doesn't block,
creating the busy while loop until there is something to read.  This is why
your CPU usage rises.

You may want to look at using something like select and/or the low level
open, read, and write (and other tools) in the os module.

Add comment

error to syslog, Gijs Molenaar, 2005/06/15
I'm using this code, and it functioning very good. But sometimes (once a week) my application crashes. I tried to try/except everything, but somehow sometimes something goes wrong. I've created a function log(), that I can use for logging. At this moment it sends me an e-mail and writes and entry in syslog. I want to redirect stderr to this function, but the problem is that strerr is a stream, and I don't really know how you can convert this to a log() call. Does anyone has an idea or am I thinking wrong?
Add comment

Gijs Molenaar, 2005/06/15
sorry, the answer is in the code I see now... I feel ashamed. I didn't implement the full code. _and_ I pushed the wrong 'add comment' button. Sorry for wasting time!
Add comment

Gijs Molenaar, 2005/06/15
The answer is quite simpel:

import sys

class LogErr:
    def write(self, data):
        print "log: " + data

t = LogErr()
sys.stderr = t
sys.stderr.write("test stderr\n")

I've put it here, maybe it can be usefull for somebody.
Add comment

Simplify, Neal Becker, 2005/09/14
I don't see why 2 forks are needed. I think all that's needed is: if (os.fork()) == 0: os.setsid() maxfd = os.sysconf("SC_OPEN_MAX") for fd in range(0, maxfd): try: os.close(fd) except OSError: # ERROR (ignore) pass # Redirect the standard file descriptors to /dev/null. os.open("/dev/null", os.O_RDONLY) # standard input (0) os.open("/dev/null", os.O_RDWR) # standard output (1) os.open("/dev/null", os.O_RDWR) # standard error (2) else: sys.exit (0)
Add comment

It always takes two forks to make a daemon. This is tradition., Noah Spurrier, 2005/09/27
Some UNIXes don't require it. It doesn't hurt to do it on all UNIXes. The reason some UNIXes require it is to make sure that the daemon process is NOT a session leader. A session leader process may attempt to aquire a controlling terminal. By definition a daemon does not have a controlling terminal. This is one of the steps that might not be strictly necessary, but it will eliminate one possible source for faults.
Add comment

Depends On Use, tuco Leone, 2005/10/08

"I don't see why 2 forks are needed."

To have a seperate, stand-alone process running that is abandoned by its parent to live its own life without ever knowing if its parant is alive or dead, for example.

Add comment

Benifit To The Parent , tuco Leone, 2005/10/11
And don't forget that the parent process who double-fork()s their daemon child never has to worry about it turning into a Zombie when they die too.
Add comment

Example of usage, jd holt, 2005/11/21
I have tried to implement this but I am having trouble. Here is what I have tried.

#!/usr/bin/env python
import pyDaemon
import time
import sys
import os
pyDaemon.createDaemon()
def logit(self):
        fp = open('test.log','w')
        fp.write('Hello\n')
        fp.close()
while true:
        time.sleep(300)
        logit()

I left the Daemon Code untouched and nothing happens.

Thanks,
         Josh holt

Add comment

Error in your program, Chris Cogdon, 2006/06/12
Because stderr is being redirected to /dev/null, you won't be informed of any errors in your program. In this case, you have 'self' as a parameter to logit, but it's not class member. Just remove 'self' and it should work fine. Also, you might want to use mode "a" rather than "w"; this way you'll see "Hello!" being added every 5 minutes, rather than just one "Hello!"
Add comment

Random file descriptor, francis giraldeau, 2006/06/17
When using the random module after daemonize, then the file description is not accessible anymore, and it produce this error:

Traceback (most recent call last):
File "/usr/share/mille-xterm/lbserver/main.py", line 332, in ?
main()
File "/usr/share/mille-xterm/lbserver/main.py", line 287, in main
random.seed()
File "/usr/lib/python2.4/random.py", line 110, in seed
a = long(_hexlify(_urandom(16)), 16)
File "/usr/lib/python2.4/os.py", line 728, in urandom
bytes += read(_urandomfd, n - len(bytes))
OSError: [Errno 9] Bad file descriptor


Should I preserve file descriptors? I don't see another way to do it. Thanks for any hint,
Francis
Add comment

sasa sasa, 2006/10/06
This is python bug 1177468, it's apparently fixed in python cvs since 4th July 2005 but may not have made it to your distribution yet. Without the fixed os.py, your only option is to leave the file descriptors open.
Add comment

sasa sasa, 2006/11/02
Actually you do need to close 0,1,2 otherwise you won't get any stdout/stderr redirects from the os.open calls.
Add comment

Can't quite get this to work, Brandon Pierce, 2006/11/10

Hello,

I've been testing this with the following code:

#!/usr/bin/python
import pyDaemon
import time

def logit():
	fp = open('test.log','a')
	fp.write('Hello\n')
	fp.close()

pyDaemon.createDaemon()
while 1:
        time.sleep(5)
        logit()

This is basically what one of the other folks earlier was doing.
I have it saves in a file called 'test.py'. I'm new to Python, and have
never done anything with daemons, so I wanted to see how it works.


If I start this using 'python test.py', it seem to start, and I can 
see it running as a process, but nothing gets written to the log
file. The file exists and I gave it permissions of 777.


If I comment out the line that executes the createDaemon() function,
it works fine, aside from not being daemonized (naturally). Any ideas?

Thanks!

Brandon

Add comment

Check /, Lloyd Carothers, 2006/11/21
The daemon code sets the current working dir of the process to /. You're file is either here or you don't have permission to write here. If you're doing serious logging check out syslog. There are some recipes on this site.
Add comment

Handling SIGTERM, greg p, 2007/10/14
How can I make my daemon using this code handle SIGTERM.

See I have it created child processes when it starts up, and when the daemon is shut down, I want it to clean up those child processes. I figured I could put the clean up code in a function.

Here's what I tried so far:

def handle_sigterm():
    """Kill all child processes to clean up."""
    logging.info('handle_sigterm called.')

signal.signal(signal.SIGTERM,handle_sigterm)

But when running $ sudo kill -15 [PID] against it, that function never gets called.
Add comment

Redirecting both stdout and stderr to the same file, Felipe Pereira, 2008/01/24
Here's what I wanted:

* print statements output should go to a logfile (even print >> sys.stderr)
* children (e.g. os.system calls) should output to the same logfile
* without touching daemonize.py code

Here's how I solved.

First of all, I needed to add:

sys.stdin.close()
sys.stdout.close()
sys.stderr.close()

To createDaemon() code. Closing underlying C buffers without closing (and thus notifying) sys.std* seems to be bad. There's also another recipe here about this. According to Python docs, if you close sys streams, the associated fd is not really closed. So we still need to close 0,1 and 2 fds.

This was the only change I made to createDaemon(). I think it's not a problem, because I was going to reassign sys.std* afterwards.

Now my code: bla.py

#!/usr/bin/python

import daemon
import os,sys,time

daemon.createDaemon()

sys.stdout.close() #we close /dev/null
sys.stderr.close()

os.close(2) # and associated fd's
os.close(1)

# now we open a new stdout
# * notice that underlying fd is 1
# * bufsize is 1 because we want stdout line buffered (it's my log file)
sys.stdout = open('/tmp/bla','w',1) # redirect stdout
os.dup2(1,2) # fd 2 is now a duplicate of fd 1
sys.stderr = os.fdopen(2,'a',0) # redirect stderr
# from now on sys.stderr appends to fd 2
# * bufsize is 0, I saw this somewhere, I guess no bufferization at all is better for stderr

# now some tests... we want to know if it's bufferized or not
print "stdout"
print >> sys.stderr, "stderr"
os.system("echo stdout-echo") # this is unix only...
os.system("echo stderr-echo > /dev/stderr")
# cat /tmp/bla and check that it's ok; to kill use: pkill -f bla.py
while 1:
        time.sleep(1)

sys.exit(0)

Ps: I had to replace "tumbler" for "aspn" in the URL to comment this!
Ps2: this is for Unix!

Add comment