ASPN ActiveState Programmer Network
ActiveState
/ Home / Perl / PHP / Python / Tcl / XSLT /
/ Safari / My ASPN /
Cookbooks | Documentation | Mailing Lists | Modules | News Feeds | Products | User Groups


Recent Messages
List Archives
About the List
List Leaders
Subscription Options

View Subscriptions
Help

View by Topic
ActiveState
.NET Framework
Open Source
Perl
PHP
Python
Tcl
Web Services
XML & XSLT

View by Category
Database
General
SOAP
System Administration
Tools
User Interfaces
Web Programming
XML Programming


MyASPN >> Mail Archive >> python-tutor
python-tutor
[Tutor] why is os.path.walk so slow?
by Garry Willgoose other posts by this author
Nov 4 2009 4:36AM messages near this date
Re: [Tutor] PolyRange -- iterator | Re: [Tutor] why is os.path.walk so slow?
I need to synchronize the files on my home and office machine and have  
been using someone else's code for this to date but have been  
frustrated by how slow it is in getting the information on files for  
the mounted drive from my office machine so I thought I'd experiment  
with a python facility for this. The code I've experimented with is as  
below

def visitfunc(arg,dirname,names):
   global filelist
   import os.path
   for name in names:
     if not os.path.isdir(dirname+'/'+name):
       fullname=dirname+'/'+name
       filelist.append([fullname,os.path.getmtime(fullname)])
   return()

def check(dir):
   global filelist
   import os.path
   filelist=[]
   os.path.walk(dir,visitfunc,'')
   print filelist
   return()

This is very fast for a directory on my local machine but  
significantly slower on the remote machine. Not surprising but I would  
have expected that the run time for the remote directory would be  
limited by my broadband speed but when I look at upload/download in  
real time it's less than 10% of maximum. Is this just par for the  
course or is there something I can do that better utilizes my  
broadband bandwidth?

====================================================================
Prof Garry Willgoose,
Australian Professorial Fellow in Environmental Engineering,
Director, Centre for Climate Impact Management (C2IM),
School of Engineering, The University of Newcastle,
Callaghan, 2308
Australia.

Centre webpage: www.c2im.org.au

Phone: (International) +61 2 4921 6050 (Tues-Fri AM); +61 2 6545 9574  
(Fri PM-Mon)
FAX: (International) +61 2 4921 6991 (Uni); +61 2 6545 9574 (personal  
and Telluric)
Env. Engg. Secretary: (International) +61 2 4921 6042

email:  garry.willgoose@newcastle.edu.au; g.willgoose@[...].com
email-for-life: garry.willgoose@[...].edu
personal webpage: www.telluricresearch.com/garry
====================================================================
"Do not go where the path may lead, go instead where there is no path  
and leave a trail"
                           Ralph Waldo Emerson
====================================================================





_______________________________________________
Tutor maillist  -  Tutor@[...].org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Thread:
Garry Willgoose
Modulok
Garry Willgoose
Wayne Werner
Hugo Arts

Privacy Policy | Email Opt-out | Feedback | Syndication
© ActiveState Software Inc. All rights reserved