ASPN ActiveState Programmer Network
ActiveState
/ Home / Perl / PHP / Python / Tcl / XSLT /
/ Safari / My ASPN /
Cookbooks | Documentation | Mailing Lists | Modules | News Feeds | Products | User Groups


Recent Messages
List Archives
About the List
List Leaders
Subscription Options

View Subscriptions
Help

View by Topic
ActiveState
.NET Framework
Open Source
Perl
PHP
Python
Tcl
Web Services
XML & XSLT

View by Category
Database
General
SOAP
System Administration
Tools
User Interfaces
Web Programming
XML Programming


MyASPN >> Mail Archive >> image-sig
image-sig
[Image-SIG] Package/Module/Recipe Versioning, Aggregation and Distrobution
by other posts by this author
Jun 16 2004 2:59PM messages near this date
[Image-SIG] Compile on Mac OS X | [Image-SIG] Fw: W h o l e $ a l e !
I have been developing a bootstrap loader to enable module/package & python interpretor vers
ioning/specification at time of import within a script.
This is primarily to encourage code re-use/portablility (satisfying dependancies on multiple
 machines & platforms), and allow revision control and coexistence of packages, a particular
 weak point of python in my experience (I could be just doing things wrong). I am interested
 now in how such versioned packages could be agregated and made avaliable through a centrali
sed service such as PyPi/CPAN, and as to wether such functionality described below will be a
 spanner in the works for distrobution techniques.

---------my actual question ------------------
To people familiar with distrobuting python projects, and associated tools, eg (py2exe, disu
tils, PyPi, freeze, etc)
Which is not my string point in python, is this going to cause headaches with the way the af
ore mentioned tools work.

Can this below  mentioned versioning/package retrieval/recipe retrieval be easily integrated
 with a CPAN like service such as PyPi & http://python.org/peps/pep-0273.html in a logical m
anner, allowing platform specific, versioned packages to be agregated and made available aut
omagically at time of import, or through some form of dependancy checker.
------------------------------------

-----functionality------------
This is similar to a technique seen in PythonMegaWidgets and a discussion I found of David A
schers on versioning.
I have made my intial version of this avaliable on aspn.
This is comprised of two parts, versioner.py & version_loader.py (http://aspn.activestate.co
m/ASPN/Cookbook/Python/Recipe/285215), versioner walks a site-packages tree identifying fold
er structures where versioned packages occur, and distrobutes version_loader.py as __init__.
py in the parent folder of this versioned packages, it also creates empty placeholder __init
__.py's & autoloaders (from foldername import * - where script name = folder name and no __i
nit__.py exists ) so that you can automagically have "from package.folder.folder.script impo
rt method, etc ,etc"
Admittedly Mark Hammonds win32, win32com, etc, didn't work at first inspection but I am happ
y for such packages to remain locally with site-packages as these are not cross platform pac
kages anyhow, however all other packages I have migrated to my versioned site packages direc
tory have played nicely in tests made. 
(for example: bsddb3, ClientCookie, DCOracle2, Ft, id3, log4py, logging, mx, OpenSSL, PIL, p
syco, psyco2, PythonMagick, soaplib, wxPython, xmlplus )
With the manual hack of dragging any built binaries (.so, .dll, .pyd, etc) for said package 
from DLLS,etc into versioned package folder (OK when this is dpendant on system services eg 
MySqld, BerkleyDB, etc this could get difficult).

The impact on python syntax for this is minimal with local variables such as:

_foo_version_ = '1.2.3.4' #where point depth is level of compatibility required
import foo

Optionally. _python_version_ = '2.2.2' #again point depth is variable

I further intend to remove all of the pollution this make of local namespace once the import
 has been performed.
And have not gone through extensive testing nor bug fixes as yet, but all seems to work quit
e nicely.

Ultimately I would love an extension to the python core syntax along the lines of:

import foo version 2.3.4.5 (with inate platform inspection suffixing on package name)#to giv
e the behaviour I have implements

I thought I had seen a PEP on this though cant seem to find this.

------------------------------------

------------extension to this------------
Ultimately I would like to achieve a CPAN like web retrieval of versioned packages/scripts w
hich are referenced yet not available, though doing this at time of import perhaps if local 
variable is declared: e.g. _PyPi_download_ = 1, where if a version is specified it will retr
ieve that version else defaulting to latest.

I would like to extend this with recipes as well,

Eg.

import recipe.aspn_python.stringutil.md5hash as md5hash
import recipe.parnassus.stringutil.utf8escape as ut8esc
import recipe.claysstuff.stringutil.utf8unescape as ut8unesc

Where one could register a base foldername, and their lookup service with a central weblooku
p services which fires a search for a packge/module/scriptname which returns standardised xm
l results, to allow retrieval of the said package, storing it in the path you have nominated
 for versioned-packages/recipes (perhaps in site.py), perhaps even allowing syntax to import
 all packages from source/category/subcategory/etc eg aspn_python.category.*

Having PyPi agregate packages (perhaps with unit tests also) and mirror seems logical, bundl
ing all immediate depandancies within a single archive also seems logical.

Having packages also nominate dependancies could be a huge benifit:

Eg. requires['package'] = '1.2.3'

Or standard dependancies.xml in base folder of package, ok this may be simplistic, modelling
 dependancies is a whole separate issue but for example
<depandancies> 
        <python version='2.2.2'/> 
                <built-against> libpthread.so.0</depend> #using ldd 	
        </python> 
        <platform os='linux' distro='shrike'> 
                <service name='gcc' version='3.22'/>  #using locate(if present) | binaryName 
-v [perhaps this is a little flimsy]
                <service name='php' version='4.2'/> 
        </platform>  
        <packages> 
                <name='proprietarycode.foo' version='1.2.4' service='http://foo.com/lookup.c
gi|php|etc' user='root' pass='secret'/>  #for private packages
                <name='aspn.util.bar' version='1.2.4'/>  #uses Pypi to get search service to 
locate/download package
        <packages> 
</depandancies> 
------------------------------------

I thought I may as well put out some of the concerns I came across with versioned package ma
nagment, and some of the enhancements I could see being of benifit for discussion.
My apologies if I have missed such discussions in these groups I have only recently joined t
hese groups.
Apologies if any of these seems unclear
Attachments:
unknown1


Privacy Policy | Email Opt-out | Feedback | Syndication
© ActiveState Software Inc. All rights reserved