This commit is contained in:
Eray Özkural
-75
@@ -1,75 +0,0 @@
|
|||||||
Like every serious project, there are guidelines.
|
|
||||||
Oooooo. "Coding Standards".
|
|
||||||
|
|
||||||
Guidelines
|
|
||||||
----------
|
|
||||||
|
|
||||||
1. When using dirnames, don't expect the dir to end
|
|
||||||
with a trailing slash, and please use the dirnames
|
|
||||||
in pisiconfig
|
|
||||||
2. Python indentation is usually 4 chars.
|
|
||||||
3. Follow python philosophy of 'batteries included'
|
|
||||||
4. Don't make the code have runtime dependencies on
|
|
||||||
a particular distribution (as much as possible)
|
|
||||||
5. Don't assume narrow use cases.
|
|
||||||
6. If you are changing something, check if that change
|
|
||||||
breaks anything and fix breakage. For instance a
|
|
||||||
name. Running the tests is not always enough!
|
|
||||||
|
|
||||||
Unit testing
|
|
||||||
------------
|
|
||||||
|
|
||||||
Unit tests are located in unittests directory. Running the tests is
|
|
||||||
trivial. But you must synchronize your code and data with the test
|
|
||||||
code, which can be a tedious work if you lose discipline.
|
|
||||||
|
|
||||||
Sample data files are located in samples/ directory.
|
|
||||||
|
|
||||||
For running the entire test suite, use the following command:
|
|
||||||
|
|
||||||
$ ./unittests/run.py
|
|
||||||
|
|
||||||
If you know what you are doing, you can run the tests seperately. But
|
|
||||||
keep in your mind that tests can depend on each other. (?) The unit test
|
|
||||||
system doesn't know about that. The following command will run tests
|
|
||||||
in specfiletests and archivetests in unittests dir:
|
|
||||||
|
|
||||||
$ ./unittests/run.py specfile archive
|
|
||||||
|
|
||||||
|
|
||||||
Misc. Suggestions
|
|
||||||
-----------------
|
|
||||||
|
|
||||||
1. Demeter's Law
|
|
||||||
|
|
||||||
In OO programming, try to invoke Demeter's law.
|
|
||||||
One of the "rules" there is not directly accessing any
|
|
||||||
objects that are further than, 2/3 refs, away. So the
|
|
||||||
following code is OK.
|
|
||||||
destroy_system(a.system().name())
|
|
||||||
but the following isn't as robust
|
|
||||||
destroy_system(object_store.root().a.system.name())
|
|
||||||
As you can tell, this introduces too many implementation
|
|
||||||
dependencies. The rule of thumb is that, in these cases
|
|
||||||
this statement must have been elsewhere.... It may be a
|
|
||||||
good idea to not count the object scope in this case,
|
|
||||||
so in Python self.a means only one level of reference,
|
|
||||||
not two.
|
|
||||||
|
|
||||||
One quibble with this: it may be preferable not to insist
|
|
||||||
on this where it would be inefficient. So if everything
|
|
||||||
is neatly packed into one object contained in another
|
|
||||||
object, why replicate everything in the upper level? If
|
|
||||||
the semantics prevents dependency changes, then chains
|
|
||||||
of 3 or even 4 could be acceptable.
|
|
||||||
|
|
||||||
OTOH, in Python and C++, it's not always good to implement
|
|
||||||
accessor/modifier pairs for every property of an object.
|
|
||||||
It would be much simpler if you are not doing any special
|
|
||||||
processing on the property (e.g. if what the type system
|
|
||||||
does is sufficient).
|
|
||||||
|
|
||||||
The main rule of thumb in Demeter's Law is avoiding
|
|
||||||
putting more than, say, 10 methods in a class. That works
|
|
||||||
really well in practice, forcing refactoring every now
|
|
||||||
and then.
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
PISI is a new package manager implemented in python.
|
|
||||||
@@ -1,23 +0,0 @@
|
|||||||
|
|
||||||
PiSi ToDo List
|
|
||||||
==============
|
|
||||||
|
|
||||||
A list of tasks to accomplish, organized into priority sections
|
|
||||||
|
|
||||||
Legend:
|
|
||||||
|
|
||||||
- Todo
|
|
||||||
? Not determined if/how we have to do
|
|
||||||
/ In progress
|
|
||||||
+ Accomplished
|
|
||||||
|
|
||||||
1. Pre-Alpha
|
|
||||||
|
|
||||||
+ implement reading spec file
|
|
||||||
/ implement install database
|
|
||||||
? transaction/locking for database
|
|
||||||
|
|
||||||
2. Alpha
|
|
||||||
|
|
||||||
3. Beta
|
|
||||||
|
|
||||||
@@ -1,58 +0,0 @@
|
|||||||
#!/usr/bin/python
|
|
||||||
# -*- coding: utf-8 -*-
|
|
||||||
|
|
||||||
# python standard library
|
|
||||||
from os.path import basename
|
|
||||||
|
|
||||||
import pisi.context
|
|
||||||
import pisi.util
|
|
||||||
import pisi.ui
|
|
||||||
from pisi.build import PisiBuild, PisiBuildError
|
|
||||||
|
|
||||||
def usage(progname = "pisi-build"):
|
|
||||||
print """
|
|
||||||
Usage:
|
|
||||||
%s [options] package-name.pspec
|
|
||||||
""" %(progname)
|
|
||||||
|
|
||||||
def main():
|
|
||||||
import sys
|
|
||||||
import getopt
|
|
||||||
|
|
||||||
# wrapper for usage(progname) function
|
|
||||||
help = lambda: usage(sys.argv[0])
|
|
||||||
|
|
||||||
# getopt magic
|
|
||||||
try:
|
|
||||||
opts, args = getopt.getopt(sys.argv[1:],"h",["help"])
|
|
||||||
except getopt.GetoptError:
|
|
||||||
help()
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
for opt, arg in opts:
|
|
||||||
if opt == "-h":
|
|
||||||
help()
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
# pspec(PISI Spec) to be used for package
|
|
||||||
pspec = ""
|
|
||||||
|
|
||||||
# TODO: We accept only one pspec file (at a time) currently,
|
|
||||||
# but pisi-build may build many packages (pspecs given in order)
|
|
||||||
# sequentially.
|
|
||||||
if len(args) != 1:
|
|
||||||
help()
|
|
||||||
raise PisiBuildError, "pisi-build expects one (and only one) pspec file currently"
|
|
||||||
else:
|
|
||||||
pspec = args[0]
|
|
||||||
|
|
||||||
# What we need to do first is create a context with our specfile
|
|
||||||
ctx = pisi.context.Context(pspec)
|
|
||||||
|
|
||||||
# don't do the real job here. this is just a CLI!
|
|
||||||
pb = PisiBuild(ctx)
|
|
||||||
pb.build()
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,43 +0,0 @@
|
|||||||
#! /usr/bin/python
|
|
||||||
# -*- coding: utf-8 -*-
|
|
||||||
|
|
||||||
# sys modules
|
|
||||||
import sys
|
|
||||||
from optparse import OptionParser
|
|
||||||
|
|
||||||
import pisi.install
|
|
||||||
import pisi.util
|
|
||||||
|
|
||||||
class ArgError(Exception):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def main():
|
|
||||||
|
|
||||||
usage = "usage: %prog [options] <package>"
|
|
||||||
parser = OptionParser(usage=usage,version="%prog " + pisi.__version__)
|
|
||||||
parser.add_option("-D", "--destdir", action="store")
|
|
||||||
parser.add_option("-v", "--verbose", action="store_true",
|
|
||||||
dest="verbose", default=False,
|
|
||||||
help="detailed output")
|
|
||||||
parser.add_option("-d", "--debug", action="store_true", default=True)
|
|
||||||
parser.add_option("-n", "--dry-run", action="store_true",
|
|
||||||
default = "do not perform any action, just show what\
|
|
||||||
would be done")
|
|
||||||
|
|
||||||
(options, args) = parser.parse_args()
|
|
||||||
|
|
||||||
# package filename
|
|
||||||
package_fn = ""
|
|
||||||
|
|
||||||
# TODO: We accept only one package file arg ATM
|
|
||||||
if len(args) != 1:
|
|
||||||
print usage
|
|
||||||
raise ArgError, "pisi-install expects one (and only one) package file currently"
|
|
||||||
else:
|
|
||||||
package_fn = args[0]
|
|
||||||
|
|
||||||
pisi.install.install_package_file(package_fn)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
|
|
||||||
# PISI package version
|
|
||||||
__version__ = "0.1"
|
|
||||||
|
|
||||||
@@ -1,103 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
# unpack magic
|
|
||||||
# maintainer baris and meren
|
|
||||||
|
|
||||||
#standart lisbrary modules
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
import tarfile
|
|
||||||
import zipfile
|
|
||||||
|
|
||||||
#pisi modules
|
|
||||||
import util
|
|
||||||
|
|
||||||
class ArchiveBase(object):
|
|
||||||
def __init__(self, ctx):
|
|
||||||
self.type = ctx.spec.source.archiveType
|
|
||||||
self.fileName = os.path.basename(ctx.spec.source.archiveUri)
|
|
||||||
self.filePath = ctx.archives_dir() + '/' + self.fileName
|
|
||||||
|
|
||||||
def unpack(self, targetDir):
|
|
||||||
self.targetDir = targetDir
|
|
||||||
# first we check if we need to clean-up our working env.
|
|
||||||
if os.path.exists(self.targetDir):
|
|
||||||
util.clean_dir(self.targetDir)
|
|
||||||
else:
|
|
||||||
os.makedirs(self.targetDir)
|
|
||||||
|
|
||||||
class ArchiveTarFile(ArchiveBase):
|
|
||||||
def __init__(self, ctx):
|
|
||||||
super(ArchiveTarFile, self).__init__(ctx)
|
|
||||||
|
|
||||||
def unpack(self, targetDir):
|
|
||||||
super(ArchiveTarFile, self).unpack(targetDir)
|
|
||||||
|
|
||||||
rmode = ""
|
|
||||||
if self.type == 'tar':
|
|
||||||
rmode = 'r:'
|
|
||||||
elif self.type == 'targz':
|
|
||||||
rmode = 'r:gz'
|
|
||||||
elif self.type == 'tarbz2':
|
|
||||||
rmode = 'r:bz2'
|
|
||||||
tar = tarfile.open(self.filePath, rmode)
|
|
||||||
oldwd = os.getcwd()
|
|
||||||
os.chdir(self.targetDir)
|
|
||||||
for tarinfo in tar:
|
|
||||||
tar.extract(tarinfo)
|
|
||||||
os.chdir(oldwd)
|
|
||||||
tar.close()
|
|
||||||
|
|
||||||
class ArchiveZip(ArchiveBase):
|
|
||||||
def __init__(self, ctx):
|
|
||||||
super(ArchiveZip, self).__init__(ctx)
|
|
||||||
|
|
||||||
def unpack(self, targetDir):
|
|
||||||
super(ArchiveZip, self).unpack(targetDir)
|
|
||||||
|
|
||||||
zip = zipfile.ZipFile(self.filePath, 'r')
|
|
||||||
for file in zip.namelist():
|
|
||||||
ofile = self.targetDir + '/' + file
|
|
||||||
|
|
||||||
# a directory is present. lets continue
|
|
||||||
if os.path.isdir(ofile):
|
|
||||||
continue
|
|
||||||
# do we need to create parent directory for our file?
|
|
||||||
if not os.path.exists(os.path.dirname(ofile)):
|
|
||||||
os.mkdir(ofile)
|
|
||||||
continue
|
|
||||||
info = zip.getinfo(file)
|
|
||||||
# O.K. we know following line is dull. What we wanted to
|
|
||||||
# do was to compare the equality to 0xa0000000. But there
|
|
||||||
# is a known problem in Python regarding the hex/oct
|
|
||||||
# constants. Please see Guido's explanation at
|
|
||||||
# http://mail.python.org/pipermail/python-dev/2003-February/033029.html
|
|
||||||
if hex(info.external_attr)[2] == 'A':
|
|
||||||
target = zip.read(file)
|
|
||||||
os.symlink(target, ofile)
|
|
||||||
else:
|
|
||||||
buff = open (ofile, 'wb')
|
|
||||||
fileContent = zip.read(file)
|
|
||||||
buff.write(fileContent)
|
|
||||||
buff.close()
|
|
||||||
|
|
||||||
zip.close()
|
|
||||||
|
|
||||||
class Archive:
|
|
||||||
"""Unpack magic for Archive files..."""
|
|
||||||
|
|
||||||
def __init__(self, ctx):
|
|
||||||
"""accepted archive types:
|
|
||||||
targz, tarbz2, zip, tar"""
|
|
||||||
|
|
||||||
handlers = {
|
|
||||||
'targz': ArchiveTarFile,
|
|
||||||
'tarbz2': ArchiveTarFile,
|
|
||||||
'tar': ArchiveTarFile,
|
|
||||||
'zip': ArchiveZip
|
|
||||||
}
|
|
||||||
|
|
||||||
type = ctx.spec.source.archiveType
|
|
||||||
self.archive = handlers.get(type)(ctx)
|
|
||||||
|
|
||||||
def unpack(self, targetDir):
|
|
||||||
self.archive.unpack(targetDir)
|
|
||||||
@@ -1,132 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
# package bulding stuff
|
|
||||||
# maintainer: baris and meren
|
|
||||||
|
|
||||||
# python standard library
|
|
||||||
import os
|
|
||||||
|
|
||||||
from fetcher import Fetcher
|
|
||||||
from archive import Archive
|
|
||||||
|
|
||||||
# import pisipackage
|
|
||||||
import util
|
|
||||||
from ui import ui
|
|
||||||
|
|
||||||
class PisiBuildError(Exception):
|
|
||||||
pass
|
|
||||||
|
|
||||||
# FIXME: this eventually has to go to ui module
|
|
||||||
# Infact all ui calls has nothing to do with this build process.
|
|
||||||
# There more of them in PisiBuild...
|
|
||||||
# And maybe we should consider moving PisiBuild.build() back to pisi-build
|
|
||||||
# CLI too.
|
|
||||||
# exa: This, like all others, will have a GUI or CLI, interchangeably
|
|
||||||
def displayProgress(pd):
|
|
||||||
out = '\r%-30.30s %3d%% %12.2f %s' % \
|
|
||||||
(pd['filename'], pd['percent'], pd['rate'], pd['symbol'])
|
|
||||||
ui.info(out)
|
|
||||||
|
|
||||||
class PisiBuild:
|
|
||||||
"""PisiBuild class, provides the package build and creation routines"""
|
|
||||||
def __init__(self, context):
|
|
||||||
self.ctx = context
|
|
||||||
self.work_dir = self.ctx.pkg_work_dir()
|
|
||||||
|
|
||||||
self.spec = self.ctx.spec
|
|
||||||
|
|
||||||
def build(self):
|
|
||||||
ui.info("Building PISI source package: %s\n" % self.spec.source.name)
|
|
||||||
|
|
||||||
ui.info("Fetching source from: %s\n" % self.spec.source.archiveUri)
|
|
||||||
self.fetchArchive(displayProgress)
|
|
||||||
ui.info("Source archive is stored: %s/%s\n"
|
|
||||||
%(self.ctx.archives_dir(), self.spec.source.archiveName))
|
|
||||||
|
|
||||||
self.solveBuildDependencies()
|
|
||||||
|
|
||||||
ui.info("Unpacking archive...")
|
|
||||||
self.unpackArchive()
|
|
||||||
ui.info(" unpacked (%s)\n" % self.ctx.pkg_work_dir())
|
|
||||||
|
|
||||||
self.applyPatches()
|
|
||||||
|
|
||||||
try:
|
|
||||||
specdir = os.path.dirname(self.ctx.pspecfile)
|
|
||||||
self.actionScript = open("/".join([specdir,self.ctx.const.actions_file])).read()
|
|
||||||
except IOError, e:
|
|
||||||
ui.error ("Action Script: %s\n" % e)
|
|
||||||
return
|
|
||||||
|
|
||||||
# FIXME: It's wrong to assume that unpacked archive
|
|
||||||
# will create a name-version top-level directory.
|
|
||||||
# Archive module should give the exact location.
|
|
||||||
# (from the assumption is evil dept.)
|
|
||||||
os.chdir(self.ctx.pkg_work_dir() + "/" + self.spec.source.name + "-" + self.spec.source.version)
|
|
||||||
locals = globals = {}
|
|
||||||
|
|
||||||
try:
|
|
||||||
exec compile(self.actionScript , "error", "exec") in locals,globals
|
|
||||||
except SyntaxError, e:
|
|
||||||
ui.error ("Error : %s\n" % e)
|
|
||||||
return
|
|
||||||
|
|
||||||
self.configureSource(locals)
|
|
||||||
self.buildSource(locals)
|
|
||||||
self.installSource(locals)
|
|
||||||
|
|
||||||
# after all, we are ready to build/prepare the packages
|
|
||||||
self.buildPackages()
|
|
||||||
|
|
||||||
def fetchArchive(self, percentHook=None):
|
|
||||||
"""fetch an archive and store to ctx.archives_dir()
|
|
||||||
using fether.Fetcher"""
|
|
||||||
fetch = Fetcher(self.ctx)
|
|
||||||
|
|
||||||
# check if source already cached
|
|
||||||
destpath = fetch.filedest + "/" + fetch.filename
|
|
||||||
if os.access(destpath, os.R_OK):
|
|
||||||
if util.md5_file(destpath) == self.spec.source.archiveMD5:
|
|
||||||
ui.info('%s [cached]\n' % self.spec.source.archiveName)
|
|
||||||
return
|
|
||||||
|
|
||||||
if percentHook:
|
|
||||||
fetch.percentHook = percentHook
|
|
||||||
|
|
||||||
fetch.fetch()
|
|
||||||
|
|
||||||
# FIXME: What a ugly hack! We should really find a cleaner way for output.
|
|
||||||
if percentHook:
|
|
||||||
ui.info('\n')
|
|
||||||
|
|
||||||
def solveBuildDependencies(self):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def unpackArchive(self):
|
|
||||||
archive = Archive(self.ctx)
|
|
||||||
archive.unpack(self.work_dir)
|
|
||||||
|
|
||||||
def applyPatches(self):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def configureSource(self, locals):
|
|
||||||
func = self.ctx.const.setup_func
|
|
||||||
if func in locals:
|
|
||||||
ui.info("Configuring %s...\n" % self.spec.source.name)
|
|
||||||
locals[func]()
|
|
||||||
|
|
||||||
def buildSource(self, locals):
|
|
||||||
func = self.ctx.const.build_func
|
|
||||||
if func in locals:
|
|
||||||
ui.info("Building %s...\n" % self.spec.source.name)
|
|
||||||
locals[func]()
|
|
||||||
|
|
||||||
def installSource(self, locals):
|
|
||||||
func = self.ctx.const.install_func
|
|
||||||
if func in locals:
|
|
||||||
ui.info("Installing %s...\n" % self.spec.source.name)
|
|
||||||
locals[func]()
|
|
||||||
|
|
||||||
def buildPackages(self):
|
|
||||||
for package in self.spec.packages:
|
|
||||||
ui.info("** Building package %s\n" % package.name);
|
|
||||||
|
|
||||||
@@ -1,48 +0,0 @@
|
|||||||
|
|
||||||
colors = {'black' : "\033[30m",
|
|
||||||
'red' : "\033[31m",
|
|
||||||
'green' : "\033[32m",
|
|
||||||
'yellow' : "\033[33m",
|
|
||||||
'blue' : "\033[34m",
|
|
||||||
'purple' : "\033[35m",
|
|
||||||
'cyan' : "\033[36m",
|
|
||||||
'white' : "\033[37m",
|
|
||||||
'brightblack' : "\033[01;30m",
|
|
||||||
'brightred' : "\033[01;31m",
|
|
||||||
'brightgreen' : "\033[01;32m",
|
|
||||||
'brightyellow' : "\033[01;33m",
|
|
||||||
'brightblue' : "\033[01;34m",
|
|
||||||
'brightmagenta' : "\033[01;35m",
|
|
||||||
'brightcyan' : "\033[01;36m",
|
|
||||||
'brightwhite' : "\033[01;37m",
|
|
||||||
'underlineblack' : "\033[04;30m",
|
|
||||||
'underlinered' : "\033[04;31m",
|
|
||||||
'underlinegreen' : "\033[04;32m",
|
|
||||||
'underlineyellow' : "\033[04;33m",
|
|
||||||
'underlineblue' : "\033[04;34m",
|
|
||||||
'underlinemagenta' : "\033[04;35m",
|
|
||||||
'underlinecyan' : "\033[04;36m",
|
|
||||||
'underlinewhite' : "\033[04;37m",
|
|
||||||
'blinkingblack' : "\033[05;30m",
|
|
||||||
'blinkingred' : "\033[05;31m",
|
|
||||||
'blinkinggreen' : "\033[05;32m",
|
|
||||||
'blinkingyellow' : "\033[05;33m",
|
|
||||||
'blinkingblue' : "\033[05;34m",
|
|
||||||
'blinkingmagenta' : "\033[05;35m",
|
|
||||||
'blinkingcyan' : "\033[05;36m",
|
|
||||||
'blinkingwhite' : "\033[05;37m",
|
|
||||||
'backgroundblack' : "\033[07;30m",
|
|
||||||
'backgroundred' : "\033[07;31m",
|
|
||||||
'backgroundgreen' : "\033[07;32m",
|
|
||||||
'backgroundyellow' : "\033[07;33m",
|
|
||||||
'backgroundblue' : "\033[07;34m",
|
|
||||||
'backgroundmagenta' : "\033[07;35m",
|
|
||||||
'backgroundcyan' : "\033[07;36m",
|
|
||||||
'backgroundwhite' : "\033[07;37m",
|
|
||||||
'default' : "\033[0m" }
|
|
||||||
|
|
||||||
def colorize(msg, color):
|
|
||||||
if colors.has_key(color):
|
|
||||||
return colors[color] + msg + colors['default']
|
|
||||||
else:
|
|
||||||
return msg
|
|
||||||
@@ -1,104 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
# PISI configuration (static and dynamic)
|
|
||||||
|
|
||||||
from specfile import SpecFile
|
|
||||||
import oo
|
|
||||||
|
|
||||||
class Constants:
|
|
||||||
"Pisi constants"
|
|
||||||
|
|
||||||
c = oo.const()
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
self.c.lib_dir_suffix = "/var/lib/pisi"
|
|
||||||
self.c.db_dir_suffix = "/var/db/pisi"
|
|
||||||
self.c.archives_dir_suffix = "/var/cache/pisi/archives"
|
|
||||||
self.c.tmp_dir_suffix = "/var/tmp/pisi"
|
|
||||||
|
|
||||||
# directory suffixes for build
|
|
||||||
self.c.work_dir_suffix = "/work"
|
|
||||||
self.c.install_dir_suffix = "/install"
|
|
||||||
|
|
||||||
# file/directory names
|
|
||||||
self.c.actions_file = "actions.py"
|
|
||||||
self.c.files_dir = "files"
|
|
||||||
|
|
||||||
# functions in actions_file
|
|
||||||
self.c.setup_func = "setup"
|
|
||||||
self.c.build_func = "build"
|
|
||||||
self.c.install_func = "install"
|
|
||||||
|
|
||||||
def __getattr__(self, attr):
|
|
||||||
return getattr(self.c, attr)
|
|
||||||
|
|
||||||
def __setattr__(self, attr, value):
|
|
||||||
return setattr(self.c, attr, value)
|
|
||||||
|
|
||||||
def __delattr__(self, attr):
|
|
||||||
return delattr(self.c, attr)
|
|
||||||
|
|
||||||
class Context(object):
|
|
||||||
"""Config/Context Singleton"""
|
|
||||||
class __impl:
|
|
||||||
def __init__(self):
|
|
||||||
self.const = Constants()
|
|
||||||
# self.c.destdir = '' # install default to root by default
|
|
||||||
self.destdir = './tmp' # only for ALPHA
|
|
||||||
# the idea is that destdir can be set with --destdir=...
|
|
||||||
|
|
||||||
def setSpecFile(self, pspecfile):
|
|
||||||
self.pspecfile = pspecfile
|
|
||||||
spec = SpecFile()
|
|
||||||
spec.read(pspecfile)
|
|
||||||
spec.verify() # check pspec integrity
|
|
||||||
self.spec = spec
|
|
||||||
|
|
||||||
# directory accessor functions
|
|
||||||
# here is how it goes
|
|
||||||
# x_dir: system wide directory for storing info type x
|
|
||||||
# pkg_x_dir: per package directory for storing info type x
|
|
||||||
|
|
||||||
def lib_dir(self):
|
|
||||||
return self.destdir + self.const.lib_dir_suffix
|
|
||||||
|
|
||||||
def db_dir(self):
|
|
||||||
return self.destdir + self.const.db_dir_suffix
|
|
||||||
|
|
||||||
def archives_dir(self):
|
|
||||||
return self.destdir + self.const.archives_dir_suffix
|
|
||||||
|
|
||||||
def tmp_dir(self):
|
|
||||||
return self.destdir + self.const.tmp_dir_suffix
|
|
||||||
|
|
||||||
def pkg_work_dir(self):
|
|
||||||
packageDir = self.spec.source.name + '-' \
|
|
||||||
+ self.spec.source.version + '-' + self.spec.source.release
|
|
||||||
|
|
||||||
return self.destdir + self.const.tmp_dir_suffix \
|
|
||||||
+ '/' + packageDir + self.const.work_dir_suffix
|
|
||||||
|
|
||||||
def install_dir(self):
|
|
||||||
return self.tmp_dir() + self.const.install_dir_suffix
|
|
||||||
|
|
||||||
def pkg_install_dir(self):
|
|
||||||
packageDir = self.spec.source.name + '-' \
|
|
||||||
+ self.spec.source.version + '-' + self.spec.source.release
|
|
||||||
|
|
||||||
return self.destdir + self.const.tmp_dir_suffix \
|
|
||||||
+ '/' + packageDir + self.const.install_dir_suffix
|
|
||||||
|
|
||||||
__instance = __impl()
|
|
||||||
|
|
||||||
def __init__(self, pspecfile = None):
|
|
||||||
if pspecfile != None:
|
|
||||||
self.__instance.setSpecFile(pspecfile)
|
|
||||||
|
|
||||||
def __getattr__(self, attr):
|
|
||||||
return getattr(self.__instance, attr)
|
|
||||||
|
|
||||||
def __setattr__(self, attr, value):
|
|
||||||
return setattr(self.__instance, attr, value)
|
|
||||||
|
|
||||||
|
|
||||||
# create a default context WITH NO PSPEC
|
|
||||||
ctx = Context()
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
# dependency analyzer
|
|
||||||
# maintainer: eray and caglar
|
|
||||||
|
|
||||||
@@ -1,139 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
# download magic
|
|
||||||
# maintainer: baris and meren
|
|
||||||
|
|
||||||
# python standard library modules
|
|
||||||
import urlparse
|
|
||||||
import urllib2
|
|
||||||
import os
|
|
||||||
|
|
||||||
# pisi modules
|
|
||||||
import util
|
|
||||||
|
|
||||||
class FetchError (Exception):
|
|
||||||
pass
|
|
||||||
|
|
||||||
class Fetcher:
|
|
||||||
"""Yet another Pisi tool for fetching files from various sources.."""
|
|
||||||
def __init__(self, ctx):
|
|
||||||
self.uri = ctx.spec.source.archiveUri
|
|
||||||
self.filedest = ctx.archives_dir()
|
|
||||||
util.check_dir(self.filedest)
|
|
||||||
self.scheme = "file"
|
|
||||||
self.netloc = ""
|
|
||||||
self.filepath = ""
|
|
||||||
self.filename = ""
|
|
||||||
self.percent = 0
|
|
||||||
self.rate = 0.0
|
|
||||||
self.percentHook = None
|
|
||||||
from string import split
|
|
||||||
u = urlparse.urlparse(self.uri)
|
|
||||||
self.scheme, self.netloc, self.filepath = u[0], u[1], u[2]
|
|
||||||
self.filename = os.path.basename(self.uri)
|
|
||||||
|
|
||||||
def fetch (self):
|
|
||||||
"""Return value: Fetched file's full path.."""
|
|
||||||
|
|
||||||
if self.filename == "":
|
|
||||||
self.err("filename error")
|
|
||||||
|
|
||||||
if os.access(self.filedest, os.W_OK) == False:
|
|
||||||
self.err("no perm to write to dest dir")
|
|
||||||
|
|
||||||
scheme_err = lambda: self.err("unexpected scheme")
|
|
||||||
|
|
||||||
handlers = {
|
|
||||||
'file': self.fetchLocalFile,
|
|
||||||
'http': self.fetchRemoteFile,
|
|
||||||
'ftp' : self.fetchRemoteFile
|
|
||||||
}; handlers.get(self.scheme, scheme_err)()
|
|
||||||
|
|
||||||
return self.filedest + "/" + self.filename
|
|
||||||
|
|
||||||
def doGrab(self, file, dest, totalsize):
|
|
||||||
symbols = [' B/s', 'KB/s', 'MB/s', 'GB/s']
|
|
||||||
from time import time
|
|
||||||
tt, oldsize = int(time()), 0
|
|
||||||
p = Progress(totalsize)
|
|
||||||
bs, size = 1024, 0
|
|
||||||
symbol, depth = "B/s", 0
|
|
||||||
st = time()
|
|
||||||
chunk = file.read(bs)
|
|
||||||
size = size + len(chunk)
|
|
||||||
self.percent = p.update(size)
|
|
||||||
while chunk:
|
|
||||||
dest.write(chunk)
|
|
||||||
chunk = file.read(bs)
|
|
||||||
size = size + len(chunk)
|
|
||||||
ct = time()
|
|
||||||
if int(tt) != int(ct):
|
|
||||||
self.rate = size / (ct - st)
|
|
||||||
while self.rate > 1000 and depth < 3:
|
|
||||||
self.rate /= 1024
|
|
||||||
depth += 1
|
|
||||||
symbol, depth = symbols[depth], 0
|
|
||||||
oldsize, tt = size, time()
|
|
||||||
if p.update(size):
|
|
||||||
self.percent = p.percent
|
|
||||||
if self.percentHook != None:
|
|
||||||
retval = {'filename': self.filename,
|
|
||||||
'percent' : self.percent,
|
|
||||||
'rate': self.rate,
|
|
||||||
'symbol': symbol}
|
|
||||||
self.percentHook(retval)
|
|
||||||
|
|
||||||
dest.close()
|
|
||||||
|
|
||||||
|
|
||||||
def fetchLocalFile (self):
|
|
||||||
from shutil import copyfile
|
|
||||||
|
|
||||||
if os.access(self.filepath, os.F_OK) == False:
|
|
||||||
self.err("no such file or no perm to read")
|
|
||||||
|
|
||||||
dest = open(self.filedest + "/" + self.filename , "w")
|
|
||||||
totalsize = os.path.getsize(self.filepath)
|
|
||||||
file = open(self.filepath)
|
|
||||||
self.doGrab(file, dest, totalsize)
|
|
||||||
|
|
||||||
|
|
||||||
def fetchRemoteFile (self):
|
|
||||||
from httplib import HTTPException
|
|
||||||
|
|
||||||
try:
|
|
||||||
file = urllib2.urlopen(self.uri)
|
|
||||||
headers = file.info()
|
|
||||||
|
|
||||||
except ValueError, e:
|
|
||||||
self.err('%s' % (e, ))
|
|
||||||
except IOError, e:
|
|
||||||
self.err('%s' % (e, ))
|
|
||||||
except OSError, e:
|
|
||||||
self.err('%s' % (e, ))
|
|
||||||
except HTTPException, e:
|
|
||||||
self.err(('(%s): %s') % (e.__class__.__name__, e))
|
|
||||||
|
|
||||||
if not headers is None and not headers.has_key('Content-Length'):
|
|
||||||
self.err('file not found')
|
|
||||||
else: totalsize = int(headers['Content-Length'])
|
|
||||||
|
|
||||||
dest = open(self.filedest + "/" + self.filename , "w")
|
|
||||||
self.doGrab(file, dest, totalsize)
|
|
||||||
|
|
||||||
|
|
||||||
def err (self, error):
|
|
||||||
raise FetchError(error)
|
|
||||||
|
|
||||||
class Progress:
|
|
||||||
def __init__(self, totalsize):
|
|
||||||
self.totalsize = totalsize
|
|
||||||
self.percent = 0
|
|
||||||
|
|
||||||
def update(self, size):
|
|
||||||
percent = (size * 100) / self.totalsize
|
|
||||||
if percent and self.percent is not percent:
|
|
||||||
self.percent = percent
|
|
||||||
return percent
|
|
||||||
else:
|
|
||||||
return 0
|
|
||||||
|
|
||||||
@@ -1,46 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
|
|
||||||
#import package
|
|
||||||
from specfile import *
|
|
||||||
from package import Package
|
|
||||||
import util
|
|
||||||
from context import ctx
|
|
||||||
from ui import ui
|
|
||||||
import installdb
|
|
||||||
import packagedb
|
|
||||||
import dependency
|
|
||||||
#import conflicts
|
|
||||||
|
|
||||||
class InstallError(Exception):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def install_package_file(package_fn):
|
|
||||||
|
|
||||||
package = Package(package_fn, 'r')
|
|
||||||
# extract control files
|
|
||||||
util.clean_dir(ctx.install_dir())
|
|
||||||
ui.info('extracting files\n')
|
|
||||||
package.extract_PISI_files(ctx.install_dir())
|
|
||||||
|
|
||||||
# verify package
|
|
||||||
# check if we have all required files
|
|
||||||
|
|
||||||
metadata = MetaData()
|
|
||||||
metadata.read(ctx.install_dir() + '/metadata.xml')
|
|
||||||
# check package semantics
|
|
||||||
if not metadata.verify():
|
|
||||||
raise InstallError("MetaData format wrong")
|
|
||||||
|
|
||||||
# check file system requirements
|
|
||||||
# what to do if / is split into /usr, /var, etc.?
|
|
||||||
|
|
||||||
# check conflicts
|
|
||||||
# check dependencies
|
|
||||||
|
|
||||||
# unzip package in place
|
|
||||||
|
|
||||||
# update databases
|
|
||||||
|
|
||||||
# installdb
|
|
||||||
installdb.install(spec.spec.install_dir() + '/files.xml')
|
|
||||||
|
|
||||||
@@ -1,52 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
# installation database
|
|
||||||
# maintainer: eray and caglar
|
|
||||||
|
|
||||||
import os
|
|
||||||
import bsddb.dbshelve as shelve
|
|
||||||
|
|
||||||
from context import ctx
|
|
||||||
import util
|
|
||||||
|
|
||||||
util.check_dir(ctx.db_dir())
|
|
||||||
d = shelve.open(ctx.db_dir() + '/install.bdb')
|
|
||||||
files_dir = ctx.archives_dir() + "/files"
|
|
||||||
|
|
||||||
class InstallDBError(Exception):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def files_name(name, version, release):
|
|
||||||
return files_dir + '/' + name + '-' + version + '-' + release
|
|
||||||
|
|
||||||
def files(n, v, r):
|
|
||||||
return file(files_name(n,v,r))
|
|
||||||
|
|
||||||
def is_recorded(name, version, release):
|
|
||||||
key = name + version + release
|
|
||||||
return d.has_key(key)
|
|
||||||
|
|
||||||
def is_installed(name, version, release):
|
|
||||||
key = name + version + release
|
|
||||||
return is_recorded(name,version,release) and d[key]=='i'
|
|
||||||
|
|
||||||
def is_removed(name, version, release):
|
|
||||||
key = name + version + release
|
|
||||||
return is_recorded(name,version,release) and d[key]=='r'
|
|
||||||
|
|
||||||
def install(name, version, release, files_xml):
|
|
||||||
key = name + version + release
|
|
||||||
if is_installed(name, version, release):
|
|
||||||
raise InstallDBError("already installed")
|
|
||||||
d[key] = 'i'
|
|
||||||
util.copy_file(files_xml, files_name(name, version, release))
|
|
||||||
|
|
||||||
def remove(name, version, release):
|
|
||||||
key = name + version + release
|
|
||||||
d[key] = 'r'
|
|
||||||
|
|
||||||
def purge(name, version, release):
|
|
||||||
os.unlink(files_name(name, version, release))
|
|
||||||
key = name + version + release
|
|
||||||
del d[key]
|
|
||||||
|
|
||||||
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
# OO extensions
|
|
||||||
# thes are really cool, you can't do this in C++ :)
|
|
||||||
|
|
||||||
class const:
|
|
||||||
"Constant members implementation"
|
|
||||||
class ConstError(TypeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def __setattr__(self, name, value):
|
|
||||||
if self.__dict__.has_key(name):
|
|
||||||
raise self.ConstError, "Can't rebind constant: %s" % name
|
|
||||||
# Binding an attribute once to a const is available
|
|
||||||
self.__dict__[name] = value
|
|
||||||
|
|
||||||
def __delattr__(self, name):
|
|
||||||
if self.__dict__.has_key(name):
|
|
||||||
raise self.ConstError, "Can't unbind constant: %s" % name
|
|
||||||
# we don't have an attribute by this name
|
|
||||||
raise NameError, name
|
|
||||||
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
# package abstraction
|
|
||||||
# provides methods to add/remove files, extract control files
|
|
||||||
# maintainer: baris and meren
|
|
||||||
|
|
||||||
class Package:
|
|
||||||
"""Package: PISI package class"""
|
|
||||||
def __init__(self, packagefn, mode):
|
|
||||||
self.filename = packagefn
|
|
||||||
self.mode = mode # bu gerekli mi?
|
|
||||||
# etc. etc.
|
|
||||||
|
|
||||||
def add_file(self, fn):
|
|
||||||
"""add a file to package"""
|
|
||||||
|
|
||||||
def extract(self, outdir):
|
|
||||||
"""extract entire package contents to directory"""
|
|
||||||
extract_dir('', outdir) # means package root
|
|
||||||
|
|
||||||
def extract_file(self, path, outdir):
|
|
||||||
"""extract file with path to outdir"""
|
|
||||||
|
|
||||||
def extract_dir(self, dir, outdir):
|
|
||||||
"""extract directory recursively"""
|
|
||||||
|
|
||||||
def extract_PISI_files(self, outdir):
|
|
||||||
"""extract PISI control files: metadata.xml, files.xml,
|
|
||||||
action scripts, etc."""
|
|
||||||
self.extract_file('metadata.xml', outdir)
|
|
||||||
self.extract_file('files.xml', outdir)
|
|
||||||
self.extract_dir('Config', outdir)
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
# package database
|
|
||||||
# interface for update/query to local package repository
|
|
||||||
# maintainer: eray and caglar
|
|
||||||
|
|
||||||
# we basically store everything in PackageInfo class
|
|
||||||
# yes, we are cheap
|
|
||||||
|
|
||||||
import bsddb.dbshelve as shelve
|
|
||||||
|
|
||||||
import util
|
|
||||||
from context import ctx
|
|
||||||
|
|
||||||
util.check_dir(ctx.db_dir())
|
|
||||||
d = shelve.open(ctx.db_dir() + '/package.bdb')
|
|
||||||
|
|
||||||
def has_package(name):
|
|
||||||
return d.has_key(name)
|
|
||||||
|
|
||||||
def get_package(name):
|
|
||||||
return d[name]
|
|
||||||
|
|
||||||
def add_package(name, package_info):
|
|
||||||
d[name] = package_info
|
|
||||||
|
|
||||||
def remove_package(name):
|
|
||||||
del d[name]
|
|
||||||
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
# package source database
|
|
||||||
# interface for update/query to local package repository
|
|
||||||
# maintainer: eray and caglar
|
|
||||||
|
|
||||||
# we basically store everything in sourceinfo class
|
|
||||||
# yes, we are cheap
|
|
||||||
|
|
||||||
import bsddb.dbshelve as shelve
|
|
||||||
|
|
||||||
util.check_dir(config.db_dir())
|
|
||||||
d = shelve.open(config.db_dir() + '/source.bdb')
|
|
||||||
|
|
||||||
def add_source(name, source_info):
|
|
||||||
d[name] = source_info
|
|
||||||
|
|
||||||
def remove_source(name):
|
|
||||||
del d[name]
|
|
||||||
@@ -1,105 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
# read/write PISI source package specification file
|
|
||||||
|
|
||||||
import xml.dom.minidom
|
|
||||||
from xmlfile import *
|
|
||||||
from os.path import basename
|
|
||||||
|
|
||||||
class PatchInfo:
|
|
||||||
def __init__(self, filenm, ctype):
|
|
||||||
self.filename = filenm
|
|
||||||
self.compressionType = ctype
|
|
||||||
|
|
||||||
def __init__(self, node):
|
|
||||||
self.filename = getNodeText(node)
|
|
||||||
self.compressionType = getNodeAttribute(node, "compressionType")
|
|
||||||
|
|
||||||
class DepInfo:
|
|
||||||
def __init__(self, node):
|
|
||||||
self.package = getNodeText(node).strip()
|
|
||||||
self.versionFrom = getNodeAttribute(node, "versionFrom")
|
|
||||||
|
|
||||||
class HistoryInfo:
|
|
||||||
def __init__(self, node):
|
|
||||||
self.date = getNodeText(getNode(node, "Date"))
|
|
||||||
self.version = getNodeText(getNode(node, "Version"))
|
|
||||||
self.release = getNodeText(getNode(node, "Release"))
|
|
||||||
|
|
||||||
class PathInfo:
|
|
||||||
def __init__(self, node):
|
|
||||||
self.pathname = getNodeText(node)
|
|
||||||
self.fileType = getNodeAttribute(node, "fileType")
|
|
||||||
|
|
||||||
# a structure to hold source information
|
|
||||||
class SourceInfo:
|
|
||||||
pass
|
|
||||||
|
|
||||||
class PackageInfo:
|
|
||||||
def __init__(self, node):
|
|
||||||
self.name = getNodeText(getNode(node, "Name"))
|
|
||||||
self.summary = getNodeText(getNode(node, "Summary"))
|
|
||||||
self.description = getNodeText(getNode(node, "Description"))
|
|
||||||
self.category = getNodeText(getNode(node, "Category"))
|
|
||||||
iDepElts = getAllNodes(node, "InstallDependencies")
|
|
||||||
self.installDeps = [DepInfo(x) for x in iDepElts]
|
|
||||||
rtDepElts = getAllNodes(node, "RuntimeDependencies")
|
|
||||||
self.runtimeDeps = [DepInfo(x) for x in rtDepElts]
|
|
||||||
self.paths = [PathInfo(x) for x in getAllNodes(node, "Files/Path")]
|
|
||||||
|
|
||||||
class SpecFile(XmlFile):
|
|
||||||
"""A class for reading/writing from/to a PSPEC (PISI SPEC) file."""
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
XmlFile.__init__(self,"PSPEC")
|
|
||||||
|
|
||||||
def read(self, filename):
|
|
||||||
"""Read PSPEC file"""
|
|
||||||
|
|
||||||
self.readxml(filename)
|
|
||||||
|
|
||||||
self.source = SourceInfo()
|
|
||||||
self.source.name = self.getChildText("Source/Name")
|
|
||||||
archiveNode = self.getNode("Source/Archive")
|
|
||||||
self.source.archiveUri = getNodeText(archiveNode).strip()
|
|
||||||
self.source.archiveName = basename(self.source.archiveUri)
|
|
||||||
self.source.archiveType = getNodeAttribute(archiveNode, "archType")
|
|
||||||
self.source.archiveMD5 = getNodeAttribute(archiveNode, "md5sum")
|
|
||||||
patchElts = self.getChildElts("Source/Patches")
|
|
||||||
if patchElts:
|
|
||||||
self.source.patches = [PatchInfo(p) for p in patchElts]
|
|
||||||
|
|
||||||
buildDepElts = self.getChildElts("Source/BuildDependencies")
|
|
||||||
if buildDepElts:
|
|
||||||
self.source.buildDeps = [DepInfo(d) for d in buildDepElts]
|
|
||||||
|
|
||||||
historyElts = self.getAllNodes("History/Update")
|
|
||||||
self.source.history = [HistoryInfo(x) for x in historyElts]
|
|
||||||
|
|
||||||
# As we have no Source/Version tag we need to get
|
|
||||||
# the last version and release information
|
|
||||||
# from the first child of History/Update. And it works :)
|
|
||||||
self.source.version = self.source.history[0].version
|
|
||||||
self.source.release = self.source.history[0].release
|
|
||||||
|
|
||||||
# find all binary packages
|
|
||||||
packageElts = self.getAllNodes("Package")
|
|
||||||
self.packages = [PackageInfo(p) for p in packageElts]
|
|
||||||
|
|
||||||
def verify(self):
|
|
||||||
"""Verify PSPEC structures, are they what we want of them?"""
|
|
||||||
return True
|
|
||||||
|
|
||||||
def write(self, filename):
|
|
||||||
"""Write PSPEC file"""
|
|
||||||
self.writexml(filename)
|
|
||||||
|
|
||||||
class MetaData(SpecFile):
|
|
||||||
"""This is a superset of the source spec definition"""
|
|
||||||
|
|
||||||
def read(self, filename):
|
|
||||||
SpecFile.read(filename)
|
|
||||||
distribution = self.getNodeText("Source/Distribution")
|
|
||||||
distributionRelease = self.getNodeText("Source/DistributionRelease")
|
|
||||||
architecture = self.getNodeText("Source/Architecture")
|
|
||||||
installSize = self.getNodeText("Source/InstallSize")
|
|
||||||
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
# generic user interface
|
|
||||||
|
|
||||||
import sys
|
|
||||||
from colors import colorize
|
|
||||||
|
|
||||||
def register(_impl):
|
|
||||||
""" Register a UI implementation"""
|
|
||||||
ui = _impl
|
|
||||||
|
|
||||||
# default UI implementation
|
|
||||||
class CLI:
|
|
||||||
def __init__(self, debuggy = True):
|
|
||||||
self.showDebug = debuggy
|
|
||||||
|
|
||||||
def info(self, msg):
|
|
||||||
sys.stdout.write(colorize(msg, 'blue'))
|
|
||||||
sys.stdout.flush()
|
|
||||||
|
|
||||||
def debug(self, msg):
|
|
||||||
if showDebug:
|
|
||||||
sys.stdout.write(msg)
|
|
||||||
sys.stdout.flush()
|
|
||||||
|
|
||||||
def error(self,msg):
|
|
||||||
sys.stdout.write(colorize(msg, 'red'))
|
|
||||||
sys.stdout.flush()
|
|
||||||
|
|
||||||
# default UI is CLI
|
|
||||||
ui = CLI()
|
|
||||||
|
|
||||||
@@ -1,69 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
# misc. utility functions, including process and file utils
|
|
||||||
# maintainer: eray and caglar and baris and meren!
|
|
||||||
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
import md5
|
|
||||||
from ui import ui
|
|
||||||
|
|
||||||
class FileError(Exception):
|
|
||||||
pass
|
|
||||||
|
|
||||||
class UtilError(Exception):
|
|
||||||
pass
|
|
||||||
|
|
||||||
# shorthand to check if a file exists
|
|
||||||
def check_file(file, mode = os.F_OK):
|
|
||||||
if not os.access(file, mode):
|
|
||||||
raise FileError("File " + file + " not found")
|
|
||||||
|
|
||||||
# check if directory exists, and create if it doesn't
|
|
||||||
# works recursively
|
|
||||||
# FIXME: could have a better name
|
|
||||||
def check_dir(dir):
|
|
||||||
dir = dir.strip().rstrip("/")
|
|
||||||
if not os.access(dir, os.F_OK):
|
|
||||||
os.makedirs(dir)
|
|
||||||
|
|
||||||
def clean_dir(top):
|
|
||||||
"""Remove all content of a directory (top)"""
|
|
||||||
for root, dirs, files in os.walk(top, topdown=False):
|
|
||||||
for name in files:
|
|
||||||
os.remove(os.path.join(root, name))
|
|
||||||
for name in dirs:
|
|
||||||
os.rmdir(os.path.join(root, name))
|
|
||||||
|
|
||||||
def copy_file(s,d):
|
|
||||||
check_file(s)
|
|
||||||
check_dir(os.path.dirname(d))
|
|
||||||
fs = file(s, 'rb')
|
|
||||||
fd = file(d, 'wb')
|
|
||||||
for l in fs:
|
|
||||||
fd.write(l)
|
|
||||||
|
|
||||||
def copy_dir():
|
|
||||||
raise UtilError("not implemented")
|
|
||||||
|
|
||||||
def md5_file(filename):
|
|
||||||
m = md5.new()
|
|
||||||
f = file(filename, 'rb')
|
|
||||||
for l in f:
|
|
||||||
m.update(l)
|
|
||||||
return m.hexdigest()
|
|
||||||
|
|
||||||
# run a command non-interactively
|
|
||||||
def run_batch(cmd):
|
|
||||||
ui.info('running ' + cmd)
|
|
||||||
a = os.popen(cmd)
|
|
||||||
lines = a.readlines()
|
|
||||||
ret = a.close()
|
|
||||||
ui.debug('return value ' + ret)
|
|
||||||
successful = ret == None
|
|
||||||
if not successful:
|
|
||||||
ui.error('ERROR: executing command: ' + cmd + '\n' + strlist(lines))
|
|
||||||
return (successful,lines)
|
|
||||||
|
|
||||||
# print a list
|
|
||||||
def strlist(l):
|
|
||||||
return string.join(map(lambda x: str(x) + ' ', l))
|
|
||||||
@@ -1,141 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
# some helper functions for using minidom
|
|
||||||
|
|
||||||
import xml.dom.minidom as mdom
|
|
||||||
|
|
||||||
class XmlError(Exception):
|
|
||||||
pass
|
|
||||||
|
|
||||||
# static functions
|
|
||||||
|
|
||||||
def getNodeAttribute(node, attrname):
|
|
||||||
for i in range(node.attributes.length):
|
|
||||||
attr = node.attributes.item(i)
|
|
||||||
if attr.name == attrname:
|
|
||||||
return attr.childNodes[0].data
|
|
||||||
|
|
||||||
def getNodeText(node):
|
|
||||||
# get the first child
|
|
||||||
try:
|
|
||||||
child = node.childNodes[0]
|
|
||||||
except IndexError:
|
|
||||||
return None
|
|
||||||
except AttributeError: # no node by that name
|
|
||||||
return None
|
|
||||||
if child.nodeType == child.TEXT_NODE:
|
|
||||||
return child.data
|
|
||||||
else:
|
|
||||||
raise XmlError("getNodeText: Expected text node, got something else!")
|
|
||||||
|
|
||||||
def getChildText(node_s, tagpath):
|
|
||||||
node = getNode(node_s, tagpath)
|
|
||||||
if not node:
|
|
||||||
return None
|
|
||||||
return getNodeText(node)
|
|
||||||
|
|
||||||
def getChildElts(node):
|
|
||||||
"""get only child elements"""
|
|
||||||
return filter(lambda x:x.nodeType == x.ELEMENT_NODE, node.childNodes)
|
|
||||||
|
|
||||||
def getNode(node, tagpath):
|
|
||||||
"""returns the *first* matching node for given tag path."""
|
|
||||||
|
|
||||||
tags = tagpath.split('/')
|
|
||||||
|
|
||||||
# iterative code to search for the path
|
|
||||||
|
|
||||||
# get DOM for top node
|
|
||||||
nodeList = node.getElementsByTagName(tags[0])
|
|
||||||
if len(nodeList) == 0:
|
|
||||||
return None # not found
|
|
||||||
|
|
||||||
node = nodeList[0] # discard other matches
|
|
||||||
for tag in tags[1:]:
|
|
||||||
nodeList = node.getElementsByTagName(tag)
|
|
||||||
if len(nodeList) == 0:
|
|
||||||
return None
|
|
||||||
else:
|
|
||||||
node = nodeList[0]
|
|
||||||
|
|
||||||
return node
|
|
||||||
|
|
||||||
def getAllNodes(node, tags):
|
|
||||||
"""retrieve all nodes that match a given tag path."""
|
|
||||||
|
|
||||||
if len(tags) == 0:
|
|
||||||
return []
|
|
||||||
|
|
||||||
nodeList = node.getElementsByTagName(tags[0])
|
|
||||||
if len(nodeList) == 0:
|
|
||||||
return []
|
|
||||||
|
|
||||||
for tag in tags[1:]:
|
|
||||||
results = map(lambda x: x.getElementsByTagName(tag),nodeList)
|
|
||||||
nodeList = []
|
|
||||||
for x in results:
|
|
||||||
nodeList.extend(x)
|
|
||||||
pass # emacs indentation error, keep it here
|
|
||||||
|
|
||||||
if len(nodeList) == 0:
|
|
||||||
return []
|
|
||||||
|
|
||||||
return nodeList
|
|
||||||
|
|
||||||
|
|
||||||
# xmlfile class that further abstracts a dom object
|
|
||||||
|
|
||||||
class XmlFile(object):
|
|
||||||
"""A class for retrieving information from an XML file"""
|
|
||||||
|
|
||||||
def __init__(self, rootTag):
|
|
||||||
self.rootTag = rootTag
|
|
||||||
|
|
||||||
def readxml(self, fileName):
|
|
||||||
self.dom = mdom.parse(fileName)
|
|
||||||
|
|
||||||
def writexml(self, fileName):
|
|
||||||
f = file(fileName,'w')
|
|
||||||
self.dom.writexml(f)
|
|
||||||
|
|
||||||
def verifyRootTag(self):
|
|
||||||
if self.dom.documentElement.tagName != self.rootTag:
|
|
||||||
raise XmlError("Root tagname not " + self.rootTag + " as expected")
|
|
||||||
|
|
||||||
def getNode(self, tagPath):
|
|
||||||
"""returns the *first* matching node for given tag path."""
|
|
||||||
self.verifyRootTag()
|
|
||||||
return getNode(self.dom.documentElement, tagPath)
|
|
||||||
|
|
||||||
def getAllNodes(self, tagPath):
|
|
||||||
"""returns all nodes matching a given tag path."""
|
|
||||||
self.verifyRootTag()
|
|
||||||
tags = tagPath.split('/')
|
|
||||||
return getAllNodes(self.dom.documentElement, tags)
|
|
||||||
|
|
||||||
def getChildren(self, tagpath):
|
|
||||||
""" returns the children of the given path"""
|
|
||||||
node = self.getNode(tagpath)
|
|
||||||
return node.childNodes
|
|
||||||
|
|
||||||
# get only elements of a given type
|
|
||||||
# BUG: this doesn't work
|
|
||||||
def getChildrenWithType(self, tagpath, type):
|
|
||||||
""" returns the children of the given path, only with given type """
|
|
||||||
node = self.getNode(tagpath)
|
|
||||||
return filter(lambda x:x.nodeType == type, node.childNodes)
|
|
||||||
|
|
||||||
# get only child elements
|
|
||||||
def getChildElts(self, tagpath):
|
|
||||||
""" returns the children of the given path, only with given type """
|
|
||||||
node = self.getNode(tagpath)
|
|
||||||
try:
|
|
||||||
return filter(lambda x:x.nodeType == x.ELEMENT_NODE, node.childNodes)
|
|
||||||
except AttributeError:
|
|
||||||
return None
|
|
||||||
|
|
||||||
def getChildText(self, tagpath):
|
|
||||||
node = self.getNode(tagpath)
|
|
||||||
if not node:
|
|
||||||
return None
|
|
||||||
return getNodeText(node)
|
|
||||||
|
|
||||||
Reference in New Issue
Block a user