Pisilinux3/pisi
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

pisi-1.1 -> trunk ...

Browse Source 
Since most of the development went on the branch and trunk did not see 
significant commits for so long...
This commit is contained in:
Faik Uygur
2007-07-11 08:58:21 +00:00
parent 0cd189a582 eda4d3d346
commit 029d1675c1
74 changed files with 6248 additions and 8146 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TODO
-189
View File
@@ -1,189 +0,0 @@
Here we take notes of the strange stuff, so we can refactor them after
Pardus 2007 release.
==> Python 2.5 support
* We should use python2.5
==> Locale support
* bindtextdomain and textdomain calls are not necessary, because
gettext.translation() dont use them.
* setlocale call is probably necessary for some stuff, but not for
message translation, it seems gettext.translation() api looks for
environment LC_ALL, LC_MESSAGES anyway.
* pygettext.py shouldn't be needed at all. Plain xgettext works with
python source.
==> Utility functions
* sha1_file and sha1_data functions has some
exception confusion. These two functions should be a lot simpler.
* get_file_hashes is overly complex and contains strange code like:
for r, d, f in os.walk(parent, topdown=False): pass
* To estimate required disk size for the packages more accurately we can
calculate package size + (nr of files * inode size of fs).
* Why in a world like this there exists *parse_package* util functions? These must be all methods of package class
* Remove all unneeded util functions, most of them belongs its classes
==> code readability
* Public functions should contain doc strings.
* Python builtins like file, list, etc should be avoided in variable names.
There is even a file.py module!
* a,b,c,d,f,r,_i,A,B,C,G are equally bad.
* some import'ed modules are not used inside the importer modules, cleanup needed.
* Would be very good if we can avoid "from x import y" form, it makes it
harder to reload modules.
* Convert all String Concatenation into "%s" % string form which is much faster, readable and correct
==> exceptions
* Current model is bad. Exception names should tell what is the error type. Instead
we have one Error exception in every module. If I call pisi.api.install("lala"),
I should get pisi.api.PackageNotFound or pisi.install.PackageNotFound or something
like that. For every kind of error, you get pisi.api.Error now, and only way to find
out exact error is try to parse error string (which is localized).
* class Exception(Exception) is evil.
* There shouldn't be a bare Except: clause in pisi modules.
==> database
* We should have a DB performance test suite to find important points to make faster,
if we cannot measure, we cannot improve.
* [FIXED] We should use "2" as pickle protocol. currently we use old binary pickle protocol "1". New
protocol generally produces more compact pickled objects (dbshelve.py)
* History, and Summary-Description objects should be kept in seperate db files. This way
we can have faster searches and probably faster access to packages and better memory footprint
* For big objects with many member objects (Package, from pisi.metadata) we might investigate
writing our own simple serialization routines for some objects like Conflicts or additionalfiles.
* context singleton is a mess. Maybe we should consider another approach to access common db objects
* ComponentDB should not hold lists of Packages that belongs to it. A new index for components to package DB
should be enough.
* Current implementation of repository system is suboptimal. We add every package to a Dictionary as value
and repository name as key and add that dictionary to db. For example if I am adding package foo to
packages db, I am actually adding, a Dictionary with ["repotype", foo] to db.
This is cheap, bloated and slow. We must have different repository databases for packages.
* We should really consider using a simple and fast relational database (sqlite) for database operations. Here
is the rationale and some issues.
- Current implementation uses berkeleydb as a storage of dictionaries, with keys and Pickled Python objects.
This approach simplifies coding requirements but leads to some problems.
1. Some operations cannot be done effectively. Searching can only be done by iterating all objects and searching
one by one. component, reverse dependencies etc are holding lists of package names that are updated on every
package operation, Multiple repository problem is
2.Performance: although fast, pickling and unpickling whole objects is still a time consuming process.
Pickled Objects contains object metadata and are larger then the data they hold, so reading, writing pickled
Objects to db is slower than reading and writing only necessary fields. large objects (Order of kilobytes)
causes berkeleydb data buckets to overflow and caches works less efficently.
4.Disk usage: Using pickled objects requires more memory and disk space. Total size of a current pisi database
is over 100Mb, if we modify files.bdb to only hold filenames and packages, size is still around 50MB.
Rational Databases offers number of advantages and disadvantages:
Advantages:
+ Relational databases offers much more flexible operations on stored data. Searching, grouping, sorting
can be done by database engine efficently.
+ There will be only a single database file and much smaller database size compared to berkeleydb files
holding pickled python objects.
+ Since database no longer holds pickled python objects, it is possible to read and modify the database
by other programs written in different languages (Java, C++ etc.)
+ SQL lite is generally very fast, probably beating "Berkeleydb with pickled objects" by the order of 4-5 on
queries.
Disadvantages
- Using a relational database will require more coding when adding and retrieving contents of objects.
- Introducing SQL language makes code more complex and harder to maintain.
- Designing a good schema, identifying required indexes etc. takes time
- Some developers are superstitious about sql and relational databases ;)
==> class attributes
* In some classes there are some attributes assigned but never
used. (see remove_unused_attributes.patch)
* PiSi uses heavy classes and creates thousands of instances of them
(Package, Metadata, Dependency, ...). These classes should define
__slots__ to reduce the heap used.
==> actionsAPI
* Get rid of ugly exception model
* Refactor/clean the code
* Add strict checks to models
* Maybe? Get rid of functional logic, switch to OO one
* ActionsAPI still needs an updated API document
==> PiSi API
* Write a real one
==> Function parameters
* Check function parameters and see if some parameters (especially the
ones named tmpDir/tmp_dir/target_dir) are redundant now. See
http://liste.uludag.org.tr/uludag-commits/2007-February/010070.html
* conflict.py
conflicts method takes a pkg_name parameter but doesn't use it at all
==> Fixes (that need API breakage)
* http://liste.uludag.org.tr/uludag-commits/2007-February/010117.html
==> Logging
* Improve logging with all needed update/downgrade/install/remove information so one can easily find the history of packages,
currently PiSi wrotes everyting into logs which is not usefull. Also this information can be used for rollback and reporting.
==> operations.py and atomicoperations.py
* These modules have gone so big and become untraceable... divide all operations to their related modules
under a pisi.operations module...
==> function code lengths
* we have some functions that goes pages long... divide all of them to small digestable chunks..
avoid writing functions longer than your editor's screen.
==> assertions
* Lots of assertions used in the code with no following descriptive string information.
==> repo order
* When trying to install a package or looking for a dependency for a package the first found package
in "repo order" is used. This design decision is not very good. When a new repo that has the latest
version of any package is added and if it is the last repo in order, pisi can not upgrade to this
package.
We can remove this order thing and in this situation we can use the latest version of the package.
This should also lead to some other problems but between these two not very good solutions this
seems to be the better one.
==> autoxml
* When setting a Long type attribute or tag of an autoxml class, value must be Long type. Otherwise,
autoxml.write() fails in x86_64
==> COMAR
* pisi-cli should use comar.
apimigration
-96
View File
@@ -1,96 +0,0 @@
Possible API Migrations
=======================
Pisi does not have an api. All the projects use its internal modules to do their
jobs. This file holds a list of these usages for api migration.
PiSi
====
Pisi itself also should not use internal db modules. It can also use this abstracted
api layer from the needed places.
* packagedb.remove_repo
* sourcedb.remove_repo
* packagedb.which_repo
* repodb.get_repo
* packagedb.get_package
* installdb.is_installed
* packagedb.get_rev_deps
* installdb.get_version
* packagedb.has_package
* packagedb.add_package
* componentdb.add_package
* filesdb.has_file
* filesdb.get_file
* installdb.get_info
* installdb.files
* installdb.pkg_dir
* installdb.install
* filesdb.add_files
* componentdb.add_spec
* sourcedb.add_spec
* componentdb.get_union_comp
* componentdb.remove_spec
* installdb.get_version
* componentdb.remove_repo
* componentdb.remove_package
* componentdb.update_component
* componentdb.add_package
* installdb.remove
* filesdb.remove_files
* sourcedb.pkgtosrc
* sourcedb.get_spec
* sourcedb.get_source
* sourcedb.get_spec_repo
* repodb.get_repo
* componentdb.has_component
* componentdb.get_component
* installdb.is_installed
* packagedb.list_packages
* installdb.list_installed
* componentdb.get_union_packages
* componentdb.list_components
* repodb.list
* sourcedb.list
* installdb.list_pending
* filesdb.match_files
Package Manager
===============
Package Manager should not use internal db modules of pisi. It should only use pisi
api.
* repodb.get_repo
* packagedb.get_package
pisi.itemsbyrepo.installed or pisi.itemsbyrepo.repos should not be used by package manager.
Instead api should have functions like api.get_installed_packages, api.get_repo_packages and
get_all_packages/get_packages.
* componentdb.list_components
* repodb.list
* componentdb.get_union_comp
* componentdb.get_union_packages
Yali
====
Yali should not use internal db modules of pisi. It should only use pisi api.
* api.add_repo
* api.update_repo
* api.remove_repo
* api.install
* packagedb.list_packages
* installdb.list_pending
* api.configure_pending
* packagedb.get_package
Buildfarm
=========
Buildfarm should not use internal db modules of pisi. It should only use pisi api.
* api.create_delta_package
pisi-cli2.5
+1 -12
View File
@@ -22,20 +22,11 @@ import pisi.context as ctx
import pisi.cli.pisicli as pisicli
import gettext
gettext.bindtextdomain('pisi', "/usr/share/locale")
gettext.textdomain('pisi')
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
def sig_handler(sig, frame):
if sig == signal.SIGTERM:
exit()
def exit():
try:
pisi.api.finalize()
except KeyboardInterrupt: # raised pending interrupt
pass
pisi.api.finalize()
sys.exit(1)
def handle_exception(exception, value, tb):
@@ -82,8 +73,6 @@ if __name__ == "__main__":
sys.excepthook = handle_exception
signal.signal(signal.SIGTERM, sig_handler)
locale.setlocale(locale.LC_ALL, '')
cli = pisicli.PisiCLI()
cli.run_command()
pisi-spec.dtd
+2 -10
View File
@@ -78,7 +78,6 @@
RuntimeDependencies |
Files |
Conflicts |
Replaces |
AdditionalFiles |
Provides
)*>
@@ -91,14 +90,6 @@
<!-- we have a problem here, this Package tag only contains a package name
while dtd treats this as another Package section, who said dtd doesnt
suck anyway -->
<!ATTLIST Package versionFrom CDATA #IMPLIED>
<!ATTLIST Package versionTo CDATA #IMPLIED>
<!ATTLIST Package version CDATA #IMPLIED>
<!ATTLIST Package releaseFrom CDATA #IMPLIED>
<!ATTLIST Package releaseTo CDATA #IMPLIED>
<!ATTLIST Package release CDATA #IMPLIED>
<!ELEMENT Replaces (Package)+>
<!ELEMENT AdditionalFiles (AdditionalFile+)>
<!ELEMENT AdditionalFile (#PCDATA)>
@@ -124,7 +115,8 @@ suck anyway -->
<!ELEMENT Update (#PCDATA | Date | Version | Comment | Name | Email)*>
<!ATTLIST Update release CDATA #REQUIRED>
<!ATTLIST Update type (security|bug) #IMPLIED>
<!-- FIXME: valid types must be used instead of CDATA -->
<!ATTLIST Update type CDATA #IMPLIED>
<!ELEMENT Date (#PCDATA)>
<!ELEMENT Version (#PCDATA)>
pisi-spec.rng
-502
View File
@@ -1,502 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- RELAX NG schema file for PISI pspec.xml files -->
<!-- http://www.pardus.org.tr/projeler/pisi/ -->
<grammar xmlns="http://relaxng.org/ns/structure/1.0">
<!-- Start Document -->
<start>
<ref name="PISI"/>
</start>
<!-- PISI Section -->
<define name="PISI">
<element name="PISI">
<ref name="Source"/>
<oneOrMore>
<ref name="Package"/>
</oneOrMore>
<ref name="History"/>
</element>
</define>
<!-- Source Section -->
<define name="Source">
<element name="Source">
<ref name="Name"/>
<ref name="Homepage"/>
<ref name="Packager"/>
<oneOrMore>
<ref name="License"/>
</oneOrMore>
<optional>
<ref name="Icon"/>
</optional>
<zeroOrMore>
<ref name="IsA"/>
</zeroOrMore>
<optional>
<ref name="PartOf"/>
</optional>
<oneOrMore>
<ref name="Summary"/>
</oneOrMore>
<zeroOrMore>
<!-- FIXME: should be oneOrMore -->
<ref name="Description"/>
</zeroOrMore>
<ref name="Archive"/>
<optional>
<ref name="BuildDependencies"/>
</optional>
<optional>
<ref name="Patches"/>
</optional>
</element>
</define>
<!-- Package Section -->
<define name="Package">
<element name="Package">
<ref name="Name"/>
<optional>
<ref name="License"/>
</optional>
<zeroOrMore>
<ref name="IsA"/>
</zeroOrMore>
<optional>
<ref name="PartOf"/>
</optional>
<zeroOrMore>
<ref name="Summary"/>
</zeroOrMore>
<zeroOrMore>
<ref name="Description"/>
</zeroOrMore>
<optional>
<ref name="RuntimeDependencies"/>
</optional>
<optional>
<ref name="Conflicts"/>
</optional>
<optional>
<ref name="Replaces"/>
</optional>
<ref name="Files"/>
<optional>
<ref name="AdditionalFiles"/>
</optional>
<optional>
<ref name="Provides"/>
</optional>
</element>
</define>
<!-- History Section -->
<define name="History">
<element name="History">
<oneOrMore>
<ref name="Update"/>
</oneOrMore>
</element>
</define>
<!-- Name -->
<define name="Name">
<element name="Name">
<data type="string" datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">
<param name="pattern">[0-9a-zA-Z\-_+]*</param>
<param name="maxLength">255</param>
</data>
</element>
</define>
<!-- Homepage -->
<define name="Homepage">
<element name="Homepage">
<text/>
</element>
</define>
<!-- Icon -->
<define name="Icon">
<element name="Icon">
<text/>
</element>
</define>
<!-- Packager -->
<define name="Packager">
<element name="Packager">
<ref name="Packager.Name"/>
<ref name="Email"/>
</element>
</define>
<!-- Packager.Name -->
<define name="Packager.Name">
<element name="Name">
<data type="string" datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">
<param name="maxLength">255</param>
</data>
</element>
</define>
<!-- Email -->
<define name="Email">
<element name="Email">
<text/>
</element>
</define>
<!-- License -->
<define name="License">
<element name="License">
<text/>
</element>
</define>
<!-- IsA -->
<define name="IsA">
<element name="IsA">
<choice>
<value>app</value>
<value>app:console</value>
<value>app:gui</value>
<value>app:web</value>
<value>library</value>
<value>service</value>
<value>data</value>
<value>data:doc</value>
<value>data:font</value>
<value>kernel</value>
<value>driver</value>
<value>locale</value>
<value>locale:tr</value>
<value>locale:en</value>
<value>locale:es</value>
<value>locale:nl</value>
<value>locale:fr</value>
<value>locale:de</value>
<value>locale:it</value>
</choice>
</element>
</define>
<!-- PartOf -->
<define name="PartOf">
<element name="PartOf">
<text/>
</element>
</define>
<!-- Summary -->
<define name="Summary">
<element name="Summary">
<ref name="attlist.Summary"/>
<text/>
</element>
</define>
<!-- attlist.Summary -->
<define name="attlist.Summary" combine="interleave">
<optional>
<attribute name="xml:lang"/>
</optional>
</define>
<!-- Description -->
<define name="Description">
<element name="Description">
<ref name="attlist.Description"/>
<text/>
</element>
</define>
<!-- attlist.Description -->
<define name="attlist.Description" combine="interleave">
<optional>
<attribute name="xml:lang"/>
</optional>
</define>
<!-- Archive -->
<define name="Archive">
<element name="Archive">
<ref name="attlist.Archive"/>
<text/>
</element>
</define>
<!-- attlist.Archive -->
<define name="attlist.Archive" combine="interleave">
<group>
<attribute name="sha1sum"/>
<attribute name="type">
<choice>
<value>targz</value>
<value>tarbz2</value>
<value>tarlzma</value>
<value>tar</value>
<value>zip</value>
<value>gzip</value>
<value>binary</value>
</choice>
</attribute>
</group>
</define>
<!-- BuildDependencies -->
<define name="BuildDependencies">
<element name="BuildDependencies">
<oneOrMore>
<ref name="Dependency"/>
</oneOrMore>
</element>
</define>
<!-- Dependency -->
<define name="Dependency">
<element name="Dependency">
<ref name="attlist.VersionReleaseToFrom"/>
<text/>
</element>
</define>
<!-- attlist.VersionReleaseToFrom -->
<define name="attlist.VersionReleaseToFrom">
<group>
<optional>
<attribute name="versionFrom"/>
</optional>
<optional>
<attribute name="versionTo"/>
</optional>
<optional>
<attribute name="version"/>
</optional>
<optional>
<attribute name="releaseFrom"/>
</optional>
<optional>
<attribute name="releaseTo"/>
</optional>
<optional>
<attribute name="release"/>
</optional>
</group>
</define>
<!-- Patches -->
<define name="Patches">
<element name="Patches">
<oneOrMore>
<ref name="Patch"/>
</oneOrMore>
</element>
</define>
<!-- Patch -->
<define name="Patch">
<element name="Patch">
<ref name="attlist.Patch"/>
<text/>
</element>
</define>
<!-- attlist.Patch -->
<define name="attlist.Patch" combine="interleave">
<group>
<optional>
<attribute name="compressionType"/>
</optional>
<optional>
<attribute name="level"/>
</optional>
<optional>
<attribute name="target"/>
</optional>
</group>
</define>
<!-- Provides -->
<define name="Provides">
<element name="Provides">
<oneOrMore>
<ref name="COMAR"/>
</oneOrMore>
</element>
</define>
<!-- COMAR -->
<define name="COMAR">
<element name="COMAR">
<ref name="attlist.COMAR"/>
<text/>
</element>
</define>
<!-- attlist.COMAR -->
<define name="attlist.COMAR">
<optional>
<attribute name="script"/>
</optional>
</define>
<!-- Conflicts -->
<define name="Conflicts">
<element name="Conflicts">
<oneOrMore>
<ref name="Conflicts.Package"/>
</oneOrMore>
</element>
</define>
<define name="Conflicts.Package">
<element name="Package">
<ref name="attlist.VersionReleaseToFrom"/>
<text />
</element>
</define>
<!-- Replaces -->
<define name="Replaces">
<element name="Replaces">
<oneOrMore>
<ref name="Replaces.Package"/>
</oneOrMore>
</element>
</define>
<define name="Replaces.Package">
<element name="Package">
<ref name="attlist.VersionReleaseToFrom"/>
<text />
</element>
</define>
<!-- AdditionalFiles -->
<define name="AdditionalFiles">
<element name="AdditionalFiles">
<oneOrMore>
<ref name="AdditionalFile"/>
</oneOrMore>
</element>
</define>
<!-- AdditionalFile -->
<define name="AdditionalFile">
<element name="AdditionalFile">
<ref name="attlist.AdditionalFile"/>
<text/>
</element>
</define>
<!-- attlist.AdditionalFile -->
<define name="attlist.AdditionalFile">
<attribute name="target"/>
<optional>
<attribute name="permission"/>
</optional>
<optional>
<attribute name="owner"/>
</optional>
</define>
<!-- RuntimeDependencies -->
<define name="RuntimeDependencies">
<element name="RuntimeDependencies">
<oneOrMore>
<ref name="Dependency"/>
</oneOrMore>
</element>
</define>
<!-- Files -->
<define name="Files">
<element name="Files">
<oneOrMore>
<ref name="Path"/>
</oneOrMore>
</element>
</define>
<!-- Path -->
<define name="Path">
<element name="Path">
<ref name="attlist.Path"/>
<text/>
</element>
</define>
<!-- attlist.Path -->
<define name="attlist.Path" combine="interleave">
<group>
<attribute name="fileType">
<choice>
<value>executable</value>
<value>library</value>
<value>data</value>
<value>config</value>
<value>doc</value>
<value>man</value>
<value>info</value>
<value>localedata</value>
<value>header</value>
</choice>
</attribute>
<optional>
<attribute name="permanent">
<choice>
<value>false</value>
<value>true</value>
</choice>
</attribute>
</optional>
</group>
</define>
<!-- Update -->
<define name="Update">
<element name="Update">
<ref name="attlist.Update"/>
<ref name="Date"/>
<ref name="Version"/>
<ref name="Comment"/>
<ref name="Packager.Name"/>
<ref name="Email"/>
</element>
</define>
<!-- attlist.Update -->
<define name="attlist.Update" combine="interleave">
<attribute name="release"/>
<optional>
<attribute name="type">
<choice>
<value>security</value>
<value>bug</value>
</choice>
</attribute>
</optional>
</define>
<!-- Date -->
<define name="Date">
<element name="Date">
<data type="date" datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes"/>
</element>
</define>
<!-- Version -->
<define name="Version">
<element name="Version">
<text/>
</element>
</define>
<!-- Comment -->
<define name="Comment">
<element name="Comment">
<text/>
</element>
</define>
</grammar>
pisi/__init__.py
+2 -2
View File
@@ -12,9 +12,9 @@
# PiSi version
__version__ = "1.1_20070305"
__version__ = "1.1.5"
__dbversion__ = "1.1_20070305"
__dbversion__ = "1.1.5"
__filesdbversion__ = "1.0.5" # yes, this is the real bottleneck
__all__ = [ 'api', 'config', 'configfile', 'packagedb', 'installdb', 'search' ]
pisi/actionsapi/pythonmodules.py
+1 -1
View File
@@ -58,7 +58,7 @@ def install(parameters = ''):
for doc in DDOCS:
if can_access_file(doc):
pisitools.dodoc(doc)
dodoc(doc)
def run(parameters = ''):
'''executes parameters with python'''
pisi/api.py
+66 -112
View File
@@ -21,19 +21,16 @@ __trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi
import pisi.db
import pisi.context as ctx
import pisi.uri
import pisi.util
import pisi.dependency as dependency
import pisi.pgraph as pgraph
import pisi.db.packagedb
import pisi.db.repodb
import pisi.db.installdb
import pisi.db.filesdb
import pisi.db.componentdb
import pisi.db.sourcedb
import pisi.db.lockeddbshelve as shelve
import pisi.packagedb
import pisi.repodb
import pisi.installdb
import pisi.sourcedb
import pisi.lockeddbshelve as shelve
import pisi.index
import pisi.config
import pisi.metadata
@@ -108,12 +105,12 @@ def init(database = True, write = True,
ctx.database = database
if database:
shelve.init_dbenv(write=write)
ctx.repodb = pisi.db.repodb.RepoDB()
ctx.installdb = pisi.db.installdb.InstallDB()
ctx.filesdb = pisi.db.filesdb.FilesDB()
ctx.componentdb = pisi.db.componentdb.ComponentDB()
ctx.packagedb = pisi.db.packagedb.PackageDB()
ctx.sourcedb = pisi.db.sourcedb.SourceDB()
ctx.repodb = pisi.repodb.init()
ctx.installdb = pisi.installdb.init()
ctx.filesdb = pisi.files.FilesDB()
ctx.componentdb = pisi.component.ComponentDB()
ctx.packagedb = pisi.packagedb.init_db()
ctx.sourcedb = pisi.sourcedb.init()
else:
ctx.repodb = None
ctx.installdb = None
@@ -131,12 +128,9 @@ def finalize():
if ctx.log:
ctx.loghandler.flush()
ctx.log.removeHandler(ctx.loghandler)
if ctx.repodb != None:
ctx.repodb.close()
ctx.repodb = None
if ctx.installdb != None:
ctx.installdb.close()
ctx.installdb = None
pisi.repodb.finalize()
pisi.installdb.finalize()
if ctx.filesdb != None:
ctx.filesdb.close()
ctx.filesdb = None
@@ -144,10 +138,10 @@ def finalize():
ctx.componentdb.close()
ctx.componentdb = None
if ctx.packagedb:
ctx.packagedb.close()
pisi.packagedb.finalize_db()
ctx.packagedb = None
if ctx.sourcedb != None:
ctx.sourcedb.close()
if ctx.sourcedb:
pisi.sourcedb.finalize()
ctx.sourcedb = None
if ctx.dbenv:
ctx.dbenv.close()
@@ -164,16 +158,18 @@ def list_installed():
"""Return a set of installed package names."""
return set(ctx.installdb.list_installed())
def list_replaces(repo = None):
"""Returns a dict of the replaced packages."""
return ctx.packagedb.get_replaces(repo = repo)
def list_available(repo = None):
"""Return a set of available package names."""
return set(ctx.packagedb.list_packages(repo = repo))
def list_upgradable():
ignore_build = ctx.get_option('ignore_build_no')
return filter(pisi.operations.is_upgradable, ctx.installdb.list_installed()) + ctx.packagedb.get_replaces().keys()
return filter(pisi.operations.is_upgradable, ctx.installdb.list_installed())
def package_graph(A, repo = pisi.db.itembyrepodb.installed, ignore_installed = False):
def package_graph(A, repo = pisi.itembyrepodb.installed, ignore_installed = False):
"""Construct a package relations graph.
Graph will contain all dependencies of packages A, if ignore_installed
@@ -197,7 +193,7 @@ def package_graph(A, repo = pisi.db.itembyrepodb.installed, ignore_installed = F
while len(B) > 0:
Bp = set()
for x in B:
pkg = get_repo_package(x, repo)
pkg = ctx.packagedb.get_package(x, repo)
#print pkg
for dep in pkg.runtimeDependencies():
if ignore_installed:
@@ -236,24 +232,24 @@ def generate_base_upgrade(A):
def generate_conflicts(A):
# returns the conflicting packages list of the to be installed packages.
# @C: conflicting and must be removed packages list to proceed
# @D: list of the conflicting packages _with each other_ in the to be installed list
# @E: dictionary that contains which package in the to be installed list conflicts
# @conflicting_pkgs: conflicting and must be removed packages list to proceed
# @conflicts_inorder: list of the conflicting packages _with each other_ in the to be installed list
# @conflicting_pairs: dictionary that contains which package in the to be installed list conflicts
# with which packages
(C, D, E) = calculate_conflicts(A, ctx.packagedb)
return (C, D, E)
(conflicting_pkgs, conflicts_inorder, conflicting_pairs) = pisi.conflict.calculate_conflicts(A, ctx.packagedb)
return (conflicting_pkgs, conflicts_inorder, conflicting_pairs)
def generate_pending_order(A):
# returns pending package list in reverse topological order of dependency
G_f = pgraph.PGraph(ctx.packagedb, pisi.db.itembyrepodb.installed) # construct G_f
G_f = pgraph.PGraph(ctx.packagedb, pisi.itembyrepodb.installed) # construct G_f
for x in A.keys():
G_f.add_package(x)
B = A
while len(B) > 0:
Bp = set()
for x in B.keys():
pkg = get_installed_package(x)
pkg = ctx.packagedb.get_package(x, pisi.itembyrepodb.installed)
for dep in pkg.runtimeDependencies():
if dep.package in G_f.vertices():
G_f.add_dep(x, dep)
@@ -321,10 +317,10 @@ def info_file(package_fn):
def info_name(package_name, installed=False):
"""Fetch package information for the given package."""
if installed:
package = get_installed_package(package_name)
package = ctx.packagedb.get_package(package_name, pisi.itembyrepodb.installed)
repo = None
else:
package, repo = ctx.packagedb.get_package_repo(package_name, pisi.db.itembyrepodb.repos)
package, repo = ctx.packagedb.get_package_repo(package_name, pisi.itembyrepodb.repos)
metadata = pisi.metadata.MetaData()
metadata.package = package
@@ -341,10 +337,10 @@ def info_name(package_name, installed=False):
files = None
return metadata, files, repo
def search_package_terms(terms, repo = pisi.db.itembyrepodb.all):
def search_package_terms(terms, repo = pisi.itembyrepodb.all):
return search_in_packages(terms, ctx.packagedb.list_packages(repo), repo)
def search_in_packages(terms, packages, repo = pisi.db.itembyrepodb.all):
def search_in_packages(terms, packages, repo = pisi.itembyrepodb.all):
def search(package, term):
term = unicode(term).lower()
@@ -355,7 +351,7 @@ def search_in_packages(terms, packages, repo = pisi.db.itembyrepodb.all):
found = []
for name in packages:
pkg = get_repo_package(name, repo)
pkg = ctx.packagedb.get_package(name, repo)
if terms == filter(lambda x:search(pkg, x), terms):
found.append(name)
@@ -364,14 +360,14 @@ def search_in_packages(terms, packages, repo = pisi.db.itembyrepodb.all):
def check(package):
md, files = info(package, True)
corrupt = []
for file in files.list:
if file.hash and file.type != "config" \
and not os.path.islink('/' + file.path):
ctx.ui.info(_("Checking /%s ") % file.path, noln=True, verbose=True)
for f in files.list:
if f.hash and f.type != "config" \
and not os.path.islink('/' + f.path):
ctx.ui.info(_("Checking /%s ") % f.path, noln=True, verbose=True)
try:
if file.hash != pisi.util.sha1_file('/' + file.path):
corrupt.append(file)
ctx.ui.error(_("\nCorrupt file: %s") % file)
if f.hash != pisi.util.sha1_file('/' + f.path):
corrupt.append(f)
ctx.ui.error(_("\nCorrupt file: %s") % f)
else:
ctx.ui.info(_("OK"), verbose=True)
except pisi.util.FileError,e:
@@ -395,6 +391,23 @@ def index(dirs=None, output='pisi-index.xml', skip_sources=False, skip_signing=F
index.write(output, sha1sum=True, compress=pisi.file.File.bz2, sign=pisi.file.File.detached)
ctx.ui.info(_('* Index file written'))
def add_repo(name, indexuri, at = None):
if ctx.repodb.has_repo(name):
raise Error(_('Repo %s already present.') % name)
else:
repo = pisi.repodb.Repo(pisi.uri.URI(indexuri))
ctx.repodb.add_repo(name, repo, at = at)
ctx.ui.info(_('Repo %s added to system.') % name)
def remove_repo(name):
if ctx.repodb.has_repo(name):
ctx.repodb.remove_repo(name)
pisi.util.clean_dir(os.path.join(ctx.config.index_dir(), name))
ctx.ui.info(_('Repo %s removed from system.') % name)
else:
ctx.ui.error(_('Repository %s does not exist. Cannot remove.')
% name)
def list_repos():
return ctx.repodb.list()
@@ -451,7 +464,7 @@ def rebuild_repo(repo):
def rebuild_db(files=False):
assert ctx.database == False
assert not ctx.database
# Bug 2596
# finds and cleans duplicate package directories under '/var/lib/pisi/package'
@@ -502,13 +515,13 @@ def rebuild_db(files=False):
percent = progress.update(processed),
info = _("Rebuilding package database"))
def reload_indices(txn):
def reload_indices():
index_dir = ctx.config.index_dir()
if os.path.exists(index_dir): # it may have been erased, or we may be upgrading from a previous version -- exa
for repo in os.listdir(index_dir):
indexuri = pisi.util.join_path(ctx.config.lib_dir(), 'index', repo, 'uri')
indexuri = open(indexuri, 'r').readline()
pisi.api.add_repository(repo, indexuri)
pisi.api.add_repo(repo, indexuri)
pisi.api.rebuild_repo(repo)
# check db schema versions
@@ -516,7 +529,7 @@ def rebuild_db(files=False):
shelve.check_dbversion('filesdbversion', pisi.__filesdbversion__, write=False)
except KeyboardInterrupt:
raise
except Exception, e: #FIXME: what exception could we catch here, replace with that.
except Exception: #FIXME: what exception could we catch here, replace with that.
files = True # exception means the files db version was wrong
shelve.init_dbenv(write=True, writeversion=True)
destroy(files) # bye bye
@@ -532,68 +545,9 @@ def rebuild_db(files=False):
clean_duplicates()
txn = ctx.dbenv.txn_begin()
reload_packages(files, txn)
reload_indices(txn)
reload_indices()
txn.commit()
############################ Under Construction ##############################
### Package Database Operations ###
## Returns the package informations of the given repository
# @param name Name of the package
# @param repo Name of the repository
def get_repo_package(name, repo=pisi.db.itembyrepodb.repos):
# FIXME: * currently we return the first package in repo order
# * we should get rid of itembyrepodb enums
# * transparent transaction support
return ctx.packagedb.get_package(name, repo, txn=None)
## Returns the installed package information
# @param name Name of the package
def get_installed_package(name):
# FIXME: * we should get rid of itembyrepodb enums
# * transparent transaction support
return ctx.packagedb.get_package(name, pisi.db.itembyrepodb.installed, txn=None)
### Repo Database Operations ###
## Adds a new repository to the pisi database
# @param name Name of the repository
# @param uri Repository location
def add_repository(name, uri):
if ctx.repodb.has_repo(name):
raise Error(_('Repo %s already present.') % name)
else:
repo = pisi.db.repodb.Repo(pisi.uri.URI(uri))
ctx.repodb.add_repo(name, repo)
ctx.ui.info(_('Repo %s added to system.') % name)
## Removes an existing repository from the pisi database
# @param name Name of the repository
def remove_repository(name):
if ctx.repodb.has_repo(name):
ctx.repodb.remove_repo(name)
pisi.util.clean_dir(os.path.join(ctx.config.index_dir(), name))
ctx.ui.info(_('Repo %s removed from system.') % name)
else:
ctx.ui.error(_('Repository %s does not exist. Cannot remove.')
% name)
### Source Database Operations ###
### Component Database Operations ###
### Files Database Operations ###
### Delta Package Operations ###
## Creates a delta package from the given .pisi package paths.
# @param old_package Path of the older package
# @param new_package Path of the newer package
def create_delta_package(old_package, new_package):
pisi.delta.create_delta_package(old_package, new_package)
############# FIXME: this was a quick fix. ##############################
# api was importing other module's functions and providing them as api functions. This is wrong.
@@ -630,7 +584,7 @@ def upgrade_base(*args, **kw):
return pisi.operations.upgrade_base(*args, **kw)
def calculate_conflicts(*args, **kw):
return pisi.operations.calculate_conflicts(*args, **kw)
return pisi.conflict.calculate_conflicts(*args, **kw)
def reorder_base_packages(*args, **kw):
return pisi.operations.reorder_base_packages(*args, **kw)
pisi/archive.py
+4 -3
View File
@@ -43,11 +43,12 @@ class ArchiveBase(object):
self.type = atype
def unpack(self, target_dir, clean_dir = False):
self.target_dir = target_dir
# first we check if we need to clean-up our working env.
if os.path.exists(target_dir) and clean_dir:
util.clean_dir(target_dir)
if os.path.exists(self.target_dir) and clean_dir:
util.clean_dir(self.target_dir)
os.makedirs(target_dir)
os.makedirs(self.target_dir)
class ArchiveBinary(ArchiveBase):
pisi/atomicoperations.py
+49 -38
View File
@@ -15,15 +15,12 @@ import gettext
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import sys
import os
import bsddb3.db as db
import shutil
import pisi
import pisi.context as ctx
import pisi.db.packagedb as packagedb
import pisi.dependency as dependency
import pisi.conflict
import pisi.util as util
import pisi.metadata
@@ -32,6 +29,7 @@ import pisi.uri
import pisi.ui
import pisi.version
import pisi.delta
import pisi.packagedb
class Error(pisi.Error):
pass
@@ -43,8 +41,12 @@ class NotfoundError(pisi.Error):
class AtomicOperation(object):
def __init__(self):
pass
def __init__(self, ignore_dep = None):
#self.package = package
if ignore_dep==None:
self.ignore_dep = ctx.config.get_option('ignore_dependency')
else:
self.ignore_dep = ignore_dep
def run(self, package):
"perform an atomic package operation"
@@ -55,14 +57,14 @@ class Install(AtomicOperation):
"Install class, provides install routines for pisi packages"
@staticmethod
def from_name(name):
def from_name(name, ignore_dep = None):
# download package and return an installer object
# find package in repository
repo = ctx.packagedb.which_repo(name)
if repo:
ctx.ui.info(_("Package %s found in repository %s") % (name, repo))
repo = ctx.repodb.get_repo(repo)
pkg = pisi.api.get_repo_package(name)
pkg = ctx.packagedb.get_package(name)
delta = None
# Package is installed. This is an upgrade. Check delta.
@@ -85,13 +87,13 @@ class Install(AtomicOperation):
ctx.ui.info(_("Package URI: %s") % pkg_path, verbose=True)
return Install(pkg_path)
return Install(pkg_path, ignore_dep)
else:
raise Error(_("Package %s not found in any active repository.") % name)
def __init__(self, package_fname, ignore_file_conflicts = None):
def __init__(self, package_fname, ignore_dep = None, ignore_file_conflicts = None):
"initialize from a file name"
super(Install, self).__init__()
super(Install, self).__init__(ignore_dep)
if not ignore_file_conflicts:
ignore_file_conflicts = ctx.get_option('ignore_file_conflicts')
self.ignore_file_conflicts = ignore_file_conflicts
@@ -145,7 +147,7 @@ class Install(AtomicOperation):
# check comar
if self.metadata.package.providesComar and ctx.comar:
import pisi.comariface as comariface
com = comariface.get_comar()
comariface.get_comar()
def check_relations(self):
# check dependencies
@@ -157,15 +159,25 @@ class Install(AtomicOperation):
# check if package is in database
# If it is not, put it into 3rd party packagedb
if not ctx.packagedb.has_package(self.pkginfo.name):
ctx.packagedb.add_package(self.pkginfo, pisi.db.itembyrepodb.thirdparty)
ctx.packagedb.add_package(self.pkginfo, pisi.itembyrepodb.thirdparty)
# If it is explicitly specified that package conflicts with this package and also
# we passed check_conflicts tests in operations.py than this means a non-conflicting
# pkg is in "order" to be installed that has no file conflict problem with this package.
# PS: we need this because "order" generating code does not consider conflicts.
def really_conflicts(pkg):
if not self.pkginfo.conflicts:
return True
return not pkg in map(lambda x:x.package, self.pkginfo.conflicts)
# check file conflicts
file_conflicts = []
for file in self.files.list:
if ctx.filesdb.has_file(file.path):
pkg, existing_file = ctx.filesdb.get_file(file.path)
dst = pisi.util.join_path(ctx.config.dest_dir(), file.path)
if pkg != self.pkginfo.name and not os.path.isdir(dst):
for f in self.files.list:
if ctx.filesdb.has_file(f.path):
pkg, existing_file = ctx.filesdb.get_file(f.path)
dst = pisi.util.join_path(ctx.config.dest_dir(), f.path)
if pkg != self.pkginfo.name and not os.path.isdir(dst) and really_conflicts(pkg):
file_conflicts.append( (pkg, existing_file) )
if file_conflicts:
file_conflicts_str = ""
@@ -187,6 +199,7 @@ class Install(AtomicOperation):
if ctx.installdb.is_installed(pkg.name): # is this a reinstallation?
#FIXME: consider REPOSITORY instead of DISTRIBUTION -- exa
#ipackage = ctx.packagedb.get_package(pkg.name, pisi.itembyrepodb.installed)
ipkg = ctx.installdb.get_info(pkg.name)
repomismatch = ipkg.distribution != pkg.distribution
@@ -273,8 +286,8 @@ class Install(AtomicOperation):
fpath = pisi.util.join_path(ctx.config.dest_dir(), config.path)
if os.path.exists(fpath) and not os.path.isdir(fpath):
if os.path.islink(fpath):
file = os.readlink(fpath)
if os.path.exists(file) and pisi.util.sha1_data(file) != config.hash:
f = os.readlink(fpath)
if os.path.exists(f) and pisi.util.sha1_data(f) != config.hash:
changed = True
else:
if pisi.util.sha1_file(fpath) != config.hash:
@@ -332,8 +345,8 @@ class Install(AtomicOperation):
# remove left over files from the old package.
def clean_leftovers():
new = set(map(lambda x: str(x.path), self.files.list))
old = set(map(lambda x: str(x.path), self.old_files.list))
new = set(str(f.path) for f in self.files.list)
old = set(str(f.path) for f in self.old_files.list)
leftover = old - new
old_fileinfo = {}
for fileinfo in self.old_files.list:
@@ -347,19 +360,19 @@ class Install(AtomicOperation):
old = filter(lambda x: x.type == 'config', self.old_files.list)
# get config path lists
newconfig = set(map(lambda x: str(x.path), new))
oldconfig = set(map(lambda x: str(x.path), old))
newconfig = set(str(x.path) for x in new)
oldconfig = set(str(x.path) for x in old)
config_overlaps = newconfig & oldconfig
if config_overlaps:
files = filter(lambda x: x.path in config_overlaps, old)
for file in files:
check_config_changed(file)
for f in files:
check_config_changed(f)
else:
for file in self.files.list:
if file.type == 'config':
for f in self.files.list:
if f.type == 'config':
# there may be left over config files
check_config_changed(file)
check_config_changed(f)
if self.package_fname.endswith(ctx.const.delta_package_suffix):
relocate_files()
@@ -369,9 +382,6 @@ class Install(AtomicOperation):
if config_changed:
rename_configs()
if self.package_fname.endswith(ctx.const.delta_package_suffix):
relocate_files()
if self.reinstall:
clean_leftovers()
@@ -414,7 +424,7 @@ class Install(AtomicOperation):
ctx.filesdb.add_files(self.metadata.package.name, self.files, txn=txn)
# installed packages
ctx.packagedb.add_package(self.pkginfo, pisi.db.itembyrepodb.installed, txn=txn)
ctx.packagedb.add_package(self.pkginfo, pisi.itembyrepodb.installed, txn=txn)
def install_single(pkg, upgrade = False):
@@ -440,10 +450,10 @@ def install_single_name(name, upgrade = False):
class Remove(AtomicOperation):
def __init__(self, package_name):
super(Remove, self).__init__()
def __init__(self, package_name, ignore_dep = None):
super(Remove, self).__init__(ignore_dep)
self.package_name = package_name
self.package = pisi.api.get_installed_package(self.package_name)
self.package = ctx.packagedb.get_package(self.package_name, pisi.itembyrepodb.installed)
try:
self.files = ctx.installdb.files(self.package_name)
except pisi.Error, e:
@@ -509,7 +519,7 @@ class Remove(AtomicOperation):
try:
if pisi.util.sha1_file(fpath) == fileinfo.hash:
os.unlink(fpath)
except pisi.util.FileError, e:
except pisi.util.FileError:
pass
else:
if os.path.isfile(fpath) or os.path.islink(fpath):
@@ -541,7 +551,8 @@ class Remove(AtomicOperation):
def remove_db(self, txn):
ctx.installdb.remove(self.package_name, txn)
ctx.filesdb.remove_files(self.files, txn)
pisi.db.packagedb.remove_tracking_package(self.package_name, txn)
# FIXME: something goes wrong here, if we use ctx operations ends up with segmentation fault!
pisi.packagedb.remove_tracking_package(self.package_name, txn)
def remove_single(package_name):
@@ -568,7 +579,7 @@ def virtual_install(metadata, files, txn):
ctx.filesdb.add_files(metadata.package.name, files, txn=txn)
# installed packages
ctx.packagedb.add_package(metadata.package, pisi.db.itembyrepodb.installed, txn=txn)
ctx.packagedb.add_package(metadata.package, pisi.itembyrepodb.installed, txn=txn)
def resurrect_package(package_fn, write_files, txn = None):
"""Resurrect the package from xml files"""
pisi/build.py
+100 -74
View File
@@ -13,7 +13,6 @@
# python standard library
import os
import sys
import glob
import copy
import stat
@@ -129,6 +128,8 @@ class Builder:
else:
self.specdir = os.path.dirname(self.specuri.get_uri())
self.read_translations(self.specdir)
self.sourceArchive = pisi.sourcearchive.SourceArchive(self.spec, self.pkg_work_dir())
self.set_environment_vars()
@@ -142,9 +143,12 @@ class Builder:
specuri = pisi.uri.URI(os.path.realpath(specuri.get_uri())) # FIXME: doesn't work for file://
self.specuri = specuri
spec = pisi.specfile.SpecFile()
spec.read(specuri, ctx.config.tmp_dir())
spec.read(self.specuri, ctx.config.tmp_dir())
self.spec = spec
def read_translations(self, specdir):
self.spec.read_translations(pisi.util.join_path(specdir, ctx.const.translations_file))
# directory accessor functions
# pkg_x_dir: per package directory for storing info type x
@@ -237,6 +241,7 @@ class Builder:
#self.location = os.path.dirname(self.url.uri)
self.fetch_actionsfile()
self.fetch_translationsfile()
self.fetch_patches()
self.fetch_comarfiles()
self.fetch_additionalFiles()
@@ -247,6 +252,14 @@ class Builder:
actionsuri = pisi.util.join_path(self.specdiruri, ctx.const.actions_file)
self.download(actionsuri, self.destdir)
def fetch_translationsfile(self):
translationsuri = pisi.util.join_path(self.specdiruri, ctx.const.translations_file)
try:
self.download(translationsuri, self.destdir)
except pisi.fetcher.FetchError:
# translations.xml is not mandatory for PiSi
pass
def fetch_patches(self):
spec = self.spec
for patch in spec.source.patches:
@@ -402,6 +415,9 @@ class Builder:
return pisi.util.join_path(self.pkg_work_dir(), workdir)
def log_sandbox_violation(self, operation, path, canonical_path):
ctx.ui.error(_("Sandbox violation: %s (%s -> %s)") % (operation, path, canonical_path))
def run_action_function(self, func, mandatory=False):
"""Calls the corresponding function in actions.py.
@@ -412,13 +428,33 @@ class Builder:
curDir = os.getcwd()
os.chdir(self.srcDir)
if func in self.actionLocals:
self.actionLocals[func]()
if ctx.get_option('ignore_sandbox'):
self.actionLocals[func]()
else:
import catbox
# stupid autoconf family needs /usr/lib/conftest* and /usr/lib/cf* for some conftest,
# http://sources.gentoo.org/viewcvs.py/portage/trunk/sandbox/files/sandbox/sandbox.c also permits these
valid_dirs = [self.pkg_dir(), "/tmp/", "/var/tmp/", "/dev/tty", "/dev/pts/", "/dev/pty", "/dev/null", "/dev/zero", "/dev/ptmx", "/proc/", "/usr/lib/conftest", "/usr/lib/cf"]
if ctx.config.values.build.buildhelper == "ccache":
valid_dirs.append("%s/.ccache" % os.environ["HOME"])
# every qt/KDE application check these
valid_dirs.append("%s/.qt/.qt_plugins_3.3rc.lock" % os.environ["HOME"])
valid_dirs.append("%s/.qt/qt_plugins_3.3rc.tmp" % os.environ["HOME"])
valid_dirs.append("%s/.qt/.qtrc.lock" % os.environ["HOME"])
valid_dirs.append("%s/.qt/.qt_designerrc.lock" % os.environ["HOME"])
valid_dirs.append("/usr/qt/3/etc/settings/.qt_plugins_3.3rc.lock")
valid_dirs.append("/usr/qt/3/etc/settings/qt_plugins_3.3rc.tmp")
valid_dirs.append("/usr/qt/3/etc/settings/qt_plugins_3.3rc")
ret = catbox.run(self.actionLocals[func], valid_dirs, logger=self.log_sandbox_violation)
if ret.code == 1:
raise RuntimeError
if ret.violations != []:
ctx.ui.error(_("Sandbox violations!"))
else:
if mandatory:
Error, _("unable to call function from actions: %s") %func
raise Error(_("unable to call function from actions: %s") % func)
os.chdir(curDir)
return True
@@ -496,6 +532,7 @@ class Builder:
ctx.ui.action(_("* Applying patch: %s") % patch.filename)
pisi.util.do_patch(self.srcDir, patchFile, level=patch.level)
return True
return True
def generate_static_package_object(self):
ar_files = []
@@ -560,7 +597,7 @@ class Builder:
except KeyError:
pisi.util.strip_directory(install_dir)
def gen_metadata_xml(self, package, build_no=None):
def gen_metadata_xml(self, package):
"""Generate the metadata.xml file for build source.
metadata.xml is composed of the information from specfile plus
@@ -634,7 +671,16 @@ class Builder:
files.write(files_xml_path)
self.files = files
def find_old_package_info(self, package_name):
def calc_build_no(self, package_name):
"""Calculate build number"""
def metadata_changed(old_metadata, new_metadata):
for key in old_metadata.package.__dict__.keys():
if old_metadata.package.__dict__[key] != new_metadata.package.__dict__[key]:
if key != "build":
return True
return False
# find previous build in packages dir
found = []
@@ -654,87 +700,71 @@ class Builder:
ctx.ui.warning('Package file %s may be corrupt. Skipping.' % old_package_fn)
for root, dirs, files in os.walk(ctx.config.compiled_packages_dir()):
for file in files:
locate_old_package(pisi.util.join_path(root,file))
for f in files:
locate_old_package(pisi.util.join_path(root,f))
outdir=ctx.get_option('output_dir')
if not outdir:
outdir = '.'
for file in [pisi.util.join_path(outdir,entry) for entry in os.listdir(outdir)]:
if os.path.isfile(file):
locate_old_package(file)
for f in [pisi.util.join_path(outdir,entry) for entry in os.listdir(outdir)]:
if os.path.isfile(f):
locate_old_package(f)
if found:
if not found:
return (1, None)
ctx.ui.warning(_('(no previous build found, setting build no to 1.)'))
else:
a = filter(lambda (x,y): y != None, found)
ctx.ui.debug(str(a))
if a:
# sort in order of increasing build number
a.sort(lambda x,y : cmp(x[1],y[1]))
old_package_fn = a[-1][0] # get the last one
old_build = a[-1][1]
return (old_package_fn, old_build)
return None
# compare old files.xml with the new one..
old_pkg = pisi.package.Package(old_package_fn, 'r')
old_pkg.read(pisi.util.join_path(ctx.config.tmp_dir(), 'oldpkg'))
def calc_build_no(self, old_package_info):
"""Calculate build number"""
changed = False
fnew = self.files.list
fold = old_pkg.files.list
fold.sort(lambda x,y : cmp(x.path,y.path))
fnew.sort(lambda x,y : cmp(x.path,y.path))
def metadata_changed(old_metadata, new_metadata):
for key in old_metadata.package.__dict__.keys():
if old_metadata.package.__dict__[key] != new_metadata.package.__dict__[key]:
if key != "build":
return True
return False
if not old_package_info:
return (1, None)
ctx.ui.warning(_('(no previous build no info found, setting build no to 1.)'))
else:
old_package_fn, old_build = old_package_info
# compare old files.xml with the new one..
old_pkg = pisi.package.Package(old_package_fn, 'r')
old_pkg.read(pisi.util.join_path(ctx.config.tmp_dir(), 'oldpkg'))
changed = False
fnew = self.files.list
fold = old_pkg.files.list
fold.sort(lambda x,y : cmp(x.path,y.path))
fnew.sort(lambda x,y : cmp(x.path,y.path))
if len(fnew) != len(fold):
changed = True
else:
for i in range(len(fold)):
fo = fold.pop(0)
fn = fnew.pop(0)
if fo.path != fn.path:
changed = True
break
else:
if fo.hash != fn.hash:
if len(fnew) != len(fold):
changed = True
else:
for i in range(len(fold)):
fo = fold.pop(0)
fn = fnew.pop(0)
if fo.path != fn.path:
changed = True
break
else:
if fo.hash != fn.hash:
changed = True
break
if metadata_changed(old_pkg.metadata, self.metadata):
changed = True
self.old_packages.append(os.path.basename(old_package_fn))
if metadata_changed(old_pkg.metadata, self.metadata):
changed = True
ctx.ui.debug('old build number: %s' % old_build)
self.old_packages.append(os.path.basename(old_package_fn))
else: # no old build had a build number
old_build = None
# set build number
if old_build is None:
ctx.ui.warning(_('(old package lacks a build no, setting build no to 1.)'))
return (1, None)
elif changed:
ctx.ui.info(_('There are changes, incrementing build no to %d') % (old_build + 1))
return (old_build + 1, old_build)
else:
ctx.ui.info(_('There is no change from previous build %d') % old_build)
return (old_build, old_build)
ctx.ui.debug('old build number: %s' % old_build)
# set build number
if old_build is None:
ctx.ui.warning(_('(old package lacks a build no, setting build no to 1.)'))
return (1, None)
elif changed:
ctx.ui.info(_('There are changes, incrementing build no to %d') % (old_build + 1))
return (old_build + 1, old_build)
else:
ctx.ui.info(_('There is no change from previous build %d') % old_build)
return (old_build, old_build)
def build_packages(self):
"""Build each package defined in PSPEC file. After this process there
@@ -814,16 +844,12 @@ class Builder:
ctx.ui.info(_("Generating %s,") % ctx.const.metadata_xml)
self.gen_metadata_xml(package)
# find the latest build info if any with build no.
# old_package_info = (old_package_file_name, old_build_no)
old_package_info = self.find_old_package_info(package.name)
# build number
if ctx.config.options.ignore_build_no or not ctx.config.values.build.buildno:
build_no = old_build_no = None
ctx.ui.warning(_('Build number is not available. For repo builds you must enable buildno in pisi.conf.'))
else:
build_no, old_build_no = self.calc_build_no(old_package_info)
build_no, old_build_no = self.calc_build_no(package.name)
self.metadata.package.build = build_no
self.metadata.write(pisi.util.join_path(self.pkg_dir(), ctx.const.metadata_xml))
pisi/cli/__init__.py
+1 -1
View File
@@ -103,7 +103,7 @@ class CLI(pisi.ui.UI):
opt = int(s)
if 1 <= opt and opt <= len(opts):
return opts(opt-1)
except (Exception,e):
except Exception:
pass
def confirm(self, msg):
pisi/cli/commands.py
+37 -37
View File
@@ -99,9 +99,9 @@ class Command(object):
@staticmethod
def commands_string():
s = ''
list = [x.name[0] for x in Command.cmd]
list.sort()
for name in list:
l = [x.name[0] for x in Command.cmd]
l.sort()
for name in l:
commandcls = Command.cmd_dict[name]
trans = gettext.translation('pisi', fallback=True)
summary = trans.ugettext(commandcls.__doc__).split('\n')[0]
@@ -173,12 +173,12 @@ class Command(object):
# make destdir absolute
if self.options.destdir:
dir = str(self.options.destdir)
d = str(self.options.destdir)
import os.path
if not os.path.exists(dir):
pisi.cli.printu(_('Destination directory %s does not exist. Creating directory.\n') % dir)
os.makedirs(dir)
self.options.destdir = os.path.realpath(dir)
if not os.path.exists(d):
pisi.cli.printu(_('Destination directory %s does not exist. Creating directory.\n') % d)
os.makedirs(d)
self.options.destdir = os.path.realpath(d)
def check_auth_info(self):
username = self.options.username
@@ -374,7 +374,7 @@ the package in graphviz format to 'pgraph.dot'.
repo = ctx.get_option('repository')
ctx.ui.info(_('Plotting packages in repository %s') % repo)
else:
repo = pisi.db.itembyrepodb.repos
repo = pisi.itembyrepodb.repos
if self.args:
a = self.args
else:
@@ -387,7 +387,7 @@ the package in graphviz format to 'pgraph.dot'.
# if A is empty, then graph all packages
ctx.ui.info(_('Plotting a graph of relations among all installed packages'))
a = ctx.installdb.list_installed()
repo = pisi.db.itembyrepodb.installed
repo = pisi.itembyrepodb.installed
g = pisi.api.package_graph(a, repo = repo,
ignore_installed = ctx.get_option('ignore_installed'))
g.write_graphviz(file(ctx.get_option('output'), 'w'))
@@ -451,8 +451,10 @@ to be downloaded from a repository containing sources.
default=False, help=_("Do not install build dependencies, fail if a build dependency is present"))
group.add_option("-F", "--package-format", action="store", default='1.1',
help=_("PiSi binary package formats: '1.0', '1.1' (default)"))
group.add_option("--use-quilt", action="store_true",
default=False, help=_("Use quilt patch management system instead of GNU patch"))
group.add_option("--use-quilt", action="store_true", default=False,
help=_("Use quilt patch management system instead of GNU patch"))
group.add_option("--ignore-sandbox", action="store_true", default=False,
help=_("Do not constrain build process inside the build folder"))
def add_steps_options(self):
group = OptionGroup(self.parser, _("build steps"))
@@ -530,7 +532,7 @@ and creates a delta pisi package with the changed files between two releases.
self.init(database=False, write=False)
if len(self.args) is not 2:
if len(self.args) != 2:
self.help()
return
@@ -730,9 +732,9 @@ expanded to package names.
import os
patterns = []
file = ctx.get_option('exclude_from')
if os.path.exists(file):
for line in open(file, "r").readlines():
f = ctx.get_option('exclude_from')
if os.path.exists(f):
for line in open(f, "r").readlines():
if not line.startswith('#') and not line == '\n':
patterns.append(line.strip())
if patterns:
@@ -752,8 +754,8 @@ expanded to package names.
if not match:
# match pattern in component names
for cmp in fnmatch.filter(ctx.componentdb.list_components(), pattern):
packages = packages - set(ctx.componentdb.get_union_packages(cmp, walk=True))
for compare in fnmatch.filter(ctx.componentdb.list_components(), pattern):
packages = packages - set(ctx.componentdb.get_union_packages(compare, walk=True))
return list(packages)
@@ -951,7 +953,7 @@ Usage: info <package1> <package2> ... <packagen>
ctx.ui.info(_('[inst] '), noln=True)
else:
ctx.ui.info(_('Installed package:'))
self.print_pkginfo(metadata, files,pisi.db.itembyrepodb.installed)
self.print_pkginfo(metadata, files,pisi.itembyrepodb.installed)
if ctx.packagedb.has_package(arg):
metadata, files, repo = pisi.api.info_name(arg, False)
@@ -959,7 +961,7 @@ Usage: info <package1> <package2> ... <packagen>
ctx.ui.info(_('[repo] '), noln=True)
else:
ctx.ui.info(_('Package found in %s repository:') % repo)
self.print_pkginfo(metadata, files, pisi.db.itembyrepodb.repos)
self.print_pkginfo(metadata, files, pisi.itembyrepodb.repos)
if not ctx.packagedb.has_package(arg):
ctx.ui.info(_("%s is not found in repositories") % arg)
@@ -968,8 +970,6 @@ Usage: info <package1> <package2> ... <packagen>
ctx.ui.info(_("%s is not installed") % arg)
def print_pkginfo(self, metadata, files, repo = None):
import os.path
if ctx.get_option('short'):
pkg = metadata.package
ctx.ui.info('%15s - %s' % (pkg.name, unicode(pkg.summary)))
@@ -1017,7 +1017,7 @@ If no packages are given, checks all installed packages.
def options(self):
group = OptionGroup(self.parser, _("check options"))
group.add_option("-c", "--component", action="store",
default=None, help=_("Check installed packages under given component"))
default=None, help=_("Check installed packages under given component"))
self.parser.add_option_group(group)
def run(self):
@@ -1146,7 +1146,7 @@ Usage: list-installed
ctx.ui.info(_('Package Name |St| Version| Rel.| Build| Distro| Date'))
print '========================================================================'
for pkg in installed:
package = pisi.api.get_installed_package(pkg)
package = ctx.packagedb.get_package(pkg, pisi.itembyrepodb.installed)
inst_info = ctx.installdb.get_info(pkg)
if self.options.long:
ctx.ui.info(unicode(package))
@@ -1273,7 +1273,7 @@ NB: We support only local files (e.g., /a/b/c) and http:// URIs at the moment
else:
name = 'pardus-2007'
indexuri = 'http://paketler.pardus.org.tr/pardus-2007/pisi-index.xml.bz2'
pisi.api.add_repository(name, indexuri, ctx.get_option('at'))
pisi.api.add_repo(name, indexuri, ctx.get_option('at'))
if ctx.ui.confirm(_('Update PiSi database for repository %s?') % name):
try:
pisi.api.update_repo(name)
@@ -1383,13 +1383,13 @@ all repositories.
component = ctx.get_option('component')
if component:
list = ctx.componentdb.get_packages(component, walk=True, repo=repo)
l = ctx.componentdb.get_packages(component, walk=True, repo=repo)
else:
list = ctx.packagedb.list_packages(repo)
l = ctx.packagedb.list_packages(repo)
installed_list = ctx.installdb.list_installed()
list.sort()
for p in list:
package = pisi.api.get_repo_package(p)
l.sort()
for p in l:
package = ctx.packagedb.get_package(p)
if self.options.long:
ctx.ui.info(unicode(package))
else:
@@ -1426,9 +1426,9 @@ repositories.
self.init(database = True, write = False)
list = ctx.componentdb.list_components()
list.sort()
for p in list:
l = ctx.componentdb.list_components()
l.sort()
for p in l:
component = ctx.componentdb.get_component(p)
if self.options.long:
ctx.ui.info(unicode(component))
@@ -1465,9 +1465,9 @@ Gives a brief list of sources published in the repositories.
self.init(database = True, write = False)
list = ctx.sourcedb.list()
list.sort()
for p in list:
l = ctx.sourcedb.list()
l.sort()
for p in l:
sf, repo = ctx.sourcedb.get_spec_repo(p)
if self.options.long:
ctx.ui.info('[Repository: ' + repo + ']')
@@ -1524,7 +1524,7 @@ Lists the packages that will be upgraded.
ctx.ui.info(_('Package Name |St| Version| Rel.| Build| Distro| Date'))
print '========================================================================'
for pkg in upgradable_pkgs:
package = pisi.api.get_installed_package(pkg)
package = ctx.packagedb.get_package(pkg, pisi.itembyrepodb.installed)
inst_info = ctx.installdb.get_info(pkg)
if self.options.long:
ctx.ui.info(package)
pisi/cli/pisicli.py
-3
View File
@@ -26,9 +26,6 @@ from pisi.cli.commands import *
class ParserError(pisi.Exception):
pass
class Error(pisi.Error):
pass
class PreParser(OptionParser):
"""consumes any options, and finds arguments from command line"""
pisi/component.py
+172
View File
@@ -15,8 +15,11 @@ __trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi
import pisi.context as ctx
import pisi.pxml.xmlfile as xmlfile
import pisi.pxml.autoxml as autoxml
import pisi.lockeddbshelve as shelve
import pisi.itembyrepodb
class Error(pisi.Error):
pass
@@ -76,4 +79,173 @@ class Component(xmlfile.XmlFile):
# the parts of this component.
# to be filled by the component database, thus it is optional.
t_Packages = [ [autoxml.String], autoxml.optional, "Parts/Package"]
t_Sources = [ [autoxml.String], autoxml.optional, "Parts/Source"]
class ComponentDB(object):
"""a database of components"""
def __init__(self):
self.d = pisi.itembyrepodb.ItemByRepoDB('component')
def close(self):
self.d.close()
def destroy(self):
self.d.destroy()
def has_component(self, name, repo = pisi.itembyrepodb.repos, txn = None):
name = str(name)
return self.d.has_key(name, repo, txn)
def get_component(self, name, repo=None, txn = None):
try:
return self.d.get_item(name, repo, txn=txn)
except pisi.itembyrepodb.NotfoundError, e:
raise Error(_('Component %s not found') % name)
def get_component_repo(self, name, repo=None, txn = None):
try:
return self.d.get_item_repo(name, repo, txn=txn)
except pisi.itembyrepodb.NotfoundError, e:
raise Error(_('Component %s not found') % name)
def get_union_comp(self, name, txn = None, repo = pisi.itembyrepodb.repos ):
"""get a union of all repository components packages, not just the first repo in order.
get only basic repo info from the first repo"""
def proc(txn):
s = self.d.d.get(name, txn=txn)
pkgs = set()
srcs = set()
for repostr in self.d.order(repo = repo):
if s.has_key(repostr):
pkgs |= set(s[repostr].packages)
srcs |= set(s[repostr].sources)
comp = self.get_component(name)
comp.packages = list(pkgs)
comp.sources = list(srcs)
return comp
return self.d.txn_proc(proc, txn)
def list_components(self, repo=None):
return self.d.list(repo)
# walk: walks through the underlying components' packages
def get_union_packages(self, component_name, walk=False, repo=pisi.itembyrepodb.repos, txn = None):
"""returns union of all repository component's packages, not just the first repo's
component's in order"""
component = self.get_union_comp(component_name, txn, repo)
if not walk:
return component.packages
packages = []
packages.extend(component.packages)
for dep in component.dependencies:
packages.extend(self.get_union_packages(dep, walk, repo, txn))
return packages
# walk: walks through the underlying components' packages
def get_packages(self, component_name, walk=False, repo=None, txn = None):
"""returns the given component's and underlying recursive components' packages"""
component = self.get_component(component_name, repo, txn)
if not walk:
return component.packages
packages = []
packages.extend(component.packages)
for dep in component.dependencies:
packages.extend(self.get_packages(dep, walk, repo, txn))
return packages
def add_child(self, component, repo, txn = None):
"""update component tree"""
parent_name = ".".join(component.name.split(".")[:-1])
if not parent_name: # root component
return
if self.has_component(parent_name, repo, txn):
parent = self.get_component(parent_name, repo, txn)
else:
parent = Component(name = parent_name)
if component.name not in parent.dependencies:
parent.dependencies.append(component.name)
self.d.add_item(parent_name, parent, repo, txn)
def update_component(self, component, repo, txn = None):
def proc(txn):
if self.has_component(component.name, repo, txn):
# preserve list of sources, packages and dependencies
current = self.d.get_item(component.name, repo, txn)
component.packages = current.packages
component.sources = current.sources
component.dependencies = current.dependencies
self.d.add_item(component.name, component, repo, txn)
self.add_child(component, repo, txn)
self.d.txn_proc(proc, txn)
def add_package(self, component_name, package, repo, txn = None):
def proc(txn):
assert component_name
if self.has_component(component_name, repo, txn):
component = self.get_component(component_name, repo, txn)
else:
component = Component( name = component_name )
if not package in component.packages:
component.packages.append(package)
self.d.add_item(component_name, component, repo, txn) # update
self.add_child(component, repo, txn)
self.d.txn_proc(proc, txn)
def remove_package(self, component_name, package, repo = None, txn = None):
def proc(txn, repo):
if not self.has_component(component_name, repo, txn):
raise Error(_('Information for component %s not available') % component_name)
if not repo:
repo = self.d.which_repo(component_name, txn=txn) # get default repo then
component = self.get_component(component_name, repo, txn)
if package in component.packages:
component.packages.remove(package)
self.d.add_item(component_name, component, repo, txn) # update
ctx.txn_proc(lambda x: proc(txn, repo), txn)
def add_spec(self, component_name, spec, repo, txn = None):
def proc(txn):
assert component_name
if self.has_component(component_name, repo, txn):
component = self.get_component(component_name, repo, txn)
else:
component = Component( name = component_name )
if not spec in component.sources:
component.sources.append(spec)
self.d.add_item(component_name, component, repo, txn) # update
self.add_child(component, repo, txn)
self.d.txn_proc(proc, txn)
def remove_spec(self, component_name, spec, repo = None, txn = None):
def proc(txn, repo):
if not self.has_component(component_name, repo, txn):
raise Error(_('Information for component %s not available') % component_name)
if not repo:
repo = self.d.which_repo(component_name, txn=txn) # get default repo then
component = self.get_component(component_name, repo, txn)
if spec in component.sources:
component.sources.remove(spec)
self.d.add_item(component_name, component, repo, txn) # update
ctx.txn_proc(lambda x: proc(txn, repo), txn)
def clear(self, txn = None):
self.d.clear(txn)
def remove_component(self, name, repo = None, txn = None):
name = str(name)
self.d.remove_item(name, repo, txn)
def remove_repo(self, repo, txn = None):
self.d.remove_repo(repo, txn=txn)
pisi/config.py
+3 -3
View File
@@ -82,9 +82,9 @@ class Config(object):
return self.destdir
def subdir(self, path):
dir = pisi.util.join_path(self.dest_dir(), path)
pisi.util.check_dir(dir)
return dir
subdir = pisi.util.join_path(self.dest_dir(), path)
pisi.util.check_dir(subdir)
return subdir
def log_dir(self):
return self.subdir(self.values.dirs.log_dir)
pisi/configfile.py
+3 -3
View File
@@ -64,9 +64,9 @@ class GeneralDefaults:
autoclean = False
distribution = "Pardus"
distribution_release = "2007"
http_proxy = (os.getenv("http_proxy") or os.getenv("HTTP_PROXY")) or None
https_proxy = (os.getenv("https_proxy") or os.getenv("HTTPS_PROXY")) or None
ftp_proxy = (os.getenv("ftp_proxy") or os.getenv("FTP_PROXY")) or None
http_proxy = os.getenv("HTTP_PROXY") or None
https_proxy = os.getenv("HTTPS_PROXY") or None
ftp_proxy = os.getenv("FTP_PROXY") or None
package_cache = False
package_cache_limit = 0
pisi/conflict.py
+44 -43
View File
@@ -16,23 +16,10 @@ import gettext
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi.context as ctx
import pisi.version
import pisi.pxml.autoxml as autoxml
import pisi.db.itembyrepodb
class Conflict:
__metaclass__ = autoxml.autoxml
s_Package = [autoxml.String, autoxml.mandatory]
a_version = [autoxml.String, autoxml.optional]
a_versionFrom = [autoxml.String, autoxml.optional]
a_versionTo = [autoxml.String, autoxml.optional]
a_release = [autoxml.String, autoxml.optional]
a_releaseFrom = [autoxml.String, autoxml.optional]
a_releaseTo = [autoxml.String, autoxml.optional]
import pisi.relation
""" Conflict relation """
class Conflict(pisi.relation.Relation):
def __str__(self):
s = self.package
if self.versionFrom:
@@ -49,39 +36,53 @@ class Conflict:
s += _(" release ") + self.release
return s
def conflicts(self, pkg_name, version, release):
"""determine if a package ver. conflicts with given conflicting spec"""
ret = True
v = pisi.version.Version(version)
if self.version:
ret &= v == pisi.version.Version(self.version)
if self.versionFrom:
ret &= v >= pisi.version.Version(self.versionFrom)
if self.versionTo:
ret &= v <= pisi.version.Version(self.versionTo)
r = pisi.version.Version(release)
if self.release:
ret &= r == pisi.version.Version(self.release)
if self.releaseFrom:
ret &= r >= pisi.version.Version(self.releaseFrom)
if self.releaseTo:
ret &= r <= pisi.version.Version(self.releaseTo)
return ret
def installed_package_conflicts(confinfo):
"""determine if an installed package in *repository* conflicts with
given conflicting spec"""
pkg_name = confinfo.package
if not ctx.installdb.is_installed(pkg_name):
return False
else:
pkg = api.get_installed_package(pkg_name)
(version, release) = (pkg.version, pkg.release)
return confinfo.conflicts(pkg_name, version, release)
return pisi.relation.installed_package_satisfies(confinfo)
def package_conflicts(pkg, confs):
for c in confs:
if pkg.name == c.package and c.conflicts(pkg.name, pkg.version, pkg.release):
if pkg.name == c.package and c.satisfies_relation(pkg.name, pkg.version, pkg.release):
return c
return None
def calculate_conflicts(order, packagedb):
# check conflicting packages in the installed system
def check_installed(pkg, order):
conflicts = []
for conflict in pkg.conflicts:
if conflict.package not in order and installed_package_conflicts(conflict):
conflicts.append(conflict)
return conflicts
B_0 = set(order)
conflicting_pkgs = conflicts_inorder = set()
conflicting_pairs = {}
for x in order:
pkg = packagedb.get_package(x)
# check if any package has conflicts with the installed packages
conflicts = check_installed(pkg, order)
if conflicts:
conflicting_pairs[x] = map(lambda c:str(c), conflicts)
conflicting_pkgs = conflicting_pkgs.union(map(lambda c:c.package, conflicts))
# now check if any package has conflicts with each other
B_i = B_0.intersection(set(map(lambda c:c.package, pkg.conflicts)))
conflicts_inorder_i = set()
for p in map(lambda x:packagedb.get_package(x), B_i):
conflicted = package_conflicts(p, pkg.conflicts)
if conflicted:
conflicts_inorder_i.add(str(conflicted))
if conflicts_inorder_i:
conflicts_inorder = conflicts_inorder.union(conflicts_inorder_i)
conflicts_inorder.add(pkg.name)
return (conflicting_pkgs, conflicts_inorder, conflicting_pairs)
pisi/constants.py
+1 -2
View File
@@ -18,8 +18,6 @@ import gettext
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi
class _constant:
"Constant members implementation"
class ConstError(TypeError):
@@ -75,6 +73,7 @@ class Constants:
self.__c.actions_file = "actions.py"
self.__c.files_dir = "files"
self.__c.metadata_dir = "metadata"
self.__c.translations_file = "translations.xml"
self.__c.comar_dir = "comar"
self.__c.files_xml = "files.xml"
self.__c.metadata_xml = "metadata.xml"
pisi/context.py
+3
View File
@@ -41,6 +41,9 @@ dbenv = None
installdb = None
packagedb = None
repodb = None
sourcedb = None
filesdb = None
componentdb = None
invidx = None
comar = None
pisi/db/__init__.py
-11
View File
@@ -1,11 +0,0 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2005 - 2007, TUBITAK/UEKAE
#
# This program is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 2 of the License, or (at your option)
# any later version.
#
# Please read the COPYING file.
#
pisi/db/componentdb.py
-190
View File
@@ -1,190 +0,0 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2005 - 2007, TUBITAK/UEKAE
#
# This program is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 2 of the License, or (at your option)
# any later version.
#
# Please read the COPYING file.
#
import types
import gettext
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi
import pisi.context as ctx
import pisi.db.itembyrepodb as itembyrepodb
import pisi.component
class ComponentDB(object):
def __init__(self):
self.d = itembyrepodb.ItemByRepoDB('component')
def close(self):
self.d.close()
def destroy(self):
self.d.destroy()
def has_component(self, name, repo = itembyrepodb.repos, txn = None):
name = str(name)
return self.d.has_key(name, repo, txn)
def get_component(self, name, repo=None, txn = None):
try:
return self.d.get_item(name, repo, txn=txn)
except itembyrepodb.NotfoundError, e:
raise Error(_('Component %s not found') % name)
def get_component_repo(self, name, repo=None, txn = None):
try:
return self.d.get_item_repo(name, repo, txn=txn)
except itembyrepodb.NotfoundError, e:
raise Error(_('Component %s not found') % name)
def get_union_comp(self, name, txn = None, repo = itembyrepodb.repos ):
"""get a union of all repository components packages, not just the first repo in order.
get only basic repo info from the first repo"""
def proc(txn):
s = self.d.d.get(name, txn=txn)
pkgs = set()
srcs = set()
for repostr in self.d.order(repo = repo):
if s.has_key(repostr):
pkgs |= set(s[repostr].packages)
srcs |= set(s[repostr].sources)
comp = self.get_component(name)
comp.packages = list(pkgs)
comp.sources = list(srcs)
return comp
return self.d.txn_proc(proc, txn)
def list_components(self, repo=None):
return self.d.list(repo)
# walk: walks through the underlying components' packages
def get_union_packages(self, component_name, walk=False, repo=pisi.db.itembyrepodb.repos, txn = None):
"""returns union of all repository component's packages, not just the first repo's
component's in order"""
component = self.get_union_comp(component_name, txn, repo)
if not walk:
return component.packages
packages = []
packages.extend(component.packages)
for dep in component.dependencies:
packages.extend(self.get_union_packages(dep, walk, repo, txn))
return packages
# walk: walks through the underlying components' packages
def get_packages(self, component_name, walk=False, repo=None, txn = None):
"""returns the given component's and underlying recursive components' packages"""
component = self.get_component(component_name, repo, txn)
if not walk:
return component.packages
packages = []
packages.extend(component.packages)
for dep in component.dependencies:
packages.extend(self.get_packages(dep, walk, repo, txn))
return packages
def add_child(self, component, repo, txn = None):
"""update component tree"""
parent_name = ".".join(component.name.split(".")[:-1])
if not parent_name: # root component
return
if self.has_component(parent_name, repo, txn):
parent = self.get_component(parent_name, repo, txn)
else:
parent = pisi.component.Component(name = parent_name)
if component.name not in parent.dependencies:
parent.dependencies.append(component.name)
self.d.add_item(parent_name, parent, repo, txn)
def update_component(self, component, repo, txn = None):
def proc(txn):
if self.has_component(component.name, repo, txn):
# preserve list of sources, packages and dependencies
current = self.d.get_item(component.name, repo, txn)
component.packages = current.packages
component.sources = current.sources
component.dependencies = current.dependencies
self.d.add_item(component.name, component, repo, txn)
self.add_child(component, repo, txn)
self.d.txn_proc(proc, txn)
def add_package(self, component_name, package, repo, txn = None):
def proc(txn):
assert component_name
if self.has_component(component_name, repo, txn):
component = self.get_component(component_name, repo, txn)
else:
component = pisi.component.Component( name = component_name )
if not package in component.packages:
component.packages.append(package)
self.d.add_item(component_name, component, repo, txn) # update
self.add_child(component, repo, txn)
self.d.txn_proc(proc, txn)
def remove_package(self, component_name, package, repo = None, txn = None):
def proc(txn, repo):
if not self.has_component(component_name, repo, txn):
raise Error(_('Information for component %s not available') % component_name)
if not repo:
repo = self.d.which_repo(component_name, txn=txn) # get default repo then
component = self.get_component(component_name, repo, txn)
if package in component.packages:
component.packages.remove(package)
self.d.add_item(component_name, component, repo, txn) # update
ctx.txn_proc(lambda x: proc(txn, repo), txn)
def add_spec(self, component_name, spec, repo, txn = None):
def proc(txn):
assert component_name
if self.has_component(component_name, repo, txn):
component = self.get_component(component_name, repo, txn)
else:
component = pisi.component.Component( name = component_name )
if not spec in component.sources:
component.sources.append(spec)
self.d.add_item(component_name, component, repo, txn) # update
self.add_child(component, repo, txn)
self.d.txn_proc(proc, txn)
def remove_spec(self, component_name, spec, repo = None, txn = None):
def proc(txn, repo):
if not self.has_component(component_name, repo, txn):
raise Error(_('Information for component %s not available') % component_name)
if not repo:
repo = self.d.which_repo(component_name, txn=txn) # get default repo then
component = self.get_component(component_name, repo, txn)
if spec in component.sources:
component.sources.remove(spec)
self.d.add_item(component_name, component, repo, txn) # update
ctx.txn_proc(lambda x: proc(txn, repo), txn)
def clear(self, txn = None):
self.d.clear(txn)
def remove_component(self, name, repo = None, txn = None):
name = str(name)
self.d.remove_item(name, repo, txn)
def remove_repo(self, repo, txn = None):
self.d.remove_repo(repo, txn=txn)
pisi/db/filesdb.py
-68
View File
@@ -1,68 +0,0 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2005 - 2007, TUBITAK/UEKAE
#
# This program is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 2 of the License, or (at your option)
# any later version.
#
# Please read the COPYING file.
#
import pisi.db.lockeddbshelve as shelve
import pisi.files
class FilesDB(shelve.LockedDBShelf):
'''
FilesDB provides db access operations for Files objects (files.Files)
'''
def __init__(self):
shelve.LockedDBShelf.__init__(self, 'files')
def add_files(self, pkg_name, files, txn = None):
def proc(txn):
for x in files.list:
path = x.path
del x.path # don't store redundant attribute in db
self.put(path, (pkg_name, x), txn)
x.path = path # store it back in
self.txn_proc(proc, txn)
def remove_files(self, files, txn = None):
def proc(txn):
for x in files.list:
if self.has_key(x.path):
self.delete(x.path, txn)
self.txn_proc(proc, txn)
def has_file(self, path, txn = None):
return self.has_key(str(path), txn)
def get_file(self, path, txn = None):
path = str(path)
def proc(txn):
if not self.has_key(path, txn):
return None
else:
(name, fileinfo) = self.get(path, txn)
fileinfo.path = path
return (name, fileinfo)
return self.txn_proc(proc, txn)
def match_files(self, glob):
# NB: avoid using, this reads the entire db
import fnmatch
glob = str(glob)
infos = []
for key in self.keys():
if fnmatch.fnmatch(key, glob):
# FIXME: Why should we assign path attribute manually
# in fileinfo? This is also done in get_file(), seems
# like a dirty workaround... - baris
name = self[key][0]
fileinfo = self[key][1]
fileinfo.path = key
infos.append((name, fileinfo))
return infos
pisi/db/dbshelve.py → pisi/dbshelve.py
+1 -1
View File
@@ -89,7 +89,7 @@ class DBShelf:
raise CodingError()
def encode(self, obj):
return cPickle.dumps(obj, 2)
return cPickle.dumps(obj, 1)
def clear(self, txn = None):
def proc(txn):
pisi/delta.py
+20 -15
View File
@@ -74,8 +74,8 @@ def create_delta_package(old_package, new_package):
ctx.build_leftover = util.join_path(ctx.config.tmp_dir(), ctx.const.install_tar_lzma)
tar = archive.ArchiveTar(util.join_path(ctx.config.tmp_dir(), ctx.const.install_tar_lzma), "tarlzma")
for file in files_delta:
tar.add_to_archive(file.path)
for f in files_delta:
tar.add_to_archive(f.path)
tar.close()
os.chdir(ctx.config.tmp_dir())
@@ -99,34 +99,39 @@ def create_delta_package(old_package, new_package):
def find_delta(oldfiles, newfiles):
hashto_files = {}
for file in newfiles.list:
hashto_files.setdefault(file.hash, []).append(file)
for f in newfiles.list:
hashto_files.setdefault(f.hash, []).append(f)
files_new = sets.Set(map(lambda x:x.hash, newfiles.list))
files_old = sets.Set(map(lambda x:x.hash, oldfiles.list))
files_delta = files_new - files_old
deltas = []
for hash in files_delta:
deltas.extend(hashto_files[hash])
for h in files_delta:
deltas.extend(hashto_files[h])
# Directory hashes are None. There was a bug with PolicyKit that should have an empty directory.
if hashto_files.has_key(None):
deltas.extend(hashto_files[None])
return deltas
def find_relocations(oldfiles, newfiles):
files_new = {}
for file in newfiles.list:
files_new.setdefault(file.hash, []).append(file)
for f in newfiles.list:
files_new.setdefault(f.hash, []).append(f)
files_old = {}
for file in oldfiles.list:
files_old.setdefault(file.hash, []).append(file)
for f in oldfiles.list:
files_old.setdefault(f.hash, []).append(f)
relocations = []
for hash in files_new.keys():
if hash and hash in files_old:
for i in range(len(files_new[hash])):
if files_old[hash][0].path != files_new[hash][i].path:
relocations.append((files_old[hash][0], files_new[hash][i]))
for h in files_new.keys():
if h and h in files_old:
old_paths = [x.path for x in files_old[h]]
for i in range(len(files_new[h])):
if files_new[h][i].path not in old_paths:
relocations.append((files_old[h][0], files_new[h][i]))
return relocations
pisi/dependency.py
+8 -47
View File
@@ -16,25 +16,11 @@ import gettext
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi
import pisi.context as ctx
import pisi.db.packagedb as packagedb
import pisi.version
import pisi.pxml.autoxml as autoxml
import pisi.db.itembyrepodb
class Dependency:
__metaclass__ = autoxml.autoxml
s_Package = [autoxml.String, autoxml.mandatory]
a_version = [autoxml.String, autoxml.optional]
a_versionFrom = [autoxml.String, autoxml.optional]
a_versionTo = [autoxml.String, autoxml.optional]
a_release = [autoxml.String, autoxml.optional]
a_releaseFrom = [autoxml.String, autoxml.optional]
a_releaseTo = [autoxml.String, autoxml.optional]
import pisi.relation
""" Dependency relation """
class Dependency(pisi.relation.Relation):
def __str__(self):
s = self.package
if self.versionFrom:
@@ -51,25 +37,6 @@ class Dependency:
s += _(" release ") + self.release
return s
def satisfies(self, pkg_name, version, release):
"""determine if a package ver. satisfies given dependency spec"""
ret = True
v = pisi.version.Version(version)
if self.version:
ret &= v == pisi.version.Version(self.version)
if self.versionFrom:
ret &= v >= pisi.version.Version(self.versionFrom)
if self.versionTo:
ret &= v <= pisi.version.Version(self.versionTo)
r = pisi.version.Version(release)
if self.release:
ret &= r == pisi.version.Version(self.release)
if self.releaseFrom:
ret &= r >= pisi.version.Version(self.releaseFrom)
if self.releaseTo:
ret &= r <= pisi.version.Version(self.releaseTo)
return ret
def dict_satisfies_dep(dict, depinfo):
"""determine if a package in a dictionary satisfies given dependency spec"""
pkg_name = depinfo.package
@@ -78,18 +45,12 @@ def dict_satisfies_dep(dict, depinfo):
else:
pkg = dict[pkg_name]
(version, release) = (pkg.version, pkg.release)
return depinfo.satisfies(pkg_name, version, release)
return depinfo.satisfies_relation(pkg_name, version, release)
def installed_satisfies_dep(depinfo):
"""determine if a package in *repository* satisfies given
dependency spec"""
pkg_name = depinfo.package
if not ctx.installdb.is_installed(pkg_name):
return False
else:
pkg = pisi.api.get_installed_package(pkg_name)
(version, release) = (pkg.version, pkg.release)
return depinfo.satisfies(pkg_name, version, release)
return pisi.relation.installed_package_satisfies(depinfo)
def repo_satisfies_dep(depinfo):
"""determine if a package in *repository* satisfies given
@@ -98,9 +59,9 @@ dependency spec"""
if not ctx.packagedb.has_package(pkg_name):
return False
else:
pkg = pisi.api.get_repo_package(pkg_name)
pkg = ctx.packagedb.get_package(pkg_name)
(version, release) = (pkg.version, pkg.release)
return depinfo.satisfies(pkg_name, version, release)
return depinfo.satisfies_relation(pkg_name, version, release)
def satisfies_dependencies(pkg, deps, sat = installed_satisfies_dep):
for dep in deps:
@@ -111,7 +72,7 @@ def satisfies_dependencies(pkg, deps, sat = installed_satisfies_dep):
return True
def satisfies_runtime_deps(pkg):
deps = pisi.api.get_repo_package(pkg).runtimeDependencies()
deps = ctx.packagedb.get_package(pkg).runtimeDependencies()
return satisfies_dependencies(pkg, deps)
def installable(pkg):
pisi/fetcher.py
+6 -5
View File
@@ -282,17 +282,17 @@ class FTPRangeHandler(urllib2.FTPHandler):
path, attrs = urllib.splitattr(req.get_selector())
dirs = path.split('/')
dirs = map(urllib.unquote, dirs)
dirs, file = dirs[:-1], dirs[-1]
dirs, f = dirs[:-1], dirs[-1]
if dirs and not dirs[0]:
dirs = dirs[1:]
try:
fw = self.connect_ftp('', '', host, port, dirs)
type = file and 'I' or 'D'
t = f and 'I' or 'D'
for attr in attrs:
attr, value = urllib.splitattr(attr)
if attr.lower() == 'type' and \
value in ('a', 'A', 'i', 'I', 'd', 'D'):
type = value.upper()
t = value.upper()
rawr = req.headers.get('Range', None)
if rawr:
@@ -300,7 +300,7 @@ class FTPRangeHandler(urllib2.FTPHandler):
else:
rest = 0
fp, retrlen = fw.retrfile(file, type, rest)
fp, retrlen = fw.retrfile(f, t, rest)
fb, lb = rest, retrlen
if retrlen is None or retrlen == 0:
@@ -327,7 +327,7 @@ class FTPRangeHandler(urllib2.FTPHandler):
raise IOError, (_('ftp error'), msg), sys.exc_info()[2]
def connect_ftp(self, user, passwd, host, port, dirs):
fw = ftpwrapper('', '', host, port, dirs)
fw = ftpwrapper(user, passwd, host, port, dirs)
return fw
class ftpwrapper(urllib.ftpwrapper):
@@ -355,6 +355,7 @@ class ftpwrapper(urllib.ftpwrapper):
if str(reason)[:3] == '501':
# workaround for REST not suported error
fp, retrlen = self.retrfile(file, type)
# WTF? No global (RangeableFileObject) found. RangeableFileObject only defined in urlgrabber / caglar
fp = RangeableFileObject(fp, (rest,''))
return (fp, retrlen)
elif str(reason)[:3] != '550':
pisi/file.py
-1
View File
@@ -144,7 +144,6 @@ class File:
"it is pointless to open a file without a URI and a mode"
self.transfer_dir = transfer_dir
self.sha1sum = sha1sum
self.compress = compress
self.sign = sign
pisi/files.py
+54
View File
@@ -15,6 +15,8 @@ during the build process of a package and used in installation.'''
import pisi.pxml.autoxml as autoxml
import pisi.lockeddbshelve as shelve
class FileInfo:
"""File holds the information for a File node/tag in files.xml"""
@@ -47,3 +49,55 @@ class Files(autoxml.xmlfile.XmlFile):
def append(self, fileinfo):
self.list.append(fileinfo)
class FilesDB(shelve.LockedDBShelf):
def __init__(self):
shelve.LockedDBShelf.__init__(self, 'files')
def add_files(self, pkg_name, files, txn = None):
def proc(txn):
for x in files.list:
path = x.path
del x.path # don't store redundant attribute in db
self.put(path, (pkg_name, x), txn)
x.path = path # store it back in
self.txn_proc(proc, txn)
def remove_files(self, files, txn = None):
def proc(txn):
for x in files.list:
if self.has_key(x.path):
self.delete(x.path, txn)
self.txn_proc(proc, txn)
def has_file(self, path, txn = None):
return self.has_key(str(path), txn)
def get_file(self, path, txn = None):
path = str(path)
def proc(txn):
if not self.has_key(path, txn):
return None
else:
(name, fileinfo) = self.get(path, txn)
fileinfo.path = path
return (name, fileinfo)
return self.txn_proc(proc, txn)
def match_files(self, glob):
# NB: avoid using, this reads the entire db
import fnmatch
glob = str(glob)
infos = []
for key in self.keys():
if fnmatch.fnmatch(key, glob):
# FIXME: Why should we assign path attribute manually
# in fileinfo? This is also done in get_file(), seems
# like a dirty workaround... - baris
name = self[key][0]
fileinfo = self[key][1]
fileinfo.path = key
infos.append((name, fileinfo))
return infos
pisi/graph.py
+3 -3
View File
@@ -44,11 +44,11 @@ class Digraph(object):
def edges(self):
"return a list of edge descriptors"
list = []
l = []
for u in self.__v:
for v in self.__adj[u]:
list.append( (u,v) )
return list
l.append( (u,v) )
return l
def from_list(self, el):
"convert a list of edges (u,v) to graph"
pisi/index.py
+1 -3
View File
@@ -23,15 +23,12 @@ import pisi
import pisi.context as ctx
import pisi.specfile as specfile
import pisi.metadata as metadata
import pisi.db.packagedb as packagedb
import pisi.db.sourcedb as sourcedb
import pisi.util as util
import pisi.package
import pisi.pxml.xmlfile as xmlfile
import pisi.file
import pisi.pxml.autoxml as autoxml
import pisi.component as component
import pisi.specfile as specfile
class Error(pisi.Error):
pass
@@ -129,6 +126,7 @@ class Index(xmlfile.XmlFile):
for comp in self.components:
ctx.componentdb.update_component(comp, repo, txn)
ctx.packagedb.remove_repo(repo, txn=txn)
ctx.packagedb.add_obsoletes(self.distribution.obsoletes, repo, txn=txn)
for pkg in self.packages:
ctx.packagedb.add_package(pkg, repo, txn=txn)
update_progress()
pisi/db/installdb.py → pisi/installdb.py
+26 -16
View File
@@ -13,10 +13,6 @@
# installation database
#
# System
import os
import fcntl
import gettext
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
@@ -24,8 +20,7 @@ _ = __trans.ugettext
# PiSi
import pisi
import pisi.context as ctx
import pisi.oo
import pisi.db.lockeddbshelve as dbshelve
import pisi.lockeddbshelve as shelve
import pisi.files
import pisi.util
@@ -68,11 +63,10 @@ class InstallInfo:
class InstallDB:
__metaclass__ = pisi.oo.Singleton
def __init__(self):
self.d = dbshelve.LockedDBShelf('install')
self.dp = dbshelve.LockedDBShelf('configpending')
self.d = shelve.LockedDBShelf('install')
self.dp = shelve.LockedDBShelf('configpending')
self.files_dir = pisi.util.join_path(ctx.config.db_dir(), 'files')
def close(self):
@@ -112,20 +106,20 @@ class InstallDB:
def list_installed(self, txn = None):
def proc(txn):
list = []
l = []
for (pkg, info) in self.d.items(txn):
if info.state=='i' or info.state=='ip':
list.append(pkg)
return list
l.append(pkg)
return l
return self.d.txn_proc(proc, txn)
def list_pending(self):
# warning: reads the entire db
dict = {}
d = {}
for (pkg, x) in self.dp.items():
pkginfo = self.d[pkg]
dict[pkg] = pkginfo
return dict
d[pkg] = pkginfo
return d
def get_info(self, pkg):
pkg = str(pkg)
@@ -198,3 +192,19 @@ class InstallDB:
if self.d.has_key(pkg, txn):
self.d.delete(pkg, txn)
self.d.txn_proc(proc, txn)
db = None
def init():
global db
if db:
return db
db = InstallDB()
return db
def finalize():
global db
if db:
db.close()
db = None
pisi/db/itembyrepodb.py → pisi/itembyrepodb.py
+1 -6
View File
@@ -10,18 +10,13 @@
# Please read the COPYING file.
#
import os
import fcntl
import types
import gettext
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi
import pisi.util as util
import pisi.context as ctx
import pisi.db.lockeddbshelve as shelve
import pisi.lockeddbshelve as shelve
installed, thirdparty, repos, all = range(1, 5)
pisi/db/lockeddbshelve.py → pisi/lockeddbshelve.py
+9 -9
View File
@@ -16,7 +16,7 @@ import types
import bsddb3.db as db
import bsddb3.dbobj as dbobj
import pisi.db.dbshelve as dbshelve
import pisi.dbshelve as shelve
import gettext
__trans = gettext.translation('pisi', fallback=True)
@@ -24,7 +24,7 @@ _ = __trans.ugettext
import pisi
import pisi.context as ctx
import pisi.util as util
import pisi.util
import pisi.version
class Error(pisi.Error):
@@ -34,7 +34,7 @@ class Error(pisi.Error):
# if write is given it knows it has write access
# if force is given it updates the specified db version
def check_dbversion(versionfile, ver, write=False, update=False):
verfn = util.join_path(ctx.config.db_dir(), versionfile)
verfn = pisi.util.join_path(pisi.context.config.db_dir(), versionfile)
firsttime = False
if os.path.exists(verfn):
verfile = file(verfn, 'r')
@@ -43,11 +43,11 @@ def check_dbversion(versionfile, ver, write=False, update=False):
dbver = pisi.version.Version(ver)
if currver < dbver:
if not update:
raise Error(_('Database version %s is obsolete. Please run the rebuild-db command.') % versionfile)
raise Error(_('Database version for %s insufficient. Please run rebuild-db command.') % versionfile)
else:
pass # continue to update, then
elif currver > dbver:
raise Error(_('Database version %s is too new for your PiSi version. You need to upgrade PiSi.') % versionfile)
raise Error(_('Database version for %s greater than PiSi version. You need a newer PiSi.') % versionfile)
elif not update:
return True # db version is OK
else:
@@ -63,7 +63,7 @@ def check_dbversion(versionfile, ver, write=False, update=False):
raise Error(_('Database version %s not present.') % versionfile)
def lock_dbenv():
ctx.dbenv_lock = file(util.join_path(pisi.context.config.db_dir(), 'dbenv.lock'), 'w')
ctx.dbenv_lock = file(pisi.util.join_path(pisi.context.config.db_dir(), 'dbenv.lock'), 'w')
try:
fcntl.flock(ctx.dbenv_lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
except IOError:
@@ -95,15 +95,15 @@ def init_dbenv(write=False, writeversion=False):
else:
ctx.dbenv = None # read-only access to database
class LockedDBShelf(dbshelve.DBShelf):
class LockedDBShelf(shelve.DBShelf):
"""A simple wrapper to implement locking for bsddb's dbshelf"""
def __init__(self, dbname, mode=0644,
filetype=db.DB_BTREE, dbenv = None):
if dbenv == None:
dbenv = ctx.dbenv
dbshelve.DBShelf.__init__(self, dbenv)
filename = util.join_path(pisi.context.config.db_dir(), dbname + '.bdb')
shelve.DBShelf.__init__(self, dbenv)
filename = pisi.util.join_path(pisi.context.config.db_dir(), dbname + '.bdb')
if dbenv and os.access(os.path.dirname(filename), os.W_OK):
flags = 'w'
elif os.access(filename, os.R_OK):
pisi/metadata.py
+1
View File
@@ -110,6 +110,7 @@ class MetaData(xmlfile.XmlFile):
# FIXME: no need to copy full history with comments
self.package.history = history
self.package.conflicts = pkg.conflicts
self.package.replaces = pkg.replaces
self.package.providesComar = pkg.providesComar
#self.package.requiresComar = pkg.requiresComar
self.package.additionalFiles = pkg.additionalFiles
pisi/oo.py
-10
View File
@@ -37,13 +37,3 @@ class Struct:
def __init__(self, **entries):
self.__dict__.update(entries)
class Singleton(type):
def __init__(cls, name, bases, dic):
super(Singleton, cls).__init__(name, bases, dic)
cls.instance = None
def __call__(cls, *args, **kwargs):
if not cls.instance:
cls.instance = super(Singleton, cls).__call__(*args, **kwargs)
return cls.instance
pisi/operations.py
+61 -61
View File
@@ -24,9 +24,8 @@ import pisi.util as util
import pisi.dependency as dependency
import pisi.conflict
import pisi.pgraph as pgraph
import pisi.db.packagedb as packagedb
import pisi.db.repodb
import pisi.db.installdb
import pisi.repodb
import pisi.installdb
import pisi.cli
import pisi.atomicoperations as atomicoperations
import pisi.ui as ui
@@ -178,7 +177,7 @@ in the respective order to satisfy extra dependencies:
while len(B) > 0:
Bp = set()
for x in B:
pkg = pisi.api.get_repo_package(x)
pkg = packagedb.get_package(x)
for dep in pkg.runtimeDependencies():
if dependency.dict_satisfies_dep(d_t, dep):
if not dep.package in G_f.vertices():
@@ -208,53 +207,37 @@ in the respective order to satisfy extra dependencies:
if 'pisi' in order and pisi_installed:
upgrade_pisi()
def check_conflict(pkg, order):
conflicts = []
# FIXME: this should be done in atomicoperations automatically.
def remove_replaced_packages(order, replaces):
for conflict in pkg.conflicts:
if conflict.package not in order and pisi.conflict.installed_package_conflicts(conflict):
conflicts.append(conflict)
replaced = []
inorder = set(order).intersection(replaces.values())
return conflicts
if inorder:
for pkg in replaces.keys():
if replaces[pkg] in inorder:
replaced.append(pkg)
def calculate_conflicts(order, packagedb):
B_0 = set(order)
C = D = set()
pkg_conflicts = {}
for x in order:
pkg = pisi.api.get_repo_package(x)
# check if any package has conflicts with the installed packages
conflicts = check_conflict(pkg, order)
if conflicts:
pkg_conflicts[x] = map(lambda c:str(c), conflicts)
C = C.union(map(lambda c:c.package, conflicts))
# now check if any package has conflicts with each other
B_i = B_0.intersection(set(map(lambda c:c.package, pkg.conflicts)))
D_i = set()
for p in map(lambda x:pisi.api.get_repo_package(x), B_i):
conflicted = pisi.conflict.package_conflicts(p, pkg.conflicts)
if conflicted:
D_i.add(str(conflicted))
if D_i:
D = D.union(D_i)
D.add(pkg.name)
return (C, D, pkg_conflicts)
if replaced:
if remove(replaced, ignore_dep=True, ignore_safety=True):
raise Error(_("Replaced package remains"))
def remove_conflicting_packages(conflicts):
if remove(conflicts, ignore_dep=True, ignore_safety=True) == False:
if remove(conflicts, ignore_dep=True, ignore_safety=True):
raise Error(_("Conflicts remain"))
def remove_obsoleted_packages():
obsoletes = filter(ctx.installdb.is_installed, ctx.packagedb.get_obsoletes())
if obsoletes:
if remove(obsoletes, ignore_dep=True, ignore_safety=True):
raise Error(_("Obsoleted packages remaining"))
def check_conflicts(order, packagedb):
"""check if upgrading to the latest versions will cause havoc
done in a simple minded way without regard for dependencies of
conflicts, etc."""
(C, D, pkg_conflicts) = calculate_conflicts(order, packagedb)
(C, D, pkg_conflicts) = pisi.conflict.calculate_conflicts(order, packagedb)
if D:
raise Error(_("Selected packages [%s] are in conflict with each other.") %
@@ -278,11 +261,12 @@ def is_upgradable(name, ignore_build = False):
return False
(version, release, build) = ctx.installdb.get_version(name)
try:
pkg = pisi.api.get_repo_package(name)
pkg = ctx.packagedb.get_package(name)
except KeyboardInterrupt:
raise
except Exception, e: #FIXME: what exception could we catch here, replace with that.
return False
if ignore_build or (not build) or (not pkg.build):
return pisi.version.Version(release) < pisi.version.Version(pkg.release)
else:
@@ -303,7 +287,7 @@ def upgrade_base(A = set(), ignore_package_conflicts = False):
if extra_upgrades:
ctx.ui.warning(_('Safety switch: Following packages in system.base will be upgraded: ') +
util.strlist(extra_upgrades))
G_f, upgrade_order = plan_upgrade(extra_upgrades, ignore_build)
G_f, upgrade_order = plan_upgrade(extra_upgrades)
# return packages that must be added to any installation
return set(install_order + upgrade_order)
else:
@@ -384,7 +368,7 @@ def plan_install_pkg_names(A, ignore_package_conflicts = False):
while len(B) > 0:
Bp = set()
for x in B:
pkg = pisi.api.get_repo_package(x)
pkg = ctx.packagedb.get_package(x)
for dep in pkg.runtimeDependencies():
ctx.ui.debug('checking %s' % str(dep))
# we don't deal with already *satisfied* dependencies
@@ -413,6 +397,9 @@ def upgrade_pkg_names(A = []):
ignore_build = ctx.get_option('ignore_build_no')
security_only = ctx.get_option('security_only')
replaced = []
replaces = ctx.packagedb.get_replaces()
if not A:
# if A is empty, then upgrade all packages
A = ctx.installdb.list_installed()
@@ -424,19 +411,27 @@ def upgrade_pkg_names(A = []):
if x.endswith(ctx.const.package_suffix):
ctx.ui.debug(_("Warning: package *name* ends with '.pisi'"))
# Handling of replacement packages
if x in replaces.values():
Ap.append(x)
continue
if x in replaces.keys():
Ap.append(replaces[x])
continue
if not ctx.installdb.is_installed(x):
ctx.ui.info(_('Package %s is not installed.') % x, True)
continue
(version, release, build) = ctx.installdb.get_version(x)
if ctx.packagedb.has_package(x):
pkg = pisi.api.get_repo_package(x)
pkg = ctx.packagedb.get_package(x)
else:
ctx.ui.info(_('Package %s is not available in repositories.') % x, True)
continue
if security_only:
history = ctx.packageDB.get_history(x)
updates = [i for i in history if pisi.version.Version(i.release) > pisi.version.Version(release)]
updates = [i for i in pkg.history if pisi.version.Version(i.release) > pisi.version.Version(release)]
if not pisi.util.any(lambda i:i.type == 'security', updates):
continue
@@ -453,7 +448,7 @@ def upgrade_pkg_names(A = []):
ctx.ui.info(_('Package %s is already at the latest build %s.')
% (pkg.name, pkg.build), True)
A = set(Ap)
if len(A)==0:
@@ -465,7 +460,7 @@ def upgrade_pkg_names(A = []):
ctx.ui.debug('A = %s' % str(A))
if not ctx.config.get_option('ignore_dependency'):
G_f, order = plan_upgrade(A, ignore_build)
G_f, order = plan_upgrade(A)
else:
G_f = None
order = list(A)
@@ -487,7 +482,7 @@ def upgrade_pkg_names(A = []):
if ctx.get_option('dry_run'):
return
if set(order) - A_0:
if set(order) - A_0 - set(replaces.values()):
if not ctx.ui.confirm(_('There are extra packages due to dependencies. Do you want to continue?')):
return False
@@ -495,7 +490,7 @@ def upgrade_pkg_names(A = []):
paths = []
for x in order:
ctx.ui.info(_("Downloading %d / %d") % (order.index(x)+1, len(order)))
ctx.ui.info(util.colorize(_("Downloading %d / %d") % (order.index(x)+1, len(order)), "yellow"))
install_op = atomicoperations.Install.from_name(x)
paths.append(install_op.package_fname)
@@ -507,15 +502,20 @@ def upgrade_pkg_names(A = []):
if conflicts:
remove_conflicting_packages(conflicts)
if replaces:
remove_replaced_packages(order, replaces)
remove_obsoleted_packages()
for path in paths:
ctx.ui.info(_("Installing %d / %d") % (paths.index(path)+1, len(paths)))
ctx.ui.info(util.colorize(_("Installing %d / %d") % (paths.index(path)+1, len(paths)), "yellow"))
install_op = atomicoperations.Install(path, ignore_file_conflicts = True)
install_op.install(True)
if 'pisi' in order:
upgrade_pisi()
def plan_upgrade(A, ignore_build = False):
def plan_upgrade(A):
# try to construct a pisi graph of packages to
# install / reinstall
@@ -534,7 +534,7 @@ def plan_upgrade(A, ignore_build = False):
while len(B) > 0:
Bp = set()
for x in B:
pkg = pisi.api.get_repo_package(x)
pkg = packagedb.get_package(x)
for dep in pkg.runtimeDependencies():
# add packages that can be upgraded
if ctx.installdb.is_installed(dep.package) and dependency.installed_satisfies_dep(dep):
@@ -547,6 +547,7 @@ def plan_upgrade(A, ignore_build = False):
else:
ctx.ui.error(_('Dependency %s of %s cannot be satisfied') % (dep, x))
raise Error(_("Upgrade is not possible."))
B = Bp
# now, search reverse dependencies to see if anything
# should be upgraded
@@ -554,7 +555,7 @@ def plan_upgrade(A, ignore_build = False):
while len(B) > 0:
Bp = set()
for x in B:
pkg = pisi.api.get_repo_package(x)
pkg = packagedb.get_package(x)
rev_deps = packagedb.get_rev_deps(x)
for (rev_dep, depinfo) in rev_deps:
# add only installed but unsatisfied reverse dependencies
@@ -633,7 +634,7 @@ def plan_remove(A):
# try to construct a pisi graph of packages to
# install / reinstall
G_f = pgraph.PGraph(ctx.packagedb, pisi.db.itembyrepodb.installed) # construct G_f
G_f = pgraph.PGraph(ctx.packagedb, pisi.itembyrepodb.installed) # construct G_f
# find the (install closure) graph of G_f by package
# set A using packagedb
@@ -643,12 +644,11 @@ def plan_remove(A):
while len(B) > 0:
Bp = set()
for x in B:
pkg = pisi.api.get_installed_package(x)
rev_deps = ctx.packagedb.get_rev_deps(x, pisi.db.itembyrepodb.installed)
rev_deps = ctx.packagedb.get_rev_deps(x, pisi.itembyrepodb.installed)
for (rev_dep, depinfo) in rev_deps:
# we don't deal with uninstalled rev deps
# and unsatisfied dependencies (this is important, too)
if ctx.packagedb.has_package(rev_dep, pisi.db.itembyrepodb.installed) and \
if ctx.packagedb.has_package(rev_dep, pisi.itembyrepodb.installed) and \
dependency.installed_satisfies_dep(depinfo):
if not rev_dep in G_f.vertices():
Bp.add(rev_dep)
@@ -668,7 +668,7 @@ def expand_src_components(A):
Ap.add(x)
return Ap
def emerge(A, rebuild_all = False):
def emerge(A):
# A was a list, remove duplicates and expand components
A = [str(x) for x in A]
@@ -684,7 +684,7 @@ def emerge(A, rebuild_all = False):
# FIXME: Errr... order_build changes type conditionally and this
# is not good. - baris
if not ctx.config.get_option('ignore_dependency'):
G_f, order_inst, order_build = plan_emerge(A, rebuild_all)
G_f, order_inst, order_build = plan_emerge(A)
else:
G_f = None
order_inst = []
@@ -725,7 +725,7 @@ installed in the respective order to satisfy dependencies:
if 'pisi' in order_build or (('pisi' in U) and pisi_installed):
upgrade_pisi()
def plan_emerge(A, rebuild_all):
def plan_emerge(A):
# try to construct a pisi graph of packages to
# install / reinstall
@@ -793,7 +793,7 @@ def plan_emerge(A, rebuild_all):
def calculate_download_sizes(order):
total_size = cached_size = 0
for pkg in [pisi.api.get_repo_package(name) for name in order]:
for pkg in [ctx.packagedb.get_package(name) for name in order]:
delta = None
if ctx.installdb.is_installed(pkg.name):
pisi/package.py
+1 -1
View File
@@ -54,7 +54,7 @@ class Package:
# Bug 3465
if ctx.get_option('reinstall'):
raise Error(_("There was a problem while fetching '%s'.\nThe package "
"may have been upgraded. Please try again to upgrade the package.") % url);
"may have been upgraded. Please try to upgrade the package.") % url);
raise
else:
ctx.ui.info(_('%s [cached]') % url.filename())
pisi/db/packagedb.py → pisi/packagedb.py
+63 -33
View File
@@ -17,20 +17,13 @@ we basically store everything in PackageInfo class
yes, we are cheap
"""
import os
import fcntl
import types
import gettext
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi
import pisi.util as util
import pisi.context as ctx
import pisi.oo
import pisi.db.itembyrepodb as itembyrepodb
import pisi.itembyrepodb
class Error(pisi.Error):
pass
@@ -43,27 +36,24 @@ class NotfoundError(pisi.Error):
class PackageDB(object):
"""PackageDB class provides an interface to the package database
using shelf objects"""
__metaclass__ = pisi.oo.Singleton
def __init__(self):
self.d = itembyrepodb.ItemByRepoDB('package')
self.dbHistory = itembyrepodb.ItemByRepoDB('package_history')
self.dr = itembyrepodb.ItemByRepoDB('revdep')
self.d = pisi.itembyrepodb.ItemByRepoDB('package')
self.dr = pisi.itembyrepodb.ItemByRepoDB('revdep')
self.do = pisi.itembyrepodb.ItemByRepoDB('obsoleted')
self.drp = pisi.itembyrepodb.ItemByRepoDB('replaces')
def close(self):
self.d.close()
self.dbHistory.close()
self.dr.close()
self.do.close()
self.drp.close()
def destroy(self):
self.d.destroy()
self.dbHistory.destroy()
self.dr.destroy()
def clear(self, txn = None):
self.d.clear()
self.dbHistory.clear()
self.dr.clear()
self.do.destroy()
self.drp.destroy()
def has_package(self, name, repo=None, txn = None):
return self.d.has_key(name, repo, txn=txn)
@@ -71,21 +61,34 @@ class PackageDB(object):
def get_package(self, name, repo=None, txn = None):
try:
return self.d.get_item(name, repo, txn=txn)
except pisi.db.itembyrepodb.NotfoundError, e:
except pisi.itembyrepodb.NotfoundError:
raise Error(_('Package %s not found') % name)
def get_history(self, name, repo=None, txn = None):
try:
return self.dbHistory.get_item(name, repo, txn=txn)
except pisi.db.itembyrepodb.NotfoundError, e:
raise Error(_('History for Package %s not found') % name)
def get_package_repo(self, name, repo=None, txn = None):
return self.d.get_item_repo(name, repo, txn=txn)
def which_repo(self, name, txn = None):
return self.d.which_repo(name, txn=txn)
def get_obsoletes(self, repo=None):
obsoletes = []
for r in self.do.list(repo):
obsoletes.extend(self.do.get_item(r, repo))
replaces = self.get_replaces(repo)
return set(str(o) for o in obsoletes) - set(replaces.keys())
# replacesdb holds the info about the replaced packages (ex. gaim -> pidgin)
def get_replaces(self, repo = None):
pairs = {}
for pkg_name in self.drp.list(repo):
replaces = self.drp.get_item(pkg_name, repo)
for r in replaces:
if pisi.replace.installed_package_replaced(r):
pairs[r.package] = pkg_name
return pairs
def get_rev_deps(self, name, repo = None, txn = None):
if self.dr.has_key(name, repo, txn=txn):
return self.dr.get_item(name, repo, txn=txn)
@@ -102,12 +105,15 @@ class PackageDB(object):
def list_packages(self, repo=None):
return self.d.list(repo)
def add_obsoletes(self, obsoletes, repo, txn = None):
def proc(txn):
self.do.add_item(repo, obsoletes, repo, txn)
ctx.txn_proc(proc, txn)
def add_package(self, package_info, repo, txn = None):
name = str(package_info.name)
def proc(txn):
self.dbHistory.add_item(name, package_info.history, repo, txn)
package_info.history = None
self.d.add_item(name, package_info, repo, txn)
for dep in package_info.runtimeDependencies():
dep_name = str(dep.package)
@@ -118,16 +124,26 @@ class PackageDB(object):
self.dr.add_item(dep_name, revdep, repo, txn)
else:
self.dr.add_item(dep_name, [ (name, dep) ], repo, txn)
if package_info.replaces:
self.drp.add_item(name, package_info.replaces, repo, txn)
# add component
ctx.componentdb.add_package(package_info.partOf, package_info.name, repo, txn)
ctx.txn_proc(proc, txn)
def clear(self, txn = None):
self.d.clear()
self.dr.clear()
self.do.clear()
self.drp.clear()
def remove_package(self, name, repo = None, txn = None):
name = str(name)
def proc(txn):
package_info = self.d.get_item(name, repo, txn=txn)
self.d.remove_item(name, repo, txn=txn)
self.dbHistory.remove_item(name, repo, txn=txn)
for dep in package_info.runtimeDependencies():
dep_name = str(dep.package)
if self.dr.has_key(dep_name, repo, txn):
@@ -152,11 +168,25 @@ class PackageDB(object):
def proc(txn):
self.d.remove_repo(repo, txn=txn)
self.dr.remove_repo(repo, txn=txn)
self.do.remove_repo(repo, txn=txn)
self.drp.remove_repo(repo, txn=txn)
self.d.txn_proc(proc, txn)
pkgdb = None
def remove_tracking_package(name, txn = None):
# remove the guy from the tracking databases
if pkgdb.has_package(name, itembyrepodb.installed, txn=txn):
pkgdb.remove_package(name, itembyrepodb.installed, txn=txn)
if pkgdb.has_package(name, itembyrepodb.thirdparty, txn=txn):
pkgdb.remove_package(name, itembyrepodb.thirdparty, txn=txn)
if pkgdb.has_package(name, pisi.itembyrepodb.installed, txn=txn):
pkgdb.remove_package(name, pisi.itembyrepodb.installed, txn=txn)
if pkgdb.has_package(name, pisi.itembyrepodb.thirdparty, txn=txn):
pkgdb.remove_package(name, pisi.itembyrepodb.thirdparty, txn=txn)
def init_db():
global pkgdb
pkgdb = PackageDB()
return pkgdb
def finalize_db():
global pkgdb
if pkgdb:
pkgdb.close()
pisi/pgraph.py
+3 -2
View File
@@ -14,14 +14,14 @@
import string
import pisi.db
import pisi
import graph
# Cache the results from packagedb queries in a graph
class PGraph(graph.Digraph):
def __init__(self, packagedb, repo = pisi.db.itembyrepodb.repos):
def __init__(self, packagedb, repo = pisi.itembyrepodb.repos):
super(PGraph, self).__init__()
self.packagedb = packagedb
self.repo = repo
@@ -76,6 +76,7 @@ class PGraph(graph.Digraph):
pkg2 = self.packagedb.get_package(conflinfo.package, self.repo)
pkg2data = (pkg2.version, pkg2.release)
# FIXME: WTF? /usr/lib/pardus/pisi/pgraph.py:80: Invalid arguments to (add_biedge), got 5, expected between 2 and 3 / caglar
self.add_biedge(str(pkg), str(conflinfo.package), ('c', conflinfo)
, pkg1data, pkg2data )
pisi/relation.py
+59
View File
@@ -0,0 +1,59 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2007, TUBITAK/UEKAE
#
# This program is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 2 of the License, or (at your option)
# any later version.
#
# Please read the COPYING file.
#
import gettext
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi.context as ctx
import pisi.version
import pisi.pxml.autoxml as autoxml
import pisi.itembyrepodb
class Relation:
__metaclass__ = autoxml.autoxml
s_Package = [autoxml.String, autoxml.mandatory]
a_version = [autoxml.String, autoxml.optional]
a_versionFrom = [autoxml.String, autoxml.optional]
a_versionTo = [autoxml.String, autoxml.optional]
a_release = [autoxml.String, autoxml.optional]
a_releaseFrom = [autoxml.String, autoxml.optional]
a_releaseTo = [autoxml.String, autoxml.optional]
def satisfies_relation(self, pkg_name, version, release):
ret = True
v = pisi.version.Version(version)
if self.version:
ret &= v == pisi.version.Version(self.version)
if self.versionFrom:
ret &= v >= pisi.version.Version(self.versionFrom)
if self.versionTo:
ret &= v <= pisi.version.Version(self.versionTo)
r = pisi.version.Version(release)
if self.release:
ret &= r == pisi.version.Version(self.release)
if self.releaseFrom:
ret &= r >= pisi.version.Version(self.releaseFrom)
if self.releaseTo:
ret &= r <= pisi.version.Version(self.releaseTo)
return ret
def installed_package_satisfies(relation):
pkg_name = relation.package
if not ctx.installdb.is_installed(pkg_name):
return False
else:
pkg = ctx.packagedb.get_package(pkg_name, pisi.itembyrepodb.installed)
(version, release) = (pkg.version, pkg.release)
return relation.satisfies_relation(pkg_name, version, release)
pisi/replace.py
+41
View File
@@ -0,0 +1,41 @@
# -*- coding: utf-8 -*-
#
# Copyright (C) 2005 - 2007, TUBITAK/UEKAE
#
# This program is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 2 of the License, or (at your option)
# any later version.
#
# Please read the COPYING file.
#
import gettext
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi.relation
""" Replace relation """
class Replace(pisi.relation.Relation):
def __str__(self):
s = self.package
if self.versionFrom:
s += _(" version >= ") + self.versionFrom
if self.versionTo:
s += _(" version <= ") + self.versionTo
if self.version:
s += _(" version ") + self.version
if self.releaseFrom:
s += _(" release >= ") + self.releaseFrom
if self.releaseTo:
s += _(" release <= ") + self.releaseTo
if self.release:
s += _(" release ") + self.release
return s
def installed_package_replaced(repinfo):
"""determine if an installed package in *repository* replaced with
given a package"""
return pisi.relation.installed_package_satisfies(repinfo)
pisi/db/repodb.py → pisi/repodb.py
+24 -9
View File
@@ -15,10 +15,8 @@ __trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi
import pisi.db.lockeddbshelve as shelve
import pisi.lockeddbshelve as shelve
import pisi.context as ctx
import pisi.db.packagedb as packagedb
import pisi.oo
class Error(pisi.Error):
pass
@@ -36,8 +34,7 @@ class Repo:
class RepoDB(object):
"""RepoDB maps repo ids to repository information"""
__metaclass__ = pisi.oo.Singleton
def __init__(self, txn = None):
self.d = shelve.LockedDBShelf("repo")
def proc(txn):
@@ -100,10 +97,28 @@ class RepoDB(object):
name = str(name)
def proc(txn):
self.d.delete("repo-" + name, txn)
list = self.d.get("order", txn)
list.remove(name)
self.d.put("order", list, txn)
l = self.d.get("order", txn)
l.remove(name)
self.d.put("order", l, txn)
ctx.packagedb.remove_repo(name, txn=txn)
ctx.sourcedb.remove_repo(name, txn=txn)
ctx.componentdb.remove_repo(name, txn=txn)
self.d.txn_proc(proc, txn)
self.d.txn_proc(proc, txn)
db = None
def init():
global db
if db:
return db
db = RepoDB()
return db
def finalize():
global db
if db:
db.close()
db = None
pisi/db/sourcedb.py → pisi/sourcedb.py
+23 -8
View File
@@ -25,19 +25,17 @@ __trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
import pisi.context as ctx
import pisi.db.repodb
import pisi.db.itembyrepodb as itembyrepodb
import pisi.oo
import pisi.repodb
import pisi.itembyrepodb
class NotfoundError(pisi.Error):
pass
class SourceDB(object):
__metaclass__ = pisi.oo.Singleton
def __init__(self):
self.d = itembyrepodb.ItemByRepoDB('source')
self.dpkgtosrc = itembyrepodb.ItemByRepoDB('pkgtosrc')
self.d = pisi.itembyrepodb.ItemByRepoDB('source')
self.dpkgtosrc = pisi.itembyrepodb.ItemByRepoDB('pkgtosrc')
def close(self):
self.d.close()
@@ -52,13 +50,13 @@ class SourceDB(object):
def get_spec(self, name, repo=None, txn = None):
try:
return self.d.get_item(name, repo, txn)
except pisi.db.itembyrepodb.NotfoundError, e:
except pisi.itembyrepodb.NotfoundError:
raise NotfoundError(_("Source package %s not found") % name)
def get_spec_repo(self, name, repo=None, txn = None):
try:
return self.d.get_item_repo(name, repo, txn)
except pisi.db.itembyrepodb.NotfoundError, e:
except pisi.itembyrepodb.NotfoundError:
raise NotfoundError(_("Source package %s not found") % name)
def pkgtosrc(self, name, txn = None):
@@ -83,6 +81,7 @@ class SourceDB(object):
for pkg in spec.packages:
self.dpkgtosrc.remove_item_repo(pkg.name, repo, txn)
ctx.componentdb.remove_spec(spec.source.partOf, spec.source.name, repo, txn)
self.d.txn_proc(proc, txn)
def remove_repo(self, repo, txn = None):
@@ -90,3 +89,19 @@ class SourceDB(object):
self.d.remove_repo(repo, txn=txn)
self.dpkgtosrc.remove_repo(repo, txn=txn)
self.d.txn_proc(proc, txn)
sourcedb = None
def init():
global sourcedb
if sourcedb:
return sourcedb
sourcedb = SourceDB()
return sourcedb
def finalize():
global sourcedb
if sourcedb:
sourcedb.close()
sourcedb = None
pisi/specfile.py
+22 -2
View File
@@ -22,12 +22,14 @@ _ = __trans.ugettext
# standard python modules
import os.path
import piksemel
# pisi modules
import pisi.pxml.xmlfile as xmlfile
import pisi.pxml.autoxml as autoxml
import pisi.context as ctx
import pisi.dependency
import pisi.replace
import pisi.conflict
import pisi.component as component
import pisi.util as util
@@ -170,8 +172,8 @@ class Package:
t_ComponentDependencies = [ [autoxml.String], autoxml.optional, "RuntimeDependencies/Component"]
t_Files = [ [Path], autoxml.optional]
t_Conflicts = [ [pisi.conflict.Conflict], autoxml.optional, "Conflicts/Package"]
t_Replaces = [ [pisi.replace.Replace], autoxml.optional, "Replaces/Package"]
t_ProvidesComar = [ [ComarProvide], autoxml.optional, "Provides/COMAR"]
#t_RequiresComar = [ [autoxml.String], autoxml.mandatory, "Requires/COMAR"]
t_AdditionalFiles = [ [AdditionalFile], autoxml.optional]
t_History = [ [Update], autoxml.optional]
@@ -181,7 +183,7 @@ class Package:
def runtimeDependencies(self):
deps = self.packageDependencies
deps += [ ctx.componentdb.get_component(x).packages for x in self.componentDependencies ]
deps += [ ctx.componentdb.get_component[x].packages for x in self.componentDependencies ]
return deps
def pkg_dir(self):
@@ -239,3 +241,21 @@ class SpecFile(xmlfile.XmlFile):
#http://liste.pardus.org.tr/gelistirici/2006-September/002332.html
self.source.description = autoxml.LocalText("Description")
self.source.description["en"] = self.source.summary["en"]
def _set_i18n(self, tag, inst):
for summary in tag.tags("Summary"):
inst.summary[summary.getAttribute("xml:lang")] = summary.firstChild().data()
for desc in tag.tags("Description"):
inst.description[desc.getAttribute("xml:lang")] = desc.firstChild().data()
def read_translations(self, path):
if not os.path.exists(path):
return
doc = piksemel.parse(path)
self._set_i18n(doc.getTag("Source"), self.source)
for pak in doc.tags("Package"):
for inst in self.packages:
if inst.name == pak.getTagData("Name"):
break
self._set_i18n(pak, inst)
pisi/ui.py
-5
View File
@@ -12,11 +12,6 @@
# generic user interface
#
import sys
import pisi
import pisi.context as ctx
(installed, upgraded, removed, installing, removing, configuring, configured, extracting,
downloading, packagestogo, updatingrepo, cached) = range(12)
pisi/util.py
+14 -22
View File
@@ -57,6 +57,9 @@ def every(pred, seq):
def any(pred, seq):
return reduce(operator.or_, map(pred, seq), False)
def unzip(seq):
return zip(*seq)
def concat(l):
"""Concatenate a list of lists."""
return reduce( operator.concat, l )
@@ -262,12 +265,12 @@ def check_file(file, mode = os.F_OK):
return True
# FIXME: check_dir is not a good name considering it can also create the dir
def check_dir(dir):
def check_dir(d):
"""Make sure given directory path exists."""
# FIXME: What is first strip doing there?
dir = dir.strip().rstrip("/")
if not os.access(dir, os.F_OK):
os.makedirs(dir)
d = d.strip().rstrip("/")
if not os.access(d, os.F_OK):
os.makedirs(d)
def clean_dir(path):
"""Remove all content of a directory."""
@@ -294,7 +297,7 @@ def dir_size(dir):
return os.path.getsize(dir)
if os.path.islink(dir):
return 0
return long(len(os.readlink(dir)))
def sizes():
for root, dirs, files in os.walk(dir):
@@ -390,8 +393,8 @@ def get_file_hashes(top, excludePrefix=None, removePrefix=None):
continue
#bug 397
for dir in dirs:
d = join_path(root, dir)
for directory in dirs:
d = join_path(root, directory)
if os.path.islink(d) and not has_excluded_prefix(d):
yield (d, sha1_sum(os.readlink(d), True))
excludePrefix.append(remove_prefix(removePrefix, d) + "/")
@@ -452,7 +455,7 @@ def sha1_data(data):
return m.hexdigest()
except KeyboardInterrupt:
raise
except Exception, e: #FIXME: what exception could we catch here, replace with that.
except Exception: #FIXME: what exception could we catch here, replace with that.
raise Error(_("Cannot calculate SHA1 hash of given data"))
def uncompress(patchFile, compressType="gz", targetDir=None):
@@ -497,8 +500,8 @@ def do_patch(sourceDir, patchFile, level = 0):
(ret, out, err) = run_batch('quilt push')
else:
# run GNU patch to apply original patch into tree
(ret, out, err) = run_batch("patch --remove-empty-files -p%d < \"%s\"" %
(level, patchFile))
(ret, out, err) = run_batch("patch --remove-empty-files -p%d < \"%s\"" % (level, patchFile))
if ret:
if out is None and err is None:
# Which means stderr and stdout directed so they are None
@@ -614,24 +617,13 @@ def is_package_name(fn, package_name = None):
# get version string, skip separator '-'
verstr = fn[len(package_name) + 1:
len(fn)-len(ctx.const.package_suffix)]
import string
for x in verstr.split('-'):
# weak rule: version components after '-' start with a digit
if x is '' or (not x[0] in string.digits):
if x == '' or (not x[0] in string.digits):
return False
return True
return False
def env_update():
import pisi.environment
ctx.ui.info(_('Updating environment...'))
env_dir = join_path(ctx.config.dest_dir(), "/etc/env.d")
if not os.path.exists(env_dir):
os.makedirs(env_dir, 0755)
pisi.environment.update_environment(ctx.config.dest_dir())
def parse_package_name(package_name):
"""Separate package name and version string.
po/de.po
+589 -476
View File
File diff suppressed because it is too large Load Diff
po/es.po
+696 -526
View File
File diff suppressed because it is too large Load Diff
po/fr.po
+757 -593
View File
File diff suppressed because it is too large Load Diff
po/it.po
+754 -598
View File
File diff suppressed because it is too large Load Diff
po/nl.po
+593 -478
View File
File diff suppressed because it is too large Load Diff
po/pisi.pot
+531 -474
View File
File diff suppressed because it is too large Load Diff
po/pt_BR.po
-2710
View File
File diff suppressed because it is too large Load Diff
po/tr.po
+529 -467
View File
File diff suppressed because it is too large Load Diff
scripts/buildno
+8 -1
View File
@@ -1,2 +1,9 @@
#!/bin/bash
pisi li -i | grep None
b=/var/lib/pisi/package
for d in `ls $b`;do
if ! grep "<Build>" $b/$d/metadata.xml &>/dev/null ; then
echo "$d"
fi
done
scripts/cleanCache.py
+1 -1
View File
@@ -60,7 +60,7 @@ def cleanPisis(clean, root = '/var/cache/pisi/packages'):
l = findUnneededFiles(list)
doit(root, l, clean, ".pisi")
def cleanBuilds(clean, root = '/var/pisi'):
def cleanBuilds(clean, root = '/var/tmp/pisi'):
# Build remnant
list = []
for f in os.listdir(root):
scripts/convert_history.py
Executable
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/python
import sys
import os
import codecs
import xml.dom.minidom as mdom
def find_pspecs(folder):
paks = []
for root, dirs, files in os.walk(folder):
if "pspec.xml" in files:
paks.append(root)
# dont walk into the versioned stuff
if ".svn" in dirs:
dirs.remove(".svn")
return paks
def addText(doc, parent, text):
cdata =doc.createTextNode(text)
parent.appendChild(cdata)
def getTags(parent, childName):
return [x for x in parent.childNodes if x.nodeType == x.ELEMENT_NODE if x.tagName == childName]
def getNodeText(node, tag, default=None):
try:
c = getTags(node, tag)[0].firstChild.data
except:
c = default
return c
def newNode(doc, tag, text):
node = doc.createElement(tag)
cdata = doc.createTextNode(text)
node.appendChild(cdata)
return node
def fixIndent(doc, node):
for x in node.childNodes:
if x.nodeType == x.ELEMENT_NODE:
if x.tagName == "Update":
fixIndent(doc, x)
else:
x.data = "\n" + x.data[5:]
def fixTags(doc, hist):
for update in hist.childNodes:
if update.nodeType == update.ELEMENT_NODE:
rno = getNodeText(update, "Release")
update.setAttribute("release", rno)
if rno == "1":
comment = newNode(doc, "Comment", "First release.")
paker = getTags(getTags(doc.documentElement, "Source")[0], "Packager")[0]
name = newNode(doc, "Name", getNodeText(paker, "Name"))
email = newNode(doc, "Email", getNodeText(paker, "Email"))
else:
comment = newNode(doc, "Comment", "FIXHISTORY")
name = newNode(doc, "Name", "FIXHISTORY")
email = newNode(doc, "Email", "FIXHISTORY")
update.replaceChild(comment, getTags(update, "Release")[0])
addText(doc, update, " ")
update.appendChild(name)
addText(doc, update, "\n ")
update.appendChild(email)
addText(doc, update, "\n ")
def fixPspec(path):
doc = mdom.parse(path)
pisi = doc.documentElement
source = getTags(pisi, "Source")[0]
history = getTags(source, "History")[0]
item = source.removeChild(history)
addText(doc, pisi, "\n ")
fixIndent(doc, item)
fixTags(doc, item)
pisi.appendChild(item)
addText(doc, pisi, "\n")
f = codecs.open(path,'w', "utf-8")
f.write(doc.toxml())
f.close()
pakages = find_pspecs(sys.argv[1])
for pak in pakages:
fixPspec(os.path.join(pak, "pspec.xml"))
scripts/newpisipackage
+11 -16
View File
@@ -14,8 +14,7 @@ import string
packagerfile = ".pisipackager"
data = {"date": time.strftime("%Y-%m-%d"),
"year": time.strftime("%Y")}
data = {"date": time.strftime("%Y-%m-%d")}
temp_pspec = '''<?xml version="1.0" ?>
<!DOCTYPE PISI SYSTEM "http://www.pardus.org.tr/projeler/pisi/pisi-spec.dtd">
@@ -28,20 +27,21 @@ temp_pspec = '''<?xml version="1.0" ?>
<Email>%(packager_email)s</Email>
</Packager>
<License>GPL-2</License>
<Icon>%(package)s</Icon>
<IsA>app:gui</IsA>
<Summary></Summary>
<Description></Description>
<Summary xml:lang="tr"></Summary>
<Description xml:lang="tr"></Description>
<Archive sha1sum="" type="targz"></Archive>
<BuildDependencies>
<Dependency versionFrom=""></Dependency>
<Dependency></Dependency>
</BuildDependencies>
<!--
<Patches>
<!--
<Patch level="1"></Patch>
</Patches>
-->
</Patches>
</Source>
<Package>
@@ -51,17 +51,15 @@ temp_pspec = '''<?xml version="1.0" ?>
<Files>
<Path fileType="data">/</Path>
</Files>
<!--
<AdditionalFiles>
<AdditionalFile owner="root" permission="0644" target="/usr/share/applications/%(package)s.desktop">%(package)s.desktop</AdditionalFile>
</AdditionalFiles>
-->
<!--
<AdditionalFile owner="root" permission="0644" target="/usr/share/applications/%(package)s.desktop">%(package)s.desktop</AdditionalFile>
-->
</AdditionalFiles>
<Provides>
<COMAR script="package.py">System.Package</COMAR>
<COMAR script="service.py">System.Service</COMAR>
</Provides>
-->
</Package>
<History>
@@ -79,7 +77,7 @@ temp_pspec = '''<?xml version="1.0" ?>
temp_actions = '''#!/usr/bin/python
# -*- coding: utf-8 -*-
#
# Copyright %(year)s TUBITAK/UEKAE
# Copyright 2006 TUBITAK/UEKAE
# Licensed under the GNU General Public License, version 2.
# See the file http://www.gnu.org/copyleft/gpl.txt.
@@ -119,13 +117,10 @@ Categories=Application;Game;ArcadeGame
'''
temp_service = '''serviceType = "local"
from comar.service import *
serviceType = "local"
serviceDesc = "%(package_cap)s"
serviceDefault = "off"
serviceDesc = _({"en": "%(package_cap)s",
"tr": "%(package_cap)s"})
from comar.service import *
def start():
if 0 == run('/sbin/start-stop-daemon --quiet --start --exec /usr/bin/%(package)s --pidfile /var/run/%(package)s.pid'):
scripts/repo-diff.py
Executable
+92
View File
@@ -0,0 +1,92 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
#
# Copyright (C) 2005, TUBITAK/UEKAE
#
# This program is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 2 of the License, or (at your option)
# any later version.
#
# Please read the COPYING file.
import os
import sys
from pisi.specfile import SpecFile
from pisi.util import join_path
def findPspec(folder):
pspecList = []
for root, dirs, files in os.walk(folder):
if "pspec.xml" in files:
pspecList.append(root)
# dont walk into the versioned stuff
if ".svn" in dirs:
dirs.remove(".svn")
return pspecList
def getVersion(pspecList):
sources = {}
for pspec in pspecList:
specFile = SpecFile(join_path(pspec, "pspec.xml"))
sources[specFile.source.name] = (specFile.source.version, specFile.source.release)
return sources
def listIntersection(firstRepo, secondRepo):
keys = list(set(firstRepo.keys()).__and__(set(secondRepo.keys())))
keys.sort()
for i in keys:
if firstRepo[i] != secondRepo[i]:
print " %s: %s (r%s) -> %s (r%s)" % (i, firstRepo[i][0], firstRepo[i][1], secondRepo[i][0], secondRepo[i][1])
def listComplement(firstRepo, secondRepo):
keys = list(set(firstRepo.keys()) - set(secondRepo.keys()))
keys.sort()
for i in keys:
print " %s" % i
def usage(miniMe):
print """Usage:
%s pathToSvn component (ex: %s /home/caglar/svn/pardus/ system/devel)
""" % (miniMe, miniMe)
sys.exit(1)
if __name__ == "__main__":
try:
svnRoot = sys.argv[1]
except IndexError:
usage(sys.argv[0])
try:
postfix = sys.argv[2]
except IndexError:
postfix = ""
tag = getVersion(findPspec(join_path(svnRoot, "tags/pardus-1.0/", postfix)))
stable = getVersion(findPspec(join_path(svnRoot,"stable/pardus-1/", postfix)))
devel = getVersion(findPspec(join_path(svnRoot, "devel/", postfix)))
print "Tag --> Stable"
listIntersection(tag, stable)
print
print "Tag has, Stable hasn't"
listComplement(tag, stable)
print
print "Stable has, Tag hasn't"
listComplement(stable, tag)
print
print "Stable --> Devel"
listIntersection(stable, devel)
print
print "Stable has, Devel hasn't"
listComplement(stable, devel)
print
print "Devel has, Stable hasn't"
listComplement(devel, stable)
print
scripts/repostats.py
Executable
+598
View File
@@ -0,0 +1,598 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
#
# Copyright (C) 2005-2007, TUBITAK/UEKAE
#
# This program is free software; you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 2 of the License, or (at your option)
# any later version.
#
import sys
import os
import codecs
import re
import getopt
import gettext
__trans = gettext.translation('pisi', fallback=True)
_ = __trans.ugettext
from svn import core, client
sys.path.append('.')
import pisi.specfile
import pisi.uri
import pisi.package
import pisi.metadata
import pisi.files
from pisi.cli import printu
# Main HTML template
html_header = """
<html><head>
<title>%(title)s</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link href="%(root)sstil.css" rel="stylesheet" type="text/css">
</head><body>
<div id='header-bugzilla'>
</div>
<div class='menu'>
<a href='%(root)sindex.html'>Genel Bilgiler</a>
| <a href='%(root)ssources.html'>Kaynak Paketler</a>
| <a href='%(root)sbinaries.html'>İkili Paketler</a>
| <a href='%(root)spackagers.html'>Paketçiler</a>
</div>
<h1 align='center'>%(title)s</h1>
<div class='content'>
%(content)s
</div>
</body></html>
"""
css_template = """
body {
margin-left:0;
margin-top:0;
margin-right:0;
background-image:url('http://www.pardus.org.tr/styles/images/HeadTile.png');
background-repeat:repeat-x;
background-color: #FFF;
}
#header-bugzilla {
background-image:url('http://www.pardus.org.tr/styles/images/HeadLogo.png');
background-repeat:no-repeat;
background-position: 0px 0px;
height:119px;
padding-bottom:5px;
}
a {
color: #F55400;
text-decoration: none;
}
a:hover {
color: #444;
background-color:#EEE;
}
.menu {
padding-left: 1em;
padding-top: 3px;
padding-bottom: 3px;
border-bottom: 1px solid #CCC;
border-top: 1px solid #CCC;
}
.content {
margin: 0.5em;
}
"""
# default html templates (now obsolete)
def_repo_sizes_html = u"""
<h3>Boyutlar</h3>
<p>Toplam kurulu boyut %(total)s</p>
<p>Dosya tiplerine göre liste:</p>
<table><tbody>
%(sizes)s
</table></tbody>
"""
def svn_uri(path):
# init
core.apr_initialize()
pool = core.svn_pool_create(None)
core.svn_config_ensure(None, pool)
# get commit date
uri = client.svn_client_url_from_path(path, pool)
# cleanup
core.svn_pool_destroy(pool)
core.apr_terminate()
return uri
def find_pspecs(folder):
paks = []
for root, dirs, files in os.walk(folder):
if "pspec.xml" in files:
paks.append(root)
# dont walk into the versioned stuff
if ".svn" in dirs:
dirs.remove(".svn")
return paks
def write_html(filename, title, content):
f = codecs.open(filename, "w", "utf-8")
root = "./"
if len(filename.split("/")) > 2:
root = "../"
dict = {
"title": title,
"content": content,
"root": root,
}
f.write(html_header % dict)
f.close()
def make_table(elements, titles=None):
def make_row(element):
return "<td>%s" % "<td>".join(map(str, element))
title_html = ""
if titles:
title_html = """
<thead><tr><th>%s</tr></thead>
""" % "<th>".join(titles)
html = """
<table>%s<tbody>
<tr>%s
</tbody></table>
""" % (title_html, "<tr>".join(map(make_row, elements)))
return html
def make_url(name, path="./"):
if not path.endswith("/"):
path += "/"
return "<a href='%s%s.html'>%s</a>" % (path, name, name)
def mangle_email(email):
return re.sub("@", " [at] ", email)
class Histogram:
def __init__(self):
self.list = {}
def add(self, name, value=None):
if value:
self.list[name] = value
else:
self.list[name] = self.list.get(name, 0) + 1
def note(self, name):
if not self.list.has_key(name):
self.list[name] = 0
def get_list(self, max=0):
items = self.list.items()
items.sort(key=lambda x: x[1], reverse=True)
if max != 0:
return items[:max]
else:
return items
# Dictionary of all source packages keyed by the source name
sources = {}
# Dictionary of all binary packages keyed by the package name
packages = {}
# Dictionary of all packagers keyed by the packager name
packagers = {}
# Dictionary of missing depended binary packages keyed by the package name
missing = {}
# List of all repository problems
errors = []
class Missing:
def __init__(self, name):
missing[name] = self
self.name = name
self.revBuildDeps = []
self.revRuntimeDeps = []
class Package:
def __init__(self, source, pakspec):
name = pakspec.name
if packages.has_key(name):
errors.append(_("Duplicate binary packages:\n%s\n%s\n") % (
source.name, packages[name].source.name))
return
packages[name] = self
self.name = name
self.source = source
self.pakspec = pakspec
self.revBuildDeps = []
self.revRuntimeDeps = []
self.installedSize = 0
def markDeps(self):
# mark reverse build dependencies
for d in self.source.spec.source.buildDependencies:
p = d.package
if packages.has_key(p):
packages[p].revBuildDeps.append(self.name)
else:
if not missing.has_key(p):
Missing(p)
missing[p].revBuildDeps.append(self.name)
# mark reverse runtime dependencies
for d in self.pakspec.packageDependencies:
p = d.package
if packages.has_key(p):
packages[p].revRuntimeDeps.append(self.name)
else:
if not missing.has_key(p):
Missing(p)
missing[p].revRuntimeDeps.append(self.name)
def report_html(self):
source = self.source.spec.source
bDeps = map(lambda x: "<a href='./%s.html'>%s</a>" % (x, x),
(map(lambda x: x.package, source.buildDependencies)))
rDeps = map(lambda x: "<a href='./%s.html'>%s</a>" % (x, x),
(map(lambda x: x.package, self.pakspec.packageDependencies)))
rbDeps = map(lambda x: "<a href='./%s.html'>%s</a>" % (x, x), self.revBuildDeps)
rrDeps = map(lambda x: "<a href='./%s.html'>%s</a>" % (x, x), self.revRuntimeDeps)
html = """
<h1>İkili paket: %s</h1>
<h2>Kaynak versiyon %s, depo sürümü %s</h2>
<h3>Kaynak paket: %s</h3>
<h3>Derlemek için gerekenler:</h3>
<p>%s</p>
<h3>Çalıştırmak için gerekenler:</h3>
<p>%s</p>
<h3>Bağımlı paketler (derlenmek için):</h3>
<p>%s</p>
<h3>Bağımlı paketler (çalışmak için):</h3>
<p>%s</p>
""" % (
self.name,
self.source.spec.getSourceVersion(),
self.source.spec.getSourceRelease(),
make_url(source.name, "../source/"),
", ".join(bDeps),
", ".join(rDeps),
", ".join(rbDeps),
", ".join(rrDeps),
)
write_html("paksite/binary/%s.html" % self.name, self.name, html)
class Source:
def __init__(self, path, spec):
name = spec.source.name
if sources.has_key(name):
errors.append(_("Duplicate source packages:\n%s\n%s\n") % (
path, sources[name].path))
return
sources[name] = self
self.spec = spec
self.name = name
self.path = path
self.uri = svn_uri(path)
for p in spec.packages:
Package(self, p)
def report_html(self):
source = self.spec.source
paks = map(lambda x: "<a href='../binary/%s.html'>%s</a>" % (x, x),
(map(lambda x: x.name, self.spec.packages)))
histdata = map(lambda x: (x.release, x.date, x.version, make_url(x.name, "../packager/"), x.comment), self.spec.history)
ptch = map(lambda x: "<a href='%s/files/%s'>%s</a>" % (self.uri,
x.filename, x.filename), source.patches)
titles = "Sürüm", "Sürüm Tarihi", "Versiyon", "Güncelleyen", "Açıklama"
hist = make_table(histdata, titles)
html = """
<h1>Kaynak paket: %s</h1>
<h2>Kaynak versiyon %s, depo sürümü %s</h2>
<h3><a href='%s'>%s</a></h3>
<h3>Açıklama</h3>
<p>%s</p>
<h3>Lisanslar:</h3>
<p>%s</p>
<h3>İşlemler:</h3>
<p><a href="%s">Paket dosyalarına bak</a></p>
<p><a href="http://bugs.pardus.org.tr/buglist.cgi?product=Paketler&component=%s&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED">Hata kayıtlarına bak</a></p>
<h3>Bu kaynaktan derlenen ikili paketler:</h3>
<p>%s</p>
<h3>Tarihçe</h3>
%s
<h3>Yamalar</h3>
%s
""" % (
self.name,
self.spec.getSourceVersion(),
self.spec.getSourceRelease(),
source.homepage,
source.homepage,
source.summary,
", ".join(source.license),
self.uri,
self.name,
"<br>".join(paks),
"".join(hist),
"<br>".join(ptch),
)
write_html("paksite/source/%s.html" % self.name, self.name, html)
class Packager:
def __init__(self, spec, update=None):
if update:
name = update.name
email = update.email
else:
name = spec.source.packager.name
email = spec.source.packager.email
if packagers.has_key(name):
if email != packagers[name].email:
e = _("Developer '%s <%s>' has another mail address '%s' in source package '%s'") % (
name, packagers[name].email, email, spec.source.name)
packagers[name].errors.append(e)
errors.append(e)
if update:
packagers[name].updates.append((spec.source.name, update.release, update.comment))
else:
packagers[name].sources.append(spec.source.name)
else:
packagers[name] = self
self.name = name
self.email = email
if update:
self.sources = []
self.updates = [(spec.source.name, update.release, update.comment)]
else:
self.sources = [spec.source.name]
self.updates = []
self.errors = []
if not update:
for update in spec.history:
Packager(spec, update)
def report_html(self):
srcs = map(lambda x: ("<a href='../source/%s.html'>%s</a>" % (x, x), ), self.sources)
srcs.sort()
upds = map(lambda x: (u"<b><a href='../source/%s.html'>%s</a> (%s)</b><br>%s<br>" % (
x[0], x[0], x[1], x[2]), ), self.updates)
html = """
<p>Paketçi: %s (%s)</p>
""" % (self.name, mangle_email(self.email))
html += """
<div class='statstat'>
<h3>Sahip olduğu paketler:</h3><p>
%s
</p></div>
""" % make_table(srcs)
html += """
<div class='statstat'>
<h3>Yaptığı güncellemeler:</h3><p>
%s
</p></div>
""" % make_table(upds)
write_html("paksite/packager/%s.html" % self.name, self.name, html)
class Repository:
def __init__(self, path):
self.path = path
self.nr_sources = 0
self.nr_packages = 0
self.nr_patches = 0
self.people = Histogram()
self.licenses = Histogram()
self.mostpatched = Histogram()
self.longpy = Histogram()
self.cscripts = Histogram()
self.total_installed_size = 0
self.installed_sizes = {}
def processPspec(self, path, spec):
# new classes
Packager(spec)
Source(path, spec)
# update global stats
self.nr_sources += 1
self.nr_packages += len(spec.packages)
self.nr_patches += len(spec.source.patches)
# update top fives
self.people.add(spec.source.packager.name)
for u in spec.history:
self.people.note(u.name)
for p in spec.packages:
for cs in p.providesComar:
self.cscripts.add(cs.om)
for L in spec.source.license:
self.licenses.add(L)
self.mostpatched.add(spec.source.name, len(spec.source.patches))
try:
f = file(os.path.join(path, "actions.py"))
L = len(f.readlines())
self.longpy.add(spec.source.name, L)
f.close()
except:
pass
def scan(self):
for pak in find_pspecs(self.path):
spec = pisi.specfile.SpecFile()
try:
spec.read(os.path.join(pak, "pspec.xml"))
except Exception, inst:
errors.append(_("Cannot parse '%s':\n%s\n") % (pak, inst.args[0]))
continue
self.processPspec(pak, spec)
for p in packages.values():
p.markDeps()
def processPisi(self, path):
p = pisi.package.Package(path)
p.extract_files(["metadata.xml", "files.xml"], ".")
md = pisi.metadata.MetaData()
md.read("metadata.xml")
self.total_installed_size += md.package.installedSize
if packages.has_key(md.package.name):
# FIXME: check version/release match too?
packages[md.package.name].installed_size = md.package.installedSize
else:
printu("Binary package '%s' has no source package in repository %s\n" % (path, self.path))
fd = pisi.files.Files()
fd.read("files.xml")
for f in fd.list:
if self.installed_sizes.has_key(f.type):
# Emtpy directories and symlinks has None size
if not f.size is None:
self.installed_sizes[f.type] += int(f.size)
else:
self.installed_sizes[f.type] = int(f.size)
def scan_bins(self, binpath):
for root, dirs, files in os.walk(binpath):
for fn in files:
if fn.endswith(".pisi"):
self.processPisi(os.path.join(root, fn))
def report_html(self):
table = (
("Kaynak paket sayısı", self.nr_sources),
("İkili paket sayısı", self.nr_packages),
("Yama sayısı", self.nr_patches),
("Paketçi sayısı", len(self.people.list)),
)
html = make_table(table)
html += """
<div class='statstat'>
<h3>En fazla yamalanmış beş kaynak paket:</h3><p>
%s
</p></div>
""" % make_table(map(lambda x: (make_url(x[0], "./source/"), x[1]), self.mostpatched.get_list(5)))
html += """
<div class='statstat'>
<h3>En uzun inşa betikli beş kaynak paket:</h3><p>
%s
</p></div>
""" % make_table(map(lambda x: (make_url(x[0], "./source/"), x[1]), self.longpy.get_list(5)))
write_html("paksite/index.html", "Genel Bilgiler", html)
titles = (
"<a href='packagers_by_name.html'>Paketçi</a>",
"<a href='packagers.html'>Paket sayısı</a>"
)
people = self.people.get_list()
people = map(lambda x: ("<a href='./packager/%s.html'>%s</a>" % (x[0], x[0]), x[1]), people)
write_html("paksite/packagers.html", "Paketçiler (paket sayısına göre)", make_table(people, titles))
people.sort(key=lambda x: x[0])
write_html("paksite/packagers_by_name.html", "Paketçiler (isme göre)", make_table(people, titles))
titles = "Paket adı", "Versiyon", "Açıklama"
srclist = map(lambda x: (make_url(x.name, "source/"), x.spec.getSourceVersion(), x.spec.source.summary), sources.values())
srclist.sort(key=lambda x: x[0])
html = make_table(srclist, titles)
write_html("paksite/sources.html", "Kaynak Paketler", html)
binlist = map(lambda x: (make_url(x.name, "binary/"), x.source.spec.getSourceVersion(), x.source.spec.source.summary), packages.values())
binlist.sort(key=lambda x: x[0])
html = make_table(binlist, titles)
write_html("paksite/binaries.html", "İkili Paketler", html)
# command line driver
def usage():
printu(_("Usage: repostats.py [OPTIONS] source-repo-path [binary-repo-path]\n"))
printu(" -t, --test-only: %s" % _("Dont generate the web site.\n"))
sys.exit(0)
if __name__ == "__main__":
try:
opts, args = getopt.gnu_getopt(sys.argv[1:], "ht", ["help", "test-only"])
except:
usage()
if args == []:
usage()
do_web = True
for o, v in opts:
if o in ("-h", "--help"):
usage()
if o in ("-t", "--test-only"):
do_web = False
repo = Repository(args[0])
printu(_("Scanning source repository...\n"))
repo.scan()
if len(args) > 1:
printu(_("Scanning binary packages...\n"))
repo.scan_bins(args[1])
if do_web:
if not os.path.exists("paksite/packager"):
os.makedirs("paksite/packager")
if not os.path.exists("paksite/binary"):
os.makedirs("paksite/binary")
if not os.path.exists("paksite/source"):
os.makedirs("paksite/source")
file("paksite/stil.css", "w").write(css_template)
repo.report_html()
for p in packagers.values():
p.report_html()
for p in packages.values():
p.report_html()
for p in sources.values():
p.report_html()
scripts/unownedfiles.py
-48
View File
@@ -1,48 +0,0 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
#
# Copyright (C) 2005 - 2007, TUBITAK/UEKAE
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2 of the License, or (at your
# option) any later version. Please read the COPYING file.
#
import os
import sys
import piksemel
def findFilesXML(path):
filesXMLs = list()
for root, dirs, files in os.walk(path):
for file in files:
if file == "files.xml":
filesXMLs.append(os.path.join(root, file))
return filesXMLs
def parseFilesXML():
files = set()
for filesXML in findFilesXML("/var/lib/pisi/package/"):
data = open(filesXML).read()
data = piksemel.parseString(data)
for filePath in data.tags("File"):
path = filePath.getTagData("Path")
if filePath.getTag("Hash"):
files.add("/%s" % path)
return files
if __name__ == "__main__":
try:
sys.argv[1]
except IndexError:
sys.exit("Unsufficient arguments...")
fileSet = parseFilesXML()
for root, dirs, files in os.walk(sys.argv[1]):
for file in files:
f = os.path.join(root, file)
if not fileSet.__contains__(f):
print f
setup.py
+6 -3
View File
@@ -64,6 +64,7 @@ class Install(install):
return
pisiconf = open(confFile, "w")
klasses = inspect.getmembers(pisi.configfile, inspect.isclass)
defaults = [klass for klass in klasses if klass[0].endswith('Defaults')]
@@ -76,8 +77,10 @@ class Install(install):
and not m[0].endswith('__')]
for member in section_members:
pisiconf.write("%s = %s\n" % (member[0], member[1]))
if member[1] == None or member[1] == "":
pisiconf.write("# %s = %s\n" % (member[0], member[1]))
else:
pisiconf.write("%s = %s\n" % (member[0], member[1]))
pisiconf.write('\n')
@@ -90,7 +93,7 @@ setup(name="pisi",
author_email="pisi@pardus.org.tr",
url="http://www.pardus.org.tr/eng/pisi/",
package_dir = {'': ''},
packages = ['pisi', 'pisi.cli', 'pisi.db', 'pisi.actionsapi', 'pisi.pxml', 'pisi.scenarioapi'],
packages = ['pisi', 'pisi.cli', 'pisi.actionsapi', 'pisi.pxml', 'pisi.scenarioapi'],
scripts = ['pisi-cli', 'scripts/lspisi', 'scripts/unpisi', 'scripts/check-newconfigs.py', 'scripts/revdep-rebuild'],
cmdclass = {'install' : Install}
)
tests/dependencytests.py
+6 -6
View File
@@ -21,27 +21,27 @@ class DependencyTestCase(unittest.TestCase):
gtkmmdep = Dependency()
gtkmmdep.versionFrom = '1.9.1'
gtkmmdep.package = 'atk'
self.assert_( not gtkmmdep.satisfies('atk', '1.8.0', '1') )
self.assert_( gtkmmdep.satisfies('atk', '1.9.1', '2') )
self.assert_( gtkmmdep.satisfies('atk', '1.10.5', '3') )
self.assert_( not gtkmmdep.satisfies_relation('atk', '1.8.0', '1') )
self.assert_( gtkmmdep.satisfies_relation('atk', '1.9.1', '2') )
self.assert_( gtkmmdep.satisfies_relation('atk', '1.10.5', '3') )
def testReleaseFrom(self):
# releaseFrom isn't taken into account #2294
dep = Dependency()
dep.releaseFrom = '121'
dep.package = 'dbus'
self.assert_(not dep.satisfies('dbus', '1.2', '2'))
self.assert_(not dep.satisfies_relation('dbus', '1.2', '2'))
def testReleaseTo(self):
dep = Dependency()
dep.releaseTo = '22'
dep.package = 'dbus'
self.assert_(not dep.satisfies('dbus', '1.2', '25'))
self.assert_(not dep.satisfies_relation('dbus', '1.2', '25'))
def testReleaseIs(self):
dep = Dependency()
dep.release = '42'
dep.package = 'dbus'
self.assert_(dep.satisfies('dbus', '1.2', '42'))
self.assert_(dep.satisfies_relation('dbus', '1.2', '42'))
suite = unittest.makeSuite(DependencyTestCase)
tests/filestests.py
+1 -1
View File
@@ -13,7 +13,7 @@ import os
import pisi.context as ctx
import pisi.api
import pisi.db.installdb
import pisi.installdb
import pisi.files as files
from pisi import util
tests/installdbtests.py
+1 -1
View File
@@ -13,7 +13,7 @@ import os
import pisi.context as ctx
import pisi.api
import pisi.db.installdb
import pisi.installdb
from pisi import util
import testcase
tests/metadatatests.py
+14 -3
View File
@@ -13,6 +13,7 @@ import os
from pisi import metadata
from pisi import util
from pisi import specfile
class MetaDataTestCase(unittest.TestCase):
@@ -20,13 +21,13 @@ class MetaDataTestCase(unittest.TestCase):
md = metadata.MetaData()
md.read('tests/popt/metadata.xml')
self.assertEqual(md.package.license, ["As-Is"])
self.assertEqual(md.package.license, ["as-is"])
self.assertEqual(md.package.version, "1.7")
self.assertEqual(md.package.installedSize, 149691)
self.assertEqual(md.package.installedSize, 15982)
return md
def testWrite(self):
md = self.testRead()
md.write('/tmp/metadata-test.xml')
@@ -36,5 +37,15 @@ class MetaDataTestCase(unittest.TestCase):
if md.errors():
self.fail("Couldn't verify!")
def testFromSpec(self):
specmd = metadata.MetaData()
spec = specfile.SpecFile('tests/popt/pspec.xml')
specmd.from_spec(spec.source, spec.packages[0], spec.history)
md = metadata.MetaData()
md.read('tests/popt/metadata.xml')
for key in md.package.__dict__.keys():
if key not in ('installedSize', 'packageFormat', 'distributionRelease',
'license', 'architecture', 'distribution', 'partOf'):
self.assertEqual(md.package.__dict__[key], specmd.package.__dict__[key])
suite = unittest.makeSuite(MetaDataTestCase)
tests/packagedbtests.py
+2 -3
View File
@@ -13,10 +13,9 @@ import os
import pisi.context as ctx
import pisi.api
from pisi.packagedb import PackageDB
from pisi import util
from pisi.specfile import SpecFile
import pisi.db.repodb
import pisi.db.packagedb
import testcase
class PackageDBTestCase(testcase.TestCase):
@@ -29,7 +28,7 @@ class PackageDBTestCase(testcase.TestCase):
def testAdd(self):
if not ctx.repodb.has_repo('test'):
ctx.repodb.add_repo('test', pisi.db.repodb.Repo(pisi.uri.URI('fakerepo.xml')) )
ctx.repodb.add_repo('test', pisi.repodb.Repo(pisi.uri.URI('fakerepo.xml')) )
ctx.packagedb.add_package(self.spec.packages[1], 'test')
self.assert_(ctx.packagedb.has_package('popt-libs'))
# close the database and remove lock
tests/packagetests.py
-1
View File
@@ -13,7 +13,6 @@ import os
from pisi import util
import pisi.context as ctx
import pisi.db.packagedb
import pisi.package
import testcase
tests/popt/metadata.xml
+35 -71
View File
@@ -1,87 +1,51 @@
<?xml version="1.0" ?>
<PISI>
<Source>
<Name>
popt
</Name>
<Name>popt</Name>
<Homepage>http://www.rpm.org/</Homepage>
<Packager>
<Name>
Kırmızı kafalar
</Name>
<Email>
hotmail@redhat.com
</Email>
<Name>Kırmızı kafalar</Name>
<Email>hotmail@redhat.com</Email>
</Packager>
</Source>
<Package>
<Name>
popt-libs
</Name>
<Summary>
Command line option parsing library
</Summary>
<Description>
library files for popt
</Description>
<License>
As-Is
</License>
<IsA>
library:util:optparser
</IsA>
<PartOf>
rpm:archive
</PartOf>
<Name>popt</Name>
<Summary xml:lang="en">Command line option parsing library</Summary>
<Summary xml:lang="tr">Komut satırı seçenekleri işleme kütüphanesi</Summary>
<Description xml:lang="en">library files for popt</Description>
<IsA>app:console</IsA>
<PartOf>system.base</PartOf>
<License>as-is</License>
<RuntimeDependencies>
<Dependency>
gettext
</Dependency>
<Dependency>gettext</Dependency>
</RuntimeDependencies>
<Files>
<Path fileType="sharedLib">
/usr/lib
</Path>
<Path fileType="doc">
/usr/share/doc
</Path>
<Path fileType="doc">
/usr/share/man
</Path>
<Path fileType="localedata">
/usr/share/locale
</Path>
<Path fileType="header">
/usr/include/popt.h
</Path>
<Path fileType="library">/usr/lib</Path>
<Path fileType="doc">/usr/share/doc</Path>
<Path fileType="man">/usr/share/man</Path>
<Path fileType="localedata">/usr/share/locale</Path>
<Path fileType="header">/usr/include/popt.h</Path>
</Files>
<History>
<Update release="3">
<Update release="1">
<Date>2005-06-14</Date>
<Version>1.7</Version>
</Update>
<Update release="2">
<Date>2005-06-10</Date>
<Version>1.7</Version>
</Update>
<Update release="3">
<Date>2005-05-05</Date>
<Version>1.7</Version>
<Comment>First release.</Comment>
<Name>Kırmızı kafalar</Name>
<Email>hotmail@redhat.com</Email>
</Update>
</History>
<Build>
0
</Build>
<Distribution>
Pardus
</Distribution>
<DistributionRelease>
0.1
</DistributionRelease>
<Architecture>
Any
</Architecture>
<InstalledSize>
149691
</InstalledSize>
<Distribution>Pardus</Distribution>
<DistributionRelease>2007</DistributionRelease>
<Architecture>Any</Architecture>
<InstalledSize>15982</InstalledSize>
<PackageFormat>1.1</PackageFormat>
<Source>
<Name>popt</Name>
<Homepage>http://www.rpm.org/</Homepage>
<Packager>
<Name>Kırmızı kafalar</Name>
<Email>hotmail@redhat.com</Email>
</Packager>
</Source>
</Package>
</PISI>
</PISI>
tests/repodbtests.py
+3 -3
View File
@@ -13,7 +13,7 @@ import os
import pisi.context as ctx
import pisi.api
import pisi.db.installdb
import pisi.installdb
from pisi import util
import testcase
@@ -24,7 +24,7 @@ class RepoDBTestCase(testcase.TestCase):
for i in range(2):
print '\nTest %d\n' % (i)
pisi.api.add_repository("foo","bar")
pisi.api.remove_repository("foo")
pisi.api.add_repo("foo","bar")
pisi.api.remove_repo("foo")
suite = unittest.makeSuite(RepoDBTestCase)
tests/sourcedbtests.py
+3 -3
View File
@@ -13,7 +13,7 @@ import os
import pisi.context as ctx
import pisi.api
import pisi.db.sourcedb as sourcedb
import pisi.sourcedb
from pisi import util
from pisi.specfile import SpecFile
@@ -23,11 +23,11 @@ class SourceDBTestCase(testcase.TestCase):
def setUp(self):
testcase.TestCase.setUp(self)
self.sourcedb = sourcedb.init()
self.sourcedb = pisi.sourcedb.init()
self.spec = SpecFile()
self.spec.read("tests/popt/pspec.xml")
if not ctx.repodb.has_repo('test'):
ctx.repodb.add_repo('test', pisi.db.repodb.Repo(pisi.uri.URI('fakerepo.xml')) )
ctx.repodb.add_repo('test', pisi.repodb.Repo(pisi.uri.URI('fakerepo.xml')) )
def testAddRemove(self):
self.sourcedb.add_spec(self.spec, 'test')
tests/utiltests.py
+3 -3
View File
@@ -29,10 +29,10 @@ def get_svn_paths(path):
return svn_paths
def move_svn_paths(paths):
[shutil.move(d, '/tmp/' + os.path.dirname(d).replace('/', '_')) for d in paths]
[shutil.move(d, 'tmp/' + os.path.dirname(d).replace('/', '_')) for d in paths]
def restore_svn_paths(paths):
[shutil.move('/tmp/' + os.path.dirname(d).replace('/', '_'), d) for d in paths]
[shutil.move('tmp/' + os.path.dirname(d).replace('/', '_'), d) for d in paths]
class UtilTestCase(testcase.TestCase):
@@ -69,7 +69,7 @@ class UtilTestCase(testcase.TestCase):
self.assertEqual(dir_size('tests/utilfiles/linktonowhere'), 23)
self.assertEqual(dir_size('tests/utilfiles/directory'), 74536)
self.assertEqual(dir_size('tests/utilfiles/linktoarfile'), 22)
self.assertEqual(dir_size('tests/utilfiles/'), 149117)
self.assertEqual(dir_size('tests/utilfiles/'), 149072)
restore_svn_paths(svn_paths)
def testGetFileHashes(self):
tools/svndist.py
+1 -1
View File
@@ -23,7 +23,7 @@ if os.path.exists('svndist/pisi-%s' % ver):
shutil.rmtree('svndist/pisi-%s' % ver)
print 'Exporting svn directory'
run('svn export http://svn.uludag.org.tr/uludag/trunk/pisi svndist/pisi-%s' % ver)
run('svn export http://svn.uludag.org.tr/uludag/branches/pisi-1.1 svndist/pisi-%s' % ver)
os.chdir('svndist')
run('tar cjvf pisi-%s.tar.bz2 pisi-%s' % (ver, ver))