
Here we take notes of the strange stuff, so we can refactor them after
Pardus 2007 release.


==> Python 2.5 support
* We should use python2.5

==> Locale support
* bindtextdomain and textdomain calls are not necessary, because
gettext.translation() dont use them.

* setlocale call is probably necessary for some stuff, but not for
message translation, it seems gettext.translation() api looks for
environment LC_ALL, LC_MESSAGES anyway.

* pygettext.py shouldn't be needed at all. Plain xgettext works with
python source.


==> Utility functions
* sha1_file and sha1_data functions has some
exception confusion. These two functions should be a lot simpler.

* get_file_hashes is overly complex and contains strange code like:
for r, d, f in os.walk(parent, topdown=False): pass

* To estimate required disk size for the packages more accurately we can
calculate package size + (nr of files * inode size of fs).

* Why in a world like this there exists *parse_package* util functions? These must be all methods of package class

* Remove all unneeded util functions, most of them belongs its classes

==> code readability
* Public functions should contain doc strings.

* Python builtins like file, list, etc should be avoided in variable names.
There is even a file.py module!

* a,b,c,d,f,r,_i,A,B,C,G are equally bad.

* some import'ed modules are not used inside the importer modules, cleanup needed.

* Would be very good if we can avoid "from x import y" form, it makes it
harder to reload modules.

* Convert all String Concatenation into "%s" % string form which is much faster, readable and correct

==> exceptions
* Current model is bad. Exception names should tell what is the error type. Instead
we have one Error exception in every module. If I call pisi.api.install("lala"),
I should get pisi.api.PackageNotFound or pisi.install.PackageNotFound or something
like that. For every kind of error, you get pisi.api.Error now, and only way to find
out exact error is try to parse error string (which is localized).

* class Exception(Exception) is evil.

* There shouldn't be a bare Except: clause in pisi modules.


==> database
* We should have a DB performance test suite to find important points to make faster,
if we cannot measure, we cannot improve. 
 
* [FIXED] We should use "2" as pickle protocol. currently we use old binary pickle protocol "1". New
protocol generally produces more compact pickled objects (dbshelve.py)

* History, and  Summary-Description objects should be kept in seperate db files. This way 
we  can have faster searches and probably faster access to packages and better memory footprint

* For big objects with many member objects (Package, from pisi.metadata) we might investigate
writing our own simple serialization routines for some objects like Conflicts or additionalfiles.

* context singleton is a mess. Maybe we should consider another approach to access common db objects 

* ComponentDB should not hold lists of Packages that belongs to it. A new index for components to package DB 
should be enough.
  
* Current implementation of repository system is suboptimal. We add every package to a Dictionary as value 
and repository name as key and add that dictionary to db. For example if I am adding package foo to 
packages db, I am actually adding, a Dictionary with ["repotype", foo] to db.
This is cheap, bloated and slow. We must have different repository databases for packages.

* We should really consider using a simple and fast relational database (sqlite) for database operations. Here
is the rationale and some issues.
- Current implementation uses berkeleydb as a storage of dictionaries, with keys and Pickled Python objects.
This approach simplifies coding requirements but leads to some problems.
1. Some operations cannot be done effectively. Searching can only be done by iterating all objects and searching
one by one. component, reverse dependencies etc are holding lists of package names that are updated on every 
package operation, Multiple repository problem is 
   
2.Performance: although fast, pickling and unpickling whole objects is still a time consuming process.
Pickled Objects contains object metadata and are larger then the data they hold, so reading, writing pickled 
Objects to db is slower than reading and writing  only necessary fields. large objects (Order of kilobytes)
causes berkeleydb data buckets to overflow and caches works less efficently.
 
4.Disk usage: Using pickled objects requires more memory and disk space. Total size of a current pisi database
is over 100Mb, if we modify files.bdb to only hold filenames and packages, size is still around 50MB.     

Rational Databases offers number of advantages and disadvantages:

Advantages:
+ Relational databases offers much more flexible operations on stored data. Searching, grouping, sorting
can be done by database engine efficently.
+ There will be only a single database file and much smaller database size compared to berkeleydb files
holding pickled python objects. 
+ Since database no longer holds pickled python objects, it is possible to read and modify the database
by other programs written in different languages (Java, C++ etc.)
+ SQL lite is generally very fast, probably beating "Berkeleydb with pickled objects" by the order of 4-5 on 
queries.

Disadvantages
- Using a relational database will require more coding when adding and retrieving contents of objects.
- Introducing SQL language makes code more complex and harder to maintain.
- Designing a good schema, identifying required indexes etc. takes time
- Some developers are superstitious about sql and relational databases ;)
 
==> class attributes
* In some classes there are some attributes assigned but never
used. (see remove_unused_attributes.patch)

* PiSi uses heavy classes and creates thousands of instances of them
  (Package, Metadata, Dependency, ...). These classes should define
  __slots__ to reduce the heap used.


==> actionsAPI
* Get rid of ugly exception model
* Refactor/clean the code
* Add strict checks to models
* Maybe? Get rid of functional logic, switch to OO one
* ActionsAPI still needs an updated API document


==> PiSi API
* Write a real one


==> Function parameters
* Check function parameters and see if some parameters (especially the
ones named tmpDir/tmp_dir/target_dir) are redundant now. See
http://liste.uludag.org.tr/uludag-commits/2007-February/010070.html

* conflict.py
conflicts method takes a pkg_name parameter but doesn't use it at all


==> Fixes (that need API breakage)
* http://liste.uludag.org.tr/uludag-commits/2007-February/010117.html

==> Logging

* Improve logging with all needed update/downgrade/install/remove information so one can easily find the history of packages, 
currently PiSi wrotes everyting into logs which is not usefull. Also this information can be used for rollback and reporting.

==> operations.py and atomicoperations.py

* These modules have gone so big and become untraceable... divide all operations to their related modules
  under a pisi.operations module...

==> function code lengths

* we have some functions that goes pages long... divide all of them to small digestable chunks.. 
  avoid writing functions longer than your editor's screen.

==> assertions

* Lots of assertions used in the code with no following descriptive string information.

==> repo order

* When trying to install a package or looking for a dependency for a package the first found package
  in "repo order" is used. This design decision is not very good. When a new repo that has the latest
  version of any package is added and if it is the last repo in order, pisi can not upgrade to this 
  package. 

  We can remove this order thing and in this situation we can use the latest version of the package.
  This should also lead to some other problems but between these two not very good solutions this 
  seems to be the better one.

==> autoxml

* When setting a Long type attribute or tag of an autoxml class, value must be Long type. Otherwise, 
  autoxml.write() fails in x86_64

==> COMAR

* pisi-cli should use comar.
