PLDWWW: TODO/FTPInfrastructure

mmazur mmazur at pld-linux.org
Mon Aug 14 18:45:00 CEST 2006


Author: mmazur   Date: Mon Aug 14 16:45:00 2006 GMT
Module: PLDWWW   URL: http://www.pld-linux.org/TODO/FTPInfrastructure
---- Log message:
Toodoo

---- Page affected: TODO/FTPInfrastructure

---- Diffs:

================================================================
New page:
#language en

= FTP Infrastructure TODO List =

Note: the whole system is written in python.

== Add "ignore removed arch" to mvpkgs ==

The title says it all. A quickie.

== Make mvpkgs and testmvpkgs handle duplicates given as arguments ==

Running the scripts with wildcards (as in "mvpkgs test PLD *") is a very bad idea, since one can't be really sure what's going to get moved, but people are lazy and under some conditions wanting to use it like that is quite acceptable. Unfortunately both (or at least the mvpkgs) scripts literally blow up most of the times when trying to be run like that, because they don't know what to do when someone is trying to move more than one version of a package (and that's what using wildcards often leads to).

The most reasonable solution would probably be for both scripts to go through the list of packages, notice any duplicates and print out a warning about having found them and that they're ignoring the older versions and working with the newer once (printing out the versions ignored and the versions used). The mvpkgs script should additionally prompt the user, to give him time to actually check those warning before deciding to give it a go.

Once again, this is not all that complicated, coding it should at most take a couple of hours.

== Making mvpkgs and testmvpkgs check for dependency errors ==

This one is a major function, but one that, if implemented correctly, would give an enormous quality boost to the distro.

The idea of the 'main' tree on our ftp server is that packages there are (more) stable, but, above all, they're just simply installable. Meaning -- anyone should be able to install any package found in the main tree. Easier said than done, since any package getting moved there has the potential to generate rpm dependency (aka dep) errors. Checking for them beforehand is possible, however it requires manual labor, is quite time consuming and currently can only be done by the person doing the move, instead of the person introducing the package that breaks deps. The idea of the 'testmvpkgs' and 'mvpkgs' is to make life easier for the ftp admins, since those two tools have some simple tests built in, however integrating dependency checking into them would be the equivalent of the Holy Grail.

And assuming the functionality gets integrated, it'd be quite easy to code up a system that does those checks periodically and actually mails the developers responsible for new packages a list of dependency errors those packages might generate even before anybody tries to move them, thereby allowing developers to fix problems earlier.

As for technical details.

Poldek currently has the ability to check dependencies this way, it's only a matter of figuring out how to run it and how to  parse the output (if there's something poldek doesn't allow or maybe should be done better, one can always mail the author, as he's willing to make changes). Delays might be a problem, since loading up indexes does take time, but since the ftp system has a daemon that needs to be running all the time, it's perfectly possible to have that daemon run a copy of poldek and never turn it off (almost never -- it would have to restart it every time indexes in the main tree get regenerated, but that's quite simple to do).

== Write complete docs ==

Describe the architecture of the ftp infrastructure for people interested in working on one of the major features plus go through the current hands-on guide and fix/broaden it where appropriate.


More information about the pld-cvs-commit mailing list