Terrible performance of Python dependency generator

[Jeffrey Johnson]

> On Nov 23, 2015, at 3:54 AM, Elan Ruusamäe <glen at pld-linux.org> wrote:
> 
> On 22.11.2015 22:39, Jacek Konieczny wrote:
>> /usr/lib/rpm/pythoneggs.py is used to find the dependencies and it is
>> not that slow by itself… but it is called twice (Provides + Requires)
>> for each file in /usr/share/pythonX.Y. And big Python packages have lots
>> of files there. Most of them not adding any extra dependency
>> information.
> i tried once to make hack to that, when facing similar problem with php dependencies generator.
> 
> the idea was simple:
> 1. first time dep generator is invoked, it becames daemon and starts listening to unix socket
> 2. further calls talk to socket instead dispatching the "requests"
> 
> but i never finished it, don't know if i have some WIP saved somewhere.
> 

The “simple” idea is fine for optimizing I/O to an external helper.

However, with an embedded python interpreter in rpm, one has all
the persistence needed to generate python (or perl or tcl or java or …)
dependencies of each file without having to “batch”.

(aside)
But any rpmdep optimization is mostly overkill, the per-interpreter dependencies
(perl perhaps the one exception) are rather primitive and hardly need to be generated
for every file every build with current packaging. Sure one can conceive of
a single python build that generates modules and dependencies for every possibly
deployed interpreter version in a single build, just that is a rather hard package
to construct and maintain … I’ve not personally seen a serious attempt to
do that yet.

> and with rpm4.5 the python dep genreator just compared pythonX.Y version  to do print "python(abi) %s" print
> that was "optimized" by providing PY_VER as env var.
> 

IIRC, something like python(abi) is still mostly what is being distributed with rpm
helper scripts (though perhaps some distro has extended the helpers to do
something more useful). 

73 de Jeff
> -- 
> glen
> 
> _______________________________________________
> pld-devel-en mailing list
> pld-devel-en at lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en