*.py packaging, again

[Jakub Bogusz]

On Thu, Jul 14, 2011 at 09:44:01AM +0200, Jacek Konieczny wrote:
> On Wed, Jul 13, 2011 at 03:07:07PM +0200, Jakub Bogusz wrote:
> > I don't like the idea of __pycache__ not managed by rpm (or not
> > maintained with rpm database in case of more robust post scripts)
> > because of at least:
> > - possible inconsistences (like leftovers in case of upgrade failures;
> >   in such case it would be even possible that after installing another
> >   version of package with *.py files having older date than pycache,
> >   pycache won't be rebuilt)
> > - more work needed to find package "owning" __pycache__/* files
> > - one more place to hide some malicious code not detectable by rpm -Va

And one more: why byte-compiling on every target machine when the result
should be exactly the same for given Python major version?
(modulo python bugs, but then - we want the same compiled code even more)
In big packages byte-compilation may take significant amount of time.

This argument is still valid when considering byte-compilation on rpm
install side.

> Ok, you may be right. But what than? We don't even have an idea how
> should the proper rpm solution work and no volunteers to design and code
> it. At the same time Python 3.2 is waiting for being included in PLD,
> Python 3.3 is on its way and PyPy could be useful too, provided
> compatible packages are available.

There is also jpython since some time.
I don't know how compatibility looks like (IIRC jpythons were released
with some delay after cpython version it was +/- compatible with).

> Maybe we should just start packing the *.py files together with the
> compiled files for the python version they are built for? Yes, I know I
> can start the old 'we don't package sources' flame war again, but '*.py'
> files are not sources, they are the executable code. '*.pyc' and '*.pyo'
> are just 'optimized versions' of those. And for Python 3.2 we have no
> option - I have already started adding both *.py and __pycache__/* to
> the python3 packages I have recently done.
> 
> For Python 2 (python-* packages), when including *.py:
> 
> We gain:
> - ability to use the installed noarch packages by other Python 2.7
>   implementations (like PyPy)

Are particular PyPy versions meant to be compatible with particular
Python 2 major version?

Even in Python 3 times noarch packages still use versioned source
directory.

Also, the question whether to package *.py refers only to noarch
Python 2 packages, as arch-dependent ones are bound to particular
cpython major version anyway.

The same gain can be reached by adding -source packages.

> For Python >= 3.2 (python3-* packages):
> 
> We gain:
> - compatibility with the language

For new Python 3 way of packaging it's unquestionable.

But there are still many packages not using setuptools(?) and thus
packaged in Python 2 way.
Should we enforce the new packaging method for them?
%py3_comp/%py3_ocomp macros use the old one.


-- 
Jakub Bogusz    http://qboosh.pl/