From baggins at pld-linux.org Thu Aug 1 07:57:24 2013 From: baggins at pld-linux.org (Jan =?utf-8?Q?R=C4=99korajski?=) Date: Thu, 1 Aug 2013 07:57:24 +0200 Subject: [packages/GLM] - initial revision - package nome in upper case b/c of conflict with existing package In-Reply-To: <51F8F999.7050308@pld-linux.org> References: <20130714143708.27204.85445@pld-linux.org> <9352f43c75a59e3a2e82e223b3ca9e9b7f5b4499_refs_heads_master@pld-linux.org> <51E3CB61.7020406@pld-linux.org> <20130715101849.GA2122@sith.mimuw.edu.pl> <51E3D26B.3090805@pld-linux.org> <51F8F999.7050308@pld-linux.org> Message-ID: <20130801055724.GA1180@home> On Wed, 31 Jul 2013, Elan Ruusam?e wrote: > On 15.07.2013 13:43, Elan Ruusam?e wrote: > > On 15.07.2013 13:18, Jan R?korajski wrote: > >> On Mon, 15 Jul 2013, Elan Ruusam?e wrote: > >> > >>> >On 14.07.2013 17:37, baggins wrote: > >>>> > >commit 9352f43c75a59e3a2e82e223b3ca9e9b7f5b4499 > >>>> > >Author: Jan R?korajski > >>>> > >Date: Sun Jul 14 16:36:26 2013 +0200 > >>>> > > > >>>> > > - initial revision > >>>> > > - package nome in upper case b/c of conflict with existing > >>>> package > >>>> > > > >>> > > >>> >this breaks github git mirror > >> Suggested fix? > > use different name for package :) > OpenGL-GLM ? Feel free to rename and update deps accordingly if you really must. > > > > > i also suggest maybe git initial push should check for such conflicts, > > i.e that there isn't already package what differs only by case > > > >> > >>> >and this breaks mirroring pld on case insensitive filesystem (vfat, > >>> hfs+) > >> Really? I bet this is the least problem with those filesystems. > > with different packages it's probably not that visible, > > i noticed it first with poppler-qt vs poppler-Qt which came from same > > package, meaning the version/release matched too. > >> Don't mirror unix fs on something that does not follow its semantics. > > when transferring rpm's to somewhere offline, using unix fs is not > > always possible! > > > > besides, there are case insensitive filesystems which are case also > > preserving filesystems, so it works most of the time (my $HOME is hfs+ > > and running MATE 1.6 desktop just fine) > > > > > -- > glen > > _______________________________________________ > pld-devel-en mailing list > pld-devel-en at lists.pld-linux.org > http://lists.pld-linux.org/mailman/listinfo/pld-devel-en -- Jan R?korajski | PLD/Linux SysAdm | http://www.pld-linux.org/ bagginsmimuw.edu.pl bagginspld-linux.org From glen at delfi.ee Thu Aug 1 11:31:45 2013 From: glen at delfi.ee (=?UTF-8?B?RWxhbiBSdXVzYW3DpGU=?=) Date: Thu, 01 Aug 2013 12:31:45 +0300 Subject: [packages/GLM] - initial revision - package nome in upper case b/c of conflict with existing package In-Reply-To: <20130801055724.GA1180@home> References: <20130714143708.27204.85445@pld-linux.org> <9352f43c75a59e3a2e82e223b3ca9e9b7f5b4499_refs_heads_master@pld-linux.org> <51E3CB61.7020406@pld-linux.org> <20130715101849.GA2122@sith.mimuw.edu.pl> <51E3D26B.3090805@pld-linux.org> <51F8F999.7050308@pld-linux.org> <20130801055724.GA1180@home> Message-ID: <51FA2B01.3040900@delfi.ee> On 01.08.2013 08:57, Jan R?korajski wrote: >>>>>> > >>> >this breaks github git mirror >>>> > >>Suggested fix? >>> > >use different name for package:) >> >OpenGL-GLM ? > Feel free to rename and update deps accordingly if you really must. > seems github has pushed last package that has changes, ie GLM: https://github.com/pld-linux/glm and the other glm is rather unfinished so perhaps the other package should be trashed/renamed? http://git.pld-linux.org/?p=packages/glm.git;a=summary libglm? aside, we have another conflict, which is file-conflict on libeio.so as well: eio vs libeio -- glen From baggins at pld-linux.org Sat Aug 3 19:51:00 2013 From: baggins at pld-linux.org (Jan =?utf-8?Q?R=C4=99korajski?=) Date: Sat, 3 Aug 2013 19:51:00 +0200 Subject: Packages scheduled for removal Message-ID: <20130803175100.GA2513@home> Hi, Just a heads up, I'm going to remove/move to obsoleted some packages fom Th. List and reasons below: java5-sun dead and obsolete java-sun dead and obsolete js replaced by js185, nothing needs it anymore module-init-tools replaced by kmod, udev now requires kmod, so no point in keeping m-i-t cpufrequtils replaced by kernel-tools-cpupower openmotif replaced by motif lesstif replaced by motif -- Jan R?korajski | PLD/Linux SysAdm | http://www.pld-linux.org/ bagginsmimuw.edu.pl bagginspld-linux.org From hawk at pld-linux.org Sat Aug 3 20:50:52 2013 From: hawk at pld-linux.org (Marcin Krol) Date: Sat, 03 Aug 2013 20:50:52 +0200 Subject: Packages scheduled for removal In-Reply-To: <20130803175100.GA2513@home> References: <20130803175100.GA2513@home> Message-ID: <51FD510C.3010309@pld-linux.org> > Hi, > Just a heads up, I'm going to remove/move to obsoleted some packages > fom Th. List and reasons below: > > java-sun dead and obsolete Any official note on Java 6 being dead and obsolete? Can't find one on Oracle site. > module-init-tools replaced by kmod, udev now requires kmod, so no point in keeping m-i-t In some situations I found module-init-tools working, while kmod was silently doing nothing. Any details on module-init-tools not working with udev 206? I haven't noticed any problems. M. From baggins at pld-linux.org Sat Aug 3 21:18:16 2013 From: baggins at pld-linux.org (Jan =?utf-8?Q?R=C4=99korajski?=) Date: Sat, 3 Aug 2013 21:18:16 +0200 Subject: Packages scheduled for removal In-Reply-To: <51FD510C.3010309@pld-linux.org> References: <20130803175100.GA2513@home> <51FD510C.3010309@pld-linux.org> Message-ID: <20130803191815.GA12673@home> On Sat, 03 Aug 2013, Marcin Krol wrote: > > Hi, > > Just a heads up, I'm going to remove/move to obsoleted some packages > > fom Th. List and reasons below: > > > > java-sun dead and obsolete > > Any official note on Java 6 being dead and obsolete? Can't find one on > Oracle site. http://java.com/en/download/faq/java_6.xml, see "Java SE 6 End of Public Updates Notice" > > module-init-tools replaced by kmod, udev now requires kmod, so no point in keeping m-i-t > > In some situations I found module-init-tools working, while kmod was > silently doing nothing. Any details on module-init-tools not working > with udev 206? I haven't noticed any problems. Some functionality has been moved from udev to kmod (see my commit 8495ab1be0d514 in systemd and read NEWS in systemd for explanations), you may end up with missing devices. I'm going to move module-init-tools to obsolete, we should find what is the problem with kmod when it does nothing for you instead of keeping outdated software in repo. -- Jan R?korajski | PLD/Linux SysAdm | http://www.pld-linux.org/ bagginsmimuw.edu.pl bagginspld-linux.org From jajcus at jajcus.net Sun Aug 4 17:53:04 2013 From: jajcus at jajcus.net (Jacek Konieczny) Date: Sun, 4 Aug 2013 17:53:04 +0200 Subject: Packages scheduled for removal In-Reply-To: <20130803191815.GA12673@home> References: <20130803175100.GA2513@home> <51FD510C.3010309@pld-linux.org> <20130803191815.GA12673@home> Message-ID: <20130804175304.25812407@lolek.nigdzie> On Sat, 3 Aug 2013 21:18:16 +0200 Jan R?korajski wrote: > On Sat, 03 Aug 2013, Marcin Krol wrote: > > > > Hi, > > > Just a heads up, I'm going to remove/move to obsoleted some > > > packages fom Th. List and reasons below: > > > > > > java-sun dead and obsolete > > > > Any official note on Java 6 being dead and obsolete? Can't find one > > on Oracle site. > > http://java.com/en/download/faq/java_6.xml, see "Java SE 6 End of > Public Updates Notice" And we have Icedtea6 ? for all platforms, with no weird license restrictions. BTW, 'java-sun' package name has been a lie for some time now, anyway ;) Greets, Jacek From glen at pld-linux.org Mon Aug 5 14:08:21 2013 From: glen at pld-linux.org (=?UTF-8?B?RWxhbiBSdXVzYW3DpGU=?=) Date: Mon, 05 Aug 2013 15:08:21 +0300 Subject: [packages/rc-scripts] - just pull mksh, C on pdksh is confusing and we do want mksh here - rel 3 In-Reply-To: References: <85f8c821b3670866698595c17a7059c65349084a_refs_heads_master@pld-linux.org> Message-ID: <51FF95B5.9060405@pld-linux.org> On 04.08.2013 13:43, baggins wrote: > commit b598c63cedbfd5a8be6b73cabae42046cc63cc23 > Author: Jan R?korajski > Date: Sun Aug 4 12:42:36 2013 +0200 > > - just pull mksh, C on pdksh is confusing and we do want mksh here > - rel 3 > > rc-scripts.spec | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > --- > diff --git a/rc-scripts.spec b/rc-scripts.spec > index 63a5077..8e8f728 100644 > --- a/rc-scripts.spec > +++ b/rc-scripts.spec > @@ -9,7 +9,7 @@ Summary(pl.UTF-8): inittab i skrypty startowe z katalogu /etc/rc.d > Summary(tr.UTF-8): inittab ve /etc/rc.d dosyalar? > Name: rc-scripts > Version: 0.4.6 > -Release: 2 > +Release: 3 > License: GPL v2 > Group: Base > #Source0: ftp://distfiles.pld-linux.org/src/%{name}-%{version}.tar.gz > @@ -57,6 +57,7 @@ Requires: hostname > Requires: iproute2 > Requires: iputils-arping > Requires: mingetty > +Requires: mksh > Requires: mktemp > Requires: mount >= 2.12 > Requires: procps >= 1:3.2.6-1.1 > @@ -79,7 +80,6 @@ Conflicts: udev-core < 1:135-2 > Conflicts: udev-core < 1:124-3 > %endif > Conflicts: lvm2 < 2.02.83 > -Conflicts: pdksh < 5.2.14-58 > Conflicts: upstart-SysVinit < 2.86-25 > Conflicts: wpa_supplicant < 0.6.3 > BuildRoot: %{tmpdir}/%{name}-%{version}-root-%(id -u -n) > is the /etc/shells clearing bug fixed now? with this change you propagate the pdksh -> mksh upgrade breaking /etc/shells automatically to lots of systems (i.e which do poldek --upgrade-dist automatically or semiautomatically)! with conflicts auto-upgrade wouldn't migrate and you had to poldek -u mksh manually, which leaves you chance to ovelook /etc/shells contents. btw, the pdksh compatibility is restored in rc-scripts svn, so that dep is no longer needed. -- glen From baggins at pld-linux.org Mon Aug 5 14:16:10 2013 From: baggins at pld-linux.org (Jan =?utf-8?Q?R=C4=99korajski?=) Date: Mon, 5 Aug 2013 14:16:10 +0200 Subject: [packages/rc-scripts] - just pull mksh, C on pdksh is confusing and we do want mksh here - rel 3 In-Reply-To: <51FF95B5.9060405@pld-linux.org> References: <85f8c821b3670866698595c17a7059c65349084a_refs_heads_master@pld-linux.org> <51FF95B5.9060405@pld-linux.org> Message-ID: <20130805121610.GX2122@sith.mimuw.edu.pl> On Mon, 05 Aug 2013, Elan Ruusam?e wrote: > On 04.08.2013 13:43, baggins wrote: > > commit b598c63cedbfd5a8be6b73cabae42046cc63cc23 > > Author: Jan R?korajski > > Date: Sun Aug 4 12:42:36 2013 +0200 > > > > - just pull mksh, C on pdksh is confusing and we do want mksh here > > - rel 3 > > > > rc-scripts.spec | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > --- > > diff --git a/rc-scripts.spec b/rc-scripts.spec > > index 63a5077..8e8f728 100644 > > --- a/rc-scripts.spec > > +++ b/rc-scripts.spec > > @@ -9,7 +9,7 @@ Summary(pl.UTF-8): inittab i skrypty startowe z katalogu /etc/rc.d > > Summary(tr.UTF-8): inittab ve /etc/rc.d dosyalar? > > Name: rc-scripts > > Version: 0.4.6 > > -Release: 2 > > +Release: 3 > > License: GPL v2 > > Group: Base > > #Source0: ftp://distfiles.pld-linux.org/src/%{name}-%{version}.tar.gz > > @@ -57,6 +57,7 @@ Requires: hostname > > Requires: iproute2 > > Requires: iputils-arping > > Requires: mingetty > > +Requires: mksh > > Requires: mktemp > > Requires: mount >= 2.12 > > Requires: procps >= 1:3.2.6-1.1 > > @@ -79,7 +80,6 @@ Conflicts: udev-core < 1:135-2 > > Conflicts: udev-core < 1:124-3 > > %endif > > Conflicts: lvm2 < 2.02.83 > > -Conflicts: pdksh < 5.2.14-58 > > Conflicts: upstart-SysVinit < 2.86-25 > > Conflicts: wpa_supplicant < 0.6.3 > > BuildRoot: %{tmpdir}/%{name}-%{version}-root-%(id -u -n) > > > > is the /etc/shells clearing bug fixed now? Worked fine for me (pdksh -> mksh-47 update). > with this change you propagate the pdksh -> mksh upgrade breaking > /etc/shells automatically to lots of systems (i.e which do poldek > --upgrade-dist automatically or semiautomatically)! > > with conflicts auto-upgrade wouldn't migrate and you had to poldek -u > mksh manually, which leaves you chance to ovelook /etc/shells contents. You have to know what to install first, I saw a few reports of confusing upgrades. > btw, the pdksh compatibility is restored in rc-scripts svn, so that dep > is no longer needed. No pdksh in Th, so that dep is needed. -- Jan R?korajski | PLD/Linux SysAdm | http://www.pld-linux.org/ bagginsmimuw.edu.pl bagginspld-linux.org From blues at pld-linux.org Tue Aug 6 15:29:44 2013 From: blues at pld-linux.org (=?ISO-8859-2?Q?Pawe=B3_Go=B3aszewski?=) Date: Tue, 6 Aug 2013 15:29:44 +0200 (CEST) Subject: [packages/rc-scripts] - just pull mksh, C on pdksh is confusing and we do want mksh here - rel 3 In-Reply-To: <20130805121610.GX2122@sith.mimuw.edu.pl> References: <85f8c821b3670866698595c17a7059c65349084a_refs_heads_master@pld-linux.org> <51FF95B5.9060405@pld-linux.org> <20130805121610.GX2122@sith.mimuw.edu.pl> Message-ID: On Mon, 5 Aug 2013, Jan R?korajski wrote: > > is the /etc/shells clearing bug fixed now? > > Worked fine for me (pdksh -> mksh-47 update). fine, but using double mksh install. I had to do this many times... -- pozdr. Pawe? Go?aszewski jid:bluesjabbergdapl -------------------------------------------------------------------------- If you think of MS-DOS as mono, and Windows as stereo, then Linux is Dolby Pro-Logic Surround Sound with Bass Boost and all the music is free. From glen at pld-linux.org Tue Aug 6 15:43:11 2013 From: glen at pld-linux.org (=?UTF-8?B?RWxhbiBSdXVzYW3DpGU=?=) Date: Tue, 06 Aug 2013 16:43:11 +0300 Subject: [packages/rc-scripts] - just pull mksh, C on pdksh is confusing and we do want mksh here - rel 3 In-Reply-To: References: <85f8c821b3670866698595c17a7059c65349084a_refs_heads_master@pld-linux.org> <51FF95B5.9060405@pld-linux.org> <20130805121610.GX2122@sith.mimuw.edu.pl> Message-ID: <5200FD6F.5070906@pld-linux.org> On 06.08.2013 16:29, Pawe? Go?aszewski wrote: > On Mon, 5 Aug 2013, Jan R?korajski wrote: >>> is the /etc/shells clearing bug fixed now? >> Worked fine for me (pdksh -> mksh-47 update). > fine, but using double mksh install. I had to do this many times... > pushed 44-8 which uses trigger to migrate. seems more common and worked here when i tested -- glen From baggins at pld-linux.org Tue Aug 6 19:23:42 2013 From: baggins at pld-linux.org (Jan =?utf-8?Q?R=C4=99korajski?=) Date: Tue, 6 Aug 2013 19:23:42 +0200 Subject: [packages/rc-scripts] - just pull mksh, C on pdksh is confusing and we do want mksh here - rel 3 In-Reply-To: References: <85f8c821b3670866698595c17a7059c65349084a_refs_heads_master@pld-linux.org> <51FF95B5.9060405@pld-linux.org> <20130805121610.GX2122@sith.mimuw.edu.pl> Message-ID: <20130806172341.GA1411@home> On Tue, 06 Aug 2013, Pawe? Go?aszewski wrote: > On Mon, 5 Aug 2013, Jan R?korajski wrote: > > > is the /etc/shells clearing bug fixed now? > > > > Worked fine for me (pdksh -> mksh-47 update). > > fine, but using double mksh install. I had to do this many times... No double install here, just worked. Current mksh has that scriptlet fixed. -- Jan R?korajski | PLD/Linux SysAdm | http://www.pld-linux.org/ bagginsmimuw.edu.pl bagginspld-linux.org From baggins at pld-linux.org Tue Aug 6 19:24:26 2013 From: baggins at pld-linux.org (Jan =?utf-8?Q?R=C4=99korajski?=) Date: Tue, 6 Aug 2013 19:24:26 +0200 Subject: [packages/rc-scripts] - just pull mksh, C on pdksh is confusing and we do want mksh here - rel 3 In-Reply-To: <5200FD6F.5070906@pld-linux.org> References: <85f8c821b3670866698595c17a7059c65349084a_refs_heads_master@pld-linux.org> <51FF95B5.9060405@pld-linux.org> <20130805121610.GX2122@sith.mimuw.edu.pl> <5200FD6F.5070906@pld-linux.org> Message-ID: <20130806172426.GB1411@home> On Tue, 06 Aug 2013, Elan Ruusam?e wrote: > On 06.08.2013 16:29, Pawe? Go?aszewski wrote: > > On Mon, 5 Aug 2013, Jan R?korajski wrote: > >>> is the /etc/shells clearing bug fixed now? > >> Worked fine for me (pdksh -> mksh-47 update). > > fine, but using double mksh install. I had to do this many times... > > > > pushed 44-8 which uses trigger to migrate. seems more common and worked > here when i tested Why don'y you push it to master then? 47 has the redirection bug fixed and works fine. -- Jan R?korajski | PLD/Linux SysAdm | http://www.pld-linux.org/ bagginsmimuw.edu.pl bagginspld-linux.org From glen at pld-linux.org Tue Aug 6 21:43:05 2013 From: glen at pld-linux.org (=?UTF-8?B?RWxhbiBSdXVzYW3DpGU=?=) Date: Tue, 06 Aug 2013 22:43:05 +0300 Subject: [packages/rc-scripts] - just pull mksh, C on pdksh is confusing and we do want mksh here - rel 3 In-Reply-To: <20130806172426.GB1411@home> References: <85f8c821b3670866698595c17a7059c65349084a_refs_heads_master@pld-linux.org> <51FF95B5.9060405@pld-linux.org> <20130805121610.GX2122@sith.mimuw.edu.pl> <5200FD6F.5070906@pld-linux.org> <20130806172426.GB1411@home> Message-ID: <520151C9.6050204@pld-linux.org> On 08/06/2013 08:24 PM, Jan R?korajski wrote: > On Tue, 06 Aug 2013, Elan Ruusam?e wrote: > >> On 06.08.2013 16:29, Pawe? Go?aszewski wrote: >>> On Mon, 5 Aug 2013, Jan R?korajski wrote: >>>>> is the /etc/shells clearing bug fixed now? >>>> Worked fine for me (pdksh -> mksh-47 update). >>> fine, but using double mksh install. I had to do this many times... >>> >> pushed 44-8 which uses trigger to migrate. seems more common and worked >> here when i tested > Why don'y you push it to master then? 47 has the redirection bug fixed > and works fine. > because i checked the tags and last th tag was on 44 so i assumed we are still on 44! -- glen From glen at pld-linux.org Tue Aug 6 21:46:53 2013 From: glen at pld-linux.org (=?UTF-8?B?RWxhbiBSdXVzYW3DpGU=?=) Date: Tue, 06 Aug 2013 22:46:53 +0300 Subject: [packages/rc-scripts] - just pull mksh, C on pdksh is confusing and we do want mksh here - rel 3 In-Reply-To: <20130806172426.GB1411@home> References: <85f8c821b3670866698595c17a7059c65349084a_refs_heads_master@pld-linux.org> <51FF95B5.9060405@pld-linux.org> <20130805121610.GX2122@sith.mimuw.edu.pl> <5200FD6F.5070906@pld-linux.org> <20130806172426.GB1411@home> Message-ID: <520152AD.2020401@pld-linux.org> On 08/06/2013 08:24 PM, Jan R?korajski wrote: > On Tue, 06 Aug 2013, Elan Ruusam?e wrote: > >> On 06.08.2013 16:29, Pawe? Go?aszewski wrote: >>> On Mon, 5 Aug 2013, Jan R?korajski wrote: >>>>> is the /etc/shells clearing bug fixed now? >>>> Worked fine for me (pdksh -> mksh-47 update). >>> fine, but using double mksh install. I had to do this many times... >>> >> pushed 44-8 which uses trigger to migrate. seems more common and worked >> here when i tested > Why don'y you push it to master then? 47 has the redirection bug fixed > and works fine. > and if it's fixed, why don't you remove the TODO from mksh.spec and write something to actual bugreport that you tested and it's fine. upstream waits for answers too! committed now to master: http://git.pld-linux.org/?p=packages/mksh.git;a=commitdiff;h=1c7e5b2c1d292665d65648f32f4e04d5eeabe496 -- glen From draenog at pld-linux.org Wed Aug 7 13:04:08 2013 From: draenog at pld-linux.org (Kacper Kornet) Date: Wed, 7 Aug 2013 13:04:08 +0200 Subject: Broken dhcpcd in th main Message-ID: <20130807110408.GA30753@camk.edu.pl> It looks like dhcpcd-6.0.2-1 from th-main has a problem with sending the correct hostname to the server. It is fixed in dhcpcd-6.0.5-1 from th-test. -- Kacper From baggins at pld-linux.org Wed Aug 7 13:38:19 2013 From: baggins at pld-linux.org (Jan =?utf-8?Q?R=C4=99korajski?=) Date: Wed, 7 Aug 2013 13:38:19 +0200 Subject: Broken dhcpcd in th main In-Reply-To: <20130807110408.GA30753@camk.edu.pl> References: <20130807110408.GA30753@camk.edu.pl> Message-ID: <20130807113819.GY2122@sith.mimuw.edu.pl> On Wed, 07 Aug 2013, Kacper Kornet wrote: > It looks like dhcpcd-6.0.2-1 from th-main has a problem with sending the > correct hostname to the server. It is fixed in dhcpcd-6.0.5-1 from th-test. Thanks, moved to th-main. -- Jan R?korajski | PLD/Linux SysAdm | http://www.pld-linux.org/ bagginsmimuw.edu.pl bagginspld-linux.org From jajcus at jajcus.net Thu Aug 8 09:56:10 2013 From: jajcus at jajcus.net (Jacek Konieczny) Date: Thu, 8 Aug 2013 09:56:10 +0200 Subject: Debugging rpm package installation order Message-ID: <20130808095610.3b4e6732@jajo.eggsoft> Hi, I am building a system image for a PLD-based distribution using poldek. It used to work well until I have upgraded my RPM from 4.5 to 5.4.12. Now the packages are installed in wrong order, %post scripts fail and the whole image build fails. poldek --verify=order gives a reasonable order: No loops -- OK Installation order: 0. FHS-2.3-35.aos1.i686 1. filesystem-4.0-11.aos1.i686 2. ldconfig-2.15-10.aos1.i686 3. libsepol-2.1.4-1.aos1.i686 4. libselinux-2.1.9-2.aos1.i686 5. setup-2.7.1-1.aos2.i686 6. pdksh-5.2.14-56.aos2.i686 7. glibc-2.15-10.aos1.i686 8. glibc-libcrypt-2.15-10.aos1.i686 9. SysVinit-tools-2.88-7.aos1.i686 10. zlib-1.2.7-1.aos1.i686 11. module-init-tools-3.16-4.aos1.i686 12. alsa-lib-1.0.25-1.aos1.i686 13. popt-1.16-1.aos1.i686 14. attr-2.4.46-1.aos1.i686 15. gmp-5.0.5-1.aos1.i686 16. libcap-libs-2.22-1.aos1.i686 17. libxcrypt-3.0.2-2.aos1.i686 18. make-3.81-1.aos2.i686 19. mawk-1.3.3-32.aos2.i686 20. cracklib-2.8.3-0.2.aos2.i686 21. cracklib-dicts-2.8.3-0.2.aos2.i686 22. gdbm-1.10-1.aos1.i686 23. sed-4.2.1-2.aos1.i686 24. pam-libs-1.1.5-7.aos2.i686 25. pam-1.1.5-7.aos2.i686 26. coreutils-8.16-1.aos1.i686 [...] 64. util-linux-2.21.1-1.aos4.i686 65. rc-scripts-0.4.5.4-2.aos1.i686 But the packages are installed in wrong order by rpm, so e.g. rc-scripts %post fails, because rc-scripts is installed long before coreutils. I guess this could be caused by some dependency problems in my packages. Poldek reports no loops. RPM 4.5 used to generate useful loop warnings in case a dependency loop was detected (the message stated exactly what packages are in the loop). The new RPM reports loops only when debug output is enabled (-vv), but I am not able to get any useful information from those messages: D: ========== tsorting packages (order, #predecessors, #succesors, tree, Ldepth, Rbreadth) D: 0 0 287 1 0 0 +FHS-2.3-35.aos1.i686 D: 1 0 0 3 0 1 +basesystem-2.99-8.aos1.i686 D: 2 0 0 2 0 2 +axeos-meta-nagios-vpbx-1.0-aos3.noarch D: 3 1 10 1 1 0 +dbus-dirs-1.6.0-1.aos1.i686 D: 4 1 2 1 1 1 +dhcp-client-dirs-4.0.2-4.aos1.i686 D: 5 1 2 1 1 2 +mibs-dirs-0.4.8-5.aos1.i686 D: 6 1 1 1 1 3 +kernel-axeos-pbx-firmware-3.7.9-aos3.i686 D: 7 1 1 1 1 4 +axeos-lsp-lib-3.1.4.79-aos1.noarch D: 8 1 1 1 1 5 +axeos-package-data-target_dir-3.1.30001-aos1.noarch D: 9 1 0 1 1 6 +busybox-static-1.19.3-1.aos4.i686 D: 10 1 0 1 1 7 +dmidecode-2.8-1.aos3.i686 D: 11 1 2 1 2 0 +mibs-net-snmp-5.6.1-4.aos3.i686 D: LOOP: D: removing iproute2-3.4.0-1.aos2.i686 "Requires(auto): /bin/sh" from tsort relations. D: removing pdksh-5.2.14-56.aos2.i686 "Requires: /usr/share/man/man1" from tsort relations. D: removing axeos-pbx-api-0.13-aos1.noarch "Requires: /etc/lighttpd/webapps.d" from tsort relations. D: removing lighttpd-1.4.32-1.aos1.i686 "Requires(pre): /bin/id" from tsort relations. D: removing coreutils-8.16-1.aos1.i686 "Requires: /usr/share/locale/es/LC_MESSAGES" from tsort relations. D: removing openssh-5.8p2-2.aos1.i686 "Requires: pam >= 0.99.7.1" from tsort relations. D: removing pam-1.1.5-7.aos2.i686 "Requires: /usr/share/locale/mr/LC_MESSAGES" from tsort relations. D: removing perl-Dahdi-2.4.1-2.aos2.rel2.i686 "Requires: dahdi-tools = 2.4.1-2.aos2.rel2" from tsort relations. D: removing dahdi-tools-2.4.1-2.aos2.rel2.i686 "Requires(auto): /bin/bash" from tsort relations. D: removing nagios-common-3.0.6-1.aos2.i686 "Requires(pre): /usr/sbin/usermod" from tsort relations. D: removing udev-197-4.aos1.i686 "Requires: udev-core = 1:197-4.aos1" from tsort relations. D: removing udev-core-197-4.aos1.i686 "Requires: /lib/udev/rules.d" from tsort relations. D: removing device-mapper-2.02.95-10.aos2.i686 "Requires: /sbin" from tsort relations. D: removing net-snmp-5.6.1-4.aos3.i686 "Requires(auto): libnetsnmp.so.25" from tsort relations. D: removing net-snmp-libs-5.6.1-4.aos3.i686 "Requires(auto): libnl.so.3" from tsort relations. D: removing libnl-3.0-3.aos2.i686 "Requires(auto): libc.so.6" from tsort relations. D: removing glibc-2.15-10.aos1.i686 "Requires: /usr/share/man/fr/man5" from tsort relations. D: removing nagios-nrpe-2.12-2.aos3.i686 "Requires(post): /sbin/chkconfig" from tsort relations. D: removing chkconfig-1.3.20-0.4.aos1.i686 "Requires: /usr/share/locale/uk/LC_MESSAGES" from tsort relations. D: removing tcpdump-4.1.1-3.aos1.i686 "Requires: libpcap >= 2:1.0.0" from tsort relations. D: removing libpcap-1.3.0-1.aos1.i686 "Requires: /usr/share/man/man7" from tsort relations. D: removing tar-1.26-1.aos1.i686 "Requires: /usr/share/man/pl/man1" from tsort relations. D: removing rc-scripts-0.4.5.4-2.aos1.i686 "Requires(auto): libglib-2.0.so.0" from tsort relations. D: removing glib2-2.34.3-2.aos3.i686 "Requires: /usr/share/locale/am/LC_MESSAGES" from tsort relations. D: removing axeos-pbx-snmp-0.3-aos1.i686 "Requires: /lib/systemd/system" from tsort relations. D: removing filesystem-4.0-11.aos1.i686 "Requires: /usr/share/man/pl" from tsort relations. How should I read those? Where is the loop? Greets, Jacek From n3npq at me.com Thu Aug 8 19:52:02 2013 From: n3npq at me.com (Jeffrey Johnson) Date: Thu, 08 Aug 2013 13:52:02 -0400 Subject: Debugging rpm package installation order In-Reply-To: <20130808095610.3b4e6732@jajo.eggsoft> References: <20130808095610.3b4e6732@jajo.eggsoft> Message-ID: <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> On Aug 8, 2013, at 3:56 AM, Jacek Konieczny wrote: > > Hi, > > I am building a system image for a PLD-based distribution using poldek. > It used to work well until I have upgraded my RPM from 4.5 to 5.4.12. > Now the packages are installed in wrong order, %post scripts fail and > the whole image build fails. > > poldek --verify=order gives a reasonable order: > > No loops -- OK > Installation order: > 0. FHS-2.3-35.aos1.i686 > 1. filesystem-4.0-11.aos1.i686 > 2. ldconfig-2.15-10.aos1.i686 > 3. libsepol-2.1.4-1.aos1.i686 > 4. libselinux-2.1.9-2.aos1.i686 > 5. setup-2.7.1-1.aos2.i686 > 6. pdksh-5.2.14-56.aos2.i686 > 7. glibc-2.15-10.aos1.i686 > 8. glibc-libcrypt-2.15-10.aos1.i686 > 9. SysVinit-tools-2.88-7.aos1.i686 > 10. zlib-1.2.7-1.aos1.i686 > 11. module-init-tools-3.16-4.aos1.i686 > 12. alsa-lib-1.0.25-1.aos1.i686 > 13. popt-1.16-1.aos1.i686 > 14. attr-2.4.46-1.aos1.i686 > 15. gmp-5.0.5-1.aos1.i686 > 16. libcap-libs-2.22-1.aos1.i686 > 17. libxcrypt-3.0.2-2.aos1.i686 > 18. make-3.81-1.aos2.i686 > 19. mawk-1.3.3-32.aos2.i686 > 20. cracklib-2.8.3-0.2.aos2.i686 > 21. cracklib-dicts-2.8.3-0.2.aos2.i686 > 22. gdbm-1.10-1.aos1.i686 > 23. sed-4.2.1-2.aos1.i686 > 24. pam-libs-1.1.5-7.aos2.i686 > 25. pam-1.1.5-7.aos2.i686 > 26. coreutils-8.16-1.aos1.i686 > [...] > 64. util-linux-2.21.1-1.aos4.i686 > 65. rc-scripts-0.4.5.4-2.aos1.i686 > > But the packages are installed in wrong order by rpm, so e.g. rc-scripts > %post fails, because rc-scripts is installed long before coreutils. > > I guess this could be caused by some dependency problems in my packages. > Poldek reports no loops. RPM 4.5 used to generate useful loop warnings > in case a dependency loop was detected (the message stated exactly what > packages are in the loop). The new RPM reports loops only when debug > output is enabled (-vv), but I am not able to get any useful > information from those messages: > > D: ========== tsorting packages (order, #predecessors, #succesors, > tree, Ldepth, Rbreadth) D: 0 0 287 1 0 0 > +FHS-2.3-35.aos1.i686 D: 1 0 0 3 0 1 > +basesystem-2.99-8.aos1.i686 D: 2 0 0 2 0 2 > +axeos-meta-nagios-vpbx-1.0-aos3.noarch D: 3 1 10 1 1 > 0 +dbus-dirs-1.6.0-1.aos1.i686 D: 4 1 2 1 1 1 > +dhcp-client-dirs-4.0.2-4.aos1.i686 D: 5 1 2 1 1 2 > +mibs-dirs-0.4.8-5.aos1.i686 D: 6 1 1 1 1 3 > +kernel-axeos-pbx-firmware-3.7.9-aos3.i686 D: 7 1 1 1 > 1 4 +axeos-lsp-lib-3.1.4.79-aos1.noarch D: 8 1 1 1 > 1 5 +axeos-package-data-target_dir-3.1.30001-aos1.noarch D: > 9 1 0 1 1 6 +busybox-static-1.19.3-1.aos4.i686 D: > 10 1 0 1 1 7 +dmidecode-2.8-1.aos3.i686 D: 11 > 1 2 1 2 0 +mibs-net-snmp-5.6.1-4.aos3.i686 What follows is a dependency LOOP with exactly the same information as before. > D: LOOP: > D: removing iproute2-3.4.0-1.aos2.i686 "Requires(auto): /bin/sh" from > tsort relations. D: removing pdksh-5.2.14-56.aos2.i686 > "Requires: /usr/share/man/man1" from tsort relations. D: removing > axeos-pbx-api-0.13-aos1.noarch "Requires: /etc/lighttpd/webapps.d" from > tsort relations. D: removing lighttpd-1.4.32-1.aos1.i686 > "Requires(pre): /bin/id" from tsort relations. D: removing > coreutils-8.16-1.aos1.i686 "Requires: /usr/share/locale/es/LC_MESSAGES" > from tsort relations. D: removing openssh-5.8p2-2.aos1.i686 "Requires: > pam >= 0.99.7.1" from tsort relations. D: removing > pam-1.1.5-7.aos2.i686 "Requires: /usr/share/locale/mr/LC_MESSAGES" from > tsort relations. D: removing perl-Dahdi-2.4.1-2.aos2.rel2.i686 > "Requires: dahdi-tools = 2.4.1-2.aos2.rel2" from tsort relations. D: > removing dahdi-tools-2.4.1-2.aos2.rel2.i686 "Requires(auto): /bin/bash" > from tsort relations. D: removing nagios-common-3.0.6-1.aos2.i686 > "Requires(pre): /usr/sbin/usermod" from tsort relations. D: removing > udev-197-4.aos1.i686 "Requires: udev-core = 1:197-4.aos1" from tsort > relations. D: removing udev-core-197-4.aos1.i686 > "Requires: /lib/udev/rules.d" from tsort relations. D: removing > device-mapper-2.02.95-10.aos2.i686 "Requires: /sbin" from tsort > relations. D: removing net-snmp-5.6.1-4.aos3.i686 "Requires(auto): > libnetsnmp.so.25" from tsort relations. D: removing > net-snmp-libs-5.6.1-4.aos3.i686 "Requires(auto): libnl.so.3" from tsort > relations. D: removing libnl-3.0-3.aos2.i686 "Requires(auto): > libc.so.6" from tsort relations. D: removing glibc-2.15-10.aos1.i686 > "Requires: /usr/share/man/fr/man5" from tsort relations. D: removing > nagios-nrpe-2.12-2.aos3.i686 "Requires(post): /sbin/chkconfig" from > tsort relations. D: removing chkconfig-1.3.20-0.4.aos1.i686 > "Requires: /usr/share/locale/uk/LC_MESSAGES" from tsort relations. D: > removing tcpdump-4.1.1-3.aos1.i686 "Requires: libpcap >= 2:1.0.0" from > tsort relations. D: removing libpcap-1.3.0-1.aos1.i686 > "Requires: /usr/share/man/man7" from tsort relations. D: removing > tar-1.26-1.aos1.i686 "Requires: /usr/share/man/pl/man1" from tsort > relations. D: removing rc-scripts-0.4.5.4-2.aos1.i686 "Requires(auto): > libglib-2.0.so.0" from tsort relations. D: removing > glib2-2.34.3-2.aos3.i686 "Requires: /usr/share/locale/am/LC_MESSAGES" > from tsort relations. D: removing axeos-pbx-snmp-0.3-aos1.i686 > "Requires: /lib/systemd/system" from tsort relations. D: removing > filesystem-4.0-11.aos1.i686 "Requires: /usr/share/man/pl" from tsort > relations. > > How should I read those? Where is the loop? > The easiest way to identify if a "fix" is functional is to use this macro to select exactly one edge to ignore in order to break the LOOP. # Relations between package names that cause dependency loops # with legacy packages that cannot be fixed. Relations are # specified as # p>q # where package p has a Requires: on something that package q Provides: # # XXX Note: that there cannot be any whitespace within the string "p>q", # and that both p and q are package names (i.e. no version/release). # %_dependency_whiteout_caos_core \ perl>perl-Filter \ pam>coreutils \ pam>initscripts \ glibc-common>glibc \ glibc>nscd \ filesystem>setup %_dependency_whiteout \ %{?_dependency_whiteout_caos_core} \ %{?_dependency_whiteout_system} \ %{nil} Note that "gibc-common>glibc" ensures that glibc-common is installed before glibc by reboving the edge in the dependency graph where glibc-common has Requires: glibc After identifying which edge to remove, then rearrange the packaging internals to avoid the dependency. E.g. combining 2 packages removes all edges by collapsing into a single package. Other rearrangements work the same way, making sure a Requirement: is matched from within some package internally. hth 73 de Jeff > Greets, > Jacek > _______________________________________________ > pld-devel-en mailing list > pld-devel-en at lists.pld-linux.org > http://lists.pld-linux.org/mailman/listinfo/pld-devel-en From glen at pld-linux.org Fri Aug 9 11:07:42 2013 From: glen at pld-linux.org (=?ISO-8859-1?Q?Elan_Ruusam=E4e?=) Date: Fri, 09 Aug 2013 12:07:42 +0300 Subject: Fwd: TEST build ERRORS: mysql.spec In-Reply-To: References: Message-ID: <5204B15E.9020105@pld-linux.org> can somebody have look and fix this? it works on carme. http://buildlogs.pld-linux.org//index.php?dist=th&arch=i686&ok=0&name=mysql&id=320df4c7-0361-4de8-9b2a-00797e67c696&action=tail -------- Original Message -------- Subject: TEST build ERRORS: mysql.spec Date: Fri, 09 Aug 2013 09:02:55 +0000 From: PLD th-i686 builder To: glen at pld-linux.org CC: pld-logs-th at lists.pld-linux.org mysql.spec (MYSQL_5_1): FAILED --- mysql.spec:MYSQL_5_1: Build-Time: user:495.22s sys:34.88s real:328.25s (faults io:107 non-io:15675910) *** buildlog for mysql.spec request from: glen checking if we should skip the build started at: Fri Aug 9 10:57:26 2013 fetching http://src.th.pld-linux.org//srpms/320df4c7-0361-4de8-9b2a-00797e67c696/mysql-5.1.70-14.8.1.src.rpm fetched 24301364 bytes, 2872.6 K/s installing srpm: mysql-5.1.70-14.8.1.src.rpm + install -d /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/RPMS + rpm -Uhv --nodeps --define '_topdir /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d' --define '_specdir %{_topdir}' --define '_sourcedir %{_specdir}' --define '_rpmdir %{_topdir}/RPMS' --define '_builddir %{_topdir}/BUILD' mysql-5.1.70-14.8.1.src.rpm Preparing... ################################################## mysql ################################################## + rm -f mysql-5.1.70-14.8.1.src.rpm + install -m 700 -d /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD/tmp + TMPDIR=/tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD/tmp + exec nice -n 0 rpmbuild -bp --short-circuit --nodeps --define '_topdir /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d' --define '_specdir %{_topdir}' --define '_sourcedir %{_specdir}' --define '_rpmdir %{_topdir}/RPMS' --define '_builddir %{_topdir}/BUILD' --target i686-pld-linux --define 'prep exit 0' /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql.spec Building target platforms: i686-pld-linux checking BuildConflict-ing packages no BuildConflicts found checking BR rpm: Building target platforms: i686-pld-linux no BR needed building RPM using: set -ex; : build-id: 320df4c7-0361-4de8-9b2a-00797e67c696; TMPDIR=/tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD/tmp exec nice -n 0 rpmbuild -bb --define '_smp_mflags -j9' --define '_topdir /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d' --define '_specdir %{_topdir}' --define '_sourcedir %{_specdir}' --define '_rpmdir %{_topdir}/RPMS' --define '_builddir %{_topdir}/BUILD' --target i686-pld-linux /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql.spec + : build-id: 320df4c7-0361-4de8-9b2a-00797e67c696 + TMPDIR=/tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD/tmp + exec nice -n 0 rpmbuild -bb --define '_smp_mflags -j9' --define '_topdir /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d' --define '_specdir %{_topdir}' --define '_sourcedir %{_specdir}' --define '_rpmdir %{_topdir}/RPMS' --define '_builddir %{_topdir}/BUILD' --target i686-pld-linux /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql.spec Building target platforms: i686-pld-linux Executing(%prep): /bin/sh -e /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD/tmp/rpm-tmp.58362 + umask 022 + cd /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD + cd /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD + rm -rf Percona-Server-5.1.70-rel14.8 + /usr/bin/gzip -dc /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/Percona-Server-5.1.70-rel14.8.tar.gz + /bin/tar -xf - + STATUS=0 + '[' 0 -ne 0 ']' + cd Percona-Server-5.1.70-rel14.8 + /usr/bin/gzip -dc /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/sphinx-0.9.9.tar.gz + /bin/tar -xf - + STATUS=0 + '[' 0 -ne 0 ']' + /bin/id -u + '[' 1001 '=' 0 ']' + /bin/id -u + '[' 1001 '=' 0 ']' + /bin/chmod -Rf a+rX,u+w,g-w,o-w . + mv sphinx-0.9.9/mysqlse storage/sphinx + echo 'Patch #18 (mysql-sphinx.patch):' Patch #18 (mysql-sphinx.patch): + '[' -f /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-sphinx.patch ']' + /bin/cat /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-sphinx.patch + /usr/bin/patch -s -p1 + echo 'Patch #0 (mysql-libs.patch):' Patch #0 (mysql-libs.patch): + '[' -f /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-libs.patch ']' + /bin/cat /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-libs.patch + /usr/bin/patch -s -p1 + echo 'Patch #5 (mysql-noproc.patch):' Patch #5 (mysql-noproc.patch): + '[' -f /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-noproc.patch ']' + /bin/cat /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-noproc.patch + /usr/bin/patch -s -p1 + echo 'Patch #6 (mysql-system-users.patch):' Patch #6 (mysql-system-users.patch): + '[' -f /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-system-users.patch ']' + /bin/cat /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-system-users.patch + /usr/bin/patch -s -p1 + echo 'Patch #8 (mysql-client-config.patch):' Patch #8 (mysql-client-config.patch): + '[' -f /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-client-config.patch ']' + /bin/cat /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-client-config.patch + /usr/bin/patch -s -p1 + echo 'Patch #9 (mysql-build.patch):' Patch #9 (mysql-build.patch): + '[' -f /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-build.patch ']' + /bin/cat /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-build.patch + /usr/bin/patch -s -p1 + echo 'Patch #11 (mysql-upgrade.patch):' Patch #11 (mysql-upgrade.patch): + '[' -f /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-upgrade.patch ']' + /bin/cat /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-upgrade.patch + /usr/bin/patch -s -p1 + echo 'Patch #12 (mysql-config.patch):' Patch #12 (mysql-config.patch): + '[' -f /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-config.patch ']' + /bin/cat /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-config.patch + /usr/bin/patch -s -p1 + echo 'Patch #14 (mysql-bug-43594.patch):' Patch #14 (mysql-bug-43594.patch): + '[' -f /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-bug-43594.patch ']' + /bin/cat /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-bug-43594.patch + /usr/bin/patch -s -p0 + echo 'Patch #15 (plugin-avoid-version.patch):' Patch #15 (plugin-avoid-version.patch): + '[' -f /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/plugin-avoid-version.patch ']' + /bin/cat /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/plugin-avoid-version.patch + /usr/bin/patch -s -p1 + echo 'Patch #16 (mysql-fix-dummy-thread-race-condition.patch):' Patch #16 (mysql-fix-dummy-thread-race-condition.patch): + '[' -f /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-fix-dummy-thread-race-condition.patch ']' + /bin/cat /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/mysql-fix-dummy-thread-race-condition.patch + /usr/bin/patch -s -p1 [...] sql_yacc.yy:38:14: note: in expansion of macro 'YYTHD' #define Lex (YYTHD->lex) ^ sql_yacc.yy:13877:7: note: in expansion of macro 'Lex' | RESUME_SYM { Lex->xa_opt=XA_RESUME; } ^ sql_yacc.yy:32:23: error: 'yythd' was not declared in this scope #define YYTHD ((THD *)yythd) ^ sql_yacc.yy:38:14: note: in expansion of macro 'YYTHD' #define Lex (YYTHD->lex) ^ sql_yacc.yy:13881:7: note: in expansion of macro 'Lex' /* nothing */ { Lex->xa_opt=XA_NONE; } ^ sql_yacc.yy:32:23: error: 'yythd' was not declared in this scope #define YYTHD ((THD *)yythd) ^ sql_yacc.yy:38:14: note: in expansion of macro 'YYTHD' #define Lex (YYTHD->lex) ^ sql_yacc.yy:13882:7: note: in expansion of macro 'Lex' | ONE_SYM PHASE_SYM { Lex->xa_opt=XA_ONE_PHASE; } ^ sql_yacc.yy:32:23: error: 'yythd' was not declared in this scope #define YYTHD ((THD *)yythd) ^ sql_yacc.yy:38:14: note: in expansion of macro 'YYTHD' #define Lex (YYTHD->lex) ^ sql_yacc.yy:13887:7: note: in expansion of macro 'Lex' { Lex->xa_opt=XA_NONE; } ^ sql_yacc.yy:32:23: error: 'yythd' was not declared in this scope #define YYTHD ((THD *)yythd) ^ sql_yacc.yy:38:14: note: in expansion of macro 'YYTHD' #define Lex (YYTHD->lex) ^ sql_yacc.yy:13889:7: note: in expansion of macro 'Lex' { Lex->xa_opt=XA_SUSPEND; } ^ sql_yacc.yy:32:23: error: 'yythd' was not declared in this scope #define YYTHD ((THD *)yythd) ^ sql_yacc.yy:38:14: note: in expansion of macro 'YYTHD' #define Lex (YYTHD->lex) ^ sql_yacc.yy:13895:7: note: in expansion of macro 'Lex' | FOR_SYM MIGRATE_SYM { Lex->xa_opt=XA_FOR_MIGRATE; } ^ sql_yacc.yy:32:23: error: 'yythd' was not declared in this scope #define YYTHD ((THD *)yythd) ^ sql_yacc.yy:38:14: note: in expansion of macro 'YYTHD' #define Lex (YYTHD->lex) ^ sql_yacc.yy:13901:23: note: in expansion of macro 'Lex' LEX *lex= Lex; ^ sql_yacc.yy:32:23: error: 'yythd' was not declared in this scope #define YYTHD ((THD *)yythd) ^ sql_yacc.yy:38:14: note: in expansion of macro 'YYTHD' #define Lex (YYTHD->lex) ^ sql_yacc.yy:13911:23: note: in expansion of macro 'Lex' LEX *lex= Lex; ^ make[3]: *** [sql_yacc.o] Error 1 make[3]: *** Waiting for unfinished jobs.... mv -f .deps/sql_prepare.Tpo .deps/sql_prepare.Po mv -f .deps/table.Tpo .deps/table.Po mv -f .deps/sql_insert.Tpo .deps/sql_insert.Po mv -f .deps/sql_base.Tpo .deps/sql_base.Po mv -f .deps/sql_parse.Tpo .deps/sql_parse.Po mv -f .deps/sql_error.Tpo .deps/sql_error.Po mv -f .deps/set_var.Tpo .deps/set_var.Po mv -f .deps/sql_select.Tpo .deps/sql_select.Po make[3]: Leaving directory `/tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD/Percona-Server-5.1.70-rel14.8/sql' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD/Percona-Server-5.1.70-rel14.8/sql' make[1]: *** [all] Error 2 make[1]: Leaving directory `/tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD/Percona-Server-5.1.70-rel14.8/sql' make: *** [all-recursive] Error 1 error: Bad exit status from /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD/tmp/rpm-tmp.21470 (%build) RPM build errors: Bad exit status from /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD/tmp/rpm-tmp.21470 (%build) ended at: Fri Aug 9 11:02:53 2013, done in 0:05:13.057317 error: No files produced. + chmod -R u+rwX /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD + rm -rf /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/tmp /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d/BUILD + rm -rf /tmp/B.5b700164-414a-47a4-a756-25b75f86d08d Begin-PLD-Builder-Info Build-Time: user:495.22s sys:34.88s real:328.25s (faults io:107 non-io:15675910) End-PLD-Builder-Info From jajcus at jajcus.net Fri Aug 9 13:55:30 2013 From: jajcus at jajcus.net (Jacek Konieczny) Date: Fri, 9 Aug 2013 13:55:30 +0200 Subject: Debugging rpm package installation order In-Reply-To: <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> References: <20130808095610.3b4e6732@jajo.eggsoft> <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> Message-ID: <20130809135530.038b36dd@jajo.eggsoft> On Thu, 08 Aug 2013 13:52:02 -0400 Jeffrey Johnson wrote: > What follows is a dependency LOOP with exactly the same information > as before. Before the RPM upgrade when I got a 'LOOP:' message, it contain two or three clearly related packages: error: LOOP: error: removing lighttpd-1.4.28-6.aos1.i686 "Requires(hint): lighttpd-mod_accesslog" from tsort relations. error: lighttpd-1.4.28-6.aos1.i686 Requires(hint): lighttpd-mod_accesslog error: removing lighttpd-mod_accesslog-1.4.28-6.aos1.i686 "Requires: lighttpd = 1.4.28-6.aos1" from tsort relations. error: lighttpd-mod_accesslog-1.4.28-6.aos1.i686 Requires: lighttpd = 1.4.28-6.aos1 Those were easy to understand and to fix. I have eventually fixed all the loop problems reported by rpm 4.5, although those problems usually were not fatal to the build process. Now I get a 'LOOP:' message with 30 packages, where relations between them are not clear. It is the same package set where RPM 4.5 reported no loops, the same package set where Poldek order verification finds no loops. Yet, RPM 5.4.12 from PLD reports a large loop and does the installation in wrong order. > The easiest way to identify if a "fix" is functional is to use this > macro to select exactly one edge to ignore in order to break the LOOP.a Exactly one edge of 30 problematic edges reported? And how is that a loop: D: LOOP: D: removing iproute2-3.4.0-1.aos2.i686 "Requires(auto): /bin/sh" from tsort relations. D: removing pdksh-5.2.14-56.aos2.i686 "Requires: /usr/share/man/man1" from tsort relations. D: removing axeos-pbx-api-0.13-aos1.noarch "Requires: /etc/lighttpd/webapps.d" from tsort relations. D: removing lighttpd-1.4.32-1.aos1.i686 "Requires(pre): /bin/id" from tsort relations. D: removing coreutils-8.16-1.aos1.i686 "Requires: /usr/share/locale/es/LC_MESSAGES" from tsort relations. D: removing openssh-5.8p2-2.aos1.i686 "Requires: pam >= 0.99.7.1" from tsort relations. D: removing pam-1.1.5-7.aos2.i686 "Requires: /usr/share/locale/mr/LC_MESSAGES" from tsort relations. [...] First two are clear: iproute2-3.4.0-1.aos2.i686 requires '/bin/sh', so it pulls pdksh? then pdksh-5.2.14-56.aos2.i686 requires '/usr/share/man/man1' and?? axeos-pbx-api-0.13-aos1.noarch does not provide '/usr/share/man/man1' (it does not even contain any manual pages), nor any of the 27 other packages listed below. Maybe the package dependencies are right, but the RPM dependency checking is broken for some reason? Any ideas? Greets, Jacek From qboosh at pld-linux.org Fri Aug 9 15:38:58 2013 From: qboosh at pld-linux.org (Jakub Bogusz) Date: Fri, 9 Aug 2013 15:38:58 +0200 Subject: Fwd: TEST build ERRORS: mysql.spec In-Reply-To: <5204B15E.9020105@pld-linux.org> References: <5204B15E.9020105@pld-linux.org> Message-ID: <20130809133858.GA18724@mail> On Fri, Aug 09, 2013 at 12:07:42PM +0300, Elan Ruusam?e wrote: > can somebody have look and fix this? it works on carme. > http://buildlogs.pld-linux.org//index.php?dist=th&arch=i686&ok=0&name=mysql&id=320df4c7-0361-4de8-9b2a-00797e67c696&action=tail [...] > > sql_yacc.yy:38:14: note: in expansion of macro 'YYTHD' > #define Lex (YYTHD->lex) > ^ > sql_yacc.yy:13877:7: note: in expansion of macro 'Lex' > | RESUME_SYM { Lex->xa_opt=XA_RESUME; } > ^ > sql_yacc.yy:32:23: error: 'yythd' was not declared in this scope > #define YYTHD ((THD *)yythd) > ^ bison 3? -- Jakub Bogusz http://qboosh.pl/ From n3npq at me.com Fri Aug 9 17:58:57 2013 From: n3npq at me.com (Jeffrey Johnson) Date: Fri, 09 Aug 2013 11:58:57 -0400 Subject: Debugging rpm package installation order In-Reply-To: <20130809135530.038b36dd@jajo.eggsoft> References: <20130808095610.3b4e6732@jajo.eggsoft> <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> <20130809135530.038b36dd@jajo.eggsoft> Message-ID: <65C3F9B2-493F-4621-986B-6A96E1506D78@me.com> On Aug 9, 2013, at 7:55 AM, Jacek Konieczny wrote: > >> The easiest way to identify if a "fix" is functional is to use this >> macro to select exactly one edge to ignore in order to break the LOOP.a > > Exactly one edge of 30 problematic edges reported? > Yes. Eliminating any edge eliminates the loop. The remaining 29 edges are usually sufficient to provide adequate ordering info to tsort. Also: Fix the 1st LOOP detected first. The errors cascade adding to complexity. > > Maybe the package dependencies are right, but the RPM dependency checking is > broken for some reason? Any ideas? > Maybe. I can say that the existing rpm implementation has been used by many for years without any change whatsoever. FIx your LOOP before guessing at what is broken. 73 de Jeff From jajcus at jajcus.net Fri Aug 9 22:39:23 2013 From: jajcus at jajcus.net (Jacek Konieczny) Date: Fri, 9 Aug 2013 22:39:23 +0200 Subject: Debugging rpm package installation order In-Reply-To: <65C3F9B2-493F-4621-986B-6A96E1506D78@me.com> References: <20130808095610.3b4e6732@jajo.eggsoft> <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> <20130809135530.038b36dd@jajo.eggsoft> <65C3F9B2-493F-4621-986B-6A96E1506D78@me.com> Message-ID: <20130809223923.34dd6d8f@lolek.nigdzie> On Fri, 09 Aug 2013 11:58:57 -0400 Jeffrey Johnson wrote: > Maybe. I can say that the existing rpm implementation has been used > by many for years without any change whatsoever. > > FIx your LOOP before guessing at what is broken. But THERE IS NO LOOP. RPM shows loop due to some imaginary dependency. Package A requires package B which requires /some/dir from package FHS. But my RPM shows: package A requires package B which requires /some/dir from package C which requires package D which requires /some/other/dir from package E which requires package A. And package C does not contain /some/dir, nor package E contains /some/other/dir. RPM sees dependencies which are not there, so it reports loop. Details in my previous mail Greets, Jacek From jajcus at jajcus.net Fri Aug 9 22:41:37 2013 From: jajcus at jajcus.net (Jacek Konieczny) Date: Fri, 9 Aug 2013 22:41:37 +0200 Subject: Debugging rpm package installation order In-Reply-To: <20130809135530.038b36dd@jajo.eggsoft> References: <20130808095610.3b4e6732@jajo.eggsoft> <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> <20130809135530.038b36dd@jajo.eggsoft> Message-ID: <20130809224137.704c6d65@lolek.nigdzie> Re-sending this mail? it was not sent for some reason? On Fri, 9 Aug 2013 13:55:30 +0200 Jacek Konieczny wrote: > First two are clear: > > iproute2-3.4.0-1.aos2.i686 requires '/bin/sh', so it pulls pdksh? > then pdksh-5.2.14-56.aos2.i686 requires '/usr/share/man/man1' and?? > > axeos-pbx-api-0.13-aos1.noarch does not provide > '/usr/share/man/man1' (it does not even contain any manual pages), > nor any of the 27 other packages listed below. > > Maybe the package dependencies are right, but the RPM dependency > checking is broken for some reason? Any ideas? Hmmm? I have added 'pdksh>axeos-pbx-api' to the '_dependency_whiteout' macro and I got a shorter loop: D: LOOP: D: removing iproute2-3.4.0-1.aos2.i686 "Requires: /sbin" from tsort relations. D: removing net-snmp-5.6.1-4.aos3.i686 "Requires(auto): libnetsnmp.so.25" from tsort relations. D: removing net-snmp-libs-5.6.1-4.aos3.i686 "Requires(auto): libnl.so.3" from tsort relations. D: removing libnl-3.0-3.aos2.i686 "Requires(auto): libc.so.6" from tsort relations. D: removing glibc-2.15-10.aos1.i686 "Requires: /usr/share/man/fr/man5" from tsort relations. D: removing nagios-nrpe-2.12-2.aos3.i686 "Requires(post): /sbin/chkconfig" from tsort relations. D: removing chkconfig-1.3.20-0.4.aos1.i686 "Requires: /usr/share/locale/uk/LC_MESSAGES" from tsort relations D: removing tcpdump-4.1.1-3.aos1.i686 "Requires: libpcap >= 2:1.0.0" from tsort relations. D: removing libpcap-1.3.0-1.aos1.i686 "Requires: /usr/share/man/man7" from tsort relations. D: removing tar-1.26-1.aos1.i686 "Requires: /usr/share/man/pl/man1" from tsort relations. D: removing rc-scripts-0.4.5.4-2.aos1.i686 "Requires: SysVinit-tools >= 2.88-1" from tsort relations. D: removing SysVinit-tools-2.88-7.aos1.i686 "Requires: /usr/share/man/cs/man8" from tsort relations. D: removing axeos-phone-configs-3.2.0-aos1.noarch "Requires(pre): axeos-services" from tsort relations. D: removing axeos-services-3.1-aos74.noarch "Requires: axeos-scripts-common >= 3.2.8" from tsort relations. D: removing axeos-scripts-common-3.2.25-aos1.i686 "Requires: /usr/bin/sensors" from tsort relations. D: removing lm_sensors-3.2.0-1.aos1.i686 "Requires: lm_sensors-config >= 3" from tsort relations. D: removing lm_sensors-config-default-3.2.0-1.aos1.i686 "Requires: /etc/sysconfig" from tsort relations. D: removing filesystem-4.0-11.aos1.i686 "Requires: /usr/share/man/pl" from tsort relations. Please note that all the 'phantom' edges (iproute2/net-snmp, glibc/nagios-nrpe, chkconfig/tcpdump, libpcap/tar/rc-scripts, SysVinit-tools/axeos-phone-configs, filesystem/iproute2) are via directories in the FHS package ? it looks like random package was used instead of 'FHS' when those dirs were required. Greets, Jacek From n3npq at me.com Fri Aug 9 22:49:28 2013 From: n3npq at me.com (Jeffrey Johnson) Date: Fri, 09 Aug 2013 16:49:28 -0400 Subject: Debugging rpm package installation order In-Reply-To: <20130809223923.34dd6d8f@lolek.nigdzie> References: <20130808095610.3b4e6732@jajo.eggsoft> <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> <20130809135530.038b36dd@jajo.eggsoft> <65C3F9B2-493F-4621-986B-6A96E1506D78@me.com> <20130809223923.34dd6d8f@lolek.nigdzie> Message-ID: <8149D072-D2E3-4A1E-BA3E-E3CF5FCD6038@me.com> On Aug 9, 2013, at 4:39 PM, Jacek Konieczny wrote: > On Fri, 09 Aug 2013 11:58:57 -0400 > Jeffrey Johnson wrote: >> Maybe. I can say that the existing rpm implementation has been used >> by many for years without any change whatsoever. >> >> FIx your LOOP before guessing at what is broken. > > But THERE IS NO LOOP. RPM shows loop due to some imaginary dependency. > Before shouting (or comparing to rpm-4.5 behavior), you need some additional information. Meanwhile you have reported a LOOP message (and you have already seen that the _dependency_whiteout macro changes/simplifies rpm behavior). > Package A requires package B which requires /some/dir from package FHS. > But my RPM shows: package A requires package B which requires /some/dir > from package C which requires package D which requires /some/other/dir > from package E which requires package A. And package C does not contain > /some/dir, nor package E contains /some/other/dir. RPM sees dependencies > which are not there, so it reports loop. Details in my previous mail > rpm-5.x has additional dependency rules: 1) every file "requires" its parent directory 2) every symlink "requires" its target end-point I put the "requires" in quote solely because yoi will _NOT_ see this information with rpm -qp --requires foo*.rpm queries. Otherwise the additional "requires" behave exactly like other Requires:, both as dependency assertions and as pre-requsites for topologically sorting. 73 de Jeff From jajcus at jajcus.net Sun Aug 11 21:19:12 2013 From: jajcus at jajcus.net (Jacek Konieczny) Date: Sun, 11 Aug 2013 21:19:12 +0200 Subject: Debugging rpm package installation order In-Reply-To: <8149D072-D2E3-4A1E-BA3E-E3CF5FCD6038@me.com> References: <20130808095610.3b4e6732@jajo.eggsoft> <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> <20130809135530.038b36dd@jajo.eggsoft> <65C3F9B2-493F-4621-986B-6A96E1506D78@me.com> <20130809223923.34dd6d8f@lolek.nigdzie> <8149D072-D2E3-4A1E-BA3E-E3CF5FCD6038@me.com> Message-ID: <20130811211912.5dc6c8b4@lolek.nigdzie> On Fri, 09 Aug 2013 16:49:28 -0400 Jeffrey Johnson wrote: > > But THERE IS NO LOOP. RPM shows loop due to some imaginary > > dependency. > > > > Before shouting (or comparing to rpm-4.5 behavior), It was a bit rude, indeed. But I am frustrated. And my mail with important details didn't get through to the list, I have not noticed it before I shouted? > Meanwhile you have reported a LOOP message (and you have already seen > that the _dependency_whiteout macro changes/simplifies rpm behavior). Yes, but I don't like using _dependency_whiteout macro applied to random packages. > rpm-5.x has additional dependency rules: > > 1) every file "requires" its parent directory > 2) every symlink "requires" its target end-point > > I put the "requires" in quote solely because yoi will _NOT_ see this > information with > rpm -qp --requires foo*.rpm > queries. Otherwise the additional "requires" behave exactly like > other Requires:, both as dependency assertions and as pre-requsites > for topologically sorting. I understand this behaviour. But it only explains the 'requires' parts of my ?phantom? dependencies. Look once more at this: D: LOOP: D: removing iproute2-3.4.0-1.aos2.i686 "Requires: /sbin" from tsort relations. D: removing net-snmp-5.6.1-4.aos3.i686 "Requires(auto): libnetsnmp.so.25" from tsort relations. D: removing net-snmp-libs-5.6.1-4.aos3.i686 "Requires(auto): libnl.so.3" from tsort relations. D: removing libnl-3.0-3.aos2.i686 "Requires(auto): libc.so.6" from tsort relations. D: removing glibc-2.15-10.aos1.i686 "Requires: /usr/share/man/fr/man5" from tsort relations. D: removing nagios-nrpe-2.12-2.aos3.i686 "Requires(post): /sbin/chkconfig" from tsort relations. [...] It is true, that iproute2 requires /sbin and that glibc requires /usr/share/man/fr/man5, as those packages contain files in this directories. But why does 'Requires: /sbin' pull 'net-snmp' and why 'Requires: /usr/share/man/fr/man5' pulls nagios-nrpe? Both '/sbin' and '/usr/share/man/fr/man5' are contained in the 'FHS' package. The requirements are valid, but RPM seems to pick random packages to satisfy them. $ rpm -qp --provides nagios-nrpe-2.14-1.aos1.i686.rpm config(nagios-nrpe) = 0:2.14-1.aos1 nagios-core nagios-nrpe = 0:2.14-1.aos1 $ rpm -qlp nagios-nrpe-2.14-1.aos1.i686.rpm /etc/nagios/nrpe.cfg /etc/nagios/nrpe.d /etc/rc.d/init.d/nrpe /usr/lib/tmpfiles.d/nagios-nrpe.conf /usr/sbin/nrpe /usr/share/doc/nagios-nrpe-2.14 /usr/share/doc/nagios-nrpe-2.14/Changelog.gz /usr/share/doc/nagios-nrpe-2.14/LEGAL.gz /usr/share/doc/nagios-nrpe-2.14/README.SSL.gz /usr/share/doc/nagios-nrpe-2.14/README.Solaris.gz /usr/share/doc/nagios-nrpe-2.14/README.gz /usr/share/doc/nagios-nrpe-2.14/SECURITY.gz /var/run/nrpe $ rpm -qp --provides net-snmp-5.7.2-1.aos2.i686.rpm config(net-snmp) = 0:5.7.2-1.aos2 snmpd net-snmp = 0:5.7.2-1.aos2 $ rpm -qlp net-snmp-5.7.2-1.aos2.i686.rpm /etc/init/snmpd.conf /etc/rc.d/init.d/snmpd /etc/snmp/snmpd.conf /etc/snmp/snmpd.local.conf /etc/sysconfig/snmpd /usr/bin/net-snmp-create-v3-user /usr/bin/sshtosnmp /usr/lib/snmp /usr/lib/snmp/dlmod /usr/sbin/snmpd /usr/share/doc/net-snmp-5.7.2 /usr/share/doc/net-snmp-5.7.2/AGENT.txt.gz /usr/share/doc/net-snmp-5.7.2/CHANGES.gz /usr/share/doc/net-snmp-5.7.2/COPYING.gz /usr/share/doc/net-snmp-5.7.2/ChangeLog.gz /usr/share/doc/net-snmp-5.7.2/EXAMPLE.conf.def.gz /usr/share/doc/net-snmp-5.7.2/EXAMPLE.conf.gz /usr/share/doc/net-snmp-5.7.2/FAQ.gz /usr/share/doc/net-snmp-5.7.2/NEWS.gz /usr/share/doc/net-snmp-5.7.2/README.agent-mibs.gz /usr/share/doc/net-snmp-5.7.2/README.agentx.gz /usr/share/doc/net-snmp-5.7.2/README.gz /usr/share/doc/net-snmp-5.7.2/README.snmpv3.gz /usr/share/doc/net-snmp-5.7.2/README.sql.gz /usr/share/doc/net-snmp-5.7.2/README.thread.gz /usr/share/doc/net-snmp-5.7.2/TODO.gz /usr/share/doc/net-snmp-5.7.2/local /usr/share/doc/net-snmp-5.7.2/local/FAQ2HTML.gz /usr/share/doc/net-snmp-5.7.2/local/Makefile.gz /usr/share/doc/net-snmp-5.7.2/local/Makefile.in.gz /usr/share/doc/net-snmp-5.7.2/local/README.mib2c.gz /usr/share/doc/net-snmp-5.7.2/local/Version-Munge.pl.gz /usr/share/doc/net-snmp-5.7.2/local/certgen-test.pl.gz /usr/share/doc/net-snmp-5.7.2/local/convertcode.gz /usr/share/doc/net-snmp-5.7.2/local/fixproc.gz /usr/share/doc/net-snmp-5.7.2/local/fixproc.made.gz /usr/share/doc/net-snmp-5.7.2/local/gittools /usr/share/doc/net-snmp-5.7.2/local/gittools/shell-functions.gz /usr/share/doc/net-snmp-5.7.2/local/html-add-header-footer.pl.gz /usr/share/doc/net-snmp-5.7.2/local/html-textfile-fix.pl.gz /usr/share/doc/net-snmp-5.7.2/local/ipf-mod.pl.gz /usr/share/doc/net-snmp-5.7.2/local/ipf-mod.pl.made.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/default-mfd-top.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/details-enums.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/details-node.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/details-table.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-ctx-copy.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-ctx-get.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-ctx-set.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-data-allocate.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-data-context.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-get-U64.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-get-char.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-get-decl-bot.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-get-decl.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-get-long.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-get-oid.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-header-bottom.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-header-top.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-source-includes.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-table-constants.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-table-enums.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-table-indexes-from-oid.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-table-indexes-set.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-table-indexes-to-oid.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-table-indexes-varbind-setup.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-table-indexes.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-table-oids.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-value-map-func.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-value-map-reverse.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/generic-value-map.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/m2c-internal-warning.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/m2c_setup_enum.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/m2c_setup_node.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/m2c_setup_table.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/m2c_table_save_defaults.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-access-container-cached-defines.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-access-unsorted-external-defines.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-data-access.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-data-get.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-data-set.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-doxygen.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-interactive-setup.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-interface.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-makefile.m2m.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-persistence.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-readme.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/mfd-top.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/node-get.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/node-set.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/node-storage.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/node-validate.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/node-varbind-validate.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/parent-dependencies.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/parent-set.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/subagent.m2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-COUNTER64-get.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-DateAndTime-get.m2d.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-DateAndTime-get.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-DateAndTime-readme.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-InetAddress-get.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-InetAddress-set.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-InetAddressType-get.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-InetAddressType-set.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-RowStatus-dependencies.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-RowStatus-get.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-RowStatus-varbind-validate.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-StorageType-dependencies.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-conf.d/syntax-TestAndIncr-get.m2i.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c-update.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.access_functions.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.array-user.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.check_values.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.check_values_local.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.column_defines.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.column_enums.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.column_storage.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.container.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.create-dataset.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.emulation.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.genhtml.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.int_watch.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.iterate.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.iterate_access.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.made.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.mfd.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.notify.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.old-api.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.perl.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.raw-table.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.row.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.scalar.conf.gz /usr/share/doc/net-snmp-5.7.2/local/mib2c.table_data.conf.gz /usr/share/doc/net-snmp-5.7.2/local/minimalist /usr/share/doc/net-snmp-5.7.2/local/minimalist/feature-check.gz /usr/share/doc/net-snmp-5.7.2/local/minimalist/feature-makedocs.gz /usr/share/doc/net-snmp-5.7.2/local/minimalist/feature-remove.gz /usr/share/doc/net-snmp-5.7.2/local/minimalist/find-unused-code.gz /usr/share/doc/net-snmp-5.7.2/local/minimalist/ignore.regexp.gz /usr/share/doc/net-snmp-5.7.2/local/minimalist/remove-unneeded-modules.gz /usr/share/doc/net-snmp-5.7.2/local/minimalist/removeifdefcode.pl.gz /usr/share/doc/net-snmp-5.7.2/local/minimalist/sizetests.gz /usr/share/doc/net-snmp-5.7.2/local/net-snmp-cert.conf.gz /usr/share/doc/net-snmp-5.7.2/local/net-snmp-cert.gz /usr/share/doc/net-snmp-5.7.2/local/net-snmp-cert.made.gz /usr/share/doc/net-snmp-5.7.2/local/pass_persisttest.gz /usr/share/doc/net-snmp-5.7.2/local/passtest.gz /usr/share/doc/net-snmp-5.7.2/local/passtest.pl.gz /usr/share/doc/net-snmp-5.7.2/local/snmp-bridge-mib.gz /usr/share/doc/net-snmp-5.7.2/local/snmp-bridge-mib.made.gz /usr/share/doc/net-snmp-5.7.2/local/snmp-ucd.sh.gz /usr/share/doc/net-snmp-5.7.2/local/snmpcheck.def.gz /usr/share/doc/net-snmp-5.7.2/local/snmpcheck.gz /usr/share/doc/net-snmp-5.7.2/local/snmpcheck.made.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmp-data /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmp-data/authopts.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmp-data/debugging.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmp-data/mibs.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmp-data/output.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmp-data/snmpconf-config.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmpd-data /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmpd-data/acl.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmpd-data/basic_setup.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmpd-data/extending.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmpd-data/monitor.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmpd-data/operation.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmpd-data/snmpconf-config.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmpd-data/system.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmpd-data/trapsinks.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmptrapd-data /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmptrapd-data/authentication.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmptrapd-data/formatting.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmptrapd-data/logging.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmptrapd-data/runtime.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmptrapd-data/snmpconf-config.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.dir/snmptrapd-data/traphandle.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.gz /usr/share/doc/net-snmp-5.7.2/local/snmpconf.made.gz /usr/share/doc/net-snmp-5.7.2/local/snmpdump.pl.gz /usr/share/doc/net-snmp-5.7.2/local/tkmib.gz /usr/share/doc/net-snmp-5.7.2/local/tkmib.made.gz /usr/share/doc/net-snmp-5.7.2/local/traptoemail.gz /usr/share/doc/net-snmp-5.7.2/local/traptoemail.made.gz /usr/share/man/man1/net-snmp-create-v3-user.1.gz /usr/share/man/man5/snmpd.conf.5.gz /usr/share/man/man5/snmpd.examples.5.gz /usr/share/man/man5/snmpd.internal.5.gz /usr/share/man/man5/variables.5.gz /usr/share/man/man8/snmpd.8.gz /usr/share/snmp/snmp_perl.pl /var/lib/net-snmp /var/log/snmpd.log Do you still think everything is ok here? Greets, Jacek From jajcus at jajcus.net Tue Aug 13 13:47:52 2013 From: jajcus at jajcus.net (Jacek Konieczny) Date: Tue, 13 Aug 2013 13:47:52 +0200 Subject: False positive dependency checks break package install order In-Reply-To: <20130809224137.704c6d65@lolek.nigdzie> References: <20130808095610.3b4e6732@jajo.eggsoft> <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> <20130809135530.038b36dd@jajo.eggsoft> <20130809224137.704c6d65@lolek.nigdzie> Message-ID: <20130813134752.78781b42@jajo.eggsoft> Hi, Trying to debug my package installation 'LOOP:' problems I have instrumented the RPM source code with many 'rpmlog(RPMLOG_DEBUG, ...)', so I could understand what is happening there. After hours of such debugging I found out what is going on: for each dependency rpmalSatisfiesDepend() is called from order.c to build the dependency graph. There, for every file dependency (package requires '/something') rpmalAllFileSatisfiesDepend() is called which finds out what packages provide the required file. And here is the problem: rpmalAllFileSatisfiesDepend() uses a Bloom filter look-up to find out if a package contain a file. But Bloom filter, to its very nature, gives false positives, which are not verified further. So, it can happen that rpmalSatisfiesDepend() returns wrong package for a required file or directory, which adds an extra edge to the dependency graph. RPM reports this as a 'LOOP' problem and tries to work-around it by changing the installation order, which totally breaks the order for me. The attached patch is a quick fix to the problem: after rpmbfChk() states a file is provided by a package, its answer is additionally verified by scanning the file index of the package. I am sure this can be implemented in a more elegant way, but I don't know the internal RPM architecture well enough and I cannot spend much more time on fixing this problem. Greets, Jacek -------------- next part -------------- A non-text attachment was scrubbed... Name: rpm-double_check_file_deps.patch Type: text/x-patch Size: 2481 bytes Desc: not available URL: From n3npq at me.com Tue Aug 13 16:44:25 2013 From: n3npq at me.com (Jeffrey Johnson) Date: Tue, 13 Aug 2013 10:44:25 -0400 Subject: False positive dependency checks break package install order In-Reply-To: <20130813134752.78781b42@jajo.eggsoft> References: <20130808095610.3b4e6732@jajo.eggsoft> <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> <20130809135530.038b36dd@jajo.eggsoft> <20130809224137.704c6d65@lolek.nigdzie> <20130813134752.78781b42@jajo.eggsoft> Message-ID: On Aug 13, 2013, at 7:47 AM, Jacek Konieczny wrote: > > Hi, > > Trying to debug my package installation 'LOOP:' problems I have > instrumented the RPM source code with many 'rpmlog(RPMLOG_DEBUG, ...)', > so I could understand what is happening there. > > After hours of such debugging I found out what is going on: for each > dependency rpmalSatisfiesDepend() is called from order.c to build the > dependency graph. There, for every file dependency (package requires > '/something') rpmalAllFileSatisfiesDepend() is called which finds out > what packages provide the required file. And here is the problem: > rpmalAllFileSatisfiesDepend() uses a Bloom filter look-up to find out > if a package contain a file. But Bloom filter, to its very nature, gives > false positives, which are not verified further. So, it can happen that > rpmalSatisfiesDepend() returns wrong package for a required file or > directory, which adds an extra edge to the dependency graph. RPM reports > this as a 'LOOP' problem and tries to work-around it by changing the > installation order, which totally breaks the order for me. > I see you had some fun. ;-) Just in case: This isn't pleasant code to debug: your analysis is largely correct, and the patch is mostly convincing. (aside) You should be calling rpmfiFree(alp->fi) but all objects have a common refcount so the patch happens to work. However the whole point of using a Bloom filter is/was to increase performance, and reduce memory, with some acceptable risk of false positives. FWIW, the file names are sorted, so a binary search could be done, if you absolutely insist on removing false positives. The linear search was attempted in rpm-3.0.4, rather slow. Meanwhile the simpler fix is to adjust the Bloom filter parameters to decrease the probability of a false positive. This can be done like so: Index: rpmfi.c =================================================================== RCS file: /v/rpm/cvs/rpm/lib/rpmfi.c,v retrieving revision 2.160.4.4 diff -p -u -w -r2.160.4.4 rpmfi.c --- rpmfi.c 4 Jun 2012 15:10:11 -0000 2.160.4.4 +++ rpmfi.c 13 Aug 2013 14:35:13 -0000 @@ -184,7 +184,7 @@ void * rpmfiFNBF(rpmfi fi) if (fi != NULL) { if (fi->_fnbf == NULL) { char * fn = (char *) alloca(fi->fnlen + 1); - static double e = 1.0e-4; + static double e = 1.0e-5; size_t n = (fi->fc > 10 ? fi->fc : 10); size_t m = 0; size_t k = 0; That reduces the probability of false positives from one in 10,000 to one in 100,000 and SHOULD fix your issue. One might also increase the estimated population "n" by 5-10%, but I would change "e" first as the intent is clearer. There is also some modest increase in memory used by the Bloom filter. See if that fixes your problem please. > The attached patch is a quick fix to the problem: after rpmbfChk() > states a file is provided by a package, its answer is additionally > verified by scanning the file index of the package. > > I am sure this can be implemented in a more elegant way, but I don't > know the internal RPM architecture well enough and I cannot spend much > more time on fixing this problem. > You pretty much know all there is to know once you have debugged a loop ordering issue like this ;-) 73 de Jeff From mmazur at kernel.pl Tue Aug 13 18:15:11 2013 From: mmazur at kernel.pl (Mariusz Mazur) Date: Tue, 13 Aug 2013 18:15:11 +0200 Subject: False positive dependency checks break package install order In-Reply-To: References: <20130808095610.3b4e6732@jajo.eggsoft> <20130813134752.78781b42@jajo.eggsoft> Message-ID: <201308131815.11680.mmazur@kernel.pl> On Tue of August 13 2013, Jeffrey Johnson wrote: > Meanwhile the simpler fix is to adjust the Bloom filter parameters to > decrease the probability of a false positive. Are you serious? You really think it's ok for rpm to be using algorithms with a non-zero chance of false positives? Are there any other places in the code with algos like this? --mmazur From n3npq at me.com Tue Aug 13 18:26:38 2013 From: n3npq at me.com (Jeffrey Johnson) Date: Tue, 13 Aug 2013 12:26:38 -0400 Subject: False positive dependency checks break package install order In-Reply-To: <201308131815.11680.mmazur@kernel.pl> References: <20130808095610.3b4e6732@jajo.eggsoft> <20130813134752.78781b42@jajo.eggsoft> <201308131815.11680.mmazur@kernel.pl> Message-ID: On Aug 13, 2013, at 12:15 PM, Mariusz Mazur wrote: > On Tue of August 13 2013, Jeffrey Johnson wrote: > >> Meanwhile the simpler fix is to adjust the Bloom filter parameters to >> decrease the probability of a false positive. > > Are you serious? You really think it's ok for rpm to be using algorithms with > a non-zero chance of false positives? > Yes. The Bloom filters been in use for about 3 years with no known problems, including randomized installations for 5-10 linux distros under a CI harness. THe performance/memory gains are significant. > Are there any other places in the code with algos like this? > Yes. Bloom filters are quite convenient. 73 de Jeff From baggins at pld-linux.org Tue Aug 13 19:55:25 2013 From: baggins at pld-linux.org (Jan =?utf-8?Q?R=C4=99korajski?=) Date: Tue, 13 Aug 2013 19:55:25 +0200 Subject: False positive dependency checks break package install order In-Reply-To: References: <20130808095610.3b4e6732@jajo.eggsoft> <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> <20130809135530.038b36dd@jajo.eggsoft> <20130809224137.704c6d65@lolek.nigdzie> <20130813134752.78781b42@jajo.eggsoft> Message-ID: <20130813175525.GA3182@home> On Tue, 13 Aug 2013, Jeffrey Johnson wrote: > > On Aug 13, 2013, at 7:47 AM, Jacek Konieczny wrote: > > > > > Hi, > > > > Trying to debug my package installation 'LOOP:' problems I have > > instrumented the RPM source code with many 'rpmlog(RPMLOG_DEBUG, ...)', > > so I could understand what is happening there. > > > > After hours of such debugging I found out what is going on: for each > > dependency rpmalSatisfiesDepend() is called from order.c to build the > > dependency graph. There, for every file dependency (package requires > > '/something') rpmalAllFileSatisfiesDepend() is called which finds out > > what packages provide the required file. And here is the problem: > > rpmalAllFileSatisfiesDepend() uses a Bloom filter look-up to find out > > if a package contain a file. But Bloom filter, to its very nature, gives > > false positives, which are not verified further. So, it can happen that > > rpmalSatisfiesDepend() returns wrong package for a required file or > > directory, which adds an extra edge to the dependency graph. RPM reports > > this as a 'LOOP' problem and tries to work-around it by changing the > > installation order, which totally breaks the order for me. > > > > I see you had some fun. ;-) > > Just in case: > This isn't pleasant code to debug: your analysis is largely correct, > and the patch is mostly convincing. > > (aside) > You should be calling rpmfiFree(alp->fi) but all objects have a common refcount > so the patch happens to work. > > However the whole point of using a Bloom filter is/was to increase performance, > and reduce memory, with some acceptable risk of false positives. > > FWIW, the file names are sorted, so a binary search could be done, if you absolutely > insist on removing false positives. The linear search was attempted in rpm-3.0.4, rather slow. > > Meanwhile the simpler fix is to adjust the Bloom filter parameters to decrease the probability > of a false positive. > > This can be done like so: > > Index: rpmfi.c > =================================================================== > RCS file: /v/rpm/cvs/rpm/lib/rpmfi.c,v > retrieving revision 2.160.4.4 > diff -p -u -w -r2.160.4.4 rpmfi.c > --- rpmfi.c 4 Jun 2012 15:10:11 -0000 2.160.4.4 > +++ rpmfi.c 13 Aug 2013 14:35:13 -0000 > @@ -184,7 +184,7 @@ void * rpmfiFNBF(rpmfi fi) > if (fi != NULL) { > if (fi->_fnbf == NULL) { > char * fn = (char *) alloca(fi->fnlen + 1); > - static double e = 1.0e-4; > + static double e = 1.0e-5; > size_t n = (fi->fc > 10 ? fi->fc : 10); > size_t m = 0; > size_t k = 0; > > That reduces the probability of false positives from one in 10,000 to one in 100,000 > and SHOULD fix your issue. One might also increase the estimated population "n" > by 5-10%, but I would change "e" first as the intent is clearer. > > There is also some modest increase in memory used by the Bloom filter. > > See if that fixes your problem please. Even if probablity reduction fixes the problem for now, I will apply Jacek's patch to rpm in PLD (with s/rpmbfFree/rpmfiFree/). Just changing Bloom filter parameters won't mean that the problem is gone, it will just make it harder to occur and I want stable installation order, always. -- Jan R?korajski | PLD/Linux SysAdm | http://www.pld-linux.org/ bagginsmimuw.edu.pl bagginspld-linux.org From jajcus at jajcus.net Tue Aug 13 20:08:38 2013 From: jajcus at jajcus.net (Jacek Konieczny) Date: Tue, 13 Aug 2013 20:08:38 +0200 Subject: False positive dependency checks break package install order In-Reply-To: References: <20130808095610.3b4e6732@jajo.eggsoft> <20130813134752.78781b42@jajo.eggsoft> <201308131815.11680.mmazur@kernel.pl> Message-ID: <20130813200838.2dc77cc4@lolek.nigdzie> On Tue, 13 Aug 2013 12:26:38 -0400 Jeffrey Johnson wrote: > Yes. The Bloom filters been in use for about 3 years with no known > problems, including randomized installations for 5-10 linux distros > under a CI harness. > > THe performance/memory gains are significant. > > > Are there any other places in the code with algos like this? > > > > Yes. Bloom filters are quite convenient. I am sure they are. For a _preliminary_ check. This can still a huge performance improvement. Let's say we have 1000 packages each 1000 files average and we are looking for 'a package containing given file'. And let's say the Bloom filter gives 1 in 1000 false positives on average and a cost of Bloom filter lookup is '2', while cost of file list scan comparision is '1'. 1. No filter: 1000 * 1000 = 1000000 comparisions, no false positives cost: 1000000 2. Bloom filter only: 1000 filter lookups, 2 false positives cost: 2000 500 times faster, but unreliable. 3. Bloom filter plus scan: 1000 filter lookups -> give us 2 'candidates' -> these require 2000 comparisions cost: 4000 Only twice slower than Bloom filter only and still 250 times faster than just scanning the file list. One could also estimate b-tree index lookup cost for this case. Greets, Jacek From n3npq at me.com Tue Aug 13 20:49:05 2013 From: n3npq at me.com (Jeffrey Johnson) Date: Tue, 13 Aug 2013 14:49:05 -0400 Subject: False positive dependency checks break package install order In-Reply-To: <20130813200838.2dc77cc4@lolek.nigdzie> References: <20130808095610.3b4e6732@jajo.eggsoft> <20130813134752.78781b42@jajo.eggsoft> <201308131815.11680.mmazur@kernel.pl> <20130813200838.2dc77cc4@lolek.nigdzie> Message-ID: <8FC2E9FD-D119-4633-843F-61882DFB8377@me.com> On Aug 13, 2013, at 2:08 PM, Jacek Konieczny wrote: > On Tue, 13 Aug 2013 12:26:38 -0400 > Jeffrey Johnson wrote: >> Yes. The Bloom filters been in use for about 3 years with no known >> problems, including randomized installations for 5-10 linux distros >> under a CI harness. >> >> THe performance/memory gains are significant. >> >>> Are there any other places in the code with algos like this? >>> >> >> Yes. Bloom filters are quite convenient. > > I am sure they are. For a _preliminary_ check. This can still a huge > performance improvement. > I've actually measured and reported the gains when implemented. Reality is always better than an estimate, though reality has a statistical probability of not achieving a desired coverage goal as well. > Let's say we have 1000 packages each 1000 files average and > we are looking for 'a package containing given file'. > > And let's say the Bloom filter gives 1 in 1000 false positives on > average and a cost of Bloom filter lookup is '2', while cost of > file list scan comparision is '1'. > > 1. No filter: > > 1000 * 1000 = 1000000 comparisions, no false positives > > cost: 1000000 > > 2. Bloom filter only: > > 1000 filter lookups, 2 false positives > > cost: 2000 > > 500 times faster, but unreliable. > > 3. Bloom filter plus scan: > > 1000 filter lookups -> give us 2 'candidates' -> these require 2000 > comparisions > > cost: 4000 > > Only twice slower than Bloom filter only and still 250 times faster than > just scanning the file list. > > One could also estimate b-tree index lookup cost for this case. > One could. But lets start with other analysis in parallele. False positives are perfectly acceptable if the consequences are minimal and the gains are significant. A false LOOP detected isn't fatal in any meaningful sense to RPM, just confusing to someone used to rpm-4.5 output. The Bloom filter implementation is also trivially tunable: I claim that there is a threshhold at which false LOOP's is indistinguishable with other errors, including packager induced failures. Whether that is one-in-a-million or one-in-a-billion needs to be decided through other means than the algorithm. But let's assume PLD package is exactly perfect (it isn't) and two false positives create a LOOP spontaneously. RPM deals with that LOOP by ignoring each of the edges while ordering, an utterly harmless failure mode. The likelier manifestation (because single rather than double incidence) is that a false positive creates a LOOP where none existed. All edges in the LOOP are ignored while ordering. The difference there is that useful information is discarded when all edges in the dependency LOOP are ignored. The core problem isn't whether Bloom filters have false positives, but that LOOP's are corrected by ignoring all edges in the LOOP, thereby discarding useful information. The other false positive failures of Bloom filters can be analyzed similarly. Meanwhile one of the major benefits of a compact/efficient sparse Bloom filter store for files is that the partial ordering -- which often fails because packagers fail to add necessary relations, aka missing data -- is augmented by the rules Every file "requires" its parent directory Every symlink "requires" its end point. The file system hierarchy is a DAG already, and using the 2 rules above adds determinism to package ordering transparently, thereby enhancing robustness and reliability of RPM installs. (aside) The first person who points out that a file system with symlinks need not be a DAG will be asked: Exactly what "real world" packaging purpose is served by adding symlinks that return ELOOP when traversed? I.e. its a packaging error to distribute a symlink LOOP. 73 de Jeff > Greets, > Jacek From n3npq at me.com Tue Aug 13 20:55:52 2013 From: n3npq at me.com (Jeffrey Johnson) Date: Tue, 13 Aug 2013 14:55:52 -0400 Subject: False positive dependency checks break package install order In-Reply-To: <20130813175525.GA3182@home> References: <20130808095610.3b4e6732@jajo.eggsoft> <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> <20130809135530.038b36dd@jajo.eggsoft> <20130809224137.704c6d65@lolek.nigdzie> <20130813134752.78781b42@jajo.eggsoft> <20130813175525.GA3182@home> Message-ID: On Aug 13, 2013, at 1:55 PM, Jan R?korajski wrote: > > Even if probablity reduction fixes the problem for now, I will apply Jacek's > patch to rpm in PLD (with s/rpmbfFree/rpmfiFree/). Just changing Bloom > filter parameters won't mean that the problem is gone, it will just make it > harder to occur and I want stable installation order, always. > See my other mail. You may do whatever you wish: the Bloom filter implementation was planned 5y ago, discussed @rpm5.org when implemented ~3y ago, and there were (at least) 2 members from PLD who had a chance to point out flaws in my reasoning, or to suggest a different approach. I also believe that you need to think through the consequences of false positives far more carefully before pledging allegience to PLD members patches. Again, you can/will do whatever you wish, I really do not care. You had timely responses with accurate information to reported problems from me. That is my only "obligation". 73 de Jeff From famille-boitelle at wanadoo.fr Thu Aug 15 13:16:23 2013 From: famille-boitelle at wanadoo.fr (Jacqueline BOITELLE) Date: Thu, 15 Aug 2013 13:16:23 +0200 (CEST) Subject: where i can download the iso of pld 3.0 x64 and pld titainium Message-ID: <472022143.4743.1376565383067.JavaMail.www@wwinf1d14> From mike at osdn.org.ua Thu Aug 15 23:00:24 2013 From: mike at osdn.org.ua (iso of pld 3.0 x64) Date: Fri, 16 Aug 2013 00:00:24 +0300 Subject: nice trolling! In-Reply-To: <472022143.4743.1376565383067.JavaMail.www@wwinf1d14> References: <472022143.4743.1376565383067.JavaMail.www@wwinf1d14> Message-ID: <20130815210024.GH24182@osdn.org.ua> From n3npq at me.com Fri Aug 16 20:31:53 2013 From: n3npq at me.com (Jeffrey Johnson) Date: Fri, 16 Aug 2013 14:31:53 -0400 Subject: False positive dependency checks break package install order In-Reply-To: References: <20130808095610.3b4e6732@jajo.eggsoft> <069BA45F-B865-4CF9-9AA7-AC4FAE783E3E@me.com> <20130809135530.038b36dd@jajo.eggsoft> <20130809224137.704c6d65@lolek.nigdzie> <20130813134752.78781b42@jajo.eggsoft> Message-ID: <4000B35B-3504-49F7-87D8-1F79CF4AE2C0@me.com> On Aug 13, 2013, at 10:44 AM, Jeffrey Johnson wrote: > > Meanwhile the simpler fix is to adjust the Bloom filter parameters to decrease the probability > of a false positive. > > This can be done like so: > > Index: rpmfi.c > =================================================================== > RCS file: /v/rpm/cvs/rpm/lib/rpmfi.c,v > retrieving revision 2.160.4.4 > diff -p -u -w -r2.160.4.4 rpmfi.c > --- rpmfi.c 4 Jun 2012 15:10:11 -0000 2.160.4.4 > +++ rpmfi.c 13 Aug 2013 14:35:13 -0000 > @@ -184,7 +184,7 @@ void * rpmfiFNBF(rpmfi fi) > if (fi != NULL) { > if (fi->_fnbf == NULL) { > char * fn = (char *) alloca(fi->fnlen + 1); > - static double e = 1.0e-4; > + static double e = 1.0e-5; > size_t n = (fi->fc > 10 ? fi->fc : 10); > size_t m = 0; > size_t k = 0; > > That reduces the probability of false positives from one in 10,000 to one in 100,000 > and SHOULD fix your issue. One might also increase the estimated population "n" > by 5-10%, but I would change "e" first as the intent is clearer. > > There is also some modest increase in memory used by the Bloom filter. > > See if that fixes your problem please. > Was the above patch ever tested? I will assume No. 73 de Jeff From qboosh at pld-linux.org Sun Aug 25 14:30:51 2013 From: qboosh at pld-linux.org (Jakub Bogusz) Date: Sun, 25 Aug 2013 14:30:51 +0200 Subject: qt5 packaging Message-ID: <20130825123051.GA23786@mail> Qt 5 is available for some time, but we don't have it yet (how can it be?). Any packaging tries or ideas so far? 1) split (submodules) or single source package? I'd use split. 2) source package names: qt5-${name}, qt5-${name#qt} or qt5${name#qt}? (i.e. qt5-qtbase, qt5-base, qt5base for qtbase tarball?) 3) binary packages Qt5* (Qt5Core, Qt5Gui, Qt5Widgets etc.) for library packages? (following pkgconfig feature names and our qt4 packaging scheme) qt5-* (qt5-qmake, qt5-build etc.) for utility packages? -- Jakub Bogusz http://qboosh.pl/ From glen at pld-linux.org Sun Aug 25 15:15:09 2013 From: glen at pld-linux.org (=?ISO-8859-1?Q?Elan_Ruusam=E4e?=) Date: Sun, 25 Aug 2013 16:15:09 +0300 Subject: qt5 packaging In-Reply-To: <20130825123051.GA23786@mail> References: <20130825123051.GA23786@mail> Message-ID: <521A035D.2070709@pld-linux.org> On 08/25/2013 03:30 PM, Jakub Bogusz wrote: > how can it > be? nobody interested working on it? -- glen From draenog at pld-linux.org Tue Aug 27 22:40:06 2013 From: draenog at pld-linux.org (Kacper Kornet) Date: Tue, 27 Aug 2013 22:40:06 +0200 Subject: [ANN] New web interface for PLD git repositories Message-ID: <20130827204005.GA21324@camk.edu.pl> There is now a cgit instance installed and configured on http://git.pld-linux.org/. So if for some reason you don't like the gitweb engine you can check the alternative at: http://git.pld-linux.org/cgi-bin/cgit.cgi -- Kacper