pld rpm 5.4.17

Jeffrey Johnson n3npq at me.com
Wed Mar 1 05:18:01 CET 2017


> On Feb 28, 2017, at 3:42 PM, Jakub Bogusz <qboosh at pld-linux.org> wrote:
> 
> On Thu, Feb 23, 2017 at 07:15:22PM -0500, Jeffrey Johnson wrote:
>> 
>>> On Feb 23, 2017, at 3:36 PM, Jakub Bogusz <qboosh at pld-linux.org> wrote:
>>> 
>>> On Thu, Feb 23, 2017 at 01:43:14PM -0500, Jeffrey Johnson wrote:
>>>> This one is left though:
>>>> 
>>>>> error: db3: header #187105280 cannot be loaded -- skipping.
>>>>> error: db3: header #4127850496 cannot be loaded -- skipping.
>>>> 
>>>> How to check what these "headers" mean?
>>>> (old, unsupported keys? some old packages with missing fields which are
>>>> now required?)
>>>> The error message is printed on a headerCopyLoad() failure.
>>>> 
>>>> The failure is usually an indication of header damage of some sort, not missing fields now required.
>>>> 
>>>> The numbers  are  primary keys into Packages printed in in decimal.
>>>> 
>>>> The values appear to be in the wrong-endian order when converted to hex
>>>> 	0xB270000
>>>> 	0xF60A0000
>>>> 
>>>> You can try finding the header by doing, say, rpm -q ???querybynumber 0xB270000
>>>> (or its reverse: its unclear what order).
>>> 
>>> Same result for bigger values, no result after swapping bytes:
>>> 
>>> -bash-4.4# rpm -q --querybynumber 0xF60A0000
>>> error: rpmdb: header #4127850496 cannot be loaded -- skipping.
>>> -bash-4.4# rpm -q --querybynumber 0x00000AF6
>>> -bash-4.4# rpm -q --querybynumber 0xB270000
>>> error: rpmdb: header #187105280 cannot be loaded -- skipping.
>>> -bash-4.4# rpm -q --querybynumber 0x0000270B
>>> -bash-4.4#
>>> 
>> 
>> So at least simply reproducible with ???querybynumber ;-)
>> 
>> If you just need a fix, db_dump will give you {KEY, VALUE} pairs in hex.
>> Find the 2 {KEY,VALUES} and delete the 2 occurrences of 2 lines of hex in a text editor.
>> Feed the result to db_load to recreate Packages.
>> rpm ???rebuild will recreate the indices.
> 
> I investigated these two entries. They doesn't seem to be invalid, but
> are not accepted after this change:
> 
> http://rpm5.org/cvs/filediff?f=rpm/rpmdb/header.c&v1=1.198.2.23&v2=1.198.2.24 <http://rpm5.org/cvs/filediff?f=rpm/rpmdb/header.c&v1=1.198.2.23&v2=1.198.2.24>
> 
> more precisely, this part:
> 
> -/*@=sizeoftype@*/
> -               rpmuint32_t * stei = (rpmuint32_t *)
> -                       memcpy(alloca(nb), dataStart + off, nb);
> +               /* XXX copy to fix alignment problems */
> +                rpmuint32_t * stei = (rpmuint32_t *)
> +                          memcpy(alloca(nb), dataStart + off, nb);
> +               if ((off + nb) > dl)
> +                   goto errxit;
>                rdl = (rpmuint32_t)-ntohl(stei[2]);     /* negative offset */
> -assert((rpmint32_t)rdl >= 0);  /* XXX insurance */
> -               ril = (rpmuint32_t)(rdl/sizeof(*pe));
> -               if (hdrchkTags(ril) || hdrchkData(rdl))
> +               if (rdl < REGION_TAG_COUNT || rdl > (rpmuint32_t)(off+nb))
>                    goto errxit;
> 
> (nb. IMO "if ((off + nb) > dl)" check should be done before memcpy…)

Ah yes … fixing problems with AFL (American Fuzzy Lop) was totally annoying there.
You can tell because of the remnant printf’s that do not align with generally accepted
C coding practices.

Meanwhile, we may be working at cross purposes, with packages that have already been installed
by older (or heavily patched) versions of rpm.

What SHOULD have prevented that problem is verifying (always) the header SHA1 while installing
which ensures the header was structurally intact, preventing the later failure of headerCopyLoad() that
was displayed as an error message.

But perhaps there is something better that can be done: O looked at that specific code path
for a couple of days with AFL fuzzed headers: what you see is my best effort (by _ALWAYS_
verifying the header SHA1. The risk of adding stricter tests is that previously known good
headers might fail.

If you have a better solution (other than disabling stricter tests), I’ll be happy to add to

> 1) package #0xB270000 the first entry of header is:
> 0000003f 00000007 00000280 00000010
> it points to data:
> dataStart+0280  0000003f 00000007 fffffca0 00000010
> so rdl=0x360, which is bigger than off+nb (0x280+0x10)
> but AFAICS rdl is pointer inside header part, not data…
> 

This is broken.

0x3f is a HEADER_IMMUTABLE marker whose length indicates the
part of a header that never (i.e. same in *.rpm and in rpmdb) changes, and is matched
by a trailer that duplicates the size of a the “immutable region”.

I cannot tell how that header was produced, nor guess if there was a big in some version of RPM,
without further information.


> 2) similarly, package #0xF60A0000 has first header entry:
> 0000003f 00000007 00000378 00000010
> which points to data:
> 
> dataStart+0378 0000003f 00000007 fffffc70 00000010
> and rdl=0x390 > 0x378+0x10
> 
> The rest of packages I've got installed seem to have longer data in
> "immutable" part and pass this check.
> 

Its always possible that there is some obscure bug in some version of RPM.

If you can pin down the value of RPMTAG_RPMVBERSION (and possibly associated patches),
I can try to do the necessary forensics.

Thanks for the report: I am not infallible (but have run like 30 billion tests over several weeks with AFL
fuzzed headers trying to ensure that rpm-5.4.17 is as fault free as possible).

hah

73 de Jeff
> 
> -- 
> Jakub Bogusz    http://qboosh.pl/ <http://qboosh.pl/>
> _______________________________________________
> pld-devel-en mailing list
> pld-devel-en at lists.pld-linux.org <mailto:pld-devel-en at lists.pld-linux.org>
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en <http://lists.pld-linux.org/mailman/listinfo/pld-devel-en>


More information about the pld-devel-en mailing list