rpm -Uhv --oldpackage loses configs

Wed Jun 8 15:50:11 CEST 2016

> On Jun 8, 2016, at 5:58 AM, Tomasz Pala <gotar at polanet.pl> wrote:
> 
> On Wed, Jun 08, 2016 at 00:10:36 -0400, Jeffrey Johnson wrote:
> 
>>> http://lists.rpm.org/pipermail/rpm-maint/2008-February/001911.html
>> 
>> OK, I???ll bite: you quote James Antill from 8 years ago ??? what are you trying to say? Please ???
>> 
>>> FS-level snapshots are shortest way to 'reboot to upgrade' insanity.
>> 
>> So you seem to not like ???FS snapshots??? ??? Good, I think FS snapshots have little to
>> do with ???rollback too, we may possibly agree. FS snapshots have a usage case,
>> but not with ???rollback for package management.
> 
> I'm just trying to figure out what mechanisms are you suggesting instead
> repackages - I'm using snapshots, but for other purposes, glad we agree!
> 

I am suggesting that storing possibly gigabytes of repackaged data on
every client disk is rather silly for largely identical redundant content.

Consider a data center, with dozens of racks and thousands of disks,
all being maintained with automated package management. The usage
case when the datacenter is hosting dozens of VM’s is even more
obvious when you look at repackaging.

Repackaging erased content into /var/spool/repackage isn’t close
to being the right engineering for a data center usage case, and
even for smaller SOHO deployments, the multiple gigabytes
of largely redundant identical content are rather pointless.

>> All I meant to suggest is that the PLD devel list is _NOT_ the pace to discuss RPM package management
>> issues (including repackaging, and ???rollback) seriously.
> 
> OK, listen - I am really only interested in working alternatives if you know any.
> Not digging into internals, glad we agree!
> 

Working alternatives to what? Repackaging or rollback?

Repackaging SHOULD be handled by returning to the original rpm content and preserving
a per-machine delta patch of modified (usually configuration) content. Recreating some
machine state from *.rpm blobs and a client delta is not rocket science with tools
like mongoldb/git.

Rollback (and ACID behavior for package/content management) is usually implemented by
serializing the sequence of invertible (i.e. rpm -e is the inverse of rpm -i and vice versa) actions
in a log and then replaying committed transactions forwards/backwards. That too has flaws:
	1) the log can/will require as much space as the content being managed
	2) replaying the log involves rewriting Gb of data and compression eats cpu cycles.
What everyone really wants is the ability to cherry pick
	Hmm that upgrade broke something, lets backup to last known good for some packages.
and the serialization implemented by ACID and logs is rather massively useless for “cherry picking”.

(aside)
There’s likely a master’s thesis (and perhaps a company or career) available to someone who can
devise a partial ACID rollback under constraints like preserving dependency closure. I’ve searched
for an answer to how to compute a subset partial ACID rollback (i.e. cherry picking just those
packages that need to be replaced to get rid of a few bad packages) without success.

>>>> This thread is/was diagnosing a problem and devising a solution, not otherwise.
>>> 
>>> Different approach to entire repackage (that YOU suggested) is also a solution.
>> 
>> What are you trying to say here? Do you like/want repackaging or not? I did successfully implement
>> repackaging (what is currently used by PLD) while @redhat in like 2003 or so.
> 
> Surely I do! In fact, if it were gone like in RH, I would be probably
> not using PLD anymore, because it would be unusable for me.
> 

I’m glad you like repackaging as implemented.

>> I???m happy you like what I implemented, but damfino what you wish changed.
> 
> "There are also better solutions than /var/spool/repackage that can be
> attempted these days"
> 
> I wish to only know these better solutions you've mentioned. Nothing
> more. Did you mean something in RPM internals? Or some other mechanism?
> Tool? Or maybe you meant %config files in repackages are not so
> important, because this particular issue should be solved via git?
> 

RPM carries the mongo-c-driver and libgit2 (and a butt load of creepy-toe to ensure
no tampering) in order to attempt a better solution to rollback and repository management.

The harder issue is setting up a server side infrastructure from which package metadata
and file content can be retrieved. The server-side schema’s are not very complex
	header’s always have a SHA1 associated that can be used as part of a retrieval key
	files always have a digest that can be used as part of a retrieval key
The digests should not be used directly as a retrieval key, but rather should be
wrapped into a UUID with an administrative name space so that, say, PLD and Poky/Yocto
could each have their own unique retrieval key even when the actual content is identical.

Given a reliable distributed server indexed as above, the need for a local /var/spool/repackage
disappears. And as long as the original content (not just the unmodified content digest) is accessible,
then recreating a *.rpm package when needed, or even creating a drpm delta package, can be done
as needed.

Most of the implementations (like the mongo-c-driver and libgit2 and the UUID generation and
the ability to generate/sign/verify signatures on blobs) to go after an exploded package Big Data
Server store exist in RPM5 for several years now, and I have prototyped various pieces of the
client puzzle as time permits.

What I haven’t done is integrate any of the development into RPM “production” code paths. Yet.

My development has been underway for quite some time now, called GUPPI
	Globally Unique Persistent Package Identifiers
Think: like a git commit, most modern packages have a SHA1, the rest is just a content store.

>> Um, a yum-history citation, and from 2010??? which depends on what is called a ???yumdb??? ???
>> is mostly useless, perhaps more useless than FS snapshots that you seem to dislike.
> 
> So, once again, we agree!
> 
>> You???ll have to remind me of what I wrote, I have written lots about RPM over the years ???
> 
> Oh, it was just one year ago - almost exactly (yet referring to May 2003):
> https://lists.pld-linux.org/mailman/pipermail/pld-devel-en/2015-June/024388.html
> But this is about %config - now just seeking if anything new has appeared in
> those years in regard for entire upgrade transaction, that would make my current workflow obsolete.
> 
>> Please don???t take my comments personally: I???ll be happy to discuss better RPM
>> implementations as you wish in an appropriate forum related to RPM, not PLD.
> 
> I like the current implementation (repackages) and had an impression,
> that you are proposing _some_ alternative. If, by saing "can be
> attempted", you had in mind "new mechanism can be _developed_", than
> yes, it's OT for pld-devel. It just wasn't clear for me, if you suggest
> attempting to USE _some_ better solution, or attempting to DEVELOP one.
> 
> We seem to have reached a consensus on this matter.
> 

I hope the above clarifies where I am headed at least somewhat.

We now return the thread to discussions of %config actions on downgrades to repackaged packages …

73 de Jeff

> -- 
> Tomasz Pala <gotar at pld-linux.org>
> _______________________________________________
> pld-devel-en mailing list
> pld-devel-en at lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en