Just like I announced last week, my binary packages repository is now history. I’m gonna remove the redirections from lighttpd and DNS in a few.
Category: Gentoo
packages.barfoo.org is going away
For those of you, still using my binary packages. It’s just a waste of disk space for me (6.8G to be exact), so I decided to remove them. I’m gonna give people one week to grab yourself a copy. I’m gonna keep the bashrc and all the other stuff I wrote back when I was still interested in binary packages, but the binary packages are gonna vanish!
Again, grab yourself a copy if you need them, at some point next week (probably on Friday), I’m simply gonna rm -rf them.
Looong time
Well, it’s been a loong time since I first thought about retiring (yes, I know that #-dev’s topic states “developer blogs” ain’t for announcing important things, but my blog has to do for this; if not, I don’t care anymore ❗ ).
But I think it’s about time for me to leave. I haven’t done much lately, I’ve been soo damn busy with work these last months like I would never have imagined. I gave it some more thought, and I finally got to the point (again I might add) where all just annoys and/or frustrates me.
All the damn bickering, the childish behaviour Josh talked about (“noooo, that’s my TOY!“), the constant abuse of power (hey #-chat, #-kde ops). I thought most of us would at least try and behave like the elderish people we are (well besides the few of us, who really are children by law’s definition – hey there welp, omp, keytoaster 😉 ). But I guess that’s just been an imaginary thought I had .. *shrug*
Retiring people
I’m not sure whether or not I blogged about this before, but here it is just for me to actually remember what, in which order I need to do. If you got the list in form of a csv file, simply do the following:
|
1 2 3 4 |
wget -q http://tinyurl.com/4atkz7 -O - | grep "(" | cut -d'(' -f2 | cut -d')' -f1 | awk '{ print tolower($1) }' > retirements for developer in $( < retirements ); do retire.py --metadata $developer /cvs/gentoo-x86/ | diffstat > ~/metadata.$developer done |
That’ll give you a detailed list of which metadata.xml need to be changed.
metadata.xml (the third)
So Petteri came up with a nifty python script (local), which in return spit out this. Which generated a rather complete list (local), that looks like this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
700: perl 569: maintainer-needed 128: media-video 126: xemacs 47: sound 32: ha-cluster 32: crypto 19: desktop-misc 16: netmon 15: forensics 13: web-apps 10: pam-bugs 8: vserver-devs 8: mips 8: embedded 8: app-backup 8: apache-bugs 8: alsa-bugs 7: net-im 7: kde 6: tcltk 6: media-tv 6: dev-embedded 5: voip 5: theology 5: samba 5: net-p2p 5: freedesktop-bugs 4: sparc 4: java 4: graphics 2: net-mail 2: ldap-bugs 2: kernel 2: fonts 2: cpp 1: x11 1: wxwidgets 1: www-servers 1: tex 1: shell-tools 1: sgml 1: sci 1: qmail-bugs 1: python 1: proaudio 1: media-optical 1: kerberos 1: hp-cluster 1: amd64 |
metadata.xml (the second)
As I was kinda bored after work today, I had a closer look at what I saw during my fuckup in the morning. Well, Steve said, that when he looked at metadata.xml it’d be “really common” .. still that isn’t making it right ..
There is a reason we do have a herds.xml (exactly for the reason to associate people with packages, and that’s what the <herd> tag is for in metadata.xml) file. So after a preliminary look through the repository, here are the winners:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
700 : perl 126 : xemacs 63 : haskell 47 : sound 32 : ha-cluster 31 : crypto 19 : desktop-misc 16 : netmon 15 : forensics 13 : web-apps 8 : mips 8 : app-backup 7 : kde 6 : tcltk 6 : net-im 6 : media-tv 6 : dev-embedded 5 : voip 5 : theology 5 : samba 5 : net-p2p 4 : sparc 4 : java 4 : graphics 2 : net-mail 2 : kernel 2 : fonts 2 : embedded 2 : cpp 1 : x11 1 : wxwidgets 1 : www-servers 1 : tex 1 : shell-tools 1 : sh 1 : sgml 1 : sci 1 : python 1 : proaudio 1 : php 1 : media-optical 1 : kerberos 1 : hp-cluster 1 : gentopia 1 : amd64 |
Don’t know how accurate that list is, but you can check it for yourself. The commands I’ve used are these:
|
1 2 3 4 5 |
for i in $( < ../herds.list ); do grep --exclude=eclass --exclude=CVS --exclude=profiles --exclude=skel.* -R "$i@gentoo.org" /cvs/gentoo-x86/* ; done > redundant-metadata-xml.list |
|
1 2 3 4 5 6 |
for i in $( < herds.list ); do echo -e " $( grep "$i@gentoo.org" ~/public_html/redundant-metadata-xml.list | wc -l )t: $i"; done | grep -v "^ 0" | sort -nr > public_html/redundant-metadata-xml.overview |
While herds.list holds a list (separated by n) with all the herds there are. The raw files are here and here and here. Knock yourself out!
metadata.xml
So I ended up cleaning out some retired (~20) people from metadata.xml, where I found this interesting piece of metadata.xml:
|
1 2 3 4 5 6 7 8 |
kernel-misc crypto@gentoo.org Crypto herd masterdriverz@gentoo.org Charlie Shephered |
And here the hint for all you people again: A DAMN HERD AIN’T NO MAINTAINER. SO IF YOUR HERD IS MAINTAINING A PACKAGE, PUT IT INTO <herd> and not into the <maintainer>. kthnxbye.
To be or not to be …
… that’s the question. I’ve been thinking lots and lots about my involvement with our “beloved” distribution.
I talked to some of the users (that is Gordon), some fellow developers (hello Christina, Łukasz, solar, Jorge, Anders) about whether or not I’m actually still wanted and/or needed. Turns out, the collective opinion is, that I am fun to have around (*shrug* don’t ask me why, I don’t find myself particularly funny/amusing) and that’d I’d be the person to have around.
That being said, I still do have some things on my agenda (they haven’t changed .. like getting healthier – as in heading to the gym; getting a better paid job; getting my own life; getting some friends), which are going to jockey with those Gentoo interests.
stages
For what it’s worth, I’ve been trying to get some stages together the last few days. Thanks to solar and Brent, the ppc-stages are now coming along quite fast.
I haven’t really tested them yet, but for what it’s worth, you’ll find stages based on Saturday’s snapshot (that is 200780105 for those not smart enough to take a look at the calendar) here for the following profiles:
- uclibc/ppc (normal/-softfloat)
- uclibc/ppc/hardened
- uclibc/x86
- uclibc/x86/hardened
- hardened/amd64
- hardened/amd64/nomultilib
- hardened/x86/2.6 (x86/i686)
Now remember, this isn’t *official* release material. This is just *MY* effort (for now) to provide current stages.
And just a side-note for those brewing their own (uClibc) soup: if you remerge system/world, you’ll have to keyword =sys-libs/uclibc.0.9.28.3-r2. Otherwise you’ll stumble on bug 195368, which is fixed thanks to solar, just not marked stable yet.
Advanced bashrc (‘Turning a simple chroot into a binpkg repository’ continued)
As I pointed out back in October, it’s rather easy to create a setup which syncs a built binary package to a remote node (which is serving them to the world – via http,rsync,ftp – pick your poison).
Now, ever since we had slight space problems on miranda (cough my binpkgs cough), I wanted to look into methods on how to get rid of storing them on the buildnode and the webnode. I think now (hehe, it’s only 7pm), I finally managed to get a “proper” bashrc which does a lot of that foo. Take a look at this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 |
... syncpkg() { # Syncing the binary tbz2 to my webhost if is_feature "buildpkg" && [[ -n $REPO_HOST && -n $REPO_BASE && -n $REPO_PATH ]] ; then REMOTE_TARGET="$REPO_HOST:$REPO_BASE/$REPO_PATH" REMOTE_ECACHE="$REPO_BASE/$REPO_PATH/settings/.ebuild.x" einfo "Publishing data to remote repository ($REPO_PATH) on ${REPO_HOST##*@}" if ! $( ssh $REPO_HOST "test -d $REPO_BASE/$REPO_PATH/settings" ) ; then ssh $REPO_HOST "mkdir -p $REPO_BASE/$REPO_PATH/settings" >> /var/log/syncpkg.log 2>&1 fi if [ -x /usr/bin/q ] ; then /usr/bin/qlist -IvCU > /etc/portage/package.list fi ebegin " [ SYNC: 1/3] /etc/make.conf, /etc/portage/package* to $REPO_PATH/settings" rssh /etc/{make.conf,portage/package*,portage/profile,portage/bin,portage/bashrc} $REMOTE_TARGET/settings/ > /var/log/syncpkg.log 2>&1 eend $? ebegin " [ SYNC: 2/3] $PKGDIR to $REPO_PATH" rssh $PKGDIR/ $REMOTE_TARGET/ >> /var/log/syncpkg.log 2>&1 eend $? ebegin " [ SYNC: 3/3] Cleaning $PKGDIR" rm -rf $PKGDIR/* >> /var/log/syncpkg.log 2>&1 eend $? # We sync our copy of .ebuild.x over to the webnode, as it might not be # synced as often / at the same time as the buildnode. Thus the # `qpkg --eclean' would remove packages, which *are* in the tree, but # the webnode hasn't synced up yet. rssh $PORTDIR/.ebuild.x $REMOTE_TARGET/settings/.ebuild.x >> /var/log/syncpkg.log 2>&1 if $( ssh $REPO_HOST "test -f $REMOTE_ECACHE" ) ; then ebegin " [MAINT: 1/3] Removing stale packages in $REPO_PATH" ssh $REPO_HOST "CACHE_EBUILD_FILE=$REMOTE_ECACHE qpkg -Eq -P $REPO_BASE/$REPO_PATH/" >> /var/log/syncpkg.log 2>&1 eend $? fi # And now remove the .ebuild.x copy again. ssh $REPO_HOST "rm -f $REMOTE_ECACHE" >> /var/log/syncpkg.log 2>&1 # The user ssh'ing, needs to be in the portage group on the remote # end. Otherwise, things *will* go wrong (like genpkgindex being # unable to write to /var/cache/edb/xpak). Also /var/cache/edb/xpak # needs to be owned by portage:portage, as well as group-writeable. if $( ssh $REPO_HOST "test -w /var/cache/edb/xpak" ) ; then ebegin " [MAINT: 2/3] Regenerating $REPO_PATH/Packages" ssh $REPO_HOST "$REPO_BASE/$REPO_PATH/settings/bin/genpkgindex $REPO_BASE/$REPO_PATH/All" >> /var/log/syncpkg.log 2>&1 eend $? fi if ! $( ssh $REPO_HOST "test -L $REPO_BASE/$REPO_PATH/Packages" ) ; then ebegin " [MAINT: 3/3] Fixing $REPO_PATH/Packages symlink" ssh $REPO_HOST "cd $REPO_BASE/$REPO_PATH; rm -f Packages; ln -s All/Packages" >> /var/log/syncpkg.log 2>&1 eend $? fi fi } ... |
As you can see, it does a lot of things, which are all connected with binary package repositories (including cleaning up old packages no longer in the tree – trying not to waste too much space). Sadly, I currently have to use a custom patched qpkg version, as the one implementing the –eclean features isn’t in the tree yet. When I talked to Ned the other day, he complained about it being slow (well, yeah — it has to go through the whole tree) which I don’t really see when you look at what it’s doing.
Also, I had a weird phenomenon today happening: the buildnode built a binary package, sent it to the webnode, which ran qpkg --eclean' afterwards. But after that the binary package was gone. "Why" you ask now ? Well, apparently the webnode isn't synced the same time the buildnode syncs (the webnode is in Germany, the buildnode in the US). So I had to come up with a trick, in order to fool qpkg into not cleaning the freshly built binary packages. See the rssh’ in front of the qpkg call ? Guess what, that’s the lil’ dirty trick …
Anyway, the full bashrc is available. The next thing I’m gonna have to look at (which Markus already did), is building packages via buildbot.
Update: as you see, I updated the bashrc a bit. That’s because after writing this, I started a new (fresh) binpkg repository (empty), and out of the sudden the thing ain’t syncing correctly (as in no Packages file, no portage settings). Turns out, rsync doesn’t create directories which ain’t there. So another extra `ssh‘ execution to create the settings/ directory inside the repo.