Kubuntu/Packaging
Getting Started
New to Kubuntu packaging?
There are just a few steps that you need to take that will get your started with packaging software for Ubuntu. The Ubuntu Packaging Guide will take you through them.
Glossary
LP = [Ubuntu Project management software) https://launchpad.net Launchpad] KA = Kubuntu Automation KCI = Kubuntu Continuous Integration] PPA = Personal Package Archive managed by Launchpad (repository for packages) git = Versioning system tool QA = Quality Assurance
What is that? (make me pretty) <ovidiu-florin> how does okular get split in okular-dbg -src -dev ..... etc... ? <ovidiu-florin> when? where? by whom? <yofel> there is no -src package, the source package is what you make yourself by hand and upload to launchpad <yofel> the thing that consists of the .orig.tar + .debian.tar + .dsc <yofel> the binary packages are then created from information inside the control file, and information from the various .install etc. files in the debian/ folder
PPA Naming scheme
PPA Naming <-- needs new graph of package flow <-- How do you like my proposed treatment?
- Staging: (before Testing) [DO NOT USE ON REGULAR SYSTEMS]
- staging-plasma:
sudo add-apt-repository ppa:kubuntu-ppa/staging-plasma - staging-frameworks
- staging-kdeapplications
- Testing: (This is usually packaged before a call to !testers in #kubuntu-devel)
- ppa-landing
sudo add-apt-repository ppa:kubuntu-ppa/ppa-landing - backports-landing
sudo add-apt-repository ppa:kubuntu-ppa/backports-landing
Public
For the release of the software after it has been tested and ready for our users
- Backports: new upstream releases
sudo add-apt-repository ppa:kubuntu-ppa/backports
- Updates: point release bug fixes
sudo add-apt-repository ppa:kubuntu-ppa/ppa
Set-up: http://packaging.ubuntu.com/html/getting-set-up.html Environment: in ~/.bashrc add: export EMAIL="[email protected]" export DEBEMAIL="[email protected]" export DEBFULLNAME="My full name is Foo Bar" <yofel> ovidiu-florin: something I just realized for the packaging docs, I strongly recommend every packager to have a ~/.dput.cf with this content: <yofel> [DEFAULT] <yofel> default_host_main = ppa <yofel> this isn't that important for beginners, but really important for every person with primary archive upload permissions SSH: Your ~/.ssh/config should be set up to use your SSH keys. Example (needed by KA): Host depot.kde.org User ftpubuntu IdentityFile ~/.ssh/my_id
pbuild: [Pbuilder Howto] Install: sudo apt install pbuilder debootstrap devscripts pigz eatmydata bzr get lp:~kubuntu-packagers/pbuilder/pbuilder-hooks and lp:kubuntu-dev-tools (bzr branch lp: ...) save http://paste.ubuntu.com/14286057 as .pbuilderrc and set HOOKDIR to point to the pbuilder-hooks clone Create animage (archive) for xenial: sudo -E dist=xenial pbuilder create To login use: sudo -E dist=xenial pbuilder login To update use: sudo -E dist=xenial pbuilder update To test get the sources of a package (e.g. apt-get source pkg-kde-tools) and then pass the .dsc file to pbuilder build. You'll find the results in /var/cache/pbuilder/xenial-amd64/result/
<clivejo> you will need to configure pbuilder to use some Launchpad PPA's <clivejo> if its plasma 5.5.2 you want, you would need to add the staging PPA's <yofel> when you add PPA's to the chroot, beware of what you add. If the PPAs you add don't match the dependencies of the PPA on LP, then you will waste your time debugging weird failures <yofel> always make sure you know what environment you're building in
For easy cloning of LP git repositories, put this in your ~/.gitconfig (where USER is your launchpad username):
[url "git+ssh://[email protected]/"]
insteadof = lp:
[url "git+ssh://[email protected]/~USER/+git/"]
insteadof = lpme:
[url "git+ssh://[email protected]/~kubuntu-packagers/kubuntu-packaging/+git/"]
insteadof = kubuntu:
Full git access documentation can be found at https://help.launchpad.net/Code/Git
Process (Packaging a New Release of frameworks, plasma or apps):
Overview: https://drive.google.com/open?id=1SoO83PepBkU_kkbLL2eIeVDYYDzv6OEA1npVoCp7wHo
- Check KDE release cycle and make sure the tarballs are available on [Depot.kde.org]. If you don't have access to depot yet, ask one of the senior packagers to add your ssh key.
- Check KCI and fix any outstanding packaging problem. This will save you a lot of time fixing problems later. Should be done on a regular basis, regardless of the KDE release cycle.
- Check in IRC channel #kubuntu-devel to make sure nobody else is working on the same release (often mentioned in the topic and on [Ninjas notes])
- Clone KA Tools, all tools implement --help for their parameter documentation and perform the steps in the README [Readme]
- KCI needs to be paused whilst executing some of the KA tools, those tools will warn you and fail if KCI is still running [Pause Integration]
- Execute the script staging-upload, an example follows: ***** please check that this is still accurate **** This script grabs the tarball source code from depot and the packaging from git. It added new entries in the changelog and runs a debuild -S for all the packages in the suite. Problem packages are moved to /manual to be fixed by hand and the staged packages are moved to /upload ready for upload to LP.
./staging-upload -d xenial -v 5.5.2 -r plasma -t ~/workspace/plasma Where -d is the distribution (xenial in this case), -v is the version, and -r is the release type (frameworks, plasma, applications), -t is the working directory
- Provided all is well, the script will complete. If the script encounters a problem it moves the problem package to a folder called manual in your working directory. These packages will have to be manually fixed and moved to the upload folder.
Upload the packages to LP. Example: (script from step 7 is called staging-upload, but it does no uploading?) dput ppa:kubuntu-ppa/staging-plasma *.changes
- Run this command and don't ask why or what it does, it's magic. Also it only works if you work with plasma.
git-push-all -t ~/workspace/plasma/
- To help with automated QA of the LP package builds, update the appropriate config file in KA build-status-conf/ and push the change, that will then generate a status overview page in http://qa.kubuntu.co.uk/ppa-status/ that will show you what packages need fixing.
- As packages build in LP they may fail because packages they depend on havent been published yet. You can retry the builds via LaunchPad's web interface or use one of the KA Tools. To retry PPA builds use "while true; do ./kubuntu-retry-builds -r plasma --ppa=kubuntu-ppa --ppaname=staging-plasma --force; sleep 1200; done" Which will basically loop retrying the builds forever!
Build status legend: Blue | not built yet (needs building, depwait, building, uploading, ...) Green | OK (or at least auto-qa ok) Orange | minor things like lintian warnings, new symbols, etc. Red | failed to build from source or otherwise very broken
Fixing a package (example)
- Get a failing package from http://qa.kubuntu.co.uk/ppa-status/applications/build_status_15.12.0_xenial.html
- Check the logs, find why it failed. For instance, Knavalbattle failed due to missing dependencies.
- Create a work directory for this package and cd into it
- Download the source for the packaging from: **** check URL**** http://anonscm.debian.org/cgit/pkg-kde/applications/knavalbattle.git/ https://git.launchpad.net/~kubuntu-packagers/kubuntu-packaging/+git/knavalbattle and cd into it
- Switch to the branch kubuntu_xenial_archive as this is the branch we're following here.
In case of dependency issues
- Add the missing dependencies to debian/control ***
- Do not use virtual packages (ask yofel why)
In case of Symbols problems
- https://pkg-kde.alioth.debian.org/symbolfiles.html **** check URL
- get the build logs from QA and extract them
- run pkgkde-symbolshelper batchpatch -v 4:15.12.0 -c amd64,i386 buildlog_ubuntu-xenial-* to auto patch the symbols files * see man pkgkde-symbolshelper for more info)
- README in kubuntu-automation has the command you need to batchpatch symbols.
- Run c++filt on any MISSING symbols.
- look up the result on KDE API and see if it is a public symbol. If it is ( this is where I cry to someone more knowledgable like yofel)
<ovidiu-florin> https://paste.kde.org/pcusi5k79/5sjxr4 <yofel> first of all, the (optional) ones are always ok to be removed <yofel> that's usually internal GCC symbols that leak into the public namespace <ovidiu-florin> ok, how do I remove them? <ovidiu-florin> manual edit ? <yofel> yes <ovidiu-florin> yofel: first one is not optional <ovidiu-florin> how do I check it? <yofel> ovidiu-florin: it indeed is not, what you missed this time is that the soname version of libokular was changed from 6 to 7 <yofel> so as the interface version changed, any kind of changes are OK <yofel> clive took care of that before you took over okular <yofel> otherwise your best bet is to run c++filt on the symbol to look where it likely came from <yofel> then look up the source history to find out what happened to it and/or talk to upstream <clivejo> yofel: does that happen much? <yofel> esp. the libs shipped in applications like to change the version every release, i.e. libmarblewidget, kdcraw, kipi, etc. <yofel> frameworks and some other cases have an ABI stability guarantee, there you need to watch out <yofel> and in the worst case, when upstream messes up we can still work around it by using the DebianAbiManager, but that's something for another day
<ovidiu-florin> it seems to be missing the vtable for Okular::ConfigInterface <ovidiu-florin> search for _ZTVN6Okular15ConfigInterfaceE <yofel> right, it does say so <yofel> +#MISSING: 4:15.12.0-0ubuntu1~ubuntu16.04~ppa3# (optional)_ZTVN6Okular15ConfigInterfaceE@Base 4:14.12.0 <yofel> so, what's the question? <ovidiu-florin> the class has virtual functions <ovidiu-florin> https://lxr.kde.org/source/kde/kdegraphics/okular/interfaces/configinterface.h <ovidiu-florin> so it should have a vtable <yofel> and are they exported by gcc as part of the public interface? (That's all the symbol files care about) <ovidiu-florin> they are public funtions <yofel> and if you build the binary, "nm -DC <libfile>" will show the symbols? <ovidiu-florin> what do I give as <libfile> <ovidiu-florin> ? <ovidiu-florin> and I assume I run that <yofel> libokularcore or how it's called <ovidiu-florin> in the pbuilder <yofel> the symbol files don't care about the specifics of the C++ API, they only care about what symbols actually get exported by g++ <yofel> which is one reason why they're insufficient to do vtable tracking, you need at least abi-compliciance-checker for that -*- ovidiu-florin researches "g++ export symbols" <yofel> it's the symbols you see if you if you run "nm -D" on a lib, like, try "nm -D /usr/lib/x86_64-linux-gnu/libKF5Archive.so.5" <yofel> if you handle symbol files, you don't need to go into API specifics. What you need to make sure is that your file matches the auto-generated output of dpkg-gensymbols <yofel> that's why you see a diff in the log, it's the diff between the file in the packaging, and the auto-generated one
Test your changes
- Use the wrap-and-sort tool to sort and cleanup the debian/control file, if you changed it
- Use uscan --download-current-version --destdir=../ to download the project sources
- Now build the package with debuild -S (Check the debuild man page to understand what it does)
- Now test your changes and build (compile) it in the pbuilder you've set up for xenial: sudo dist=xenial pbuilder build ../knavalbattle_15.12.0-0ubuntu1.dsc
- If it fails read what it says (why it failed). Fix the issue and try again.
- Usually if it fails, you are dropped down to a shell inside the pbuilder environment.
if you get an error like "cp: cannot stat 'debian/tmp/usr/share/applications/kde4/knavalbattle.desktop': No such file or directory" and you're building a KF5 app, it might be because the .desktop file for KDE4 does not exist anymore, So in this case comment the mention of that file in debian/*.install files. Ask a senior packager for guidance here.
Finishing
- When you're all done and all works, run dch to update the changelog file
- Get a patch of your changes and send it to the devel team
- the procedure for getting something uploaded that you cannot upload yourself is Sponsorship Process
IDEAS: Perhaps we should update these pages: https://help.ubuntu.com/community/KMailGPGAgent https://userbase.kde.org/KMail/gpg and point to them for setting up gpg-agent (yofel)
Useful links: Porting to KF5 status page http://developer.kde.org/~cfeck/portingstatus.html
Once we release, announce on: Kub.org/Kub-wire/KDE::Kub-Release, Social Media (please list the account-names and who has access), https://community.kde.org/Plasma/Packages#Kubuntu
Packaging Reference
Below are some commands that are helpful for new and experienced packagers alike:
Git
- git clone
- clone git repo
- git checkout dir/file
- copy file from master to current branch
- git checkout branch-name
- switch to branch-name
- git diff
- see what changed in all the files
Bzr
- bzr co lp:~kubuntu-packagers/+junk/kubuntu-automation
- grab kubuntu-automation to ignore errors
Apt
- apt-get source packagename
- grab source for packagename
- apt-get build-dep packagename
- grab depends of packagename
- apt-cache policy packagename
- check installed version and available version
Debian command
- dh_install --list-missing
- check missing files in debian/.install file
- debuild -nc
- (no clean) builds again but without compiling everything over
- debsign -r host@ipaddress:path/to/changes
- sign package from remote
- dput -u ppa:username/ppa <source.changes>
- uploads to Launchpad while ignoring the local keyring search
pbuilder
pbuilder-dist wily create - make a wily pbuilder in that directory
pbuilder-dist vivid create - make a vivid pbuilder in that directory