Kexi/Porting to Qt&KF 5

Note

Draft. Based on notes from the Calligra 2014 Sprint. Each step leads to compiling (running?) state.
Started by jstaniek (talk) 19:06, 7 July 2014 (CET).

The grand plan:

1. [100%, builds, example and autotests work] Port PredicateKDb↓ lib to Qt5/KF5
   1. [100%] Port the SQLite driver
   2. [100%] Port the MySQL driver
   3. [100%] Port the PostgreSQL driver
   4. [10%] Port the xBase driver
   5. [10%] Port the Sybase driver
2. [100%, builds, example and autotests work] Move koproperty lib from calligra to a separate KProperty↓ repo, port to Qt5/KF5
3. [100%, builds, example and autotests work] Move koreport lib from calligra to a separate KReport↓ repo, port to Qt5/KF5
   1. Remove scripting or port scripting?
4. [100%] Port Kexi (without KexiDB/calligradb, reporting, koproperty, koreport) to Qt 5 and use KDb/KProperty/KReport
   1. [100%] Remove unused calligradb lib and parts of KexiDB lib
   2. [100%] Port the Table plugin
   3. [100%] Port the Query plugin
   4. [100%] Port the Form plugin
   5. [100%] Port the Report plugin
   6. [100%] Port the Import/Export plugin
   7. [20%] Port the Migration module
   8. [0%] Reimplement report scripting using a simple solution to keep backward compatibility with Kexi 2 (see Scripting::*, KRScriptConstants, KRScriptDraw, KRScriptDebug)
5. [40%] Improve Kexi stability after porting
6. -- Kexi 3.0 beta release here. To indicate the nature of this version we never release 3.0 stable. --
7. Add tests and/or examples to KReport, make it easier to understand as a framework
   1. Port some code from Kexi Reports (e.g. KexiReportView [1]) to KReport
8. Make KReport lib dependent on KDb and remove code/API that becomes redundant (e.g. KoReportData)
9. Add a new scripting module
10. Use the module in Kexi Reports too, keeping backward compatibility with Kexi 2 scripting (add unit tests before porting to be very sure that compatibility is kept)

Rules

• To build Kexi, first build & install KDb, KProperty, KReport (master branches), i.e. for each:

cd <<BUILDDIR>>
cmake -DPRODUCTSET=kexi -DCMAKE_INSTALL_PREFIX=<<PREFIX>> -DCMAKE_BUILD_TYPE=Debug <<SRCDIR>>

• Then use calligra.git 'master' branch: type "git checkout master".
• Use personal git scratch repos to avoid stress with buildbreaks and unnecessary BUG/review notifications
• Boud's advice: My approach is more like, run each of montel's scripts, fix what's wrong, then go on. It's not something that's easy to paralellize.
• Don't run astyle on the code, it pollutes history, and astyle-kdelibs is a bit broken
• Based on "Porting notes" below, keep porting of one aspect in one commit to simplify reviews
• Use the Phabricator for reviews
• Use kde-dev-scripts.git porting scripts, in particular
  • convert-to-cmake-automoc.pl
• Libs that are separate repos should use the framework template (see Links) and be ported to this layout (for now it's just kdb, kproperty, kreport).

Notes and Status

• TODO: publish detailed checklist (in a spreadsheet?)

Dependencies

Please read all this section carefully before taking action.

• Qt 5. Get >=5.2 stable version from your distro, not forgetting about development packages (headers). If you feel you want it, build Qt 5, but it's usually not necessary extra work.
• CMake: version >=2.8.12+ and 3.x are both supported. Get it from your distro (recommended) or build it.
• KDE Frameworks 5 (KF5) are needed as build-time and run-time dependencies for Calligra, Kexi and our frameworks (KProperty, etc).
• Use the kdesrc-build tool to conveniently build KF5. Building by hand in an option if you accept extra work usually not needed for our development (see Frameworks/Building/Details).
• There is the Adjust the paths step. For Kexi development (and Calligra in general) it's good to re-use paths that we already use except the install prefix (kdedir), see the Recommended setup of the Calligra build instructions. This means that recommendation for the kdesrc-buildrc config file is:

 source-dir /home/YOU/kde/src
 build-dir /home/YOU/kde/build
 kdedir /home/YOU/kde/inst5

Note

If you used /home/YOU/kde/inst as a dir the Calligra 2.x and other kdelibs4-based software, do not mix it with the Calligra 3.x+ and other kf5-based software, otherwise you can experience crashes or other strange behaviour. Use /home/YOU/kde/inst5 for example.

• Speed up: before using the KF5 with kdesrc-build, edit the kf5-qt5-build-include file and comment these lines if you don't want to build the workspace, applications and KDEPIM. Thus only "include kf5-frameworks-build-include" is left.

include kf5-workspace-build-include

include kf5-applications-build-include

include kf5-kdepim-build-include

• In particular, the kdesrc-build will install extra CMake Modules: [2], which is needed to make our software able to find the KF5 components at build time.

KDb

• Download/Build/Release info → KDb.
• Use master branch, official for Qt 5/KF 5, for all platforms, always stable
• Our target version is 3.0.0.
• To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
• Depends only on system libraries and Qt, so it's a tier 1 framework
• Windows and Mac-specific matters.
• Porting to Qt 5 started by Wojtek Kosowicz (Jan 2015)
• Porting to Qt 5/Mac OS X started by Wojtek Kosowicz too (Jan 2015)
• Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)

KProperty

• Our target version is 3.0.0 and KProperty project in Playground/Libs category of KDE projects
• Depends on system libraries and Qt, plus minor deps on KF5WidgetsAddons/KF5GuiAddons for some editors; later we may move this dependency to a plugin (so the core would stay Qt-only)
• Official git repo: kproperty, master branch (use: git clone kde:kproperty)
• Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)

KReport

• Our target version is 3.0.0 and KReport project in Playground/Libs category of KDE projects
• To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
• Depends on system libraries, Qt, and KProperty (tier 1), so it's a tier 2 framework, some deps on KF5WidgetsAddons (may be moved to a plugin so the core would stay Qt-only)
• Official git repo: kreport, master branch (use: git clone kde:kreport)
• Directory structure reorganized according to recommendations --Jstaniek (talk) 20:59, 23 February 2015 (UTC)
• Deps from calligra libs (copied, renamed): KoRuler, KoUnit

Build instructions

Kexi 3/Calligra 3 builing is explained in a (work-in-progress) doc: Calligra/Building/3. Our dependencies (KDb, KPropert, KReport) can be built in a usual way (cmake, make install). Don't forget to set environment variables in the same way as you do for Kexi/Calligra builds.

Porting scripts usage status

These porting scripts from the kf5 dir of the kde:kde-dev-scripts repo would be applied.
Legend: - = not run, + = run, N = not applicable, T = TODO(we know it's needed)

KDb | KProperty | KReport | Kexi | script-name
+ + + + adapt_cmakelists_file.pl
+ + + + adapt_knewstuff3_includes.pl
+ + + + add_missing_kpart_include.pl
+ + + + clean-forward-declaration.sh
+ + + + clean-includes.sh
N N N - cmakelists_install_vars.pl (N/A because KDb/KProperty/KReport don't use KDE_*DIR variables)
N N N N convert-kabc-to-kcontacts.pl (unused)
+ + + + convert-kaction.pl
+ + + - convert-kbuttongroup.pl
T + + + convert-kcmdlineargs.pl
+ + + N convert-kcolordialog.pl
+ + + + convert-kdebug.pl
N N N N convert-kdebug-with-argument.sh (N/A because KDb/KProperty/KReport don't use that)
N + N + convert-kdialog.pl (helps but KProperty is largery broken by this script)
N N N N convert-kdoublenuminput.pl
N N N N convert-kfiledialog.pl
N N N N convert-kfontcombobox.pl
N N N T convert-kicon.pl
N N N N convert-kimageio.pl
N N N N convert-kintnuminput.pl
N N N N convert-kintspinbox.pl
N N N N convert-kjob.pl
N N N + convert-klineedit.pl
N N N + convert-klistwidget.pl
N N N N convert-klocale-formatdate.pl
N N N + convert-kmd5.pl
N N N N convert-kmenubar.pl
N N N + convert-kmenu.pl
N N + + convert-kmimetype.pl
N N N + convert-kprogressdialog.pl
N N N + convert-kpushbutton.pl
N N N + convert-ksavefile.pl
N N N + convert-ksharedptr.pl
N N N + convert-kshortcut.pl
N N N N convert-ksplashscreen.pl
N N N + convert-kstandarddirs.pl
N N N + convert-ktabwidget.pl
N N N + convert-ktempdir.pl
+ N + + convert-ktemporaryfile.pl
N N N + convert-ktextbrowser.pl
N + + + convert-kurl.pl
N N N + convert-kvbox.pl
+ + + - convert-qkdebug-to-qcdebug.sh
+ + + + convert-qt5.pl
+ + + + convert-to-cmake-automoc.pl
T N N T convert-to-k4aboutdata.pl
- - - - convert-to-new-signal-slot-signal.pl
N N N N dnssd-to-kdnssd.sh
+ + + + fix-ecm-install-icons.pl
- - - - fix-kcm-module-desktop-file.sh
- - - - install_forwarding_headers.pl
N N N N port-kauthactions.pl
N N N + port_to_autogenerate_export_header.sh
+ + + + remove-kde4support.pl
N N N N remove_virtual_prefix.pl
N N N + resolve_kuit.py
T T T T search-kdelibs4support-header.sh
+ + + + frameworks/plasma-framework/src/tools/port-cmake-style.sh
+ + + - frameworks/plasma-framework/src/tools/port-qslots.sh
+ + + - frameworks/plasma-framework/src/tools/port-includes.sh

Git surgery

1. For editing large history you may need a fast filesystem. So a Linux RAM disk such as /dev/shm is your friend, deep-copy your repo temporarily there: [3].

• To deep-copy use:

cd /path/to/ramdisk git clone /path/to/the/orig/repo --no-hardlinks --no-local

• Consider switching off swap ('sudo swapoff -a') as swapping makes no sense, when we want to use RAM exclusively.
• Don't forget to create a backup of the original repository and then replace the original repository with the copy edited, using git clone again:

cd /path/to/the/orig
rm -rf repo
git clone /path/to/ramdisk/repo --no-hardlinks --no-local

2. (Jstaniek (talk) 09:54, 23 February 2015 (UTC)) Repos are prepared by history rewrites such as:

git filter-branch --prune-empty -f --tree-filter \
'(mkdir -p src;mv Predicate src/predicate;mv Drivers src/drivers) 2> /dev/null || true' -- HEAD

3. Before your push your branch to a new repo, you may need some postprocessing if the commits contain non-Unix line endings, git push refuses to work then:

remote: Audit failure - Commit 9460d32f6f2f51c2a182b8faa40ff7537b7ac5c6 - End of Line Style (non-Unix)

To remove the non-Unix line endings (see also [4]).

3.1. Create this script and call it fix-eol.sh:

#!/bin/bash
find . -type f -exec grep -Iq . {} \; -and -print | xargs dos2unix -k -q

(the find and grep commands here try very hard to find text files)

3.2. Run the actual rewrite. Note, we call this postprocessing because it's time-consuming so it's worth to run it after the step #2 only on files that we really need in the edited repo.

cd /path/to/the/repo
git filter-branch --tree-filter fix-eol.sh  -- --all

Links

• Calligra 3.0 Porting Plan
• Calligra 3.0 Porting Notes
• Kexi tasks at todo.kde.org (KDE Identity needed), public read-only view
• OLD: Predicate tasks at todo.kde.org (KDE Identity needed), public read-only view
• Framework repository template - use for creating new frameworks
  • real world examples: KDiagram (recommended because it's a Calligra project -- next-gen kdchart, kdgantt), KArchive (tier 1), KIO (tier 3), KImageFormats plugins
• http://www.proli.net/2014/06/21/porting-your-project-to-qt5kf5/
  • CMake porting script port-cmake.sh

Docs

• Frameworks
  • Policies
  • Porting Notes
  • Build instructions
  • KF5 API docs
• Porting to Qt 5
  • Qt 5 C++ API changes
  • Qt4to5 tool

Background

• Notes from the 2014 Sprint Meeting