View source for Kexi/Porting to Qt&KF 5

{{Note|Based on [http://mail.kde.org/pipermail/calligra-devel/2014-July/011603.html notes] from the [[Calligra/Meetings/Spring_2014_Sprint|Calligra 2014 Sprint]]. Each step leads to compiling (running?) state.<br/>Started by [[User:Jstaniek|jstaniek]] ([[User talk:Jarosław|talk]]) 19:06, 7 July 2014 (CET).}}

__TOC__

The grand plan:
# [100%, builds, example and autotests work] Port <strike>Predicate</strike>[[#KDb|KDb]]&darr; lib to Qt5/KF5
## [100%] Port the SQLite driver
## [100%] Port the MySQL driver
## [100%] Port the PostgreSQL driver
## [10%] Port the xBase driver
## [10%] Port the Sybase driver
# [100%, builds, example and autotests work] Move koproperty lib from calligra to a separate [[#KProperty|KProperty]]&darr; repo, port to Qt5/KF5
# [100%, builds, example and autotests work] Move koreport lib from calligra to a separate [[#KReport|KReport]]&darr; repo, port to Qt5/KF5
## Remove scripting or port scripting?
# [100%] Port Kexi (without KexiDB/calligradb, reporting, koproperty, koreport) to Qt 5 and use KDb/KProperty/KReport
## [100%] Remove unused calligradb lib and parts of KexiDB lib
## [100%] Port the Table plugin
## [100%] Port the Query plugin
## [100%] Port the Form plugin
## [100%] Port the Report plugin
## [100%] Port the Import/Export plugin
## [20%] Port the Migration module
## [0%] Reimplement report scripting using a simple solution to keep backward compatibility with Kexi 2 (see [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/namespaceScripting.html Scripting::*], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptConstants.html KRScriptConstants], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptDraw.html KRScriptDraw], [http://api.kde.org/bundled-apps-api/calligra-apidocs/libs/koreport/html/classKRScriptDebug.html KRScriptDebug])
# [40%] Improve Kexi stability after porting
# '''-- Kexi 3.0 beta release here. To indicate the nature of this version we never release 3.0 stable. --'''
# Add tests and/or examples to KReport, make it easier to understand as a framework 
## Port some code from Kexi Reports (e.g. KexiReportView [https://todo.kde.org/?controller=task&action=show&task_id=1206]) to KReport
# Make KReport lib dependent on KDb and remove code/API that becomes redundant (e.g. KoReportData)
# Add a new [[Kexi/Plugins/Scripts|scripting module]]
# Use the module in Kexi Reports too, keeping backward compatibility with Kexi 2 scripting (add unit tests before porting to be very sure that compatibility is kept)

==Rules==
* To build Kexi, first build & install KDb, KProperty, KReport (master branches), i.e. for each:
 cd <<BUILDDIR>>
 cmake -DPRODUCTSET=kexi -DCMAKE_INSTALL_PREFIX=<<PREFIX>> -DCMAKE_BUILD_TYPE=Debug <<SRCDIR>>
* Then use calligra.git 'master' branch: type "git checkout master". 
*Use [[Sysadmin/GitKdeOrgManual#Personal_scratch_repositories|personal git scratch repos]] to avoid stress with buildbreaks and unnecessary BUG/review notifications
*Boud's advice: ''My approach is more like, run each of montel's scripts, fix what's wrong, then go on. It's not something that's easy to paralellize.
*Don't run astyle on the code, it pollutes history, and astyle-kdelibs is a bit broken 
*Based on "Porting notes" below, keep porting of one aspect in one commit to simplify reviews
*Use the Phabricator for reviews
*Use [http://quickgit.kde.org/?p=kde-dev-scripts.git kde-dev-scripts.git] porting scripts, in particular
**[http://kfunk.org/2015/01/11/new-porting-helper-convert-to-cmake-automoc-pl convert-to-cmake-automoc.pl]
*Libs that are separate repos should use the framework template (see Links) and be ported to this layout (for now it's just kdb, kproperty, kreport).

==Notes and Status==

*TODO: publish detailed checklist (in a spreadsheet?)

===Dependencies===
Please read all this section carefully before taking action.

*[https://community.kde.org/Frameworks/Building/Details#Qt5-based_dependencies Qt 5]. Get >=5.2 stable version from your distro, not forgetting about development packages (headers). If you feel you want it, build Qt 5, but it's usually not necessary extra work.
*CMake: version >=2.8.12+ and 3.x are both supported. Get it from your distro (recommended) or [https://community.kde.org/Frameworks/Building/Details#CMake_from_Git build it].
*KDE Frameworks 5 (KF5) are needed as build-time and run-time dependencies for Calligra, Kexi and our frameworks (KProperty, etc).
*[https://community.kde.org/Frameworks/Building Use the kdesrc-build] tool to conveniently build KF5. Building by hand in an option if you accept extra work usually not needed for our development (see [[Frameworks/Building/Details]]).
*There is the [[Frameworks/Building#Adjust_the_paths|Adjust the paths]] step. For Kexi development (and Calligra in general) it's good to re-use paths that we already use except the install prefix (kdedir), see the [[Calligra/Building/3#Recommended_Setup|Recommended setup]] of the Calligra build instructions. This means that recommendation for the <tt>kdesrc-buildrc</tt> config file is:
  source-dir /home/YOU/kde/src
  build-dir /home/YOU/kde/build
  kdedir /home/YOU/kde/inst5

{{Note|If you used ''/home/YOU/kde/inst'' as a dir the Calligra 2.x and other kdelibs4-based software, '''do not mix it with the Calligra 3.x+ and other kf5-based software''', otherwise you can experience crashes or other strange behaviour. '''Use ''/home/YOU/kde/inst5'' for example.}}

*Speed up: before using the KF5 with kdesrc-build, edit the kf5-qt5-build-include file and comment these lines if you don't want to build the workspace, applications and KDEPIM. Thus only "include kf5-frameworks-build-include" is left.
:include kf5-workspace-build-include
:include kf5-applications-build-include
:include kf5-kdepim-build-include


*In particular, the kdesrc-build will install extra CMake Modules: [https://community.kde.org/Frameworks/Building/Details#Extra_CMake_Modules], which is needed to make our software able to find the KF5 components at build time.

===KDb===
*Download/Build/Release info &rarr; [[KDb]].
*Use '''master''' branch, official for Qt 5/KF 5, for all platforms, always stable
*Our target [[KDb/Releases|version]] is 3.0.0.
*To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
*Depends only on system libraries and Qt, so it's a tier 1 framework
*[[KDb/Windows|Windows]] and [[KDb/Mac|Mac]]-specific matters.
*Porting to Qt 5 started by Wojtek Kosowicz (Jan 2015)
*Porting to Qt 5/Mac OS X started by Wojtek Kosowicz too (Jan 2015)
*Directory structure reorganized according to [[Frameworks/Policies#Framework_directory_structure|recommendations]] --[[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 20:59, 23 February 2015 (UTC)

===KProperty===
*Our target version is 3.0.0 and '''KProperty''' [https://projects.kde.org/projects/playground/libs/kreport project in Playground/Libs category of KDE projects]
*Depends on system libraries and Qt, plus minor deps on KF5WidgetsAddons/KF5GuiAddons for some editors; later we may move this dependency to a plugin (so the core would stay Qt-only)
*Official git repo: kproperty, master branch (use: git clone kde:kproperty)
*Directory structure reorganized according to [[Frameworks/Policies#Framework_directory_structure|recommendations]] --[[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 20:59, 23 February 2015 (UTC)

===KReport===
*Our target version is 3.0.0 and '''KReport''' [https://projects.kde.org/projects/playground/libs/kreport project in Playground/Libs category of KDE projects]
*To avoid conflicts, Qt 4 version kept for reference, has version 0.0.0.
*Depends on system libraries, Qt, and KProperty (tier 1), so it's a tier 2 framework, some deps on KF5WidgetsAddons (may be moved to a plugin so the core would stay Qt-only)
*Official git repo: kreport, master branch (use: git clone kde:kreport)
*Directory structure reorganized according to [[Frameworks/Policies#Framework_directory_structure|recommendations]] --[[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 20:59, 23 February 2015 (UTC)
*Deps from calligra libs (copied, renamed): KoRuler, KoUnit

==Build instructions==
Kexi 3/Calligra 3 builing is explained in a (work-in-progress) doc: [[Calligra/Building/3]].
Our dependencies (KDb, KPropert, KReport) can be built in a usual way (cmake, make install). Don't forget to set environment variables in the same way as you do for Kexi/Calligra builds.

==Porting scripts usage status==
These porting scripts from the kf5 dir of the kde:kde-dev-scripts repo would be applied. <br/>
Legend: - = not run, + = run, N = not applicable, T = TODO(we know it's needed)
<pre>
KDb | KProperty | KReport | Kexi | script-name
+ + + + adapt_cmakelists_file.pl
+ + + + adapt_knewstuff3_includes.pl
+ + + + add_missing_kpart_include.pl
+ + + + clean-forward-declaration.sh
+ + + + clean-includes.sh
N N N - cmakelists_install_vars.pl (N/A because KDb/KProperty/KReport don't use KDE_*DIR variables)
N N N N convert-kabc-to-kcontacts.pl (unused)
+ + + + convert-kaction.pl
+ + + - convert-kbuttongroup.pl
T + + + convert-kcmdlineargs.pl
+ + + N convert-kcolordialog.pl
+ + + + convert-kdebug.pl
N N N N convert-kdebug-with-argument.sh (N/A because KDb/KProperty/KReport don't use that)
N + N + convert-kdialog.pl (helps but KProperty is largery broken by this script)
N N N N convert-kdoublenuminput.pl
N N N N convert-kfiledialog.pl
N N N N convert-kfontcombobox.pl
N N N T convert-kicon.pl
N N N N convert-kimageio.pl
N N N N convert-kintnuminput.pl
N N N N convert-kintspinbox.pl
N N N N convert-kjob.pl
N N N + convert-klineedit.pl
N N N + convert-klistwidget.pl
N N N N convert-klocale-formatdate.pl
N N N + convert-kmd5.pl
N N N N convert-kmenubar.pl
N N N + convert-kmenu.pl
N N + + convert-kmimetype.pl
N N N + convert-kprogressdialog.pl
N N N + convert-kpushbutton.pl
N N N + convert-ksavefile.pl
N N N + convert-ksharedptr.pl
N N N + convert-kshortcut.pl
N N N N convert-ksplashscreen.pl
N N N + convert-kstandarddirs.pl
N N N + convert-ktabwidget.pl
N N N + convert-ktempdir.pl
+ N + + convert-ktemporaryfile.pl
N N N + convert-ktextbrowser.pl
N + + + convert-kurl.pl
N N N + convert-kvbox.pl
+ + + - convert-qkdebug-to-qcdebug.sh
+ + + + convert-qt5.pl
+ + + + convert-to-cmake-automoc.pl
T N N T convert-to-k4aboutdata.pl
- - - - convert-to-new-signal-slot-signal.pl
N N N N dnssd-to-kdnssd.sh
+ + + + fix-ecm-install-icons.pl
- - - - fix-kcm-module-desktop-file.sh
- - - - install_forwarding_headers.pl
N N N N port-kauthactions.pl
N N N + port_to_autogenerate_export_header.sh
+ + + + remove-kde4support.pl
N N N N remove_virtual_prefix.pl
N N N + resolve_kuit.py
T T T T search-kdelibs4support-header.sh
+ + + + frameworks/plasma-framework/src/tools/port-cmake-style.sh
+ + + - frameworks/plasma-framework/src/tools/port-qslots.sh
+ + + - frameworks/plasma-framework/src/tools/port-includes.sh
</pre>

==Git surgery==
1. For editing large history you may need a fast filesystem. So a Linux '''RAM disk''' such as /dev/shm is your friend, deep-copy your repo temporarily there: [http://kvz.io/blog/2007/07/18/create-turbocharged-storage-using-tmpfs/]. 
* To deep-copy use:
 cd /path/to/ramdisk git clone /path/to/the/orig/repo --no-hardlinks --no-local
* Consider switching off swap ('sudo swapoff -a') as swapping makes no sense, when we want to use RAM exclusively. 
* Don't forget to create a backup of the original repository and then replace the original repository with the copy edited, using git clone again:
 cd /path/to/the/orig
 rm -rf repo
 git clone /path/to/ramdisk/repo --no-hardlinks --no-local
2. ([[User:Jstaniek|Jstaniek]] ([[User talk:Jstaniek|talk]]) 09:54, 23 February 2015 (UTC)) Repos are prepared by history rewrites such as:
 git filter-branch --prune-empty -f --tree-filter \
 '(mkdir -p src;mv Predicate src/predicate;mv Drivers src/drivers) 2> /dev/null || true' -- HEAD
3. Before your push your branch to a new repo, you may need some '''postprocessing''' if the commits contain non-Unix line endings, git push refuses to work then:
 remote: Audit failure - Commit 9460d32f6f2f51c2a182b8faa40ff7537b7ac5c6 - End of Line Style (non-Unix)

To remove the non-Unix line endings (see also [http://blog.gyoshev.net/2013/08/29/normalizing-line-endings-in-git-repositories/]).

3.1. Create this script and call it ''fix-eol.sh'':
 #!/bin/bash
 find . -type f -exec grep -Iq . {} \; -and -print | xargs dos2unix -k -q

(the ''find'' and ''grep'' commands here try very hard to find text files)

3.2. Run the actual rewrite. Note, we call this ''postprocessing'' because it's time-consuming so it's worth to run it after the step #2 only on files that we really need in the edited repo.
 cd /path/to/the/repo
 git filter-branch --tree-filter fix-eol.sh  -- --all

==Links==
*'''[[Calligra/Schedules/3.0/Porting_Plan|Calligra 3.0 Porting Plan]]'''
*'''[[Calligra/Schedules/3.0/Porting_Notes|Calligra 3.0 Porting Notes]]'''
*'''[https://todo.kde.org/?controller=board&action=show&project_id=7 Kexi tasks at todo.kde.org] (KDE Identity needed), [https://todo.kde.org/?controller=board&action=readonly&token=d39ee567c32511a5f32e69054902b64a4118930be455b4cf7a9c7b83fe87 public read-only view]'''
*OLD: '''[https://todo.kde.org/?controller=board&action=show&project_id=7 Predicate tasks at todo.kde.org] (KDE Identity needed), [https://todo.kde.org/?controller=board&action=readonly&token=d39ee567c32511a5f32e69054902b64a4118930be455b4cf7a9c7b83fe87 public read-only view]'''
*[http://quickgit.kde.org/?p=kdeexamples.git&a=tree&f=framework-template Framework repository template] - use for creating new frameworks
**real world examples: [https://projects.kde.org/projects/extragear/graphics/kdiagram KDiagram] (recommended because it's a Calligra project -- next-gen kdchart, kdgantt), [https://projects.kde.org/projects/frameworks/karchive/repository KArchive] (tier 1), [https://projects.kde.org/projects/frameworks/kio/repository KIO] (tier 3), [https://projects.kde.org/projects/frameworks/kimageformats/repository KImageFormats plugins]
*http://www.proli.net/2014/06/21/porting-your-project-to-qt5kf5/
**CMake porting script [http://quickgit.kde.org/?p=plasma-framework.git&a=blob&h=4d49e9ac199817dbf05398b89acf69791aedb530&hb=bc26ac13349a7a057621143d75e9c7b2518f5486&f=src%2Ftools%2Fport-cmake.sh port-cmake.sh]
Docs
*[[Frameworks]]
**[[Frameworks/Policies|Policies]]
**[[Frameworks/Porting Notes|Porting Notes]]
**[[Frameworks/Building|Build instructions]]
**[http://api.kde.org/frameworks-api/frameworks5-apidocs/ KF5 API docs]
*[http://doc.qt.io/qt-5/portingcppapp.html Porting to Qt 5]
**[http://doc.qt.io/qt-5/sourcebreaks.html Qt 5 C++ API changes]
**[https://github.com/KDAB/Qt4to5 Qt4to5 tool]
Background
*[https://lists.kde.org/?l=calligra-devel&m=140480621024543&w=2 Notes from the 2014 Sprint Meeting]

[[Category:Kexi Porting]]