KDE PIM/History

↑ Main page

KDE PIM History

Before 2000, in the dark ages, what is currently known as KDE PIM was developed as separate applications by different people. KDE PIM as a module was formed by Rik Hemsley around 1999 but did not contain all that is currently KDE PIM.

Around 2001 time Matthias Hölzer-Klüpfel from Caldera developed "Kaplan" in an attempt to integrate the KDE PIM applications, inspired by the KParts useage of KDevelop at that time.

Kolab Client

In 2002 the Kolab initiative was started with a commercial development project around KDE PIM where Intevation and KDAB were working on integrating KOrganizer and KMail.

Shortly after, work was done on KPart-ifying KMail, making Kaplan a viable solution and forming the base of what became the Kontact shell.

First KDE PIM Meeting

The first KDE PIM meeting in 2003 brought together developers independently working on their respective applications. The earlier integration attempts were discussed and it was decided that the Kontact/KParts solution was the way to go.

It was also decided at that meeting to move KMail from the KDE Network group to the brand new KDE PIM CVS location, in a sense marking the creation of the real 'team' dedicated to maintain and develop KDE PIM.

Unifying the PIM backend

File:KDE PIM small.png

Changes to the KDE PIM architecture over the years

At the KDE PIM sprint in 2004, deeper integration was decided upon: the creation of one unified layer instead of the separate data storage locations. This was to be part of the "pillars of KDE 4", the re-architecturing going on for the major new release from the KDE community.

It took significant time to develop this new PIM backend named "Akonadi". During the KDE SC 4.x series up to 4.4, a 'hybrid' solution was shipped to KDE users, with some applications using the new backend while others still were their monolithic selves.

The first release of the KDE PIM backend

Finally, in 2011, KDE SC 4.7 brought 5 years of hard work to the users with the first "Akonadified" release of KDE PIM. Mainly Akregator, which has been barely maintained over the last year, was still except from using the new PIM backend but there were plans in development.

The first release had some growing pains but in the 6 months following the release the PIM team managed to fix the vast majorities of issues and the 2012 release of KDE PIM 4.8.0 brought a quite stable and useable product to KDE's userbase.

Since then, development has been focusing on performance and stability. The PIM release part of KDE's Software Compliation 4.9 and 4.10 introduced vast improvements to indexing speed and loading of folders, in part due to better utilization of the Nepomuk-provided search and indexing functionality and improvements in Nepomuk itself.

Historical Findings

This piece collects antique bits and pieces that were saved from the bit-bucket during the long years of development on KDE PIM. This is surely not a complete historical document, but it collects some notes and odds and ends that were interesting at some time.

Kandy

Kandy was the attempt to provide a KDE application for syncing data between the desktop and mobile phones. It worked reasonably well for a small set of phones but never left the ground to become a general-purpose framework or application. See the original Kandy home page for some more details.

Pre-Kontact

The current wrapper-application that puts all the KDE PIM applications together is Kontact. Sometime in pre-history, something similar existed:

twister is a quick sketch of an Outlook-style PIM application. It is basically a top-level window which embeds KParts. It was used to test embedding Empath and KOrganizer components and currently serves no useful purpose.

Data Storage

Data storage (addresses, mail, etc.) has been a persistent theme within KDE PIM (hey, it's all about data management), and one proposal that has come up fairly often is the centralized data storage -- either as a database server, or a daemon, or something else. (Portions elided -- original dated 2002-01-09)

Nick Papadonis posted KPIM-RFC 1 which covers KDE PIM Data Access.

Background

KPIM has progressed much over time such that data management is achieved with multiple applications; KMail, KAddressBook, KOrganizer, KPilot, Kandy, KArm, KNotes, KAlarm. Each application stores its configuration and PIM data using a different file formats and unique datastore. This is currently very functional and appreciated by all, however, it would be desirable to place all PIM data in single file with a common format and provide an easy API for extracting data.

General Data Access

There should be a common methods for ALL applications to access PIM data. I envision the use of a lightweight client/server relational database.

DATABASE SERVER PROCESS <----> DATABASE CLIENT PROCESS

Let the DB server be present on the machine running PIM applications. Each application uses the appropriate DB client methods to access data in the datastore.

When using a database for accessing and storing data, many uncertainties and debug efforts are elimanted from development.

Class Hierarchy

                                +-->KORG DATA PXY
                                |
SQL CLIENT API-->KPIM DATA PXY--+-->KNOTES DATA PXY
                                |
                                +-->KMAIL DATA PXY

Proposal for libksync

This is what Cornelius Schumacher posted on Jun, 19 2001 on the kde-pim mailing list concerning "Proposal for a KDE syncing Library - libksync":

I would like to throw in yet another proposal for the kde-pim architecture. I think it is complementary to the other proposals, but is more focused at what we have now and how to evolve from there. I couldn't resist to also draw a diagram, see attachment.

What I propose is libKSync, a library for handling synchronisation of data like calendars and addressbooks. This lib provides synchronisation logic operating on abstract Sync classes (e.g. an algorithms to resolve conflict by scanning data entries, checking if they are equal or have changed, resolving conflicts according to the preferences of the user, logging data modifications etc.). The abstract Sync classes provide functions for determining, if entries are equal, return unique identifiers, copy data, etc.

To be useful the abstract Sync classes have to be subclassed by concrete Sync classes operating on the specific data objects for calendar, addressbook or any other concrete data. That means the overall sync logic is only implemented once and the data specific logic is implemented once per data type.

For representing a data type there is one class for each type. There is a class for calender data, a class for addressbook data etc and the good thing is, that we already have this classes in libkcal and libkab.

The KDE-PIM applications operate on these data classes. There are apps that have full viewing and manipulation capabilities like KOrganizer and KAddressBook and others, which are mainly for reading/writing data from other devices. But they are all similar in one respect, they take data from a source and store it in an object of the corresponding data type. The source can be a Palm Pilot, an iCalendar file, libkab, a server, whatever and the access to this data can also vary, e.g. per serial interface or USB, per kioslave, per some sort of server access over the network. The mechansim how to access the data in combination with the handling of the format of the data and the conversion to the standard object for a type is a conduit.

Let's take an example of a synchronisation process: KOrganizer would like to synchronize a local calendar file with a file on an FTP server. So it loads both files into Calendar objects and then uses libKSync to bring them in sync, it then stores them back to their original locations. Another example would be syncing of Pilot data with KOrganizer. Here KOrganizer would ask KPilot for the data, would then perform the sync with libKSync and ask KPilot to write the synced data back.

The control of the syncing process needs not to be in a specific application. Currently it is implemented this way, but it is possible to put this functionality into a separate Syncer application (KitchenSync?), which is accessed via DCOP or something similar.

What are the benefits of this architecture?

• It reduces duplication of code. Syncing algorithms and functions to convert data formats would be seperated and only implemented once per data type
• It is extensible. Adding a new device or format (e.g. XML) is only a matter of implementing the conversion to the standard data object
• It is reusable. The syncing lib could also be used in a server, which has to sync data (e.g. a SyncML server)

What is necessary to implement this?
Most of the code is already there. In some cases it even is there multiple times like the syncing algorithms used by the different KPilot conduits and Kandy. We only have to rearrange and clean up the code, add the abstract interface classes and implement what is missing in terms of syncing functionality.

That's it for now. What do you think? Questions and comments are very welcome.

The road to Kolab

About the Kroupware contract

In 2002 Erfrakon, Intevation and Klarälvdalens Datakonsult AB have won a bid to write a Free software groupware server and client for the German "Bundesamt für Sicherheit in der Informationstechnik" (Federal Agency for IT Security, BSI). It aims to work in a heterogenous environment and provide email, contacts, appointments and tasks lists.

Kroupware is the current contract title for the contract named above. After finishing the Kroupware contract, the resulting software will be further integrated in the K Desktop Environment and in the KDE-PIM family.

Matthias Kalle Dalheimer posted a preannounce of the Kroupware contract to several KDE developer mailinglists. You find Kalle's preannounce in the kroupware mailinglist archive.

For further information and discussion, please refer to the official webpage of the contract at [1].

The road to KDE 3.0

Cornelius, ever watching the direction we are going, posted a roadmap in august of 2001. This was updated on the 21st.

Areas of development for kdepim 3.0 (details follow below):

• Group Scheduling
• KDE-PIM Web Site
• Address Book API
• Syncing
• KArm
• KAlarm
• Plugin Interface for KOrganizer
• KPilot
• KOrganizer todo list

Group Scheduling

RFC 2445-2447 specify an email-based protocol to exchange calendar information about appointments, todos etc. This allows to announce events, organize meetings, browse calendars of colleagues for free time, let a schedule be managed by a secretary and many other things. The protocol and used formats are widely accepted as a standard and allow interoperability with a wide range of other calendar tools, from Evolution to Outlook. Note that this does not involve a calendar server.

STATUS: Most of it is already implemented in libical, libkcal and KOrganizer. First iteration of integration with KMail is done.

TODO: Implement a free-busy view, gradually add support for the full specification.

Address Book API

As we all know, the KDE address book API needs a refresh. I propose to design it as a set of value-based, implicitly shared classes along the lines of the entries specified in the vCard standard (RFC 2426) and implement this on top of a backend using standard vCard text files as storage method. This can be extended in the future by more sophisticated backends like the ones provided by kdepim/kab2.

STATUS: Planning, a vCard parser exists in kdepim/vCard.

TODO: Decide on an interface design and implement it. I'm not sure if that can be done in time for kde 3, but I would consider this as one of the most important projects within kdepim.

Syncing

Syncing calendar and addressbook data between different computers, PDAs or other devices is a recurring task within the kdepim area. To maximise code reuse and make the development of support for new devices as easy as possible, syncing should be unified in a common architecture. This is also interesting for other data types like bookmarks, config files etc. It might involve some daemon process. We have to consider how to handle multiple users.

STATUS: There is kdepim/ksync, which contains a library with implementations for calendar and bookmark syncing and a standalone-app to perform network transparent syncing on these data types. The lib needs more work and more data types have to be implemeted.

TODO: Finish design and API of the general syncing lib, implement address book syncing based on the new address book API.

KArm

The time tracker application KArm is now part of kdepim. The new maintainer Tomas Pospisek already has some plans to achieve a better integration with kdepim, especially with KOrganizer.

STATUS: KArm is nice and stable, currently lacks the intergration with kdepim.

TODO: Do the integration.

KAlarm

KAlarm is another new application in kdepim written by David Jarvie. It is a small tool to quickly setup alarms without the need for a full blown calendar app. It shares the alarm daemon with KOrganizer. Adding multiple alarms for events and having triggered alarms send an e-mail message would be nice. One other thing for the alarm daemon, which was originally Davids idea: We could split the alarm daemon in two parts, one low level part handling the parsing of the alarm data and and abstract kind of notification and another part doing the presentation of the alarm like opening a message box or sending the mail as response to the notification by the low level alarm daemon. This would it make easier to exchange the presentation layer and would allow to make the low-level daemon a non-gui version, which would allow to run the daemon independent of the user being logged in.

STATUS: The app is complete. It probably needs some testing, in particular, if the alarmd daemon works well for both KAlarm and KOrganizer.

TODO: Testing

Plugin Interface for KOrganizer

One can think of many extensions to a calendar, not all of them useful for everybody. To make addition of such extensions more easy, the CVS version of KOrganizer now has a plugin interface. The holidays are now provided as a plugin and I have a hacked version of kmoon on my harddisk, which provides moon phases for each day within KOrganizer. Other applications would be things like historic data, daily cartoons, astrological data, monthly pictures or other things known by paper calendars. This data could also be taken directly from the web. Another type of plugin could call scripts in a generic way and put their results into the calendar.

STATUS: A first iteration of the plugin interface exists.

TODO: Complete the interface API, implement more plugins.

KPilot

• re-using gnome-pilot conduits
• check if jpilot conduits can be reused
• making one device-sync systray applet
• figuring out devfs support for kpilot
• evaluating KDE apps for suitability for syncing with PDA's

KOrganizer Todo List

From Scott:
Being able to save the state of my todo configuration would be great. If I reorder the columns or have certain sub todo groups expanded or compacted it would be nice if this would stay the same after I add a new todo or if I close and restart the app (like every morning after I sync up my korganizer at work w/ the changes I made at home). Color coding for todos due soon or of a certain category would be cool as well. Adding recurrence to todos is also something I would like to see.

KDE PIM branches

There have been a few "big" branches in KDE PIM history, most notably ones coinciding with Osnabrück meetings during a freeze. Bo Thorsen wrote an explanation:

kroupware_branch is for the old release only. It runs with KDE 3.1.4 and you can see the build instructions on http://www.kroupware.org/howto-kolab-kde-client.html. With this, you get the kroupware project packages. Almost no development has been done on these packages for the last six months though.

KDE 3.2 can be downloaded with the released tarballs or with the cvs branch KDE_3_2_BRANCH (cvs checkout -r KDE_3_2_BRANCH module, where module is kdebase, kdelibs, kdepim...). KDE releases are always tagged like KDE_3_2_0_RELEASE, so you can fetch that one with cvs checkout -r KDE_3_2_0_RELEASE module. With this, you get the Kontact that has the new KDE 3.2 developments and most kroupware_project features, but misses the important groupware scheduling parts.

kdepim development branch is done in cvs HEAD (cvs checkout kdepim), and compiles and runs with all KDE 3.2 releases/branches. This is the development we all think of as a "Kontact 1.0" release, where one of the features is that Kontact will be a full Kolab client.

These are the three options you have. All others are irrelevant now.

One Billion Seconds

Old versions of KMail will stop working correctly on September 9, 2001. On that date, a number overflow may cause KMail mail folders to be corrupted when using KMail version < 1.0.29.1. If you are using version 1.0.29.1 or 1.0.29.2, the dates of new mails will not be shown correctly anymore, but no mail will be lost. This is actually the problem raised by the passage of one billion seconds since the beginning of the Unix epoch at 00:00 on 1 January 1970.