The KDE PIM community has been working hard in March and April to fix various bugs and bring new features and improvements to Kontact, Itinerary and other KDE PIM projects. Here is a summary of some of the changes that landed in the past two months.

KDE PIM Sprint

The team has met for a weekend for a traditional spring PIM sprint in Toulouse at the beginning of April. We focused on solving some long-standing issues in Akonadi, prepared some of the KDE PIM libraries to be moved to KDE Frameworks and discussed outreach and how to attract more people to KDE PIM.

You can read David’s report from the sprint.

KMail

KMail has received a nice batch of bugfixes for the KDE Applications 19.04 release as well as some small features, which will be available in the 19.08 release.

Improvements and New Features

The email composer has received improvements to support UTF-8 emojis (Laurent Montel, KDE Applications 19.08) and to support composing rich-text emails using Markdown with syntax highlighting (Laurent Montel, KDE Applications 19.08).

Additionally, notification when sending email fails is now less intrusive (Laurent Montel, KDE Applications 19.08).

Bugfixes

• Fixed crash on shutdown due to mishandling of QWebEngineProfile (Jonathan Marten, D19559)
• Fixed reconnecting to IMAP server when connection is lost during login (David Faure, D20028)
• Fixed identity not appearing after creation (Laurent Montel, bug 391631)
• Fixed deleting spam messages (Laurent Montel, bug 406324)
• Fixed custom header search query failing with inconclusive error (Laurent Montel, bug 405456)
• Fixed trying to send a message without any recipients fails quitely (Laurent Montel, bug 402577)
• Fixed a crash switching on an ad-blocked (Laurent Montel, bug 405434)
• Fixed a crash when opening a vCard attachment (Laurent Montel, bug 405791)

KOrganizer

During the PIM sprint, Volker Krause did a major work on cleaning up KCalCore, a library that implements the iCal standard to store events and tasks information, so that the library can be moved to KDE Frameworks. You can read more about Volker’s effort in his blog post.

New Features and Improvements

It is now possible to move event from one calendar to another in KOrganizer by changing the calendar in the incidence editor (David Faure, KDE Applications 19.08).

KAddressBook

Volker also worked on preparing the KContacts library, a vCard standard implementation, to be moved to KDE Frameworks alongside KCalCore. You can read about this on his blog as well.

Improvements

As part of his code cleanup, Volker has removed the DataMatrix from contact display, so we only display the QR code now (Volker Krause, KDE Applications 19.08), and did massive clean up of the contact display code. When displaying contact address on a map, KAddressBook now defaults to OpenStreetMaps (Volker Krause, KDE Applications 19.08).

Thanks to Laurent Montel it is now also possible to send SMS messages from KAddressBook through KDE Connect (KDE Applications 19.08).

Bugfixes

• Fixed dialing contact’s phone-number from KAddressBook through KDE Connect (Volker Krause, D20353)

Akregator

Bugfixes

• Make the feed URI the baseUrl for article previews (Pierre Ducroquet, D19739)
• Fixed Delete key not working after using search (Laurent Montel, bug 394946)

Common Infrastructure

Improvements

David Faure and I have spent large part of the PIM sprint investigating and digging into two major issues we currently have in Akonadi: database deadlocks and a bug known as “multiple merge candidates”. We were unable to determine the real cause for the “multiple merge candidates” bug, so it remains unfixed for now. As a result of digging through the code base, however, David has produced a set of patches to hugely improve handling of database deadlocks and transaction rollbacks in the Akonadi server. He also did improve the sync scheduling code. I have removed a large chunk of code by removing mostly unused code to handle Collection references and LDAP/Kolab contact merging.

New Features

• LibKGAPI: support for Team Drives API for Google Drive (David Barchiesi, T10521)
• KItinerary: support for parsing train booking from SNCF confirmation emails (Volker Krause, bug 404451)

Bugfixes

• Akonadi: fixed sync getting stuck after failure (David Faure, D19487, bug 399167)
• Akonadi: fixed race conditions in Attribute handling (David Faure, D19556, D19632)
• Akonadi: fixed crash when in resources when handling a collection change (Daniel Vrátil, bug 403642)
• Akonadi: fixed crash when an Akonadi client unexpectedly disconnects from the server (Filipe Azevedo, D19983)

Help us make Kontact even better!

Take a look at some of the junior jobs that we have! They are simple, mostly programming tasks that don’t require any deep knowledge or understanding of Kontact, so anyone can work on them. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you and answer all your questions. Read more here…

Last year I wrote about Plasma Pass, a Plasma applet for the Pass password manager. Over the couple last months I got some emails from packagers from various distributions asking for a proper release so they can package it…so here it is, proudly announcing Plasma Pass 1.0.0.

Here’s a video of how it works:

Tarball:

https://download.kde.org/stable/plasma-pass/plasma-pass-1.0.0.tar.xz

Checksum:

SHA-256: 9821c5ad80e4370dce002855bd0300d234feec249258b01fafd9e3a9241cbc66
SHA-1:   8d877b4d4bbbbf12890ba77c03c5cdd62b8279d6
MD-5:    69928b6df32ba82d0436a6d0abf73a8a

Signature:

0ABDFA55A4E6BEA99A83EA974D69557AECB13683 Daniel Vrátil <dvratil@kde.org>

Feel free to report any issues or feature requests to KDE Bugzila.

Here’s a little tip how to get DrKonqi, the KDE crash handler to work in applications that use QtWebEngine.

If your application uses QtWebEngine, you probably noticed that DrKonqi doesn’t pop up when the program crashes. This is because QtWebEngine installs its own crash handler, overriding the one DrKonqi has set up.

The workaround is quite simple but is not trivial to find because all of it is undocumented (and not everyone wants to dig into Chromium code…). The trick is to add --disable-in-process-stack-traces to QTWEBENGINE_CHROMIUM_FLAGS environment variable before initializing QtWebEngine:

const auto chromiumFlags = qgetenv("QTWEBENGINE_CHROMIUM_FLAGS");
if (!chromiumFlags.contains("disable-in-process-stack-traces")) {
    qputenv("QTWEBENGINE_CHROMIUM_FLAGS", chromiumFlags + " --disable-in-process-stack-traces");
}
...
auto view = new QtWebEngineView(this);
...

Here’s a full example of how we fixed this in Kontact

What is Flatpak

Flatpak is a new way of distributing applications.  Each application runs in its own isolated environment called sandbox with all its dependencies being provided as part of the Flatpak and with no access to other programs. This way every user runs the exact same application in the exact same environment no matter what Linux distribution they use.  The applications inside the sandbox are also limited to what system resources they can access, which provides greater security.

Kontact Stable on Flathub

The latest stable version of Kontact is now available on Flathub, the official app store for Flatpaks. So even if your distribution does not ship the latest version of KDE Applications or Kontact you can now easily install it from the Kontact Flatpak without having to worry about breaking your system.

How to get it? Follow this setup guide on Flathub to install Flatpak and to set up the Flathub repository.

Then you can install Kontact Flatpak from Discover or you can install it manually from terminal:

flatpak install flathub org.kde.kontact

You should see Kontact icon in your installed apps, or you can run it manually from terminal again:

flatpak run org.kde.kontact

Your feedback is welcomed, taming the entire Kontact to run inside of a tiny sandbox wasn’t an easy task and we may have missed something. You can report issues in the Kontact Flathub repository, reach us on the KDE PIM mailing list or via IRC on the #kontact channel on Freenode.

Kontact Nightly

We also have nightly builds of the Kontact development branch. You can use the development Kontact Flatpak to see if your bug has been fixed, to help us with testing Kontact and to provide early feedback on new features before they are released. The nightly Kontact Flatpak runs completely isolated from the stable Flatpak as well as from system-wide installation, which means that your data and configuration are completely safe. We hope that offering this easy and safe way of running the latest development builds of Kontact to a wide audience we will receive more feedback and early testing, leading to better and more polished releases in the future.

To install the nightly build you first need to add the kdeapps-testing Flatpak repository:

flatpak remote-add --if-not-exists kdeapps-testing --from https://distribute.kde.org/flatpak-apps-testing/kdeapps.flatpakrepo

And install Kontact Nightly:

flatpak install kdeapps-testing org.kde.kontact//master

To run the nightly version, use

flatpak run org.kde.kontact//master

You can read a more detailed guide to Flatpak on our community wiki.

Remember to run flatpak update` from time to time to always get the latest version.

Help us make Kontact even better!

Take a look at some of the junior jobs that we have! They are simple, mostly programming tasks that don’t require any deep knowledge or understanding of Kontact, so anyone can work on them. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you and answer all your questions. Read more here…

Today KMail has gained a new cool feature that has been repeatedly requested in the User survey last year as well as on forums and social networks: Unified mailboxes.

Unified mailboxes offer not only a unified inbox - a single “Inbox” folder showing emails from inboxes of all your accounts, it also provides unified sent and drafts folders by default. But we did not stop there: you can create completely custom unified mailboxes consisting of any folders you choose. You can even customize the default ones (for example exclude an Inbox from a particular account).

Some obligatory screenshots:

The feature will be present in the December release of KDE Applications.

Do you want to help us bring more cool features like this to Kontact?

Then take a look at some of the junior jobs that we have! They are simple mostly programming tasks that don’t require any knowledge of Akonadi or all the complexities of Kontact. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you. Read more here…

Do you want to help us improve your favorite PIM suite but you were always scared by its size and complexity? Well, fear no more! We have collected a bunch of simple and isolated tasks in various parts of the PIM suite that require none or just very basic understanding of how the entire Kontact and Akonadi machinery works. We have documented them and we are prepared to guide you and help you to accomplish the tasks. Those are small simple tasks, but they will make many users (and PIM developers) very very happy.

I’m in! What do I need to know?

Just some C++ and maybe a bit of QML if you want to go for the QML tasks. We don’t expect you to know anything about Akonadi or how the entire PIM thing works as most of the tasks are pretty self-contained (although you can read up on the basic concepts and architecture if you are interested).

Cool, what do you offer?

We have tasks to improve the look of KAddressbook’s contact list, contact view, and contact editor. If you prefer working on KOrganizer, you can help us to make the event view look more modern. We would also like to improve the Account Wizard experience by porting it to QML and improving Gmail/Google Calendar and Contacts integration. Of course, the key part of Kontact is KMail and even there we have a few places that can be improved: we would like to improve the IMAP quota warning and add support for Autocrypt. And finally, you can also make life easier for other KDE PIM developers by improving our debugging tool, the Akonadi Console: we want to be able to save the output into JSON and load it again, alphabetically sort some of the lists, make working with the DB console a bit more comfortable and be able to restart Akonadi agents whenever we want to.

There’s also a very cool effort ongoing to allow integration between Kontact and MyCroft, the opensource voice assistant. For this, we need help improving a command line tool that’s used as a bridge between MyCroft and Kontact.

If you don’t know any programming but you would still like to help, we have some non-programming tasks as well! Sure! We are working on a new website for Kontact and we could use help with both design and writing content for it! We also need help improving our user documentation, cleaning and updating our wikis on community.kde.org and userbase.kde.org or cleaning up our bug tracker. If you want to help with any of that, get in touch with us on the kde-pim mailing list!

You can find the full list of junior jobs on Phabricator.

Haven’t found anything interesting? Don’t worry, we will keep adding more over the time, so just check the list every now and then. Or do you have your own idea how to improve KDE PIM and you just don’t know where to begin? Get in touch with us and we will help you!

Now how do I get started?

1) Get in touch with us

To make sure several people won’t try to solve the same thing, it is the best to get in touch with the PIM community first so we can look at the single topics in more details. Some of the descriptions of the tasks are intentionally a bit vague as there are multiple ways how to approach or solve the problem. It’s always better to talk about the options first so that no time is wasted on approaches that won’t work.

2) Get your development environment set up

The KDE PIM community wiki contains articles on how to develop KDE PIM inside a Docker container. Alternatively, as most of the changes are pretty isolated, you should be able to compile just a single component from source against your distribution packages (you will just need to install some development packages first).

3) Pick a task

Pick one of the tasks linked above, or just look at all the junior job in Phabricator. They span different topics, different components and are of different complexity and size. If you find a particular task that you would like to work on, assign it to yourself and get working! If someone else already has the task assigned, you can ask if they maybe want some help, or just look for another task.

4) Get to work!

Fire up your favorite IDE and start working! If you need any help with the task - from finding the right repository and code, through getting the program compiled to being stuck on a bug or something not working - just ask us! You can ask in the Phabricator task or send an email to the kde-pim mailing list and some of the PIM devs will help you.

Also, don’t feel limited by the description of the tasks - feel free to do only part of the task, or do even more than what’s in the task description. If you think you have a better idea how to solve something, let us know in the Phabricator task.

I have the code, what’s next?

Awesome! Now it’s time to upload the code for review. You can use the arcanist command line tool, or you can just generate a diff and upload it manually via the web interface. Don’t worry if you don’t know whom to assign for review, Phabricator sends the notification the entire PIM team automatically.

You may have heard about pass, the standard Unix password manager. I learned about it from Milian Wolf some months ago and I really liked it for its simplicity, respect for privacy and multiplatform support. And so over the past months, I started to slowly change my passwords to randomly generated ones stored in pass.

To get a password from pass, you simply type pass -c SomePath/SomeService into console and pass will copy the password straight to your clipboard. Super simple. Slightly less comfortable when you are dealing with websites though. Luckily there’s a wonderful browser extension called browserpass that can fill online login forms with a single click and has automatic password matching based on the current domain.

But sometimes even I am simply too lazy to open Yakuake and type in a command, so I started looking for some GUI. There’s qtpass, but that’s not exactly what I was looking for. And so I dusted off my QML knowledge and wrote Plasma Pass: a systray Plasma applet to quickly find your password and copy it into the clipboard with a single mouse click. The applet also takes care of removing the password from the X11 clipboard as well as Klipper after 45 seconds so it won’t leak accidentally through your clipboard history.

The source code is currently available in my scratch repo: https://cgit.kde.org/scratch/dvratil/plasma-pass.git/ plasma-pass.git repo: https://cgit.kde.org/plasma-pass.git.

And now back to fixing Akonadi ;-)

This blog post is long overdue, but now that I’m back home from the KDE PIM Sprint in Toulouse, which took place last weekend, there’s some more news to report.

Akonadi Improvements

On the sprint, I finally finished and merged a new improvement in Akonadi called Notification Payloads. I will not go into the technical details here, the most important thing is that this new improvement will notably reduce the CPU and disk load in Akonadi, especially during intensive operations like email sync. It should also help with the long-standing issue regarding errors and email duplication when using POP3 and local mail filters. Finally, this new feature opens doors to further improvements and optimizations like server-side change recording (technicalities here) and ultimately being able to shut down Akonadi Resources when they are not needed and start them on-demand, thus saving some more resources.

As I was touching the internal notification system in Akonadi I also improved the relevant debugging tools in Akonadi Console, our developer and debugging tool for Akonadi. Based on input from Sandro I also added Logs view. Thanks to that it’s now possible to see debug output from all running Akonadi applications straight in the Akonadi Console without the need to restart Akonadi or the application from the terminal to see the debug output. This will make it easier for users to provide us with relevant information to help us debug and solve their Akonadi issues.

Kontact Improvements

This was just a minor change, but it finally solved my long-standing issue with Kontact and Breeze: the side-pane icons to choose between different Kontact modules were colorful - the only non-monochromatic part of Kontact which was so obviously not fitting into the rest of the UI. With a tiny change, the icons are now also monochromatic, making the Kontact window look more uniform.

Native Gmail authentication for IMAP and SMTP

For a while now the IMAP resource supports logging into Gmail accounts using the so-called OAuth method, where you provide your credentials into the Gmail login window which also supports two-factor authentication. The IMAP Resources was forcing the OAuth method with Gmail for everyone, but this requirement has now been relaxed. Although the IMAP resource will choose this method by default it’s possible now to also choose the traditional authentication methods like with any other email provider.

Secondly, the OAuth support has finally landed also into our SMTP module which is used for sending emails, so if you select this method in your Outgoing account configuration with Gmail, you no longer need to use “App-specific passwords” from Gmail.

Syndication Cleanup

The Syndication library is used to retrieve and parse RSS and ATOM feeds and is used among others by Akregator. We have now cleaned up the library and removed some redundant dependencies so that we will eventually be able to move it into KDE Frameworks so that even more applications can benefit from it.

Going to Windows

Thanks to a huge effort from Hannah we are now able to build Akonadi and other parts of the KDE PIM stack on Windows. While we are still a long way away from having Kontact properly running on Windows, we managed to get Akonadi to work on Windows with some other programs. Windows is a huge platform and Kontact with all its features and functionality could be a good competition to established PIM solutions there and a huge potential to grow our user and developer base. While we still focus primarily on Linux, we are slowly looking forward to extending our reach to Windows.

Bugfixes

A lot of them. Big thanks to David Faure who spent a big part of the weekend debugging his IMAP resource to figure out why it keeps getting stuck on occasions. He fixed several issues in the IMAP resource so that it properly reconnects after server connection is lost or times out (due to poor internet connectivity usually) and also found and fixed some issues in Akonadi syncing code.

Future

What’s next then? We will continue to work towards a stable release for Windows, and hopefully soon finish the rewrite of the indexing and search infrastructure in KDE PIM to make it faster, reliable and more useful again. There’s also a lot of smaller tasks and improvements to look into during the year.

Back in August, we ran a survey to get input from our users and get a better understanding of how they use KMail. First, let me start by thanking everyone who took their time to fill in the survey. We collected over 3000 responses which is much more than we expected. Thank you very much! We got some interesting numbers and data from the survey, which I’ll analyze later, but to my big surprise, the most interesting part was the comments that many of you left at the end of the survey. We got over 1000 comments which provided us with a consistent feedback from the userbase. In this and the next blog posts, I want to address the common themes, complaints, and remarks that appeared in the comments, address the concerns raised and present some action plans that we are going to take to address those.

What’s going to happen to KMail when Kube is released

Nothing. Kube is not going to replace KMail. Both projects will co-exist and be continuously developed and improved. Each project targets a slightly different audience - KMail focuses on power users with several accounts and high volume email traffic, while Kube focuses on less advanced users. Kube will not, and is not attempting to, provide many of the advanced features that KMail has, instead, it will help KDE to provide a friendly PIM solution for all users.

“Akonadi should die”

We hear this a lot: “Akonadi sucks”, “Akonadi should be replaced”, etc. While we understand the frustration of many users who are having huge troubles with Akonadi, we are committed to it and we are convinced it is the right way to go. Akonadi got a lot of bad reputation in the early days, but we worked very hard to fix the bugs and improve the performance over the years and we will continue to do so in the future. I took some more detailed notes, so I have an idea what people dislike about it the most and will try to focus towards solving that. There also seems to be a lot of misconception as to what Akonadi really does and it is often blamed for things it’s not directly responsible for. I might write a detailed blog post about that sometime in the future.

Some of you also mentioned Sink: Sink (formerly known as Akonadi-next, but no longer has anything to do with the current Akonadi) is the backend used by Kube. At this moment Sink is not mature enough and lacks the capabilities and features we need in order to be viable as a replacement for Akonadi. I personally like the concept and some of the ideas behind Sink and I will try to get some of them into Akonadi and I hope that in the future we will be able to co-operate with Sink more closely.

Search is broken/useless

I was honestly surprised with the amount of complaints about the search feature of KMail - generally it boils down to searching not being reliable and returning wrong or no results. I took a good look into our indexing and searching code and indeed found numerous issues with the way we index data and query them. I already created a plan how to solve this and make search fast, reliable and giving more precise results. The work on this is already under way, the fixes to actual indexing and searching code will be available in the December release, with more high-level fixes (indexing speed, better search UI, result presentation etc.) coming next year.

Account management

This issue has been on our radar for some time now and it’s good to know that it’s something you want us to target and fix as well. Adding a new email account to KMail is super hard, you have to basically do three things - create the incoming email account, create an outgoing email account, create the identity, and then you need to re-visit each of the accounts to configure them. We have the Account Wizard (Settings -> Add account…, or Tools -> Account Wizard in older KMail versions) which simplifies adding new accounts, but it’s not fully there yet and adjusting a configuration later on is still a painful experience spread across three different dialogs and many mouse clicks.

I took a look at what Thunderbird and Evolution do, and I also looked at K9, an opensource email client for Android, which is very close to KMail in the terms of account configurability (different identities per account, different outgoing accounts per identity etc.) and how their UI works. All in all, we have a lot of inspiration and examples to follow and build on, and we will work with the KDE VDG and our UX experts to overhaul the account management in KMail, making it much smoother and nicer experience.

Feature Discoverability

KMail has myriads of features, but many of them are well hidden from our users, almost as if we did not want anyone to use them. I’ve often seen comments like “It would be nice if KMail could do X” or “Until this survey, I did not even know KMail could do Y”. So, we need to fix that too. There are several ways how to do that, I think the best one is a combination of all: we need to improve our UI and make features easily discoverable and ready at hand so that they are easy to use. Secondly, we need to bring our documentation up to date. Couple years ago Scarlett did a great job at updating our documentation, but there’s still so much that is not covered. Finally, we need a good and easily accessible feature overview, ideally on a web page or wiki. We would welcome any help with writing and updating the documentation - this does not require any programming skills, just some free time and will to help :-) Get in touch with us on the kde-pim@kde.org mailing list!

More coming soon in part 2…

Let me start by annoying you with some pictures:

And now for the serious part: in my [last blog post]({% post_url 2017-09-14-randa-report-the-fall-of-kdatetime %}), I talked about achieving our main goal for this year’s Randa meetings - we successfully ported the entire Kontact away from the obsoleted KDateTime class. Since we finished this on Thursday, there was still enough time left to start working on something new and exciting.

Volker and Frederik went on to work on a KWin plugin to simulate various kinds of color blindness which will help developers to see how visually impaired users see their software, I did a bit more code clean-up after the porting and a bit of code-review.

On Friday morning Volker and I discussed search in KDE PIM. Broken and unreliable search was one of the most often mentioned issues in the KMail User Survey, which we ran for the past month and a half, so I decided to tackle the problem and make our indexing and searching infrastructure fast and reliable.

The final plan consists of several phases - starting with reorganizing our current code to put related pieces of code (like email indexing and querying code) into a single place to make the code easier to maintain. This phase is already progressing and I hope to finish it within the next week. The second phase will involve moving the code that is responsible for indexing data into the backend Resources - whenever a backend retrieves an email from an IMAP server or an event from Google Calendar it will also index it and will send the index data alongside the raw data to Akonadi. Akonadi will then take care for just writing the data into the Xapian database. This will speed up indexing, reduce the IO load and will ensure that all data are reliably indexed and stored before they are presented in Kontact. The third phase will involve changing Kontact to query the index database directly, instead of asking Akonadi to do it for us. This will speed up the search and provide results faster. The final phase will focus on which data we are actually indexing. As they say - less is sometimes more - so having fewer, but better-defined data will allow us to provide better and more exact search results to the user.

Once this is settled, we can make applications to depend on the search index - for example KOrganizer will be able to query the index to only get events from e.g. December 2017 instead of fetching all events from the calendar and then figuring out if they should be displayed or not, making calendars of even the busiest people to load instantaneously.

All in all, it was an extremely productive hackfest for the PIM team and I’d again like to thank Mario, Christian and the rest of the Randa team for organizing the hackfest. You guys rock!

And remember, you can still donate to the Randa fundraiser to make future productive sprints possible!