Today KMail has gained a new cool feature that has been repeatedly requested in the User survey last year as well as on forums and social networks: Unified mailboxes.
Unified mailboxes offer not only a unified inbox - a single “Inbox” folder showing emails from inboxes of all your accounts, it also provides unified sent and drafts folders by default. But we did not stop there: you can create completely custom unified mailboxes consisting of any folders you choose. You can even customize the default ones (for example exclude an Inbox from a particular account).
Some obligatory screenshots:
Unified Mailboxes in KMail
Unified Mailboxes configuration
The feature will be present in the December release of KDE Applications.
Do you want to help us bring more cool features like this to Kontact?
Then take a look at some of the junior jobs that we have! They are simple mostly programming tasks that don’t require any knowledge of Akonadi or all the complexities of Kontact. Feel free to pick any task from the list and reach out to us! We’ll be happy to guide you. Read more here…
Do you want to help us improve your favorite PIM suite but you were always scared by its size and complexity? Well, fear no more! We have collected a bunch of simple and isolated tasks in various parts of the PIM suite that require none or just very basic understanding of how the entire Kontact and Akonadi machinery works. We have documented them and we are prepared to guide you and help you to accomplish the tasks. Those are small simple tasks, but they will make many users (and PIM developers) very very happy.
I’m in! What do I need to know?
Just some C++ and maybe a bit of QML if you want to go for the QML tasks. We don’t expect you to know anything about Akonadi or how the entire PIM thing works as most of the tasks are pretty self-contained (although you can read up on the basic concepts and architecture if you are interested).
There’s also a very cool effort ongoing to allow integration between Kontact and MyCroft, the opensource voice assistant. For this, we need help improving a command line tool that’s used as a bridge between MyCroft and Kontact.
If you don’t know any programming but you would still like to help, we have some non-programming tasks as well! Sure! We are working on a new website for Kontact and we could use help with both design and writing content for it! We also need help improving our user documentation, cleaning and updating our wikis on community.kde.org and userbase.kde.org or cleaning up our bug tracker. If you want to help with any of that, get in touch with us on the kde-pim mailing list!
Haven’t found anything interesting? Don’t worry, we will keep adding more over the time, so just check the list every now and then. Or do you have your own idea how to improve KDE PIM and you just don’t know where to begin? Get in touch with us and we will help you!
Now how do I get started?
1) Get in touch with us
To make sure several people won’t try to solve the same thing, it is the best to get in touch with the PIM community first so we can look at the single topics in more details. Some of the descriptions of the tasks are intentionally a bit vague as there are multiple ways how to approach or solve the problem. It’s always better to talk about the options first so that no time is wasted on approaches that won’t work.
2) Get your development environment set up
The KDE PIM community wiki contains articles on how to develop KDE PIM inside a Docker container. Alternatively, as most of the changes are pretty isolated, you should be able to compile just a single component from source against your distribution packages (you will just need to install some development packages first).
3) Pick a task
Pick one of the tasks linked above, or just look at all the junior job in Phabricator. They span different topics, different components and are of different complexity and size. If you find a particular task that you would like to work on, assign it to yourself and get working! If someone else already has the task assigned, you can ask if they maybe want some help, or just look for another task.
4) Get to work!
Fire up your favorite IDE and start working! If you need any help with the task - from finding the right repository and code, through getting the program compiled to being stuck on a bug or something not working - just ask us! You can ask in the Phabricator task or send an email to the kde-pim mailing list and some of the PIM devs will help you.
Also, don’t feel limited by the description of the tasks - feel free to do only part of the task, or do even more than what’s in the task description. If you think you have a better idea how to solve something, let us know in the Phabricator task.
I have the code, what’s next?
Awesome! Now it’s time to upload the code for review. You can use the arcanist command line tool, or you can just generate a diff and upload it manually via the web interface. Don’t worry if you don’t know whom to assign for review, Phabricator sends the notification the entire PIM team automatically.
Back in August, we ran a survey to get input from our users and get a better understanding of how they use KMail. First, let me start by thanking everyone who took their time to fill in the survey. We collected over 3000 responses which is much more than we expected. Thank you very much! We got some interesting numbers and data from the survey, which I’ll analyze later, but to my big surprise, the most interesting part was the comments that many of you left at the end of the survey. We got over 1000 comments which provided us with a consistent feedback from the userbase. In this and the next blog posts, I want to address the common themes, complaints, and remarks that appeared in the comments, address the concerns raised and present some action plans that we are going to take to address those.
What’s going to happen to KMail when Kube is released
Nothing. Kube is not going to replace KMail. Both projects will co-exist and be continuously developed and improved. Each project targets a slightly different audience - KMail focuses on power users with several accounts and high volume email traffic, while Kube focuses on less advanced users. Kube will not, and is not attempting to, provide many of the advanced features that KMail has, instead, it will help KDE to provide a friendly PIM solution for all users.
“Akonadi should die”
We hear this a lot: “Akonadi sucks”, “Akonadi should be replaced”, etc. While we understand the frustration of many users who are having huge troubles with Akonadi, we are committed to it and we are convinced it is the right way to go. Akonadi got a lot of bad reputation in the early days, but we worked very hard to fix the bugs and improve the performance over the years and we will continue to do so in the future. I took some more detailed notes, so I have an idea what people dislike about it the most and will try to focus towards solving that. There also seems to be a lot of misconception as to what Akonadi really does and it is often blamed for things it’s not directly responsible for. I might write a detailed blog post about that sometime in the future.
Some of you also mentioned Sink: Sink (formerly known as Akonadi-next, but no longer has anything to do with the current Akonadi) is the backend used by Kube. At this moment Sink is not mature enough and lacks the capabilities and features we need in order to be viable as a replacement for Akonadi. I personally like the concept and some of the ideas behind Sink and I will try to get some of them into Akonadi and I hope that in the future we will be able to co-operate with Sink more closely.
Search is broken/useless
I was honestly surprised with the amount of complaints about the search feature of KMail - generally it boils down to searching not being reliable and returning wrong or no results. I took a good look into our indexing and searching code and indeed found numerous issues with the way we index data and query them. I already created a plan how to solve this and make search fast, reliable and giving more precise results. The work on this is already under way, the fixes to actual indexing and searching code will be available in the December release, with more high-level fixes (indexing speed, better search UI, result presentation etc.) coming next year.
Account management
This issue has been on our radar for some time now and it’s good to know that it’s something you want us to target and fix as well. Adding a new email account to KMail is super hard, you have to basically do three things - create the incoming email account, create an outgoing email account, create the identity, and then you need to re-visit each of the accounts to configure them. We have the Account Wizard (Settings -> Add account…, or Tools -> Account Wizard in older KMail versions) which simplifies adding new accounts, but it’s not fully there yet and adjusting a configuration later on is still a painful experience spread across three different dialogs and many mouse clicks.
I took a look at what Thunderbird and Evolution do, and I also looked at K9, an opensource email client for Android, which is very close to KMail in the terms of account configurability (different identities per account, different outgoing accounts per identity etc.) and how their UI works. All in all, we have a lot of inspiration and examples to follow and build on, and we will work with the KDE VDG and our UX experts to overhaul the account management in KMail, making it much smoother and nicer experience.
Feature Discoverability
KMail has myriads of features, but many of them are well hidden from our users, almost as if we did not want anyone to use them. I’ve often seen comments like “It would be nice if KMail could do X” or “Until this survey, I did not even know KMail could do Y”. So, we need to fix that too. There are several ways how to do that, I think the best one is a combination of all: we need to improve our UI and make features easily discoverable and ready at hand so that they are easy to use. Secondly, we need to bring our documentation up to date. Couple years ago Scarlett did a great job at updating our documentation, but there’s still so much that is not covered. Finally, we need a good and easily accessible feature overview, ideally on a web page or wiki. We would welcome any help with writing and updating the documentation - this does not require any programming skills, just some free time and will to help :-) Get in touch with us on the kde-pim@kde.org mailing list!
Akademy is over and so here’s a short summary of what the PIMsters have worked on during the past week.
Wiki Cleanup
Me and Volker sat down and went through all KDE PIM wikipages on community.kde.org, userbase.kde.org and techbase.kde.org. Most of our wiki pages are horribly outdated, so we tried to clean them up, remove pages that are no longer relevant or useful. With fewer pages to take care of and better overview of what all content we have, we should be able to keep them more up-to-date than we did in the past years.
Developer Story
Contributing to KDE PIM is hard and we know that. Getting all the dependencies and environment set up correctly is not trivial, and you can’t run stable and development Kontact alongside each other easily.
We decided to address all those issues and make contributing to KDE PIM substantially easier. We are working on a Docker image that has all the dependencies and environment set up, so developers just need to run a single command to build entire KDE PIM. And thanks to the containerization, it’s also possible to use the development version of Kontact in parallel with the stable version.
We hope that having ready-made development environment it will be easier for new contributors to get involved with KDE PIM. We will post a more details once the Docker image is ready.
Kontact Homepage
Right now kontact.kde.org and kontact.org just redirect to our page on the Userbase wiki. We decided that we want a simple, but professionally-looking web site to market Kontact as an actual product so that it appears more attractive to new users, especially those who will be coming from Windows in the future and contains comprehensive information for both users and developers.
KMail User Survey
During QtCon 2016 we started working on KMail user survey to get a better idea of what our user base is like, how they use KMail and what their impressions of it are. And now the survey is finally live, so please go and fill it if you haven’t done so yet.
Wayland Support
Volker has finished porting Kontact to Wayland, so if you have Qt 5.9, you can now run Kontact natively on Wayland. Our main limitation was Wayland support in QWebEngine, which we use to render emails, but that has been resolved in Qt 5.9.
Merging Exchange Support
Krzysztof Nowicki has been working on Microsoft Exchange support for Kontact for a while now. We now have plans to merge his code into kdepim-runtime repository, so if everything goes right the Exchange support will be available out-of-the-box to all our users starting with the December release of KDE Applications.
Next Sprint
We will be meeting soon again in Randa. Our main plan for the sprint is to continue with removal of KDateTime from our code, and thus making KDE PIM free of kdelibs4support.
There’s some more that I did not mention here, you can check the full notes for details.
If you can’t synchronize a folder in KMail and you are seeing “Multiple Merge Candidates” error after the synchronization fails, here’s a how to fix the folder to make it synchronize again - basically you force KMail to forget and re-sync the folder again.
Open Akonadi Console.
Go to the Browser tab.
Right-click the broken folder and select “Clear Akonadi Cache” - this will remove all emails from the folder in Akonadi. This will NOT delete your emails on the server.
Akonadi Console will freeze for a while, wait until it unfreezes (sorry, it’s just a developer tool, we don’t have a very good UX there :-)).
Logout and login to make sure all PIM components are restarted.
After login start KMail (or Kontact) and hit “Check mail". KMail will now re-download all emails from the previously broken folder. This may take a while depending on how large the folder is and how fast your internet connection is. After that the synchronization should work as expected.
In the upcoming KDE Applications 16.12.1 release Akonadi will have a fix that fixes the reason why the “Multiple Merge Candidates” error occurs, so hopefully in the future you should not see this error anymore.
The KDE PIM Sprint is over (unfortunately…I could do this every day :-)), so now it’s time for some recap of what has been done. I’ll try to cover the Akonadi side, and leave the rest up to others to cover their projects ;-)
Akonadi Server optimizations
We finished and reviewed Volker’s old branch with a big optimization of the database schema. On my computer it reduces size of the file with the largest table by 30% and it speeds up all queries on that database, because the WHERE condition now has to perform only integer comparision, instead of string.
This however means, that we have to migrate user’s database on start. During the migration it is not be possible to use any Akonadi-based applications. We improved the code so that the migration takes about 10 minutes on my computer (used to take 20 and more). I personally think that it’s acceptable “downtime” for a one-time migration, so after I finish testing the migration code on other backends, I’ll merge the branch to master and we’ll ship it with KDE 4.13.
There's always time for a beer!
Server-side Search
When using online IMAP, only headers are in Akonadi, the body is downloaded on-demand when the message is opened in KMail. This means that Nepomuk can’t index these emails and thus can’t include them in search results. To fix this case, we want to make use of IMAP’s SEARCH functionality. We simply ask Nepomuk to search it’s database of indexed emails, at the same time we send IMAP server the same search query and then we just merge results and show them to users. Most of the infrastructure in Akonadi Server has been in place for a long time now, so I’ll just undust it, adopt it to our current needs and we should be good to go ;-)
Using Akonadi to store tags
Working on tags!
I already mentioned this in the previous report: we want to cache tags in the Akonadi database and write them to storage backends if they support it (for instance as additional flags to emails on IMAP server, as CATEGORIES into events in iCal, etc.). Thanks to it it will be possible to share tags between multiple computers, yay! We just need to modify the Nepomuk libraries, so that when you ask Nepomuk for all data tagged with “Holiday”, Nepomuk can search it’s own database and also query Akonadi. Another benefit will be that filtering emails in KMail by tags will be much much faster, because the relation will be stored locally in Akonadi and we won’t have to talk to Nepomuk, which is very slow (mostly because of Virtuoso).
[…] KMail has the second best threading in the world, I think, second only to mutt because that is faster. […]
Why can’t KMail just have the very best threading in the world? Because right now it has to fetch the entire folder from Akonadi in order to be able to perform Subject comparision when building threads. That’s both very slow and CPU-intensive operation. So we thought: let’s store information about relations between emails in Akonadi, and when KMail asks for content of a folder, we give back only first few conversations just to fill the screen, and then fetch remaining conversations on demand when user scrolls down. This should make opening even massive folders superfast and should save a lot of memory, too.
Akonadi and KDE Frameworks
Hopefully David's code is more stable than his towers :)
The most-awaited discussion of the entire sprint was about KDE PIM and KDE Frameworks. When should we start? What has to be done? What can we use this opportunity for? From Akonadi point of view I want to do several things: remove deprecated API, change some API so that we use consistent naming and separate UI and non-UI stuff. Volker Krause suggested that we could move the client libraries into Akonadi repository with Akonadi server, so thatwe could share some of the code (protocol parsers for example), which I like, so we’ll go for that, too.
A bit unrelated, but still: the Akonadi server already compiles with Qt 5 for a while, so possibly during this development cycle we might switch to supporting only Qt 5 (and making use of all the C++11 awesomeness). There’s a little library that kdepimlibs link against, so we just build both Qt 4 and Qt 5 versions of it. Akonadi depends only on QtCore and QtDBus, so we only need distros to ship qt5-qtbase, which we believe most of them do by now.
Gmail resource
Vishesh and Àlex (Photo by Lukáš Tinkl)
I’ve been promising this for ages, now I finally discussed this with others, got some input and can start hacking :-) Let’s see if I can do something before Christmas ;-) Gmail resource would store all your mails in one folder and would create virtual folders for each label and just link emails from the “All mails” folder into respective labels. This way the emails will share flags (read/unread), and you will even be able to manage the labels by linking or unlinking emails from label folders in KMail.
Here I’d like to thank everyone for coming to Brno - if was a lot of fun and great pleasure to see all of you again, and also thank Red Hat for letting us use the office. Looking forward to see you all again on FOSDEM, next sprint, Akademy or anywhere else :-)
we have a little favor to ask from you :-) On the KDE PIM Sprint we discussed how to improve email threading in KMail by using Akonadi to store the information, so that KMail does not have to compute it every time. This would make opening a folder almost instant, all threads would be reconstructed immediately and it would massively improve CPU and memory consumption (so it’s totally something worth helping us with ;-)) More details on what else we discussed and implemented will follow in another blog post tomorrow.
To implement the threading caching, we need to know, whether in these days it still makes sense to support threading by Subject. It’s used as a fallback when grouping by standardized email headers (In-Reply-To, References) are missing, which used to be a case with buggy email clients years ago, but hopefully is better now, so we could drop it, which would massively simplify the algorithms.
So we would like you to disable threading by Subject, observe how much it breaks your threading (and potentially your workflow) and report back to us. To disable it, go to View ->Message List ->Aggregation -> Configure. There go to Groups & Threading tab and in Threading combobox select Perfect and by Reference. If the combo boxes are disabled, you have to click Clone Aggregation to clone the default settings, and use the clone.
View->Message List->Aggregation->Configure…
Aggregation Configuration
Aggregation Configuration
If removing threading by subject would break threading and workflow for too many users, we will keep the settings and we will try to figure out another way to do it.
So please configure your KMails, and let us know in comments below this post, on IRC, kde-pim mailing list or through any other communication means (just please try to avoid using smoke signals and pigeons ;-))
