KDevelop/ClangRoadmap

This document is supposed to lay out the roadmap towards proper Clang integration into KDevelop. This is about Clang as a backend for our C++ language support, not related to using Clang as a compiler for your project (you can already do that).

NOTE: To make this clear, we _do not want to break existing language plugins_ !!

Why Clang

Maintenance

Our existing C++ language plugin might work reasonably well for C++03 code. But for more complicated features from C++11 (e.g. variadic templates, constexpr, ...) are not yet supported. Due to lack of time and manpower, it is an open question when, if ever, we come around to implementing this. Furthermore one can note that the C++ standards committee picked up speed. With C++14 concepts e.g. will become a technical preview which again will be lots of work to implement properly.

Using Clang as a backend will give us kickass support of C++ "for free".

Static Analysis

Using an IDE like KDevelop for writing C++ should reduce the amount of time you need to compile your code, just to see what errors you made. Clang comes with excellent error reporting tools. These we will then be able to show directly inside the editor.

Refactoring

KDevelop so far comes with only very rudimentary support for refactoring. The clang community is very actively writing new tools though which we can then integrate.

Why Not Clang

File-based Workflow

So far, KDevelop uses a file-based workflow which is different from what compilers do: They work on translation units and a given file is not necessarily compiling standalone. We will need to figure out how to support this, such that one can still open a file in KDevelop without first loading a project and still getting _some_ language support. Also, working on headers or new files in a project should "work" just like it does now.

Performance

It is left to be seen how clang works in terms of performance. So far it looks "OK", especially since one should be able to write code that leverages multiple CPU cores much better. We will probably have to look into precompiled headers though. Potentially we will also have to spearhead the effort behind the clangd idea, where one could share information acquired for the IDE with the compiler and other external tools.

Update: Performance looks exceptionally good, even without PCH support. While more cycles are used, we leverage multicore systems much better leading to far better performance overall. And with PCH's in place, the whole thing just flies off the charts!

API/ABI

The C++ API of clang is still not stable. Probably the AST part will never become stable. It is left to be seen how this will be problematic. As long as source compatibility is given, or at least source changes are minimal, I think this will work out OK. It might mean requiring new versions of Clang and it will definitely mean that we need to get in contact with the Clang community. We need to be aware of changes they make and make them aware of what we are using it for.

Update: So far, we just use clang-c, which comes with API/ABI compatibility guarantees.

Tight Coupling between Abstraction and Storage

The classes in the DUChain are so far our abstraction of a language. Things like common types, contexts, declarations, definitions and uses. The big trouble there is that all these classes are tightly coupled to the storage.

To gain more performance, esp. to reduce lock contention, we should refactor our API and decouple the language abstraction from the storage and make it possible for a given language to implement a custom storage layer. This would also be good in future if we have clangd or similar and do not want to duplicate the caching inside KDevelop.

Roadmap

• refactor KDevplatform code, decouple language abstraction from storage
  • instead of asking the storage for contexts, declarations, etc. pp. in a given file/project/... ask the language plugin of that file
  • don't put lookup code and similar that is only required to implement a _language plugin_ into the new api
  • add API for tasks such as:
    • create outline of a given document
    • quickopen of classes, functions
    • classbrowser functionality, basically project outline
    • context browsing functionality
• keep existing DUChain API as common implementation of new language abstraction for reuse in existing language plugins (php, ruby, python, ...)
• move existing, usable refactoring tools from C++ to kdevplatform language, make it possible for other language tools to adapt them to their needs (i.e. to get renaming working properly for PHP)
  • don't show language specific refactoring tools in a different language, i.e. 'move to source' when working on a PHP/Ruby/Python file or such

Random TODO

• For better performance, ensure we don't add "useless" stuff into the PersistentSymbolTable. Things that are currently still put in there but should not:
  • anything in *.cpp files, which is arguably something we never want, but must be checked.
  • private members of classes/structs/...
  • contents of anonymous namespaces
• note on the above: The Declaration should get something similar to "anonymousInContext" which just prevents that a e.g. private data member is ever added to the symbolTable, even when the parent context is in the symbol table.
• extend dtor range over full identifier, i.e. not just associate use with "~", but with full "~foo" or similar.

Positive Side Effects

• decoupling of storage will allow it to more easily exchange the storage with a different implementation. This should then allow us to test e.g. NoSQL or SQL databases for storage.
• once we get clang working, we can remove a ton of C++ specific code from the existing DUChain API, thus simplifying it.
• with the storage decoupled, it should be the next goal to further simplify the existing DUChain API. The goals should be:
  • simple to use, no magic knowledge required
  • rethink (i.e. get rid of) the locking concept, improve performance scaling with multiple cores
  • simple to extend for different languages
• with the language plugins being asked for outline, quickopen, ... it will finally become possible to make the output language specific:
  • PHP namespaces
  • ruby/python function signatures

technical details

• use libclang (the C api) for higher stability and more features, like indexing and USR
• clangd idea: http://lists.cs.uiuc.edu/pipermail/cfe-dev/2012-June/022028.html