Goals/Automate and systematize internal processes

One of KDE's three overarching goals is to improve institutional memory by automating and systematizing internal processes, expanding documentation, and working more as teams rather than isolated individuals. See the original proposal at https://phabricator.kde.org/T15627.

Follow our progress

Weekly progress is posted at https://pointieststick.com/category/automation-systematization!

Communicate with the team

Join the #kde-institutional-memory:kde.org room on Matrix!

There is also a team you can join at https://invent.kde.org/teams/automation. Once you join the team, don't forget to set your notification preferences to "Watch" or else you'll miss things. Long-term task tracking will be done at https://invent.kde.org/teams/automation/issues/-/issues.

How you can help

Here are some examples of things you can do to help make this a reality.

For items that involve deploying software on KDE.org infrastructure (such as running bots) or making requests to KDE.org infrastructure (such as the Gitlab API - needed for items such as a CI dashboard) please discuss your approach with the Sysadmin team in #kde-sysadmin:kde.org to ensure it can be deployed into production.

Code

• Implement a "System Misconfiguration Detection Hub"
• Remove AppStream metadata for our Plasma widgets; it's non-useful and confusing to users

Documentation

• Create documentation docs for how to write good tests
• Add and improve API documentation for library components with poor or missing documentation.
• Codify and document the steps required for adequately QAing merge requests at https://community.kde.org/Infrastructure/GitLab#Perform_QA
• Improve developer and internal process documentation to be good enough that contributors can generally answer their own questions rather than asking in real-time chatrooms. Move our specialized knowledge out of our heads!
• Create and document an "offboarding" process so that people transitioning away from KDE can hand off their work to any other interested community members, and their knowledge can be preserved.
• Move documentation into the git repo for the software it describes, and then require that every commit and merge request that changes anything be accompanied by a change to the relevant documentation, so that it always remains accurate.

Continuous integration

• CI status dashboard (in progress -- sitter)
• For repos that have CI which does not require all tests to pass, change that and then fix or remove all tests that are persistently or randomly failing, as they have no value. (in progress -- nico)
• Add a new CI job to check only for new REUSE license regressions, then enable it everywhere the more strict "reuse-lint" runner isn't enabled due to lack of license data somewhere. (in progress -- sitter)
• Enable the "reuse-lint" CI job for repos that don't currently have it active, and then fix the license issues it finds by contacting the authors of files with no license information and asking them to decide on a license.
• Add a new CI job to check for English spelling and grammar mistakes. (in progress -- nico)
• Add a new CI job to check for translated strings without sufficient context--particularly single-word strings.
• Add a CI runner to do fuzzing; eventually make it a mandatory-pass thing after fixing all issues.
• Make the cppcheck CI runner actually run clang-format and fail for code that isn't compliant. If this is annoying, fix our clang-format rules so it isn't!
• Find missing i18nc calls in qml text properties (in progress -- redondo)

Bugzilla and Sentry

• Change Bugzillas internal rules to require posting a comment when re-opening a resolved bug report.
• Make DrKonqi stricter about accepting un-actionable backtraces without debug symbols. Don't let it! (in progress -- sitter)
• Work on the bugzilla bot, making it handle more things. Ideas:
  • Auto-close old UNCONFIRMED bugs after a few warnings (https://bugs.kde.org/show_bug.cgi?id=224641).
  • If a bug report with the crash severity doesn't have a backtrace attached or pasted inline, nag for one.
  • If a backtrace was attached, paste it inline.
  • If an inline backtrace lacks debug symbols (search for " in () at "), nag the reporter to fix it; maybe this can be done by https://bugs.kde.org/show_bug.cgi?id=354217?
  • If a commit references the number of the bug it fixes, infer the version fixed in from the milestone or the target branch of the related merge request

Chat

• Write a chat bot to paste the following into the #plasma room once per day (in progress -- https://invent.kde.org/bbonacci/bug-safari):

X VHI priority regressions (https://tinyurl.com/plasma-vhi-regressions)
X VHI priority bugs (https://tinyurl.com/plasma-vhi-bugss)
X HI priority bugs (https://tinyurl.com/plasma-15-minute-bugs)
X Regressions in general: (https://tinyurl.com/plasma-regressions) n fewer than yesterday etc

GitLab

• Create a merge request template like the Bugzilla bug report template which prompts people to do things like write testing steps and make documentation changes alongside any UI changes.
• Change the culture around QA to adopt the mindset of "it's ready when it stops failing, not when it starts working." Encourage merge requests to be thoroughly tested before they are submitted, as well as during the review process.
• Nag people who make changes in code to also get the accompanying documentation up to date.
• Write a bot to automatically cherry-pick the commits from merge-requests with the "cherry-pick" keyword attached, targeting the branch specified in the milestone. If it tries and fails, it should loudly yell!
  • PROBLEM: what if cherry pick fails/does not apply? comment on MR 'this has failed to pick!!!!'? maybe write a comment in general also on success?
• Write a bot to automatically update the milestones for open merge requests that are targeting old milestones when the Plasma version they're for becomes ineligible for new features.
• Write a bot to automatically check whether a commit to be cherry-picked onto a stable branch contains any changes to translatable strings when the Plasma version it targets has become ineligible for string changes. If it does, it should loudly yell!
  • IDEA: run Messages.sh before and after, possibly as part of a pipeline?
• Some sort of information on how long the queue is and / or which position a given job has in the queue

Translation system

• When reviewing merge requests, make sure developers provide context in translatable strings.
• Implement a way for translators to directly jump to the commit that introduced a string, so they have an easier time to directly give feedback to the author in case of any uncertainties.

Release process

• Change kdereview process to have a CI runner for everything in the process's checklist, and then getting through kdereview amounts to keeping the CI green.
• Unify on one set of release tooling for Frameworks, Gear and Plasma, so that any release person becomes capable of making a new release of any of those pieces of software!
• Encourage apps to join KDE Gear and adopt its version numbering schema, so that they don't have to be self-released.

Teams

• Encourage projects to make use of the QA team for merge request review.
• Create project-specific teams on invent.kde.org for large projects that span multiple repos (e.g. Plasma, KDE PIM, System Settings) and encourage people to join those teams when contributing to their projects. Contributing to KDE should feel like being a member of a group, not an individual feeding inputs into a giant machine.
• Encourage people to join the volunteer Gardening team and help coordinate the efforts of volunteers and sponsored contributors.
• Encourage people to join the volunteer QA team and perform QA on open merge requests to make sure they do what they say they do and don't cause regressions.
• I move my "This week in KDE" blog series to KDE infrastructure and open it up to other contributors, standardizing around an open process that has a bus factor greater than 1.