Plasma/TestDays

What is Plasma Test Days?

The Plasma Test Days will happen the 19th and 20th of June 2016 in preparation for the Plasma 5.7 release. During that time we will stress test the upcoming release with the help of the KDE community and we want to invite everyone, no matter what level of experience you have to be part in this project to ensure the stability and quality of Plasma 5.7 for all.

Our overarching goal is not just to ensure Plasma's future stability but to set up a new path for involvement in KDE where in members of the wider community can easily be a part as well as having a better relationship with distro's shipping Plasma.

After the Test Days our hope is that everyone involved will feel that they got something out of it. That developers feel that they have a better and more clear grasp of common bugs, community members feel that they have greater access to "the inner workings of Plasma" and distro maintainers feel that they have a better relationship with the KDE community and it's members.

I'm in! I'm a community member! How does it work?

Awesome! Ok so what you need to do is get a KDE account which, if you don't have one already, you can do here [Link to KDE accounts] this simply to have a clear way of staying in contact with you if there are any questions and for later evaluation of your experiences from Test Day.

After this the method for testing is to pick an ISO from the list below, pick one closest to the distro you would most probably want to use if you weren't bugtesting (this to ensure that the any bugs aren't connected to "New Confusing Distro Syndrome" and choose one of the scenario's below. You see if it plays out the way it's intended and then report back using the report form [Link to report form].

I'm in! I'm a distro maintainer! How does it work?

I'm in! I'm a Plasma developer! How does it work?

Communication Channels for Test Day

[Link to IRC] [Link to Email List] [Link to Telegram Group] [Link to Forum Thread]

Test Iso's

Here we will display the available iso's hopefully using stable bases and the beta of Plasma 5.7 on top. Our goal is to have a wide range of different distro's to be a part of the testing to easier ensure that whatever bug found is Plasma related. Our hope is to have one from KDE neon, Opensuse, Fedora, KaOS and hopefully one using Arch as base.

Test Scenarios

Report Form

[Link to report form]

What Happens Afterwards?

This is where the fun starts, after the scenarios have been tested, the information handed over to the test day team, the test day team have written, triaged and checked the bugs and passed them off to the developers for whom they are relevant; we evaluate the Test Day and present the results and possible future Test Days.

Evaluation

After the Test Day we will go through the available information for a proper evaluation of what happened, how it happened and if the end results where good. We wont just look at what bugs where found, where cleared or where fixed - but how many took part, how many NEW people took part (as that is one of the goals mentioned above), and perhaps see how distro maintainers felt it worked.

All to ensure that everyone found it worthwhile.

If it was a bust we will have to ask ourselves why it was a bust, what we could have done different, how we can improve it to next Test Day and if they are worthwhile. If on the other hand it was a thundering success, we need to ask the same questions: Why was it a success? How can we reproduce this success? When is the next Test Day?

As the test day comes closer we will hopefully be able to formalize and fill out this segment so that evaluation can begin the second Test Day is over!

Results

To be announced after the testing is done.

FAQ

Why can I only report bugs connected to these scenarios? - Because we need to ensure some very common use cases and scenarios and focus on those. Essentially putting a lot of individual sets of eyes on these common scenarios as a way to find the most commonly occurring bugs and issues.
Why isn't [distro X] used instead of [distro Y]? - We have contacted several distributions and tried to find distro packagers who have time to set up a test ISO with a vanilla version of Plasma 5.7. The only reason we have excluded anyone is because they simply didn't have time which isn't that uncommon since distro creation is complex and takes time. Hopefully all distributions will benefit.
Why can't I use my already installed distro instead of the provided ISO's? - The reason is so we can ensure that the issue's that may crop up are connected to Plasma 5.7. In your already installed system you will probably have made some edits and changes and these would affect the test by making it more complex.
Is this a recurring thing? - Hopefully, it all depends on how well it goes. If for example the Test Day works as intended, plenty of bugs gets tested and squashed, if everyone feels that they gain from the Test Day then it will happen again for sure! On the other hand if the results are underwhelming we will have to look at the reasons why before doing another one.
I have bug that's been on bugzilla for years, what about that one? - This test is about these specific situations. A test that would involve all bugs in bugzilla would be beyond our capacity.

Articles and External Links

Dario's Guide to Bug Triaging - Extensive guide on how to perform bug triagers tasks

Quick introduction to Bugzilla - This article explains the basics about the bugtracking software that KDE uses: "Bugzilla". It includes the description of a bug reports fields and the workflow of most common tasks like searching into the database.

A Bug's Life Cycle - This article describes the possible status of a bug report and when each should be used.

How to triage bugs - This article gives step-by-step what you do during a BugDay, or how to start triaging on your own for "ongoing triage"

Preparing a testing environment - This article from the BugWeeks initiative describes how to properly configure and setup a KDE SC environment in order to test the bugs.

How to create useful crash reports - This article helps users to prepare their KDE packages such they can create detailed backtraces.

Basic guide about crash reports - This article is a basic/simplified version of the previous one.

Extra Mile - This article presents the Extra Mile initiative, whose goal is to help KDE applications and workspaces "walk the extra mile". The Extra Mile initiative uses Bugzilla to track extra mile bugs.