Get Involved/Quality/Akademy2012BoF: Difference between revisions
< Get Involved | Quality
→Improving Tester numbers: added real name |
m Entriken moved page Getinvolved/Quality/Akademy2012BoF to Get Involved/Quality/Akademy2012BoF |
||
| (7 intermediate revisions by one other user not shown) | |||
| Line 1: | Line 1: | ||
KDE Quality Team BOF | KDE Quality Team BOF | ||
==Review of 4.9 testing== | == Review of 4.9 testing == | ||
* Was really only a trial, went well. | * Was really only a trial, went well. | ||
| Line 8: | Line 8: | ||
* We need to scale up what we're doing into more areas, without doing a worse job on it | * We need to scale up what we're doing into more areas, without doing a worse job on it | ||
==Improving User Testing== | == Improving User Testing == | ||
* The checklists in Plasma worked well, found (and fixed) many issues. | * The checklists in Plasma worked well, found (and fixed) many issues. | ||
* This needs to be extended to everything | * This needs to be extended to everything | ||
* | * Aurélien Gateau, Copy Ubuntu's "checkboxes" (TODO find a link) | ||
* It tells the user "do this [worked, broken, skip]" | * It tells the user "do this [worked, broken, skip]" | ||
* Then submits a report | * Then submits a report | ||
| Line 17: | Line 18: | ||
* Less work to write than automated tests, and more flexible. | * Less work to write than automated tests, and more flexible. | ||
==EBN== | == EBN == | ||
* too many false positives / minor issues hide the big issues | * too many false positives / minor issues hide the big issues | ||
* Aleix Poi apparently has a static analyser written for a thesis we could look at. | * Aleix Poi apparently has a static analyser written for a thesis we could look at. | ||
==Jenkins== | == Jenkins == | ||
* Needs to be more open | * Needs to be more open | ||
* We need examples/docs. What it does + how to use it. | * We need examples/docs. What it does + how to use it. | ||
== Valgrind (memcheck) == | |||
* This is a useful test, however it's quite tricky to use/analyse, so reports should only be done by developers testing (to identify which lib a leak is in for example) | * This is a useful test, however it's quite tricky to use/analyse, so reports should only be done by developers testing (to identify which lib a leak is in for example) | ||
* Someone should write a wiki page on how to memcheck, filter out nonsense that we can't fix. | * Someone should write a wiki page on how to memcheck, filter out nonsense that we can't fix. | ||
* Then submit leaks as bug reports with valgrind backtraces | * Then submit leaks as bug reports with valgrind backtraces | ||
==Improving Tester numbers== | == Improving Tester numbers == | ||
* Rohan Garg to make VirtualBox images of ProjectNeon. Has some issues, hopefully can all be fixed before 4.10 | * Rohan Garg to make VirtualBox images of ProjectNeon. Has some issues, hopefully can all be fixed before 4.10 | ||
==Release numbers== | == Release numbers == | ||
* | |||
* Aurélien Gateau suggested modifying the release numbers to something more easier. | |||
** 4.9.71 = alpha1 | ** 4.9.71 = alpha1 | ||
** 4.9.72 = alpha2 | ** 4.9.72 = alpha2 | ||
| Line 46: | Line 51: | ||
* suggest changing for 4.10 | * suggest changing for 4.10 | ||
==Other== | == Other == | ||
* everyone should listen to Jeroen on bugzilla management. | * everyone should listen to Jeroen on bugzilla management. | ||
==Extra Mile== | == Extra Mile == | ||
* [http://community.kde.org/Akademy/2012/Extra_Mile_BoF Extra Mile BoF] | * [http://community.kde.org/Akademy/2012/Extra_Mile_BoF Extra Mile BoF] | ||
* Was agreed to put it under the banner of "KDE Quality" and do the discussions/planning on our mailing list/IRC channel. If you have any objections to that, please raise them now. | * Was agreed to put it under the banner of "KDE Quality" and do the discussions/planning on our mailing list/IRC channel. If you have any objections to that, please raise them now. | ||
==Actions== | == Actions == | ||
* Write an email to the kde-release-team with the proposed release numbers | |||
* Write an email to the kde-release-team with the proposed release numbers (Albert attended the BoF, he already knows about) | |||
* Write a tool for showing "UI test steps" | * Write a tool for showing "UI test steps" | ||
* Write some test examples (suggest | * Write some test examples (suggest Aurélien Gateau does gwenview, David Edmundson to do KTP, then we can encourage users to write tests for the others) | ||
* Sort out Jenkins | * Sort out Jenkins | ||
* find a way to highlight the important issues in EBN | * find a way to highlight the important issues in EBN | ||
* Tools page in wiki needs splitting into debuggers/profilers (such as gammaray) and actual testing tools | * Tools page in wiki needs splitting into debuggers/profilers (such as gammaray) and actual testing tools | ||
[[Category:Testing]] | |||
Latest revision as of 01:42, 16 February 2015
KDE Quality Team BOF
Review of 4.9 testing
- Was really only a trial, went well.
- Definitely better than if we hadn't done anything
- Regression email to Plasma worked well (though should probably stop now)
- We need to scale up what we're doing into more areas, without doing a worse job on it
Improving User Testing
- The checklists in Plasma worked well, found (and fixed) many issues.
- This needs to be extended to everything
- Aurélien Gateau, Copy Ubuntu's "checkboxes" (TODO find a link)
- It tells the user "do this [worked, broken, skip]"
- Then submits a report
- Don't be too specific in what to check, that way the user can spot bugs that are hard for automated testing to find.
- Less work to write than automated tests, and more flexible.
EBN
- too many false positives / minor issues hide the big issues
- Aleix Poi apparently has a static analyser written for a thesis we could look at.
Jenkins
- Needs to be more open
- We need examples/docs. What it does + how to use it.
Valgrind (memcheck)
- This is a useful test, however it's quite tricky to use/analyse, so reports should only be done by developers testing (to identify which lib a leak is in for example)
- Someone should write a wiki page on how to memcheck, filter out nonsense that we can't fix.
- Then submit leaks as bug reports with valgrind backtraces
Improving Tester numbers
- Rohan Garg to make VirtualBox images of ProjectNeon. Has some issues, hopefully can all be fixed before 4.10
Release numbers
- Aurélien Gateau suggested modifying the release numbers to something more easier.
- 4.9.71 = alpha1
- 4.9.72 = alpha2
- 4.9.73 = alpha3
- 4.9.81 = beta1
- 4.9.82 = beta2
- 4.9.91 = RC1
- 4.9.92 = RC2
- suggest changing for 4.10
Other
- everyone should listen to Jeroen on bugzilla management.
Extra Mile
- Extra Mile BoF
- Was agreed to put it under the banner of "KDE Quality" and do the discussions/planning on our mailing list/IRC channel. If you have any objections to that, please raise them now.
Actions
- Write an email to the kde-release-team with the proposed release numbers (Albert attended the BoF, he already knows about)
- Write a tool for showing "UI test steps"
- Write some test examples (suggest Aurélien Gateau does gwenview, David Edmundson to do KTP, then we can encourage users to write tests for the others)
- Sort out Jenkins
- find a way to highlight the important issues in EBN
- Tools page in wiki needs splitting into debuggers/profilers (such as gammaray) and actual testing tools