KDb/Tests: Difference between revisions

Latest revision as of 08:54, 23 June 2015

Unit tests

Warning

The current unit test coverage in KDb isn't bad. Volunteers wanted!

Introduction

KDb has complex functionality and should be covered by unit tests. It has layered architecture:

[      KDb             ]
[Driver A] [Driver B] ....

There are two areas to test:

Areas independent of drivers (e.g. data structures and high-level logic correctness, proper modeling).
Driver-dependent areas, e.g. correctness of translation of high-level data definitions to any driver that supports given feature. Because of this, following matrix is handy:

Test	Driver A	Driver B	...
Test 1	PASSED	PASSED	...
Test 2	PASSED	N/A	...
...	...	...

Certain unit tests have small number of input test data (example: testing data correctness of data structures, when it is enough to cover typical and corner cases). Other tests need massive test data (example: SQL parser, data migration/conversion features). In the latter case it is useful to clearly separate in order to minimize repetitive work and risk of mistakes.

Summing up, for any new set of test (e.g. for whole class), it is handy to develop test plan first, and reach agreement that the plan addresses the needs. Having a test that look as compete but in fact isn't, can have negative consequences for the project - defects can be still hidden.

Proposed workflow for given existing class includes initial test plan, initial set of tests and iterative plan updates, prioritization of tasks, and further tests implementation.

Summing up, tests are consisted of three main artifacts: plans, test data and implementation. All these should be provided for review on the Review Board. Massive data can provided for review using hyperlink references.

For new new classes/functions (or after extending functionality of existing ones) it is proposed add unit tests to ASAP, and publish them for review as a part of the review for the feature implementation itself.

Running unit tests

$ cd {BUILDDIR}
$ ctest

You can also use 'ctest -V' for verbose output.

HOWTOs

Functional tests

Functional tests are located in tests/features/.

kdbfeaturestest uses fundamental parts of the APIs. They can be also used by newcomers to learn how to use KDb APIs. kdbfeaturestest is built during the default build of KDb, although it is not installed on make install.

To run the kdbfeaturestest test:

$ cd {BUILDDIR}/tests/features

1. Full test for database, tables and data creation for SQLite:

$ ./kdbfeaturestest -t tables org.kde.kdb.sqlite mydb

On success, mydb file will be created in the current directory and the test says:

main: "tables" test: PASSED

2. Full test for a database, tables and data creation for MySQL server:

$ ./kdbfeaturestest -p {mysql_password} -u {mysql_user} -t tables org.kde.kdb.mysql mydb

On success, mydb database will be created on the server and the test says:

main: "tables" test: PASSED

Similarly to MySQL, kdbfeaturestest can run with PostgreSQL databases, in this case the driver ID is called org.kde.kdb.postgresql).

@@ Line 1: / Line 1: @@
 ==Unit tests==
-{{Warning|The current unit test coverage in Predicate is '''poor'''. Volunteers wanted!}}
+{{Warning|The current unit test coverage in KDb isn't bad. Volunteers wanted!}}
 ===Introduction===
-Predicate has complex functionality and should be covered by unit tests.
+KDb has complex functionality and should be covered by unit tests.
 It has layered architecture:
-  [      Predicate         ]
+  [      KDb             ]
   [Driver A] [Driver B] ....
 There are two areas to test:
 *Areas independent of drivers (e.g. data structures and high-level logic correctness, proper modeling).
 *Driver-dependent areas, e.g. correctness of translation of high-level data definitions to any driver that supports given feature. Because of this, following matrix is handy:
 {{TableStart}}
 !Test
@@ Line 30: / Line 32: @@
 |...
 |}
 '''Certain unit tests have small number of input test data''' (example: testing data correctness of data structures, when it is enough to cover typical and corner cases). '''Other tests need massive test data''' (example: SQL parser, data migration/conversion features). In the latter case it is useful to clearly separate in order to minimize repetitive work and risk of mistakes.
@@ Line 35: / Line 38: @@
 Summing up, for any new set of test (e.g. for whole class), '''it is handy to develop test plan first''', and reach agreement that the plan addresses the needs. Having a test that look as compete but in fact isn't, can have negative consequences for the project - defects can be still hidden.
-Proposed workflow for given existing class includes initial test plan, initial set of tests and iterative plan updates, prioritization, and further tests implementation.
+Proposed workflow for given existing class includes initial test plan, initial set of tests and iterative plan updates, prioritization of tasks, and further tests implementation.
-Summing up, tests are consisted of three main artifacts: plans, test data and implementation. All these should be provided for review on the Review Board. Massive data can provided for review using hyperlink references.
+Summing up, tests are consisted of three main artifacts: '''plans, test data and implementation'''. All these should be provided for review on the Review Board. Massive data can provided for review using hyperlink references.
-For new new classes/functions (or after extending functionality of existing ones) it is proposed add unit tests to ASAP, and publish them for review as a part of the review for the feature implementation itself.
+For new new classes/functions (or after extending functionality of existing ones) it is proposed '''add unit tests to ASAP''', and publish them for review as a part of the review for the feature implementation itself.
 ===Running unit tests===
-  $ cd {BUILDDIR}/Predicate/tests
+  $ cd {BUILDDIR}
-  $ ctest --verbose
+  $ ctest
+You can also use 'ctest -V' for verbose output.
 ===HOWTOs===
 *[http://techbase.kde.org/Development/Tutorials/Unittests Techbase tutorial]
+*[http://qt-project.org/wiki/Writing_Unit_Tests Qt Project hints]
 ==Functional tests==
-Functional test '''predicatefeaturestest''' uses many parts of the APIs. They can be also used by newcomers to learn how to use Predicate APIs.
+Functional tests are located in tests/features/.
-predicatefeaturestest is built during the [[../Build/|default build]] of Predicate, although it is not installed on ''make install''.
+'''kdbfeaturestest''' uses fundamental parts of the APIs. They can be also used by newcomers to learn how to use KDb APIs. kdbfeaturestest is built during the [[../Build/|default build]] of KDb, although it is not installed on ''make install''.
-To run the predicatefeaturestest test:
+To run the ''kdbfeaturestest'' test:
   $ cd {BUILDDIR}/tests/features
- $ less README # read README file to
 . Full test for database, tables and data creation for SQLite:
-  $ ./predicatefeaturestest -t tables sqlite sqlitetest
+  $ ./kdbfeaturestest -t tables org.kde.kdb.sqlite mydb
-On success, sqlitetest file will be created and the test says:
+On success, mydb file will be created in the current directory and the test says:
   main: "tables" test: PASSED
-. Full test for database, tables and data creation for MySQL server:
+. Full test for a database, tables and data creation for MySQL server:
-  $ ./predicatefeaturestest -p {mysql_password} -u {mysql_user} -t tables mysql mysqltest
+  $ ./kdbfeaturestest -p {mysql_password} -u {mysql_user} -t tables org.kde.kdb.mysql mydb
-On success, mysqltest server database will be created and the test says:
+On success, mydb database will be created on the server and the test says:
   main: "tables" test: PASSED
-Similarly to MySQL, predicatefeaturestest can run on top of PostgreSQL database (the driver is called ''postgresql'').
+Similarly to MySQL, kdbfeaturestest can run with PostgreSQL databases, in this case the driver ID is called ''org.kde.kdb.postgresql'').