Add test case IDs to increase tracebility

Each test case should have unique UUID to help with traceability of test runs. With UUID for each test case, it is easier to follow trends in stability over time. Signed-off-by: Pavol Brilla <[email protected]>
eclipse-bluechi · Oct 19, 2023 · ffa2452 · ffa2452
1 parent fff3fce
commit ffa2452
Show file tree

Hide file tree

Showing 36 changed files with 46 additions and 0 deletions.
diff --git a/tests/README.md b/tests/README.md
@@ -155,6 +155,17 @@ exit_code, output = ctrl.run_python("python/monitor.py")
 
 A full example of how to use the python bindings can be found in the [monitor open-close test](./tests/tier0/monitor-open-close/).
 
+### Generating test ID
+
+Every test should be distinctly identified with a unique ID. Therefore, when adding a new test, please execute the following command to assign an ID to the new test:
+
+```shell
+cd ~/bluechi/tests
+tmt test id .
+New id 'UUID' added to test '/tests/path_to_your_new_test'.
+...
+```
+
 ## Usage of containers
 
 The integration tests rely on containers as separate compute entities. These containers are used to simulate BlueChi's

diff --git a/tests/tests/main.fmf b/tests/tests/main.fmf
@@ -1,3 +1,4 @@
+component: bluechi
 test: |
     pytest -svv \
     --confcutdir=../../../ \

diff --git a/tests/tests/tier0/bluechi-agent-invalid-port-configuration/main.fmf b/tests/tests/tier0/bluechi-agent-invalid-port-configuration/main.fmf
@@ -1 +1,2 @@
 summary: Test if bluechi agent can handle invalid port in the configuration file
+id: 5645dcdf-acaa-4a04-8c0d-d478c8a6f2a3
diff --git a/tests/tests/tier0/bluechi-agent-logisquiet/main.fmf b/tests/tests/tier0/bluechi-agent-logisquiet/main.fmf
@@ -1 +1,2 @@
 summary: Test if bluechi agent can handle different types of values for LogIsQuiet in the configuration
+id: 96aa0e17-5e23-4cc3-bc34-88368b8cc07b
diff --git a/tests/tests/tier0/bluechi-agent-loglevel/main.fmf b/tests/tests/tier0/bluechi-agent-loglevel/main.fmf
@@ -1 +1,2 @@
 summary: Test if bluechi agent can handle different types of values for LogLevel in the configuration
+id: f3dda8c1-ceb7-446c-9d65-2aa8ca798099
diff --git a/tests/tests/tier0/bluechi-agent-logtarget/main.fmf b/tests/tests/tier0/bluechi-agent-logtarget/main.fmf
@@ -1 +1,2 @@
 summary: Test if bluechi agent can handle different types of values for LogTarget in the configuration
+id: 5cf92c54-e073-475c-970f-b8e090ec4da2
diff --git a/tests/tests/tier0/bluechi-agent-resolve-fqdn/main.fmf b/tests/tests/tier0/bluechi-agent-resolve-fqdn/main.fmf
@@ -1 +1,2 @@
 summary: Test if a local bluechi-agent on the same node as the controller can connect via the localhost name
+id: 48a9f92e-19f4-4f1b-ae34-a0d05f8ed16e
diff --git a/tests/tests/tier0/bluechi-agent-started-and-connected/main.fmf b/tests/tests/tier0/bluechi-agent-started-and-connected/main.fmf
@@ -1 +1,2 @@
 summary: Test if bluechi-agent on node foo started successfully and is connected to bluechi controller
+id: 15f5ced1-dcb4-4575-b649-1b12152c7fe7
diff --git a/tests/tests/tier0/bluechi-and-agent-on-same-machine/main.fmf b/tests/tests/tier0/bluechi-and-agent-on-same-machine/main.fmf
@@ -1 +1,2 @@
 summary: Test if a bluechi-agent can run alongside bluechi controller on the same machine
+id: d5f4a7d4-7019-423a-9ad6-2d7d3ecb7512
diff --git a/tests/tests/tier0/bluechi-anonymous-node/main.fmf b/tests/tests/tier0/bluechi-anonymous-node/main.fmf
@@ -1 +1,2 @@
 summary: Test if a monitor can be created and closed successfully
+id: bb9d4394-bd53-4d36-b8ba-ee91697c4850
diff --git a/tests/tests/tier0/bluechi-enable-service/main.fmf b/tests/tests/tier0/bluechi-enable-service/main.fmf
@@ -1 +1,2 @@
 summary: Test if a disabled systemd service can be enabled using bluechictl
+id: e3ec3cb3-1010-4205-9b99-507233c6a8f1
diff --git a/tests/tests/tier0/bluechi-freeze-and-thaw-service/main.fmf b/tests/tests/tier0/bluechi-freeze-and-thaw-service/main.fmf
@@ -1 +1,2 @@
 summary: Test if a systemd service can be frozen and thawed using bluechictl
+id: 0744f5b6-cc47-4f1a-bdb2-cb3bcbca0439
diff --git a/tests/tests/tier0/bluechi-invalid-port/main.fmf b/tests/tests/tier0/bluechi-invalid-port/main.fmf
@@ -1 +1,2 @@
 summary: Test if bluechi can handle invalid values for the port in the configuration properly
+id: f72ec296-bb6a-4f34-b9ef-5317fb097bdf
diff --git a/tests/tests/tier0/bluechi-long-multiline-config-setting/main.fmf b/tests/tests/tier0/bluechi-long-multiline-config-setting/main.fmf
@@ -1 +1,2 @@
 summary: Test if bluechi can properly parse a very long setting value spread across multiple lines, e.g. for the allowed node names
+id: 9a157fb9-71b0-4293-99b2-fd7a4f21b4ba
diff --git a/tests/tests/tier0/bluechi-service-startup/main.fmf b/tests/tests/tier0/bluechi-service-startup/main.fmf
@@ -1 +1,2 @@
 summary: Test if bluechi service started successfully
+id: bd3fd928-b5d8-4bb2-9df4-f59824316c19
diff --git a/tests/tests/tier0/monitor-agent-loses-connection/main.fmf b/tests/tests/tier0/monitor-agent-loses-connection/main.fmf
@@ -1 +1,2 @@
 summary: Test if the bluechi agent emits a status changed signal when it disconnects from the controller, e.g. when the controller goes down
+id: 7b5c59d0-f40e-4b29-b7c9-4fdeef9d079d
diff --git a/tests/tests/tier0/monitor-multiple-nodes-and-units/main.fmf b/tests/tests/tier0/monitor-multiple-nodes-and-units/main.fmf
@@ -1 +1,2 @@
 summary: Test if a montor with a subscription to a list of units on a single node receives the relevant signals, but none from a different node with the same unit
+id: 0d86b4bb-8e92-488a-8a17-bef9cfe1d2ce
diff --git a/tests/tests/tier0/monitor-node-disconnect/main.fmf b/tests/tests/tier0/monitor-node-disconnect/main.fmf
@@ -1 +1,2 @@
 summary: Test if the bluechi controller emits a message when a bluechi-agent disconnects
+id: 5f7b55f6-3b12-48e7-8462-dddc1e8af598
diff --git a/tests/tests/tier0/monitor-node-reconnect/main.fmf b/tests/tests/tier0/monitor-node-reconnect/main.fmf
@@ -1 +1,2 @@
 summary: Test if the bluechi-agent automatically reconnects on a successful heartbeat after the controller has been stopped and restarted
+id: 7db7738e-b1bf-458d-896a-5633b005d4dd
diff --git a/tests/tests/tier0/monitor-open-close/main.fmf b/tests/tests/tier0/monitor-open-close/main.fmf
@@ -1 +1,2 @@
 summary: Test if a monitor can be created and closed successfully
+id: 0df54792-aee3-4b66-8809-1a4b2b6d70b4
diff --git a/tests/tests/tier0/monitor-specific-node-and-unit/main.fmf b/tests/tests/tier0/monitor-specific-node-and-unit/main.fmf
@@ -1 +1,2 @@
 summary: Test if all signals for a subscription on a specific node and unit are emitted
+id: 4fb7abd3-fa4a-47ad-9c9e-a59c39da7f6c
diff --git a/tests/tests/tier0/monitor-wildcard-node-reconnect/main.fmf b/tests/tests/tier0/monitor-wildcard-node-reconnect/main.fmf
@@ -1 +1,2 @@
 summary: Test if the proper virtual signals are emitted when a monitor with wildcard subscription is active and a node disconnects and reconnects again
+id: 071d8a13-aee6-4dc0-9a91-0fb9a2e38a93
diff --git a/tests/tests/tier0/monitor-wildcard-unit-changes/main.fmf b/tests/tests/tier0/monitor-wildcard-unit-changes/main.fmf
@@ -1 +1,2 @@
 summary: Test if the proper signals are emitted when a monitor with wildcard subscription is active
+id: a6f2cc45-731f-467d-abb0-52c4147963fb
diff --git a/tests/tests/tier0/properties-get/main.fmf b/tests/tests/tier0/properties-get/main.fmf
@@ -1 +1,2 @@
 summary: Test if the properties for a systemd unit can be retrieved
+id: 0157e8aa-56bc-46ca-8971-41b00adf0877
diff --git a/tests/tests/tier0/property-get/main.fmf b/tests/tests/tier0/property-get/main.fmf
@@ -1 +1,2 @@
 summary: Test if a property value for a systemd unit can be retrieved
+id: 1e3ce729-8e6e-4263-8644-b0375e79f1bd
diff --git a/tests/tests/tier0/property-set/main.fmf b/tests/tests/tier0/property-set/main.fmf
@@ -1 +1,2 @@
 summary: Test if a property value for a systemd unit can be set
+id: 47ad938f-5f3b-467e-997d-ebf9d028b81d
diff --git a/tests/tests/tier0/proxy-service-fails-on-execstart/main.fmf b/tests/tests/tier0/proxy-service-fails-on-execstart/main.fmf
@@ -1 +1,2 @@
 summary: Test if proxy service fails on failure of ExecStart of dependency service
+id: 4ef7c787-e59e-4ac1-8d55-20b9c4017036
diff --git a/tests/tests/tier0/proxy-service-fails-on-non-existent-service/main.fmf b/tests/tests/tier0/proxy-service-fails-on-non-existent-service/main.fmf
@@ -1 +1,2 @@
 summary: Test if proxy service fails on requesting non-existent dependency service
+id: 20a2e52d-b09a-49ea-ad22-18168ed06614
diff --git a/tests/tests/tier0/proxy-service-fails-on-typo-in-file/main.fmf b/tests/tests/tier0/proxy-service-fails-on-typo-in-file/main.fmf
@@ -1 +1,2 @@
 summary: Test if proxy service fails if dependency service can't be loaded due to typo
+id: 240fcd55-a436-4301-97c6-f4d6426811a9
diff --git a/tests/tests/tier0/proxy-service-multiple-services-multiple-nodes/main.fmf b/tests/tests/tier0/proxy-service-multiple-services-multiple-nodes/main.fmf
@@ -1 +1,2 @@
 summary: Test proper lifetimes of all services when multiple services on multiple nodes request the same service on the same node
+id: 4b5a0d32-4b85-404d-9a02-f58531bf1957
diff --git a/tests/tests/tier0/proxy-service-multiple-services-one-node/main.fmf b/tests/tests/tier0/proxy-service-multiple-services-one-node/main.fmf
@@ -1 +1,2 @@
 summary: Test proper lifetimes of all services when multiple services on the same node request the same service on the same node
+id: b5db1d85-811e-497a-8dd5-c698051839c4
diff --git a/tests/tests/tier0/proxy-service-start/main.fmf b/tests/tests/tier0/proxy-service-start/main.fmf
@@ -1 +1,2 @@
 summary: Test if requesting.service's dependency on simple.service from node bar can be resolved
+id: cad16b0d-ed2a-42bd-b6e7-d6cd7a00ae23
diff --git a/tests/tests/tier0/proxy-service-stop-bluechi-dep/main.fmf b/tests/tests/tier0/proxy-service-stop-bluechi-dep/main.fmf
@@ -1 +1,2 @@
 summary: Test if stopping the bluechi-dep service results in a stopped bluechi-proxy service, but both actual services are still running
+id: cb13e1ed-d692-4466-9ff8-389c90c29454
diff --git a/tests/tests/tier0/proxy-service-stop-requesting-with-unneeded/main.fmf b/tests/tests/tier0/proxy-service-stop-requesting-with-unneeded/main.fmf
@@ -1 +1,2 @@
 summary: Test if stopping the requesting service results in a stopped dependency service if its marked with StopWhenUnneeded
+id: 18cca4fa-953c-4436-a22c-85ed51122fab
diff --git a/tests/tests/tier0/proxy-service-stop-requesting/main.fmf b/tests/tests/tier0/proxy-service-stop-requesting/main.fmf
@@ -1 +1,2 @@
 summary: Test if stopping the requesting service results in a stopped bluechi-proxy and bluechi-dep service, but dependency service is still running
+id: 6ebd2116-eb64-43b8-b346-8eb632122549
diff --git a/tests/tests/tier0/proxy-service-stop-target/main.fmf b/tests/tests/tier0/proxy-service-stop-target/main.fmf
@@ -1 +1,2 @@
 summary: Test if stopping the dependency service results in a stopped bluechi-dep and bluechi-proxy service, but the requesting service is still running
+id: b005b0fd-4a8e-492b-b916-300c5bae1d02
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		summary: Test if bluechi agent can handle invalid port in the configuration file
		id: 5645dcdf-acaa-4a04-8c0d-d478c8a6f2a3
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		summary: Test if bluechi agent can handle different types of values for LogIsQuiet in the configuration
		id: 96aa0e17-5e23-4cc3-bc34-88368b8cc07b
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		summary: Test if bluechi agent can handle different types of values for LogLevel in the configuration
		id: f3dda8c1-ceb7-446c-9d65-2aa8ca798099
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		summary: Test if bluechi agent can handle different types of values for LogTarget in the configuration
		id: 5cf92c54-e073-475c-970f-b8e090ec4da2
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		summary: Test if a local bluechi-agent on the same node as the controller can connect via the localhost name
		id: 48a9f92e-19f4-4f1b-ae34-a0d05f8ed16e
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		summary: Test if bluechi-agent on node foo started successfully and is connected to bluechi controller
		id: 15f5ced1-dcb4-4575-b649-1b12152c7fe7
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		summary: Test if a bluechi-agent can run alongside bluechi controller on the same machine
		id: d5f4a7d4-7019-423a-9ad6-2d7d3ecb7512
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		summary: Test if a monitor can be created and closed successfully
		id: bb9d4394-bd53-4d36-b8ba-ee91697c4850
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		summary: Test if a disabled systemd service can be enabled using bluechictl
		id: e3ec3cb3-1010-4205-9b99-507233c6a8f1
Original file line number	Diff line number	Diff line change
		@@ -1 +1,2 @@
		summary: Test if a systemd service can be frozen and thawed using bluechictl
		id: 0744f5b6-cc47-4f1a-bdb2-cb3bcbca0439