Document experiment groups (#27)

* fixing readme and contrib Signed-off-by: Srinivasan Parthasarathy <[email protected]> * fix group docs Signed-off-by: Srinivasan Parthasarathy <[email protected]> * fix percentile computations Signed-off-by: Srinivasan Parthasarathy <[email protected]> * fix edit uri bug Signed-off-by: Srinivasan Parthasarathy <[email protected]> * fix community meeting link Signed-off-by: Srinivasan Parthasarathy <[email protected]> * format namespace Signed-off-by: Srinivasan Parthasarathy <[email protected]> * updated roadmap Signed-off-by: Srinivasan Parthasarathy <[email protected]>
iter8-tools · Apr 27, 2022 · f55f9d2 · f55f9d2
1 parent 542133d
commit f55f9d2
Show file tree

Hide file tree

Showing 6 changed files with 72 additions and 23 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -29,7 +29,7 @@ The best ways to reach us with a question is to ask...
 
 * On the original GitHub issue
 * In the `#development` channel in the [Iter8 Slack workspace](https://join.slack.com/t/iter8-tools/shared_invite/zt-awl2se8i-L0pZCpuHntpPejxzLicbmw)
-* During our [community meetings](https://iter8.tools/community/)
+* During our [community meetings](https://iter8.tools/latest/community/community/)
 
 ## Find an issue
 

diff --git a/README.md b/README.md
@@ -10,16 +10,17 @@
 <img alt-text="Iter8 experiment" src="https://iter8-tools.github.io/docs/0.9/images/iter8-intro-dark.png" width="70%" />
 </p>
 
-## :rocket: Usage examples
-1.  [Load test, benchmark and validate HTTP services with SLOs](https://iter8.tools/0.9/tutorials/load-test-http/usage/).
-2.  [Load test, benchmark and validate gRPC services with SLOs](https://iter8.tools/0.9/tutorials/load-test-grpc/usage/).
-3.  Performance testing and SLO validation using Iter8 GitHub Action: [HTTP](https://iter8.tools/0.9/tutorials/load-test-http/ghaction/) and [gRPC](https://iter8.tools/0.9/tutorials/load-test-grpc/ghaction/).
-4.  Performance testing and SLO validation for Knative services: [HTTP](https://iter8.tools/0.9/tutorials/integrations/knative/load-test-http/) and [gRPC](https://iter8.tools/0.9/tutorials/integrations/knative/load-test-grpc/).
+## :rocket: Features at a glance
+1. **Load test, benchmark and validate HTTP services with SLOs**
+    * [In your local environment](https://iter8.tools/0.10/tutorials/load-test-http/basicusage/)
+    * [Inside a Kubernetes cluster](https://iter8.tools/0.10/tutorials/load-test-http/kubernetesusage/)
+2. **Load test, benchmark and validate gRPC services with SLOs**
+    * [In your local environment](https://iter8.tools/0.10/tutorials/load-test-grpc/basicusage/)
+    * [Inside a Kubernetes cluster](https://iter8.tools/0.10/tutorials/load-test-grpc/kubernetesusage/)
 
 Please see [https://iter8.tools](https://iter8.tools) for the complete documentation.
 
 ## :maple_leaf: Issues
-
 Iter8 issues are tracked [here](https://github.com/iter8-tools/iter8/issues).
 
 ## :tada: Contributing

diff --git a/docs/roadmap.md b/docs/roadmap.md
@@ -8,12 +8,13 @@ hide:
 # Roadmap
 
 1. Kubernetes experimentation
-2. Readiness checks in experiments for Kubernetes resources
-3. Iter8 GitHub Action
-4. Traffic mirroring with SLO validation experiments
-5. Iter8 Tekton task
-6. A/B(/n) testing example
-7. AutoX service for auto launching Iter8 experiments, app version tracking, and business telemetry
-8. Spike/ramp testing
-9. Notifications
-10. DevSecOps experiments
+  * Readiness checks in experiments for Kubernetes resources
+2. Iter8 GitHub Action
+3. Traffic mirroring with SLO validation experiments
+4. Iter8 Tekton task
+5. A/B(/n) testing example
+6. AutoX service for auto launching Iter8 experiments, app version tracking, and business telemetry
+7. Spike/ramp testing
+8. Notifications
+9. DevSecOps experiments
+10. MLOps concept drift detection experiments
diff --git a/docs/tutorials/load-test-http/basicusage.md b/docs/tutorials/load-test-http/basicusage.md
@@ -48,28 +48,25 @@ The following metrics are collected by default by this experiment:
 
 All latency metrics have `msec` units.
 
-Latency percentile values beyond the default ones can be collected and used in SLO assessments by setting the `percentiles` parameter.
-
 ***
 
 Launch the following experiment. The `--noDownload` flag reuses the Iter8 experiment `charts` folder downloaded during the previous `iter8 launch` invocation.
 
 ```shell
 iter8 launch -c load-test-http --noDownload \
---set percentiles={45\,97.5} \
 --set url=http://127.0.0.1/get \
 --set SLOs.http/error-rate=0 \
 --set SLOs.http/latency-mean=50 \
 --set SLOs.http/latency-p90=100 \
---set SLOs.http/latency-p'97\.5'=200
+--set SLOs.http/latency-p95=200
 ```
 
 In the experiment above, the following SLOs are validated.
 
 - error rate is 0
 - mean latency is under 50 msec
-- 90th percentile latency is under 100 msec
-- 97.5th percentile latency is under 200 msec
+- 90^th^ percentile latency is under 100 msec
+- 95^th^ percentile latency is under 200 msec
 
 ***
 

diff --git a/docs/user-guide/topics/group.md b/docs/user-guide/topics/group.md
@@ -0,0 +1,49 @@
+---
+template: main.html
+---
+
+# Experiment namespace and group
+
+Iter8 experiments like [`load-test-http`](../../tutorials/load-test-http/kubernetesusage.md) and [`load-test-grpc`](../../tutorials/load-test-grpc/kubernetesusage.md) can be run within Kubernetes. Such experiments are launched within a Kubernetes namespace, and associated with a unique *group* within that namespace.
+
+For example, consider the following invocation:
+
+```shell
+iter8 k launch -c load-test-http -g hbin \
+--set url=http://httpbin.default \
+--set SLOs.http/latency-mean=50
+```
+
+In the above invocation, the `iter8 k launch` implicitly specifies the namespace as `default`, and explicitly specifies the group as `hbin`. If the group name is not specified explicitly, then it is set to `default`.
+
+The following example illustrates the relationship between namespaces, groups, and experiments.
+
+```shell
+.
+├── namespace1
+│   ├── group-a
+│   │   └── experiment
+│   ├── group-b
+│   │   └── experiment
+│   └── group-c
+│       └── experiment
+├── namespace2
+│   ├── group-a
+│   │   └── experiment
+│   ├── group-b
+│   │   └── experiment
+│   └── group-c
+│       └── experiment
+└── namespace3
+    └── group-x
+        └── experiment
+```
+
+## Use-cases
+
+1.  Run multiple experiments concurrently within a Kubernetes namespace. These experiments may be associated with the same app or with different apps.
+2.  Replace a currently running experiment in Kubernetes with a new one. When you invoke `iter8 k launch`, any previous experiment runs within the group are wiped out and replaced with a new run.
+
+## How groups work
+
+Under the covers, Iter8 implements each experiment group as a [Helm release](https://helm.sh/docs/glossary/#release) and each new experiment run within the group as an [update](https://helm.sh/docs/glossary/#release-number-release-version) of that release.
diff --git a/mkdocs.yml b/mkdocs.yml
@@ -6,7 +6,7 @@ site_description: >-
 # Repository
 repo_name: iter8-tools/iter8
 repo_url: https://github.com/iter8-tools/iter8
-# edit_uri: edit/master/docs
+edit_uri: ""
 
 # Configuration
 theme:
@@ -151,6 +151,7 @@ nav:
 - User guide:
   - Topics:
     - Chart values: user-guide/topics/values.md
+    - Namespace and group: user-guide/topics/group.md
   - Commands:
     - iter8: user-guide/commands/iter8.md
     - iter8 launch: user-guide/commands/iter8_launch.md