Add test protocol as step.

content/analysis.tex

@@ -44,11 +44,8 @@ The steps that are introduced by \ridm are covered in more detail.
         To find the best solution it is important to explore the different solutions and design space.
         Often, there are many possible alternatives but they must be narrowed down to the solutions that fit within the schedule and available resources.
         The best alternative is materialized in a design document together with the system requirements.
-        To verify that the system works in line with the system requirements, a test protocol is written for the system.
-        This protocol explains how the tests are preformed and what the pass conditions are.
         This design document is used in the next phase of the design.
 
-
 \section{Rapid Iterative Design Method}
     From this point, the design plan is based on the \ridm and not anymore on the waterfall model.
     The first step is the feature definition, which prepares the required features based on the initial design.
@@ -56,13 +53,20 @@ The steps that are introduced by \ridm are covered in more detail.
     The definition of the feature contains a description and a set of sub-requirements which is used to implement and test the feature.
     During the feature definition, the dependencies, risks and time resources are determined as well, this establishes the order of implementation in the feature selection step.
 
-    The second step is the feature selection, where one of the features is selected. 
-    This selection is based on the dependencies, risk, and time requirements in the feature definitions.
-    The third step is the rapid development cycle, which uses the sub-requirements and description of the selected feature to create an initial design, a minimal implementation and tests.
-    In the last step, the variable detail approach is used to add detail to the minimal implementation over multiple iterations.
+    Based on the requirements of \ridm as explained in \autoref{chap:background}, the next step is the feature selection.
+    However, it became apparent that the number of tests related to a specific feature is a good metric for the selection step.
+    Because, at the point that a feature is implemented, the tests are completed as well, and when the tests of the complete system pass, the system meets the specifications.
+    Following the \ridm, these tests are specified at the start of the rapid development cycle.
+    This makes it impossible to use the tests during the feature selections.
+    Therefore, a test protocol step is added after the feature definition and before the feature selection step.
+
+    The third step is the feature selection, where one of the features is selected. 
+    This selection is based on the dependencies, tests, risk, and time requirements in the feature definitions.
+    The fourth step is the rapid development cycle, which uses the sub-requirements and description of the selected feature to create an initial design and a minimal implementation.
+    In the last step, the variable detail approach is used to add detail to the minimal implementation over the course of multiple iterations.
     The tests are used to determine if the added detail does not introduce any unexpected behavior.
     This cycle of adding detail and testing is repeated till the feature is fully implemented.
-    From this point, the \ridm is repeated from the second step until all features are implemented.
+    From this point, the \ridm is repeated from the third step until all features are implemented.
 
 \subsection{Feature Definition}
     \label{sec:featuredefinition}
@@ -71,9 +75,6 @@ The steps that are introduced by \ridm are covered in more detail.
     To achieve these short cycles, the features that are implemented in these cycles, are as small as possible. 
     However, the features must still be implemented and tested individually during the implementation and can thus not be split indefinitely.
     Together with the definition of the features, the requirements are divided along the features as well.
-    During the initial design, a test protocol is made to evaluate the system-wide specifications.
-    For the feature requirements, tests are added to the test protocol as well.
-    These tests only require the corresponding feature to be implemented and must all pass to finish the implementation of the feature.
     The optimal strategy on splitting features and specifications is strongly dependent on the type of system.
     Therefore, the best engineering judgement of the developer the best tool available.
 
@@ -95,7 +96,14 @@ The steps that are introduced by \ridm are covered in more detail.
     Due to these dependencies it is possible that the division of requirements changes, because the result of the implemented feature was not as expected.
     This is not directly a problem, but a good administration of the requirements makes an update of these requirements easier. 
 
-
+\subsection{Test protocol}
+    \label{sec:systemtesting}
+    During the rapid development cycle and the variable detail approach, the system is tested constantly.
+    This is to make sure that the design still performs as expected.
+    The tests are based on the specifications.
+    Each specification must be covered with at least one test.
+    The tests consist of a description which specifies how to perform the test and what the result of the test must of must not be.
+    Together with the description, there is a list of required features to perform the test and a list of specifications that are met if the test passes.
 
 \subsection{Feature Selection}
     \label{sec:feature_selection}
@@ -194,15 +202,6 @@ The electrical motors have also internal states, but store significantly less en
 An basic model would in this case only consists of the arms, possibly even without any dynamic behavior.
 The dynamic behavior, motor characteristics, resistance, or gravitational force are examples of detail elements that can be added to increase the detail.
 
-Now with initial design and the basic model, all that is left is to describe a list of tests.
-The goal of these tests is to verify if the design meets the specifications of the feature.
-The tests have a short description on how to perform the tests and what should be achieved.
-It is important that all the specifications are covered by at least one test.
-This relatively simple approach of testing is possible due to the limited scope of this thesis.
-For a complexer design, the testing needs to be automated.
-The \ac{amt} \autocite{jansen_automated_2019} is an interesting method to perform the testing of the models.
-However, at the time of writing, the software is in a proof of concept state and not usable for this thesis.
-
 \subsection{Variable Detail Approach}
 With the variable detail approach the basic model is developed into a refined model of the feature.
 This is done by implementing the detail elements over the course of multiple iterations.
@@ -233,7 +232,7 @@ The developer must evaluate if there are feasible alternatives left for this ele
 When all detail elements are implemented and the basic model has evolved into a refined model of the feature, the design cycle moves back to the feature selection.
 In the case that this is the last feature to implement, this concludes the development.
 
-\section{Summation}
+\section{Summary of Design Plan}
 \begin{marginfigure}
     \centering
     \includegraphics[width=6cm]{graphics/design_flow_analysis.pdf}
@@ -241,11 +240,12 @@ In the case that this is the last feature to implement, this concludes the devel
     \label{fig:design_plan_analysis}
 \end{marginfigure}
 The waterfall model from \ac{se} and the \ridm \autocite{broenink_rapid_2019} are combined to create the design plan as shown in \autoref{fig:design_plan_analysis}.
-The first four steps of the design process form the preparation phase: problem description, specifications, initial design, and feature definition.
+The first five steps of the design process form the preparation phase: problem description, specifications, initial design, feature definition, and test protocol.
 The initial design step creates a holistic design based on the prior problem description and specifications step.
 The last step of the preparation is the feature definition, where the initial design is split into different features.
 The resulting initial design and its features together form the design proposal for the development steps.
-The development step consists of the feature selection, rapid development, and variable detail steps.
+The last step of the preparation phase is the test protocol step, where the tests are defined to monitor the design process and validate that the system meets the specifications.
+The development cycle consists of the feature selection, rapid development, and variable detail steps.
 These three steps are applied to each feature in the initial design individually.
 
 With each iteration of the development cycle a new feature is added to the complete system.

graphics/design_flow_analysis.tex

@@ -8,13 +8,14 @@ draw=black!50, thick, font=\footnotesize, fill=white},
 
 \begin{document}
 \begin{tikzpicture}[on grid,y=1.2cm,x=3.2cm]
-    \draw[fill=lightgray] (-1.7cm, 1.5cm) rectangle (1.7cm, -4.1cm);
-    \draw[fill=lightgray] (-1.7cm,-4.3cm) rectangle (5cm, -8.2cm);
+    \draw[fill=lightgray] (-1.7cm, 1.5cm) rectangle (1.7cm, -5.3cm);
+    \draw[fill=lightgray] (-1.7cm,-5.5cm) rectangle (5cm, -9.2cm);
     \node (pd) {Problem Description};
     \node (sp)[below=1 of pd]  {Specifications};
     \node (id)[below=1 of sp]  {Initial Design};
     \node (fs)[below=1 of id]  {Feature Definition};
-    \node (ss)[below=1 of fs]  {Feature Selection};
+    \node (tp)[below=1 of fs]  {Test Protocol};
+    \node (ss)[below=1 of tp]  {Feature Selection};
     \node (a1)[below=0.8 of ss,draw=none, fill=none] {};
     \node (rd)[below=0.8 of a1]  {Rapid Development};
     \node (va)[right=1 of a1]  {Variable Approach};
@@ -23,7 +24,8 @@ draw=black!50, thick, font=\footnotesize, fill=white},
     \path[->] (pd) edge (sp)
               (sp) edge (id)
               (id) edge (fs)
-              (fs) edge (ss)
+              (fs) edge (tp)
+              (tp) edge (ss)
               (ss) edge (rd)
               (rd.east) edge[bend right] (va)
               (va) edge[bend right] (ss.east);