Process feedback case evaluation

content/case_evaluation.tex

@@ -6,30 +6,32 @@ This chapter focusses on the evaluation of the development during the case study
 The design method itself is evaluated in the next chapter.
 However, some of the topics discussed in this chapter have a strong overlap with those in the next chapter.
 
-The first section gives a short overview of results of the case study.
-Then a section is about the time spend on the development.
+The first section is about the time spend on the development.
 Followed by two sections on the role of stake holders and the use of modelling languages during a development.
 The last section is a more personal reflection about the development.
 
-%    \section{Result}
-%    \input{content/case_evaluation_result.tex}
-
     \section{Time Investment}
+    \label{sec:time_investment}
         Prior to each step in the development, I made an estimation on the workload of that particular step.
         In \autoref{fig:time_spend} the planned and spend time on each step are plotted next to each other.
-        Five of these steps were completed in the planned number of days.
-        However, three steps required more time than expected.
-        As evaluated in \autoref{sec:case_featuredefinition_evaluation}, the proposed design method for the feature definition was not feasible.
-        Documenting and solving the problem resulted in a delay of seven days.
-        The second development cycle experienced a delay of four days.
-        This was a underestimation of the time needed to complete the step.
+        In addition to the steps, time was also spend on the hardware construction and software development.
+
+        The original implementation of the feature definition was not feasible and had to be reviewed.
+        As the new approach for the feature definition is based on the expected result.
+        The time spend on performing the feature definition is not representative as there is not clear starting moment.
+
         \begin{figure}
             \centering
             \includegraphics{graphics/time_table.pdf}
-            \caption{Overview of the planned and spend number of days for each step during the case study. For Development Cycle 1 three days were planned for the initial development, based on the outcome I decided to abandon this cycle. Therefore, no additional time was planned nor spend on the development.}
+            \caption{Overview of the planned and spend number of days for each step during the case study. 
+            Some of the values in this do not represent the time requirements of this design method:\newline
+            \textsuperscript{1} During the feature definition the design method was reviewed. 13 days were spend on this review and execution, obfuscating the actual execution time. The execution time is an estimated 3 to 5 days.\newline
+            \textsuperscript{2} The first cycle was cut short due to its complexity.\newline
+            }
             \label{fig:time_spend}
         \end{figure}
-        Furthermore, there is a significant difference between the planned number of days for both development cycles.
+
+        Furthermore, there is a significant time difference both development cycles.
         Prior to the first development cycle I was not confident about the feasibility of the end-effector implementation.
         Based on that, I decided to spend about three days on the basic model of the end-effector to collect more information.
         This let me to the conclusion that the end-effector was too time-consuming for this case study.
@@ -37,28 +39,20 @@ The last section is a more personal reflection about the development.
         This time, the basic model was finished within a couple of hours.
         Based this early success and prior experience, I planned an additional two weeks of development time for this cycle.
 
-        Although not directly part of the design method, I did build a prototype.
+        To validate the design of the \ac{scara}, I build a prototype.
         This consisted of acquiring and assembling the hardware, and writing software.
         Acquiring and assembling the hardware took about two days.
         This was mainly due to CoViD-19 restrictions which made part ordering and printing more challenging.
         Without these restrictions I think it would be a day of work.
-        However, the time required to get the software to a viable state was four weeks.
 
-        Even though, the focus was not on the software, this timespan of four weeks is too significant to ignore.
+        The time required to get the software to a viable state was four weeks.
+        Even though the focus was not on the software, this timespan of four weeks was too significant to ignore.
         Especially when the software is compared to the developed models.
         As explained in the previous section, I build a total of eight models.
         Each of these models includes documentation and an evaluation of the design process.
         The software, on the other hand, is in a bare minimum state; I skipped documentation and evaluation; and the code quality relatively low.
         Still, the software was more time consuming than the hardware modeling and development.
 
-        \textcite{royce_managing_1970} also acknowledges this difference in complexity for soft- and hardware.
-        He expects 50 pages of software documentation for each page of hardware documentation in projects with comparable budget.
-        \textcite{sheard_718_1998} discusses multiple points about the difference between soft- and hardware.
-        The point that is most applicable to this case study is that pure-hardware solution are relatively simple in their problem-space perspective.
-        However, the hardware solution is often complex in the solution space perspective.
-        The inverse is true for software.
-        Thus, a complex system is easier to implement with software.
-
     \section{One-man development team}
         The case study was performed by me, as a single developer.
         Against all expectations, this one-man development team made the preparation phase more difficult instead of easier.
@@ -68,7 +62,7 @@ The last section is a more personal reflection about the development.
         Part of this is that there is no penalty for future reviews of the requirements, as I already agreed.
 
         Furthermore, specific details and decisions were often made subconsciously, while I was commuting, waiting in line, or even showering.
-        Making structured documentation of these decisions at a later point in time without missing any of them was impossible.
+        Making structured documentation of these decisions at a later point in time without missing any of them is impossible.
         The social interaction within a design team stimulates this documenting process as it improves the recall and interpretation of information.
         It also improves the judgement and selection of design alternatives \autocite{lamb_221_2008}.
 
@@ -84,12 +78,12 @@ The last section is a more personal reflection about the development.
         Resulting in a fourth model which represents the mechanical component design, in the form of a CAD drawing.
         
         There are a couple of problems with this approach.
-        Implementing the same model with a different modelling approach, makes both models incompatible with each other.
+        Implementing the same model with two different modelling approaches, makes both models incompatible with each other.
         This removes the possibility to switch back to a lower detail implementation.
         Additionally, it creates the possibility to transfer parameters incorrectly from one model to another.
         Such a switch is also labor intensive as the complete model has to be build from scratch again.
         Furthermore, there is the possibility that this new model has been for nothing, as the planned detail proves to be unfeasible.
-        The point is, a future iteration of the design method must avoid these type of model switches.
+        The point is, a future iteration of the design method must minimize these type of model switches to reduce the chance of implementation errors.
 
     \section{Reflection}
         In the following section, I reflect on my own impact on the development.
@@ -97,7 +91,7 @@ The last section is a more personal reflection about the development.
 
         \subsection{Preparation phase}
             During the preparation phase often I had difficulty with getting the required information.
-            The information was often not specific enough or it it was overlooked.
+            The information was often not specific enough or it was overlooked.
             Even though attempting to be thorough, requirements were never really specific.
             As explained in the previous section, the lack of stake-holders is one of the reasons for information not being specific.
             Furthermore, during the preparation information was often overlooked.
@@ -123,8 +117,7 @@ The last section is a more personal reflection about the development.
             For example, the \ac{scara} and the \ac{cdc} both include two stepper motors.
             When I add detail to the stepper motor model, the \ac{scara} and the \ac{cdc} would then be updated as well.
             However, each sub-model has to be updated manually.
-            In total four times in case of the stepper motor.
-            Which makes this workflow very labor intensive.
+            In total four times in case of the stepper motor, which makes this workflow very labor intensive.
 
             A workflow that enables easy combination and interchange of sub-models is beneficial with this design method.
             It makes it easy to evaluate the latest changes, by comparing them with previous versions.
@@ -134,12 +127,15 @@ The last section is a more personal reflection about the development.
 
         \subsection{Continuation of this Case Study}
             \label{sec:evaluation_reflection_protoype}
-            At the point that the SCARA was implemented, I gathered so much new information that the some of the design choices felt obsolete.
-            Although, the current design method does not incorporate a prototype, the current state of the design is more or less a prototype.
+            At the point that the \ac{scara} was implemented, I gathered so much new information that the some of the design choices felt obsolete.
+            In this case study, the prototype is used to validate the design.
+            However, the current prototype contains so much information that it would improve the requirements and initial design significantly.
+            
+            Following the current design plan, the next step would be to develop the \ac{cdc}.
             In theory, if I would continue the case study, my proposal is to consider the current design as an actual prototype and revisit the preparation phase.
-            With the current knowledge, the specifications and initial design would improve significantly.
             However, it is very important to note that this decision relies on the fact that the prototype is already created.
             In other words, the work is already done and resulted in useful information for a next design iteration.
-            But, in case of a different system, I doubt that creating a prototype, followed by a full repeat of the design method is an efficient approach.
-            Therefore, the choice to revisit the preparation phase must not be considered as an improved design method but as an argument to improve the preparation phase itself.
 
+            In case of a different system, I doubt that creating a prototype, followed by a full repeat of the design method is an efficient approach.
+            Therefore, the choice to revisit the preparation phase must not be considered as an improved design method but as an argument to improve the preparation phase itself.
+            However, I think that an improved preparation phase must be shorter and incorporate a prototype.

content/case_evaluation_result.tex

@@ -1,4 +1,6 @@
 %&tex
+\section{Result}
+\label{sec:result}
 In the end, the development produced eight models with increasing levels of detail and one prototype.
 The different levels of detail and how they are modelled are shown in \autoref{fig:levels}.
 The assembled \ac{scara} prototype is shown in \autoref{fig:prototype}.
@@ -14,6 +16,7 @@ In addition, it was possible to write three characters. Therefore, passing \auto
     }
     \label{fig:levels}
 \end{marginfigure}
+
 \begin{figure}[t]
     \hspace{5mm}
     \includegraphics[width=96mm]{graphics/prototype.JPG}

content/case_experiment.tex

@@ -37,4 +37,3 @@ Splitting the initial design into features is done in the feature definition ste
 
 \section{System Design Validation}
 \input{content/case_experiment_prototype.tex}
-

content/case_experiment_prototype.tex

@@ -75,6 +75,5 @@ The position from the interpolation is then converted to angles using the look-u
 The angles are pushed to the software stepper driver, which are used to calculate the interval between steps.
 The data path for drawing a line is shown in \autoref{fig:objects}.
 
-\section{Result}
 \input{content/case_evaluation_result.tex}
 

content/reflection.tex

@@ -76,6 +76,39 @@
         Probably resulting in a situation where specific design details are spread over different sub-models.
         Such a subdivision of details across different models is, without any doubt, undesired.
 
+    \section{System Complexity}
+        \autoref{sec:time_investment} explains the time resources required for the development of the software in the system.
+        Even though the focus was creating a hardware focussed solution for the "Tweet on a whiteboard", the complexity of the software required for this system was underestimated.
+        \textcite{royce_managing_1970} also acknowledges this difference in complexity for soft- and hardware.
+        He expects 50 pages of software documentation for each page of hardware documentation in projects with comparable budget.
+        
+        Although the focus was on complex hardware solution, this solution was only possible with the use of software.
+        The interaction between the \ac{scara} and \ac{cdc} is only possible with software that can switch states.
+        Furthermore, the path planning used to write characters on the board is completely software dependent as well.
+        \textcite{sheard_718_1998} discusses that pure-hardware solution are relatively simple in their problem-space perspective.
+        However, the hardware solution is often complex in the solution space perspective.
+        And indeed, during the initial design in the case study, the choice was made for the most complex hardware solution.
+        But without software, the \ac{scara} and \ac{cdc} have no functionality.
+
+        Another point on system complexity is prototyping.
+        Because hardware tends to be relatively simple, building a hardware prototype such as the \ac{scara} is cheap and quick.
+        An initial hardware prototype is easily constructed with \ac{ots} readily available.
+        Because the hardware transfers power the interfacing between components is straight forward.
+        For example, linear actuation can be achieved with a rack and pinion construction, linear motor, gear and chain link, or a connecting rod.
+        This might not be part of the final product, but it is useful to investigate the feasibility of the project.
+
+        Furthermore, the changes are also easily made to hardware.
+        It is possible to weld or glue new parts on or remove them with the angle grinder.
+        Adding components to software is tedious and can lead to unwanted behavior.
+        However, this is difficult to test because the software is more complex.
+        Moreover, unwanted behavior of the hardware is discoverable, and when it breaks it is often destructive.
+        The software can run for multiple days before crashing, as a result of integer, stack or buffer overflows for example.
+
+        As long as the development is still in progress, one hardware system is more malleable than the software in terms of resources.
+        When the production of a product starts, changing multiple hardware systems becomes economically unviable.
+        A design method for \ac{cps} must acknowledge that software has a high \emph{cost of change} and has also a high \emph{chance of failure}.
+        Additionally, the design method must use the hardware prototype low \emph{cost of change} to its advantage.
+
     \section{Preparation Phase}
         The start of this chapter explains the reason to prepend the preparation phase to the \ac{ridm}.
         Where the preparation phase aims to produce the requirements and features, based on the waterfall method.

graphics/time_table.tex

@@ -13,21 +13,23 @@
     xmin=0,
     xmax=21,
     width=85mm,
-    height=75mm,
+    height=80mm,
     enlarge y limits=0.15,
     legend style={at={(0.98,0.95)},
         anchor=north east,legend columns=-1},
     xlabel={Days},
-    symbolic y coords={Software Development,Development Cycle 2,Development Cycle 1,Test Protocol,Feature Definition,Initial Design,Specifications,Problem Description},
+    symbolic y coords={Software Development,Hardware Construction,Development Cycle 2,Development Cycle 1,Test Protocol,Feature Definition,Initial Design,Specifications,Problem Description},
     ytick=data,
 ]
-    \addplot[draw=red,fill=red!40] coordinates {(2,Problem Description) (3,Specifications) (3.5,Initial Design) (13,Feature Definition) (0.5,Test Protocol) (3,Development Cycle 1) (17,Development Cycle 2) (20,Software Development)};
-    \addplot[draw=blue,fill=blue!40] coordinates {(2,Problem Description) (3,Specifications) (5,Initial Design) (6,Feature Definition) (1,Test Protocol) (3,Development Cycle 1) (13,Development Cycle 2)};
+    \addplot[draw=red,fill=red!40] coordinates {(2,Problem Description) (3,Specifications) (3.5,Initial Design) (2.5,Feature Definition) (0.5,Test Protocol) (3,Development Cycle 1) (14,Development Cycle 2) (2,Hardware Construction) (20,Software Development)};
+    \addplot[draw=blue,fill=blue!40] coordinates {(2,Problem Description) (3,Specifications) (5,Initial Design) (5,Feature Definition) (1,Test Protocol) (3,Development Cycle 1) (11,Development Cycle 2)};
     \legend{Time Spend,Time Planned};
     \pgfmathsetlengthmacro{\baroffset}{\barwidthvalue+\xbarvalue/2}
     \pgfmathsetlengthmacro{\smeer}{7mm}
-    \node (a) at ($(axis cs:3,Development Cycle 1)-(0.1mm,0mm)+(0mm,\baroffset)$) {};
-    \node (b) at ($(axis cs:3,Development Cycle 1)-(0.1mm,0mm)-(0mm,\baroffset)$) {};
+    \pgfmathsetlengthmacro{\superscriptoffset}{8.5mm}
+    \node (d) at ($(axis cs:3,Development Cycle 1)+(\superscriptoffset,1mm)$) {\textsuperscript{2}};
+    \coordinate (a) at ($(axis cs:3,Development Cycle 1)-(0.1mm,0mm)+(0mm,\baroffset)$);
+    \coordinate (b) at ($(axis cs:3,Development Cycle 1)-(0.1mm,0mm)-(0mm,\baroffset)$);
     \fill[white] (a) rectangle ++(\smeer,-\barwidthvalue-1mm);
     \fill[red!40,path fading=east] (b) rectangle ++(\smeer,1mm);
     \fill[blue!40,path fading=east] (a) rectangle ++(\smeer,-\barwidthvalue);
@@ -35,6 +37,13 @@
     \draw[white] ($(b)$) -- +(\smeer,0mm); 
     \draw[blue,path fading=east] ($(a)$) -- +(\smeer,0mm) ++(0mm,-\barwidthvalue) -- +(\smeer,0mm);
     \draw[red,path fading=east] ($(b)$) -- +(\smeer,0mm); 
+    
+    \node (e) at ($(axis cs:3,Feature Definition)+(\superscriptoffset,1mm)$) {\textsuperscript{1}};
+    \coordinate (c) at ($(axis cs:2.5,Feature Definition)-(0.1mm,0mm)-(0mm,\baroffset)$);
+    \fill[red!40,path fading=east] (c) rectangle ++(\smeer,1mm);
+    \draw[white] ($(c)$) -- +(\smeer,0mm); 
+    \draw[red,path fading=east] ($(c)$) -- +(\smeer,0mm); 
+    \draw[blue,path fading=east] ($(c)+(0pt,1mm)$) -- +(\smeer,0mm);% ++(0mm,-\barwidthvalue) -- +(\smeer,0mm);
 \end{axis}
 \end{tikzpicture}
 \end{document}