|
|
@@ -1,17 +1,35 @@ |
|
|
%&tex |
|
|
%&tex |
|
|
\chapter{Case Study: Evaluation} |
|
|
\chapter{Case Study: Evaluation} |
|
|
\label{chap:case_evaluation} |
|
|
\label{chap:case_evaluation} |
|
|
|
|
|
\begin{marginfigure} |
|
|
|
|
|
\centering |
|
|
|
|
|
\includegraphics[width=51mm]{graphics/model_versions.pdf} |
|
|
|
|
|
\caption{ |
|
|
|
|
|
Levels of detail of the design are shown on the right, starting with the least detail at the top and most detail at the bottom. |
|
|
|
|
|
Through out the development different types of models are used, these are shown on the left. |
|
|
|
|
|
} |
|
|
|
|
|
\label{fig:levels} |
|
|
|
|
|
\end{marginfigure} |
|
|
|
|
|
The previous chapter described the development and implementation process of the Whiteboard Writer. |
|
|
|
|
|
This chapter focusses on the evaluation of the development during the case study. |
|
|
|
|
|
The design method itself is evaluated in the next chapter. |
|
|
|
|
|
However, some of the topics discussed in this chapter have a strong overlap with those in the next chapter. |
|
|
|
|
|
|
|
|
|
|
|
The first section gives a short overview of results of the case study. |
|
|
|
|
|
Then a section is about the time spend on the development. |
|
|
|
|
|
Followed by two sections on the role of stake holders and the use of modelling languages during a development. |
|
|
|
|
|
The last section is a more personal reflection about the development. |
|
|
|
|
|
|
|
|
\section{Result} |
|
|
\section{Result} |
|
|
\input{content/case_evaluation_result.tex} |
|
|
\input{content/case_evaluation_result.tex} |
|
|
|
|
|
|
|
|
\section{Time Investment} |
|
|
\section{Time Investment} |
|
|
Prior to each step in the development, I made an estimation on the workload of that particular step. |
|
|
Prior to each step in the development, I made an estimation on the workload of that particular step. |
|
|
In \autoref{fig:time_spend} the planned and spend time on each step is plotted next to each other. |
|
|
|
|
|
|
|
|
In \autoref{fig:time_spend} the planned and spend time on each step are plotted next to each other. |
|
|
Five of these steps were completed in the planned number of days. |
|
|
Five of these steps were completed in the planned number of days. |
|
|
However, three steps required more time than expected. |
|
|
However, three steps required more time than expected. |
|
|
As evaluated in \autoref{sec:case_featuredefinition_evaluation}, the proposed design method for the Feature Definition was not feasible. |
|
|
|
|
|
Solving this problem resulted in a delay of seven days. |
|
|
|
|
|
|
|
|
As evaluated in \autoref{sec:case_featuredefinition_evaluation}, the proposed design method for the feature definition was not feasible. |
|
|
|
|
|
Documenting and solving the problem resulted in a delay of seven days. |
|
|
The second development cycle experienced a delay of four days. |
|
|
The second development cycle experienced a delay of four days. |
|
|
This was a underestimation of the time needed to complete the step. |
|
|
This was a underestimation of the time needed to complete the step. |
|
|
\begin{figure} |
|
|
\begin{figure} |
|
|
@@ -32,8 +50,9 @@ |
|
|
This consisted of acquiring and assembling the hardware, and writing software. |
|
|
This consisted of acquiring and assembling the hardware, and writing software. |
|
|
Acquiring and assembling the hardware took about two days. |
|
|
Acquiring and assembling the hardware took about two days. |
|
|
This was mainly due to CoViD-19 restrictions which made part ordering and printing more challenging. |
|
|
This was mainly due to CoViD-19 restrictions which made part ordering and printing more challenging. |
|
|
Without these restrictions I think it would it would be a day of work. |
|
|
|
|
|
|
|
|
Without these restrictions I think it would be a day of work. |
|
|
However, the time required to get the software to a viable state was four weeks. |
|
|
However, the time required to get the software to a viable state was four weeks. |
|
|
|
|
|
|
|
|
Even though, the focus was not on the software, this timespan of four weeks is too significant to ignore. |
|
|
Even though, the focus was not on the software, this timespan of four weeks is too significant to ignore. |
|
|
Especially when the software is compared to the developed models. |
|
|
Especially when the software is compared to the developed models. |
|
|
As explained in the previous section, I build a total of eight models. |
|
|
As explained in the previous section, I build a total of eight models. |
|
|
@@ -41,66 +60,83 @@ |
|
|
The software, on the other hand, is in a bare minimum state; I skipped documentation and evaluation; and the code quality relatively low. |
|
|
The software, on the other hand, is in a bare minimum state; I skipped documentation and evaluation; and the code quality relatively low. |
|
|
Still, the software was more time consuming than the hardware modeling and development. |
|
|
Still, the software was more time consuming than the hardware modeling and development. |
|
|
|
|
|
|
|
|
|
|
|
\textcite{royce_managing_1970} also acknowledges this difference in complexity for soft- and hardware. |
|
|
|
|
|
He expects 50 pages of software documentation for each page of hardware documentation in projects with comparable budget. |
|
|
|
|
|
\textcite{sheard_718_1998} discusses multiple points about the difference between soft- and hardware. |
|
|
|
|
|
The point that is most applicable to this case study is that pure-hardware solution are relatively simple in their problem-space perspective. |
|
|
|
|
|
However, the hardware solution is often complex in the solution space perspective. |
|
|
|
|
|
The inverse is true for software. |
|
|
|
|
|
Thus, a complex system is easier to implement with software. |
|
|
|
|
|
|
|
|
\section{One-man development team} |
|
|
\section{One-man development team} |
|
|
The case study was performed by me, as a single developer. |
|
|
The case study was performed by me, as a single developer. |
|
|
Against all expectations, this one-man development team made the preparation phase more difficult instead of easier. |
|
|
Against all expectations, this one-man development team made the preparation phase more difficult instead of easier. |
|
|
The goal of the problem description and the specifications step is to get the stakeholders on the same line \autocite{shafaat_exploring_2015}. |
|
|
|
|
|
|
|
|
The goal of the problem description and the requirements step is to get the stakeholders on the same line \autocite{shafaat_exploring_2015}. |
|
|
This involves creating agreed-upon requirements for the system, but with only one stakeholder, this agreement is implicit. |
|
|
This involves creating agreed-upon requirements for the system, but with only one stakeholder, this agreement is implicit. |
|
|
Moreover, it undermines the incentive of the problem description and specifications step. |
|
|
|
|
|
Part of this is that there is no penalty for future reviews of the specifications, as I already agreed. |
|
|
|
|
|
|
|
|
Moreover, it undermines the incentive of the problem description and requirements step. |
|
|
|
|
|
Part of this is that there is no penalty for future reviews of the requirements, as I already agreed. |
|
|
|
|
|
|
|
|
Furthermore, specific details and decisions were often made subconsciously, while I was commuting, waiting in line, or even showering. |
|
|
Furthermore, specific details and decisions were often made subconsciously, while I was commuting, waiting in line, or even showering. |
|
|
Making structured documentation of these decisions at a later point in time without missing any of them was impossible. |
|
|
Making structured documentation of these decisions at a later point in time without missing any of them was impossible. |
|
|
The social interaction within a design team stimulates this documenting process as it improves the recall and interpretation of information. |
|
|
The social interaction within a design team stimulates this documenting process as it improves the recall and interpretation of information. |
|
|
It also improves the judgement and selection between design alternatives \autocite{lamb_221_2008}. |
|
|
|
|
|
|
|
|
It also improves the judgement and selection of design alternatives \autocite{lamb_221_2008}. |
|
|
|
|
|
|
|
|
|
|
|
\section{Switching Modelling Language} |
|
|
|
|
|
The initial idea of the development was to start with a basic model and extend that model by adding more detail. |
|
|
|
|
|
Meaning that one design and one model would develop in parallel with each other. |
|
|
|
|
|
However, the development of the \ac{scara} resulted in four major model versions. |
|
|
|
|
|
The basic model started with a kinematics model. |
|
|
|
|
|
To take the physics of the design into account, a 2D dynamics model was created. |
|
|
|
|
|
Multiple steps of detail into the development, the 2D model was not adequate anymore. |
|
|
|
|
|
Therefore, the design was remodeled with 3D physics. |
|
|
|
|
|
Although this 3D physics model was able to implement the dynamic behavior, the modeling language was not suitable to design the shape of the mechanical components. |
|
|
|
|
|
Resulting in a fourth model which represents the mechanical component design, in the form of a CAD drawing. |
|
|
|
|
|
|
|
|
|
|
|
There are a couple of problems with this approach. |
|
|
|
|
|
Implementing the same model with a different modelling approach, makes both models incompatible with each other. |
|
|
|
|
|
This removes the possibility to switch back to a lower detail implementation. |
|
|
|
|
|
Additionally, it creates the possibility to transfer parameters incorrectly from one model to another. |
|
|
|
|
|
Such a switch is also labor intensive as the complete model has to be build from scratch again. |
|
|
|
|
|
Furthermore, there is the possibility that this new model has been for nothing, as the planned detail proves to be unfeasible. |
|
|
|
|
|
The point is, a future iteration of the design method must avoid these type of model switches. |
|
|
|
|
|
|
|
|
\section{Reflection} |
|
|
\section{Reflection} |
|
|
In the following section, I will reflect on my own impact on the development. |
|
|
|
|
|
|
|
|
In the following section, I reflect on my own impact on the development. |
|
|
The preparation and development phase are discussed separately. |
|
|
The preparation and development phase are discussed separately. |
|
|
|
|
|
|
|
|
\subsection{Preparation phase} |
|
|
\subsection{Preparation phase} |
|
|
During the preparation phase often I had difficulty with getting the required information. |
|
|
During the preparation phase often I had difficulty with getting the required information. |
|
|
The information was often not specific enough or it it was overlooked. |
|
|
The information was often not specific enough or it it was overlooked. |
|
|
For the case of information not being specific, can be explained by the lack of stake-holders as explained in the previous section. |
|
|
|
|
|
Even though attempting to be thorough, requirements were never really specific. |
|
|
Even though attempting to be thorough, requirements were never really specific. |
|
|
|
|
|
As explained in the previous section, the lack of stake-holders is one of the reasons for information not being specific. |
|
|
Furthermore, during the preparation information was often overlooked. |
|
|
Furthermore, during the preparation information was often overlooked. |
|
|
Resulting in a situation where I needed information that should have been the result of a previous step. |
|
|
|
|
|
|
|
|
Resulting in a situation where I needed information that should have been the result of a previous step, which was not the case. |
|
|
In most situations it was possible to continue with the execution of the step. |
|
|
In most situations it was possible to continue with the execution of the step. |
|
|
However, during the test protocol step (\autoref{sec:test_protocol}) it was not possible to continue. |
|
|
However, during the test protocol step (\autoref{sec:test_protocol}) it was not possible to continue. |
|
|
|
|
|
Resulting in additional requirements added to the design, before continuing with the design process. |
|
|
|
|
|
|
|
|
One of the main causes that attribute to the information shortage is that I am relatively inexperienced in design. |
|
|
|
|
|
The experience that I have is a graduate course and two extracurricular projects. |
|
|
|
|
|
Being inexperienced does definitely not make the design process easier. |
|
|
|
|
|
However, I think that more experience would improve the information situation, it would not solve the problem. |
|
|
|
|
|
The biggest issue is the linear approach of the waterfall model. |
|
|
|
|
|
A spiral model would be more appropriate, especially as each step results in more information that can improve the design. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
One of the main causes that attributes to the information shortage is that I am a novice designer/developer. |
|
|
|
|
|
The experience that I have is from a graduate course and two extracurricular projects. |
|
|
|
|
|
Being inexperienced does definitely not aid the design process. |
|
|
|
|
|
Needless to say, more experience would improve the information situation. |
|
|
|
|
|
However, it does not solve the problem. |
|
|
|
|
|
Further improvements for the design method is required, to improve the information process during development. |
|
|
|
|
|
|
|
|
\subsection{Development phase} |
|
|
\subsection{Development phase} |
|
|
For the development phase I posses significantly more relevant experience compared to the preparation phase. |
|
|
|
|
|
Creating models is something that I really enjoy and this smooths the process significantly. |
|
|
|
|
|
Nevertheless, there is room for improvement in this system. |
|
|
|
|
|
|
|
|
For the development phase I have significantly more experience compared to the preparation phase. |
|
|
|
|
|
Creating models is something that I really enjoy and this improves the process significantly. |
|
|
|
|
|
Even though the development phase went smoother than the preparation phase, there is still room for improvement. |
|
|
Originally I attempted to create separate sub-models for each component. |
|
|
Originally I attempted to create separate sub-models for each component. |
|
|
These sub-models could then be combined into larger models. |
|
|
|
|
|
For example, the SCARA and the cable bot both use two stepper motors. |
|
|
|
|
|
When I add detail to the stepper motor model, the SCARA and the cable bot would then be updated as well. |
|
|
|
|
|
However, the workflow to get this working is labor intensive, as each sub-model has to be updated manually. |
|
|
|
|
|
|
|
|
These sub-models can then be combined into larger models. |
|
|
|
|
|
For example, the \ac{scara} and the \ac{cdc} both include two stepper motors. |
|
|
|
|
|
When I add detail to the stepper motor model, the \ac{scara} and the \ac{cdc} would then be updated as well. |
|
|
|
|
|
However, each sub-model has to be updated manually. |
|
|
|
|
|
In total four times in case of the stepper motor. |
|
|
|
|
|
Which makes this workflow very labor intensive. |
|
|
|
|
|
|
|
|
A workflow that enables easy combination and interchange of sub-models is beneficial with this design method. |
|
|
A workflow that enables easy combination and interchange of sub-models is beneficial with this design method. |
|
|
It makes it easy to evaluate the latest changes, by comparing them with previous versions. |
|
|
It makes it easy to evaluate the latest changes, by comparing them with previous versions. |
|
|
In addition, it makes it possible to lower the detail on some models during the development. |
|
|
In addition, it makes it possible to lower the detail on some models during the development. |
|
|
The lower detail of the sub-models can improve the simulation speed significantly. |
|
|
The lower detail of the sub-models can improve the simulation speed significantly. |
|
|
And during the final test use the full detail to ensure that every thing is still functioning as expected. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
And during the final test use the full detail to ensure that every thing is performing as expected. |
|
|
|
|
|
|
|
|
\section{Switching Modelling Language} |
|
|
|
|
|
The initial idea of the development was to start with a basic model and extend that model by adding more detail. |
|
|
|
|
|
Meaning that one design and one model would develop in parallel with each other. |
|
|
|
|
|
However, the development of the SCARA resulted in four major model versions. |
|
|
|
|
|
The basic model started with a kinematics model. |
|
|
|
|
|
To take the physics of the design into account, a 2D dynamics model was created. |
|
|
|
|
|
Multiple steps of detail into the development, the 2D model was not adequate anymore. |
|
|
|
|
|
Therefore, the design was remodeled with 3D physics. |
|
|
|
|
|
Although this 3D physics model was able to implement the dynamic behavior, the modeling language was not suitable to design the shape of the mechanical components. |
|
|
|
|
|
Resulting in a fourth model which represents the mechanical component design, in the form of a CAD drawing. |
|
|
|
|
|
|
|
|
|