Process feedback conclusion

content/conclusion.tex

@@ -1,14 +1,13 @@
 %&tex
 %    \begin{itemize}
-%        \item Extend the \ac{ridm} with a preliminary design phase.
-%            This makes it possible develop a system for a given problem or idea, using this design method.
-%        \item Refine the \ac{ridm} to make the execution of the different design steps explicit and unambiguous.
-%        \item Develop and perform a case study that tests and evaluates the \ac{ridm}.
+%        \item Extend the \ac{ridm} with a preliminary design phase, focussing on the physical part of \ac{cps}.
+%        \item Refine the \ac{ridm} to make the design steps more explicit with improved instructions.
+%        \item Develop and perform a case study that tests and evaluates the \ac{ridm} as a design method for the physical part of \ac{cps}.
 %    \end{itemize}
-%    Based on the results of the case study I will answer the following research questions:
+%    Evaluation of the \ac{ridm} as a design method is done with the results of the case study as the following objectives:
 %    \begin{itemize}
-%        \item Which techniques of the \ac{ridm} can be applied developing the physical part of \ac{cps}?
-%        \item Which adaptations are required to make the \ac{ridm} suitable for developing the computation and physical part of \ac{cps}?
+%        \item Assess the influence that applying the \ac{ridm} has on the design process for \ac{cps}.
+%        \item Describe which adaptations are required for both the \ac{ridm} and the design method to establish a competent design process for \ac{cps}.
 %    \end{itemize}
 
 
@@ -17,54 +16,166 @@
 \label{chap:conclusion}
 % Intro: end goal
 
-% Reflect Extend the RIDM with a preliminary design phase. This makes it possible develop a system for a given problem or idea, using this design method.
-The first research objective is to extend the \ac{ridm} with a preliminary design phase. 
-This makes it possible develop a system for a given problem or idea, using the \ac{ridm}.
-Based on the waterfall model as described by \ac{se}, the steps for problem description, requirements, and initial design were prepended to the \ac{ridm}.
+\section{Case Study}
+
+% Reflect: Extend the RIDM with a preliminary design phase. This makes it possible develop a system for a given problem or idea, using this design method.
+\emph{Extend the \ac{ridm} with a preliminary design phase, focussing on the physical part of \ac{cps}.}\newline
+To get from a given problem or idea, to an initial design that can be used by the \ac{ridm}, a linear set of steps was added.
+This set consists of a problem definition, requirements and initial design step.
+These steps are based on the \ac{se}-approach.
 
 % Reflect: Refine the RIDM to make the execution of the different design steps explicit and unambiguous.
-The second research objective is to refine the \ac{ridm} to make the execution of the different design steps explicit and unambiguous.
-The \ac{ridm} specify the development cycle and the variable detail approach with enough detail, making them ready to use.
+\emph{Refine the \ac{ridm} to make the design steps more explicit with improved instructions.}\newline
+To perform a reproducible evaluation of the \ac{ridm}, the method of the different design steps were defined more explicit.
+The \ac{ridm} specifies the development cycle and the variable-detail approach with sufficient detail, making them ready to use.
 How to define features and tests for the development cycle, were not as clearly defined.
-Two design steps are added in this thesis that describe a method to define the set of features and create a test protocol.
-Furthermore, a feature selection step is added to aid with the development.
+Two design steps were added in this thesis that describe a method to define the set of features and create a test protocol.
+Furthermore, a feature selection step was added to aid with the development.
 
 % Reflect: Develop and perform a case study that tests and evaluates the RIDM.
-The third and last research objective is to develop and perform a case study that tests and evaluates the design plan.
-The case study consists of a development that is performed according to the design plan.
-A set of requirements is made to ensure the optimal system of design.
-With this the \emph{Tweet on a Writeboard} system is chosen.
-The progress of the case study is monitored according to a list of questions.
-
-
-% Answer:  Which design techniques of the design method by Broenink and Broenink (2019) can be applied developing the physical part of CPS?
-With three research objectives fulfilled, it is possible to answer the two research questions:\\
-\emph{Which techniques of the \ac{ridm} can be applied developing the physical part of CPS?}\\
-
-To answer this question I must put emphasis on the difference between the design and modelling process.
-The design process embodies the development of a product or system as an answer to a problem or need.
-The modelling process allows the developers to gain insight of the inner workings of a product.
-By creating and simulating models for the system under design, the modelling process improves the design process tremendously.
-Looking at the \ac{ridm}, the fact that the first research objective is to prepend design steps to the \ac{ridm} highlights its shortcoming as a design method.
-
-Despite its exploratory nature, the case study offers some insight into the \ac{ridm} as a technique for rapid prototyping.
-The segmentation of the design provides a structured and organized approach.
-Moreover, the in this thesis proposed feature selection procedure contribute to the risk management of the development.
-By implementing high risk-per-time features first, the design problems are more likely to be found in the early stage of the design.
-
-The variable-detail approach is promising for the implementation of individual features.
-Similar to the design begin split in features, this approach implements one feature as multiple levels of detail.
-One benefit is that the structured addition of detail enables intermediate testing, allowing the development to continue when all tests are satisfied.
-Another benefit is that having one model available in different level of detail is that these models can be reused with the minimum detail possible.
-This keeps the complexity of models to a minimum and can be useful to improve simulation speed of large systems.
-The major limitation in this thesis is that the model represented the design.
-Therefore, the stopping at a certain level of detail or reusing lower detail models did not occur during the case study.
-Notwithstanding these limitations, the variable-detail approach does offer a structured approach providing feedback during the implementation.
-
-\emph{Which adaptations are required to make the \ac{ridm} suitable for developing the computation and physical part of CPS?}\\
-The most obvious finding to emerge from this study is that the \ac{ridm} without any additions is not a valid design method.
-The findings of the case study suggest that a worthwhile solution is to make \ac{ridm} part of a existing design method.
-The existing method provides a basis wherein the \ac{ridm} can come to its own, which is to tackle complexity.
+\emph{Develop and perform a case study that tests and evaluates the \ac{ridm} as a design method for the physical part of \ac{cps}.}\newline
+The case study consisted of the development of a \emph{Tweet on a Whiteboard} writer.
+This development is performed according to the design plan, that was the result of the first two research objectives.
+The \emph{tweet on a whiteboard} writer was chosen as subject of design based on a set of requirements.
+The aim of these requirements is to find a subject of design that would optimize the evaluation of the \ac{ridm}.
+
+A list of questions was formed to monitor the progress of the case study.
+The questions are answered before and after each step of the design process.
+The list was created to ensure a consistent flow of information that can be used to compare the expected result with the actual result of each step.
+
+\section{\acl{ridm}}
+\emph{Assess the influence that applying the \ac{ridm} has on the design process for \ac{cps}.}\newline
+%De kern van het RIDM bestaat uit de development cycle en de variable-detail approach.
+%Beide hebben hun eigen specifieke invloed op het design process.
+The core of the \ac{ridm} consists of the development cycle and the variable-detail approach.
+Both of these methods have specific influence on the design process.
+
+The development cycle introduces a feature-based approach to the development process.
+With the development cycle the system is implemented feature by feature.
+This requires the development team to split the functionality of the system into features.
+It forces the developers to go through the design in a structured manner.
+Furthermore, to determine in what order the features are implemented, the developers must establish the \emph{cost of change} and \emph{chance of failure}-metric for each feature.
+
+With the \emph{chance of failure} and \emph{cost of change} metrics, the features are order with the goal to reduce the impact of a design failure.
+Even though the case study only applied the feature selection twice, it proved itself useful by selecting the end-effector feature first.
+By prioritizing the end-effector, its failure had only a minor impact on the design.
+
+During each iteration of the development cycle, the selected feature is implemented according to the variable-detail approach.
+However, the ability to assess the influence of the variable-detail approach is limited by the absence of tooling for model orginization and testing.
+Without the tooling it is difficult to switch between model versions, undo design changes or run automated testing.
+Furthermore, as the development did not distinct between design and model, the models used often contained more detail than strictly necessary.
+Both these limitations resulted in models that would surpase the minimal required level of detail; therefore, it is not possible to assess whether the minimum required level of detail can be established with passing all the tests.
+Nevertheless, the variable-detail approach introduces a step wise addition of detail that enforces a structered method similar to the development cycle.
+
+It is unfortunate that the development cycle did not include a structured method to define the features nor their order of implementation.
+The performance of the variable-detail approach is currently hindered by the absence of tooling.
+Consequently, this limits the accuracy of the assessment on the actual influence of the \ac{ridm}.
+Notwithstanding these limitations, the results of the case study suggest that the structured approach of \ac{ridm} reduces the impact of design failures and reduces the development cost for \ac{cps} design.
+
+\emph{Describe which adaptations are required for both the \ac{ridm} and the design method to establish a competent design process for \ac{cps}.}\newline
+At the start of this thesis it was clear that the \ac{ridm} required adaptations to make it suitable for the development of physical part of a \ac{cps}.
+A new design method was created by adding a preparation phase and refining the steps that \ac{ridm} provided.
+The case study showed that it is possible to create a set of features and implement those features with the new design method.
+However, the adaptations show variable degrees of success. 
+
+%When designing from scratch the preparation phase fullfills a far more important role than I initialy expected.
+
+In the design method in this thesis, the goal of the preparation phase is to define the features of the system.
+These features stem from spliting the functionality and each of the feature is then developed using the \ac{ridm}.
+However, the functionality of the system is dictated by the design choices made in the system.
+In the case of developing systems from scratch, therefore, it seems that the design steps of the preparation phase play a critical role in the success of the design process.
+
+The \ac{ridm} must have a design process to get from the problem description to the features or the \ac{ridm} must be incorporated into an existing design model, in order to use the \ac{ridm} as a design process for \ac{cps}.
+In either situation, the functionality, components and requirements of the system must be incorporated together in the design.
+These three elements together form the features of the system that are implemented using the \ac{ridm}.
+
+The evaluation of the case study suggest that the feature selection method described in this thesis is an effective approach to establish the order of implementation.
+The current metrics used to establish the order leave much room for improvement.
+Both the \emph{cost of change} and the \emph{chance of failure} metric must be improved in order to be used more reliable.
+
+Apart from the lack of preparation phase, the \ac{ridm} has to a couple of obstacles to fully utilize the advantages that it provides.
+The models from the development cycle and the variable-detail approach must inherit all their properties from the design documents.
+To make design changes easier, there must be a machine readable database for all design parameters, where all models download the required parameters from.
+
+Especially the variable-detail approach is currently hindered by the lack of tooling.
+This is even more apparent when the model is correctly separated from the design, as it allows for more specific models.
+This results in more models that are smaller.
+The large set of models improves the testing results, but tooling for automated testing is required to handle the increasing amount of models.
+Furthermore, to deal with the large set of models, the modelling software must be compatible with version control.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+%Er moet een preparation phase komen
+%%Software en hardware mogen niet los van elkaar gezien worden.
+%%Meenemen dat een feature uit functie, component en requirement bestaat.
+
+%Improve feature selection
+%% Betere berekening van coc en cof.
+
+%Organize development cycle
+%%Deze moet rekening houden met het verschil tussen design en methode
+%%Tooling voor design parameters
+%%Version control
+
+%Improve testing.
+%% De modellen moeten automatisch te testen zijn.
+
+
+
+
+
+%Omdat het RIDM geen complete methode aandraagt om dit gestructureerd te doen is in deze thesis een lineare set aan ontwikkelings stappen gebruikt.
+A method to obtain the features for the system is not provided by the \ac{ridm}.
+Therefore, in this thesis a linear set of design steps is used instead.
+%Helaas boden deze stappen te weinig structuur om tot een degelijke set aan features te komen.
+Although the case study shows a set of features that are partially implemented, the evaluation shows that the current method is flawed.
+
+%De feature based approach kan daadwerkelijk bijdragen aan het ontwerpproces, maar dan moet het RIDM een structurele methode bevatten om volledig tot zijn recht te komen.
+Further research should be carried out to improve the estimation for the \emph{cost of change} and \emph{chance of failure}.
+
+
+
+%With each cycle a single feature is implemented and tested.
+%The most obvious finding to emerge from this study is that the \ac{ridm} without any additions is not a valid design method.
+%The findings of the case study suggest that a worthwhile solution is to make \ac{ridm} part of a existing design method.
+%The existing method provides a basis wherein the \ac{ridm} can come to its own, which is to tackle complexity.
+
+%To answer this question I must put emphasis on the difference between the design and modelling process.
+%The design process embodies the development of a product or system as an answer to a problem or need.
+%The modelling process allows the developers to gain insight of the inner workings of a product.
+%By creating and simulating models for the system under design, the modelling process improves the design process tremendously.
+%Looking at the \ac{ridm}, the fact that the first research objective is to prepend design steps to the \ac{ridm} highlights its shortcoming as a design method.
+%
+%Despite its exploratory nature, the case study offers some insight into the \ac{ridm} as a technique for rapid prototyping.
+%The segmentation of the design provides a structured and organized approach.
+%Moreover, the in this thesis proposed feature selection procedure contribute to the risk management of the development.
+%By implementing high risk-per-time features first, the design problems are more likely to be found in the early stage of the design.
+%
+%The variable-detail approach is promising for the implementation of individual features.
+%Similar to the design begin split in features, this approach implements one feature as multiple levels of detail.
+%One benefit is that the structured addition of detail enables intermediate testing, allowing the development to continue when all tests are satisfied.
+%Another benefit is that having one model available in different level of detail is that these models can be reused with the minimum detail possible.
+%This keeps the complexity of models to a minimum and can be useful to improve simulation speed of large systems.
+%The major limitation in this thesis is that the model represented the design.
+%Therefore, the stopping at a certain level of detail or reusing lower detail models did not occur during the case study.
+%Notwithstanding these limitations, the variable-detail approach does offer a structured approach providing feedback during the implementation.