diff --git a/content/conclusion.tex b/content/conclusion.tex index 3a8ea23..1b95c9e 100644 --- a/content/conclusion.tex +++ b/content/conclusion.tex @@ -1,14 +1,13 @@ %&tex % \begin{itemize} -% \item Extend the \ac{ridm} with a preliminary design phase. -% This makes it possible develop a system for a given problem or idea, using this design method. -% \item Refine the \ac{ridm} to make the execution of the different design steps explicit and unambiguous. -% \item Develop and perform a case study that tests and evaluates the \ac{ridm}. +% \item Extend the \ac{ridm} with a preliminary design phase, focussing on the physical part of \ac{cps}. +% \item Refine the \ac{ridm} to make the design steps more explicit with improved instructions. +% \item Develop and perform a case study that tests and evaluates the \ac{ridm} as a design method for the physical part of \ac{cps}. % \end{itemize} -% Based on the results of the case study I will answer the following research questions: +% Evaluation of the \ac{ridm} as a design method is done with the results of the case study as the following objectives: % \begin{itemize} -% \item Which techniques of the \ac{ridm} can be applied developing the physical part of \ac{cps}? -% \item Which adaptations are required to make the \ac{ridm} suitable for developing the computation and physical part of \ac{cps}? +% \item Assess the influence that applying the \ac{ridm} has on the design process for \ac{cps}. +% \item Describe which adaptations are required for both the \ac{ridm} and the design method to establish a competent design process for \ac{cps}. % \end{itemize} @@ -17,54 +16,166 @@ \label{chap:conclusion} % Intro: end goal -% Reflect Extend the RIDM with a preliminary design phase. This makes it possible develop a system for a given problem or idea, using this design method. -The first research objective is to extend the \ac{ridm} with a preliminary design phase. -This makes it possible develop a system for a given problem or idea, using the \ac{ridm}. -Based on the waterfall model as described by \ac{se}, the steps for problem description, requirements, and initial design were prepended to the \ac{ridm}. +\section{Case Study} + +% Reflect: Extend the RIDM with a preliminary design phase. This makes it possible develop a system for a given problem or idea, using this design method. +\emph{Extend the \ac{ridm} with a preliminary design phase, focussing on the physical part of \ac{cps}.}\newline +To get from a given problem or idea, to an initial design that can be used by the \ac{ridm}, a linear set of steps was added. +This set consists of a problem definition, requirements and initial design step. +These steps are based on the \ac{se}-approach. % Reflect: Refine the RIDM to make the execution of the different design steps explicit and unambiguous. -The second research objective is to refine the \ac{ridm} to make the execution of the different design steps explicit and unambiguous. -The \ac{ridm} specify the development cycle and the variable detail approach with enough detail, making them ready to use. +\emph{Refine the \ac{ridm} to make the design steps more explicit with improved instructions.}\newline +To perform a reproducible evaluation of the \ac{ridm}, the method of the different design steps were defined more explicit. +The \ac{ridm} specifies the development cycle and the variable-detail approach with sufficient detail, making them ready to use. How to define features and tests for the development cycle, were not as clearly defined. -Two design steps are added in this thesis that describe a method to define the set of features and create a test protocol. -Furthermore, a feature selection step is added to aid with the development. +Two design steps were added in this thesis that describe a method to define the set of features and create a test protocol. +Furthermore, a feature selection step was added to aid with the development. % Reflect: Develop and perform a case study that tests and evaluates the RIDM. -The third and last research objective is to develop and perform a case study that tests and evaluates the design plan. -The case study consists of a development that is performed according to the design plan. -A set of requirements is made to ensure the optimal system of design. -With this the \emph{Tweet on a Writeboard} system is chosen. -The progress of the case study is monitored according to a list of questions. - - -% Answer: Which design techniques of the design method by Broenink and Broenink (2019) can be applied developing the physical part of CPS? -With three research objectives fulfilled, it is possible to answer the two research questions:\\ -\emph{Which techniques of the \ac{ridm} can be applied developing the physical part of CPS?}\\ - -To answer this question I must put emphasis on the difference between the design and modelling process. -The design process embodies the development of a product or system as an answer to a problem or need. -The modelling process allows the developers to gain insight of the inner workings of a product. -By creating and simulating models for the system under design, the modelling process improves the design process tremendously. -Looking at the \ac{ridm}, the fact that the first research objective is to prepend design steps to the \ac{ridm} highlights its shortcoming as a design method. - -Despite its exploratory nature, the case study offers some insight into the \ac{ridm} as a technique for rapid prototyping. -The segmentation of the design provides a structured and organized approach. -Moreover, the in this thesis proposed feature selection procedure contribute to the risk management of the development. -By implementing high risk-per-time features first, the design problems are more likely to be found in the early stage of the design. - -The variable-detail approach is promising for the implementation of individual features. -Similar to the design begin split in features, this approach implements one feature as multiple levels of detail. -One benefit is that the structured addition of detail enables intermediate testing, allowing the development to continue when all tests are satisfied. -Another benefit is that having one model available in different level of detail is that these models can be reused with the minimum detail possible. -This keeps the complexity of models to a minimum and can be useful to improve simulation speed of large systems. -The major limitation in this thesis is that the model represented the design. -Therefore, the stopping at a certain level of detail or reusing lower detail models did not occur during the case study. -Notwithstanding these limitations, the variable-detail approach does offer a structured approach providing feedback during the implementation. - -\emph{Which adaptations are required to make the \ac{ridm} suitable for developing the computation and physical part of CPS?}\\ -The most obvious finding to emerge from this study is that the \ac{ridm} without any additions is not a valid design method. -The findings of the case study suggest that a worthwhile solution is to make \ac{ridm} part of a existing design method. -The existing method provides a basis wherein the \ac{ridm} can come to its own, which is to tackle complexity. +\emph{Develop and perform a case study that tests and evaluates the \ac{ridm} as a design method for the physical part of \ac{cps}.}\newline +The case study consisted of the development of a \emph{Tweet on a Whiteboard} writer. +This development is performed according to the design plan, that was the result of the first two research objectives. +The \emph{tweet on a whiteboard} writer was chosen as subject of design based on a set of requirements. +The aim of these requirements is to find a subject of design that would optimize the evaluation of the \ac{ridm}. + +A list of questions was formed to monitor the progress of the case study. +The questions are answered before and after each step of the design process. +The list was created to ensure a consistent flow of information that can be used to compare the expected result with the actual result of each step. + +\section{\acl{ridm}} +\emph{Assess the influence that applying the \ac{ridm} has on the design process for \ac{cps}.}\newline +%De kern van het RIDM bestaat uit de development cycle en de variable-detail approach. +%Beide hebben hun eigen specifieke invloed op het design process. +The core of the \ac{ridm} consists of the development cycle and the variable-detail approach. +Both of these methods have specific influence on the design process. + +The development cycle introduces a feature-based approach to the development process. +With the development cycle the system is implemented feature by feature. +This requires the development team to split the functionality of the system into features. +It forces the developers to go through the design in a structured manner. +Furthermore, to determine in what order the features are implemented, the developers must establish the \emph{cost of change} and \emph{chance of failure}-metric for each feature. + +With the \emph{chance of failure} and \emph{cost of change} metrics, the features are order with the goal to reduce the impact of a design failure. +Even though the case study only applied the feature selection twice, it proved itself useful by selecting the end-effector feature first. +By prioritizing the end-effector, its failure had only a minor impact on the design. + +During each iteration of the development cycle, the selected feature is implemented according to the variable-detail approach. +However, the ability to assess the influence of the variable-detail approach is limited by the absence of tooling for model orginization and testing. +Without the tooling it is difficult to switch between model versions, undo design changes or run automated testing. +Furthermore, as the development did not distinct between design and model, the models used often contained more detail than strictly necessary. +Both these limitations resulted in models that would surpase the minimal required level of detail; therefore, it is not possible to assess whether the minimum required level of detail can be established with passing all the tests. +Nevertheless, the variable-detail approach introduces a step wise addition of detail that enforces a structered method similar to the development cycle. + +It is unfortunate that the development cycle did not include a structured method to define the features nor their order of implementation. +The performance of the variable-detail approach is currently hindered by the absence of tooling. +Consequently, this limits the accuracy of the assessment on the actual influence of the \ac{ridm}. +Notwithstanding these limitations, the results of the case study suggest that the structured approach of \ac{ridm} reduces the impact of design failures and reduces the development cost for \ac{cps} design. + +\emph{Describe which adaptations are required for both the \ac{ridm} and the design method to establish a competent design process for \ac{cps}.}\newline +At the start of this thesis it was clear that the \ac{ridm} required adaptations to make it suitable for the development of physical part of a \ac{cps}. +A new design method was created by adding a preparation phase and refining the steps that \ac{ridm} provided. +The case study showed that it is possible to create a set of features and implement those features with the new design method. +However, the adaptations show variable degrees of success. + +%When designing from scratch the preparation phase fullfills a far more important role than I initialy expected. + +In the design method in this thesis, the goal of the preparation phase is to define the features of the system. +These features stem from spliting the functionality and each of the feature is then developed using the \ac{ridm}. +However, the functionality of the system is dictated by the design choices made in the system. +In the case of developing systems from scratch, therefore, it seems that the design steps of the preparation phase play a critical role in the success of the design process. + +The \ac{ridm} must have a design process to get from the problem description to the features or the \ac{ridm} must be incorporated into an existing design model, in order to use the \ac{ridm} as a design process for \ac{cps}. +In either situation, the functionality, components and requirements of the system must be incorporated together in the design. +These three elements together form the features of the system that are implemented using the \ac{ridm}. + +The evaluation of the case study suggest that the feature selection method described in this thesis is an effective approach to establish the order of implementation. +The current metrics used to establish the order leave much room for improvement. +Both the \emph{cost of change} and the \emph{chance of failure} metric must be improved in order to be used more reliable. + +Apart from the lack of preparation phase, the \ac{ridm} has to a couple of obstacles to fully utilize the advantages that it provides. +The models from the development cycle and the variable-detail approach must inherit all their properties from the design documents. +To make design changes easier, there must be a machine readable database for all design parameters, where all models download the required parameters from. + +Especially the variable-detail approach is currently hindered by the lack of tooling. +This is even more apparent when the model is correctly separated from the design, as it allows for more specific models. +This results in more models that are smaller. +The large set of models improves the testing results, but tooling for automated testing is required to handle the increasing amount of models. +Furthermore, to deal with the large set of models, the modelling software must be compatible with version control. + + + + + + + + + + + + + + + + + + + + +%Er moet een preparation phase komen +%%Software en hardware mogen niet los van elkaar gezien worden. +%%Meenemen dat een feature uit functie, component en requirement bestaat. + +%Improve feature selection +%% Betere berekening van coc en cof. + +%Organize development cycle +%%Deze moet rekening houden met het verschil tussen design en methode +%%Tooling voor design parameters +%%Version control + +%Improve testing. +%% De modellen moeten automatisch te testen zijn. + + + + + +%Omdat het RIDM geen complete methode aandraagt om dit gestructureerd te doen is in deze thesis een lineare set aan ontwikkelings stappen gebruikt. +A method to obtain the features for the system is not provided by the \ac{ridm}. +Therefore, in this thesis a linear set of design steps is used instead. +%Helaas boden deze stappen te weinig structuur om tot een degelijke set aan features te komen. +Although the case study shows a set of features that are partially implemented, the evaluation shows that the current method is flawed. + +%De feature based approach kan daadwerkelijk bijdragen aan het ontwerpproces, maar dan moet het RIDM een structurele methode bevatten om volledig tot zijn recht te komen. +Further research should be carried out to improve the estimation for the \emph{cost of change} and \emph{chance of failure}. + + + +%With each cycle a single feature is implemented and tested. +%The most obvious finding to emerge from this study is that the \ac{ridm} without any additions is not a valid design method. +%The findings of the case study suggest that a worthwhile solution is to make \ac{ridm} part of a existing design method. +%The existing method provides a basis wherein the \ac{ridm} can come to its own, which is to tackle complexity. + +%To answer this question I must put emphasis on the difference between the design and modelling process. +%The design process embodies the development of a product or system as an answer to a problem or need. +%The modelling process allows the developers to gain insight of the inner workings of a product. +%By creating and simulating models for the system under design, the modelling process improves the design process tremendously. +%Looking at the \ac{ridm}, the fact that the first research objective is to prepend design steps to the \ac{ridm} highlights its shortcoming as a design method. +% +%Despite its exploratory nature, the case study offers some insight into the \ac{ridm} as a technique for rapid prototyping. +%The segmentation of the design provides a structured and organized approach. +%Moreover, the in this thesis proposed feature selection procedure contribute to the risk management of the development. +%By implementing high risk-per-time features first, the design problems are more likely to be found in the early stage of the design. +% +%The variable-detail approach is promising for the implementation of individual features. +%Similar to the design begin split in features, this approach implements one feature as multiple levels of detail. +%One benefit is that the structured addition of detail enables intermediate testing, allowing the development to continue when all tests are satisfied. +%Another benefit is that having one model available in different level of detail is that these models can be reused with the minimum detail possible. +%This keeps the complexity of models to a minimum and can be useful to improve simulation speed of large systems. +%The major limitation in this thesis is that the model represented the design. +%Therefore, the stopping at a certain level of detail or reusing lower detail models did not occur during the case study. +%Notwithstanding these limitations, the variable-detail approach does offer a structured approach providing feedback during the implementation.