mar/content/conclusion.tex

\chapter{Conclusion}
\label{chap:conclusion}
% Intro: end goal

\section{Case Study}

% Reflect: Extend the RIDM with a preliminary design phase. This makes it possible develop a system for a given problem or idea, using this design method.
\emph{Extend the \ac{ridm} with a preliminary design phase, focussing on the physical part of \ac{cps}.}\newline
To get from a given problem or idea, to an initial design that can be used by the \ac{ridm}, a linear set of steps was added.
This set consists of a problem definition, requirements and initial design step.
These steps are based on the \ac{se}-approach.

% Reflect: Refine the RIDM to make the execution of the different design steps explicit and unambiguous.
\emph{Refine the \ac{ridm} to make the design steps more explicit with improved instructions.}\newline
To perform a reproducible evaluation of the \ac{ridm}, the method of the different design steps were defined more explicit.
The \ac{ridm} specifies the development cycle and the variable-detail approach with sufficient detail, making them ready to use.
How to define features and tests for the development cycle, were not as clearly defined.
In this thesis, two steps were added to the design method: one with a method to define the set of features and one that is used to specify the test protocol.
Two design steps were added in this thesis that describe a method to define the set of features and create a test protocol.
Furthermore, a feature selection step was added to aid with the development.

% Reflect: Develop and perform a case study that tests and evaluates the RIDM.
\emph{Develop and perform a case study that tests and evaluates the \ac{ridm} as a design method for the physical part of \ac{cps}.}\newline
The case study consisted of the development of a \emph{Tweet on a Whiteboard} writer.
This development is performed according to the design plan, that was the result of the first two research objectives.
The \emph{tweet on a whiteboard} writer was chosen as subject of design based on a set of requirements.
The goal of these requirements is to find a subject of design that evaluates most aspects of the \ac{ridm} when implemented.

A list of questions was formed to monitor the progress of the case study.
The questions are answered before and after each step of the design process.
The list was created to ensure a consistent documentation of the expected outcome and the actual outcome of each step.
Both the expected and actual outcome are used to evaluate the design step.

\section{\acl{ridm}}
\emph{Assess the influence that applying the \ac{ridm} has on the design process for \ac{cps}.}\newline
The core of the \ac{ridm} consists of the development cycle and the variable-detail approach.
Both of these methods have specific influence on the design process.

The development cycle introduces a feature-based approach to the development process.
With the development cycle the system is implemented feature by feature.
This requires the development team to split the functionality of the system into features.
It forces the developers to go through the design in a structured manner.
Furthermore, to determine in what order the features are implemented, the developers must establish the \emph{cost of change} and \emph{chance of failure}-metric for each feature.

Based on the \emph{chance of failure} and \emph{cost of change} metrics, the features are ordered with the aim to reduce the impact of a design failure.
Even though the case study only applied the feature selection twice, it proved itself useful by selecting the end-effector feature first.
By prioritizing the end-effector, its failure had only a minor impact on the design.

During each iteration of the development cycle, the selected feature is implemented according to the variable-detail approach.
However, the ability to assess the influence of the variable-detail approach is limited by the absence of tooling for model organization and testing.
Without tooling that is compatible with version control, it is difficult to switch between model versions, undo design changes, or run automated testing.
Furthermore, as the development did not distinct between design and model, the models used often contained more detail than strictly necessary.
Both these limitations resulted in models that would surpass the minimal required level of detail; therefore, it is not possible to assess whether the minimum required level of detail can be established with passing all the tests.
Nevertheless, the variable-detail approach introduces a step wise addition of detail that enforces a structured method similar to the development cycle.

It is unfortunate that the development cycle did not include a structured method to define the features nor their order of implementation.
The performance of the variable-detail approach is currently hindered by the absence of tooling.
Consequently, this limits the accuracy of the assessment on the actual influence of the \ac{ridm}.
Notwithstanding these limitations, the results of the case study suggest that the structured approach of \ac{ridm} reduces the impact of design failures and reduces the development cost for \ac{cps} design.

\emph{Describe which adaptations are required for both the \ac{ridm} and the design method to establish a competent design process for \ac{cps}.}\newline
The \ac{ridm} required adaptations before it could be evaluated.
The adaptations made in this thesis showed variable degrees of success during the case study.
To create a competent design process, some adaptations must be improved and some new ones must be added.

The produced result by the development cycle depends strongly on the provided features to implement.
To ensure a consistent result for the design of \ac{cps}, \ac{ridm} must incorporate a design process to define these features.
Moreover, the features must not only describe functionality, but each feature must also include a physical component and a requirement.
These three elements together make it that features can be implemented and tested individually.

The design process must describe a complete method from problem description till the set of features.
In this design process, the solution to the problem is established in the form of the functionality of the system.
The design process determines what components perform that functionality, and puts requirements on the components and the functionality.
All design decisions made during this process shape the final product.
Therefore, the design process to determine the features is urgently important in the ability of \ac{ridm} to successfully develop \ac{cps} from scratch.

The variable-detail approach requires adaptations to fully utilize the advantages that the short cycle and testing provide.
Therefore, the models must be separated from the design.
This requires a centralized design including a database for all design parameters.
Models are no longer required to represent the complete design, allowing for more specific models.
Moreover, the models can conform to the general model properties.
Because the models are more specific, more of them are required to cover all aspects of the design.

To manage the increased number of models a form of version control is needed.
The version control makes it possible to organize, and if necessary to combine and integrate, different models.
Furthermore, it makes it possible to revert design changes and switch to different model versions.

The final adaptation is the ability for automated testing.
Automated testing provides a major advantage on top of the previous adaptations.
As the models inherit their parameters directly from the centralized design database, every design change propagates to all models.
With automated testing, all model are simulated after a design change. 
This highlights any unwanted behavior caused by the design change.
As the models are made more specific, a failed simulation of a model automatically pin points the area where the problem occurs.

Implementing and evaluating the adaptations as described above are required to determine if these adaptations are sufficient.
The next section describes recommendations that must be considered before implementing these adaptations.

\section{Recommendations}
Before any of the adaptations are applied to the \ac{ridm}, further research on the exact format of these adaptations is recommended.
The recommended steps taken in further research are:

\begin{itemize}
    \item \emph{Make the application area and purpose of the \ac{ridm} specific}: To design a good design method, the design process must start with a clear problem description.
        Currently, the \ac{ridm} does provide a design method, but does not state clear requirements such as:
        \begin{itemize}
            \item \emph{Type of \ac{cps}}: mainly hardware, software or control?
            \item \emph{Design focus}: improve reliability, real-time guarantee, reduce complexity, shorten development-time?
            \item \emph{Internal or external use}: is the client directly involved?
            \item \emph{Development team}: number of developers from what background?
        \end{itemize}
        These requirements improve the focus of further research.
        This focus could also help to attract and involve other organizations.
        Above all, it prevents the \ac{ridm} of becoming a "master of none".
    \item \emph{Explore existing design projects that share the application area and purpose}:
        To avoid inventing the wheel or or provide a solution none wants, it is recommended to explore existing design project. 
        Involve projects from companies and universities, successful and unsuccessful. 
        Evaluate all the projects with at least the following questions:
        \begin{itemize}
            \item What type of design paradigm or model is being applied?
            \item Where is the complexity in the project and how is it dealt with?
            \item How are the metrics of \emph{cost of change} and \emph{chance of failure} defined?
            \item How are the design and models connected?
            \item Which design tools are used by the design team? Why are they used?
            \item Are there common design problems between the different projects?
            \item How is the client involved in the development process?
            \item What considerations are made to chose between modelling or hardware prototyping?
        \end{itemize}

    \item \emph{Hypothesize the improvements provided by \ac{ridm} for existing design projects, and vice versa}: 
        Based on the evaluation of the design projects:
        \begin{itemize}
            \item How could the \ac{ridm} improve those existing design projects? 
            \item What lessons can be drawn from the existing design projects?
        \end{itemize}
\end{itemize}

Depending on what the results and conclusions of the recommended research topics are, a strategy has to be created to further develop the \ac{ridm}.
Currently, there are two likely scenarios:
\begin{itemize}
    \item Make the \ac{ridm} part of an existing design model, such that the advantages are integrated with existing design models.
    \item Develop the \ac{ridm} into a complete design method, such that it can be used for the development of the complete product life-cycle.
\end{itemize}
Independent of what strategy is chosen, it is recommended to:
\begin{itemize}
    \item Implement the adaptations as described in this thesis.
    \item Perform the adaptations and improvements of the \ac{ridm} with a multi-disciplinary design team.
    \item Evaluate the \ac{ridm} with projects that are within the application area of the \ac{ridm}.
\end{itemize}

The recommendations result in a more focussed development of the \ac{ridm}.
But these recommendations are only the top of the iceberg of what is required to develop \ac{ridm} as a complete design method for \ac{cps}
Expected is that a full development of the \ac{ridm} takes multiple years and many developers and researchers to complete.

The \ac{ridm} does bring some techniques that show potential.
These techniques could improve existing design methods.
Based on this thesis, the following research topics are recommended:

\begin{itemize}
    \item \emph{A technique or protocol for to organize the parameters of a design, such that the parameters can be used in modelling}: Can the current modelling tools be adapted to read parameters from a database and can design tools be adapted to write parameters to a database?
    \item \emph{Tooling for modelling software, to allow for unit testing}: Software development applies unit testing, where behavior of each function is tested separately.
        In modelling this would allow every sub-model to be tested separately.
\end{itemize}