mar/content/analysis.tex

%&tex
\chapter{Analysis}
\label{chap:analysis}
The previous chapter introduced how two design methods are combined to form the bases for one complete design method.
In this chapter, a design plan is created from this combined design method.
The goal is to have a concrete design plan that is used in the case study.
All of the steps in the design plan must be specific such that each of these steps can be evaluated after the case study is finished.
The first three steps of the design phase are based on the \ac{se} approach and are already described with great extend by \textcite{blanchard_systems_2014}.
As the evaluation of \ac{se} is not in the scope of this thesis, this chapter only covers the minimal description of the design steps in \ac{se}.
The steps that are introduced by \ac{ridm} are covered in more detail.

\section{Systems Engineering}
    The goal of the preliminary design is to setup system requirements and an initial design according to the problem definition.
    Although these design steps in \ac{se} play a crucial roll in the success of the development, they are, however, very exhaustive.
    A major part of this complete design process is the required documentation to ensure agreement about the design between the different stakeholders.
    Resulting in a process that can take months or even years, which is not feasible for this thesis.
    In this thesis, this design plan is only used for evaluation and has only one stakeholder, the author.
    This allows for a simple implementation of the \ac{se} approach, as it not possible to create a false start due to misunderstanding, saving valuable time.
    For each of these \ac{se} steps is explained what is involved with a full implementation, and what part of the step is used in the design plan.

    \subsection{Problem Definition}
        Before any design process can start, the "problem" has to be defined.
        In other words, why is the function of the system needed?
        This is described in a \emph{statement of the problem}.
        In this statement of the problem it is important to describe "what" has to be solved, not directly "how".
        \textcite{blanchard_systems_2014} also note that "defining the problem is often the most difficult part of the process".
        It is important to ensure good communication and understanding between the different stakeholders.
        Otherwise, it is possible that the designed product is not up to the customers expectations.
        It furthermore involves defining the subjects like what are the primary and secondary functions? When must this be accomplished? What is not a function?
        For this thesis, however, the problem definition is limited to a short statement of the problem, covering some required functions with corresponding requirements.

    \subsection{System Requirements}
        The system requirements are derived from the problem definition, and describe the characteristics of the system.
        As these characteristics form the foundation of the system, the requirements must be defined without any ambiguity, vagueness or complexity.
        The requirements are written according to the \ac{ears} \autocite{mavin_easy_2009}.
        \ac{ears} was chosen for this design method due to its simplicity, which fits the scope of this thesis.
        Later in the design, these requirements are divided over the subsystems.
        Any issues, like ambiguity, in the requirements, propagate through these subsystems.
        This might lead to a redesign of multiple sub-systems when these requirements have to be updated.

    \subsection{Initial Design}
        \label{sec:se_initial_design}
        In the initial design step, the "what has to be solved", is expanded with a solution on "how it is solved".
        To find the best solution it is important to explore the different solutions and design space.
        Often, there are many possible alternatives but they must be narrowed down to the solutions that fit within the schedule and available resources.
        The best alternative is materialized in a design document together with the system requirements.
        This design document is used in the next phase of the design.

\section{Rapid Iterative Design Method}
    From this point, the design plan is based on the \ac{ridm} and not anymore on the waterfall model.
    The first step is the feature definition, which prepares the required features based on the initial design.
    The features are defined by splitting the system in such a way that the results of each implemented feature are testable.
    The definition of the feature contains a description and a set of sub-requirements which is used to implement and test the feature.
    During the feature definition, the dependencies, risks and time resources are determined as well, this establishes the order of implementation in the feature selection step.

    Based on the requirements of \ac{ridm} as explained in \autoref{chap:background}, the next step is the feature selection.
    However, it became apparent that the number of tests related to a specific feature is a good metric for the selection step.
    Because, at the point that a feature is implemented, the tests are completed as well, and when the tests of the complete system pass, the system meets the specifications.
    Following the \ac{ridm}, these tests are specified at the start of the rapid development cycle.
    This makes it impossible to use the tests during the feature selections.
    Therefore, a test protocol step is added after the feature definition and before the feature selection step.

    The third step is the feature selection, where one of the features is selected. 
    This selection is based on the dependencies, tests, risk, and time requirements in the feature definitions.
    The fourth step is the rapid development cycle, which uses the sub-requirements and description of the selected feature to create an initial design and a minimal implementation.
    In the last step, the variable detail approach is used to add detail to the minimal implementation over the course of multiple iterations.
    The tests are used to determine if the added detail does not introduce any unexpected behavior.
    This cycle of adding detail and testing is repeated till the feature is fully implemented.
    From this point, the \ac{ridm} is repeated from the third step until all features are implemented.

\subsection{Define features of the design}
    \label{sec:featuredefinition}
    During the feature definition step, the system is split into features as preparation for the rapid development cycle and the variable-detail approach.
%
%    \subsubsection{Functions and Components}
%        \textcite{broenink_rapid_2019} define the features of their system by dividing the functionality of the system.
%        However, the design method in this thesis also includes the development of the hardware.
%        This creates an extra dimension to the features in the design.
%        In addition of splitting the functionality, the hardware has to be split as well.
%        Therefore, the design is described with two types of features.
%        The functionality is split into functions and the hardware is split in to components.
%
%    \subsubsection{Making the Cut}
        The \ac{ridm} does not provide a particular approach to define the features of the design.
        But, the goal is to have features that can be implemented and tested individually.
        Resulting in short development cycles and earlier feedback about the design.
        The approach in this design plan aims to provide more guided and structured way to split the features.
        \begin{marginfigure}
            \centering
            \includegraphics[width=21mm]{graphics/robmosys_levels.pdf}
            \caption{}
            \label{fig:robmosys_levels}
        \end{marginfigure}
        
        The approach to define features in this design plan is based on the separation of levels principle \autocite{noauthor_robmosys_2017}.
        This principle defines different levels of abstraction.
        This starts from the top with the \emph{mission}, serving coffee for example.
        Followed by less abstract levels such as: a \emph{task} to fill the coffee mug; a \emph{skill} to hold the coffee mug; and a \emph{service} that opens or closes the hand.

        The different levels allow the features to be split multiple times in a structured way.
        Take the coffee serve example, to fill the coffee mug, it is not sufficient to only hold the mug.
        The system also has to pour coffee into the mug, and maybe add sugar or milk.
        This results in a hierarchical tree of functions as shown in \autoref{fig:robmosys_levels}.

        With this approach, features are defined top-down and are implemented bottom-up.
        Thus a \emph{skill} is defined as one or more \emph{services}.
        When all the \emph{services} are implemented, they are combined into a \emph{skill}.
        The advantage of this is that the \emph{skill} defines a kind of milestone, at the moment that relevant \emph{services} are combined.
        Or looking at the example: the system must at least be able to grab, stir, and pour before it can fill a mug with coffee, milk and sugar.
        Another advantages is that two or more \emph{skills} can have a \emph{service} in common.
        This would be the case if our system also needs to serve tea. The system can already hold a mug, but still needs to add a teabag.
        Even though there is no exact level of abstraction required for each of the features, it does create a structure for the developer.
        In the end, the developer must rely on its engineering judgement to chose the optimal division between features.

        The bottom level of the hierarchy is a special case as it describes the hardware instead of functionality.
        The components are used to perform the functionality of the system with.
        For example, having a mobile robot arm near a coffee machine does meet the hardware requirements, it does not have any functionality if that is not yet implemented.
        This also creates a clear division for the developer as the functions cannot be mixed with the hardware.

%        
%
%
%        
%
%        
%        
%
%        
%    
%
%        As explained in the previous chapter, the goal of the \ac{ridm} is to get feedback on the design as early as possible. 
%        To achieve this, the design is split into features, and each feature is implemented and tested sequentially.
%        Resulting in smaller development cycles that are tested individually.
%        If a feature fails its test it occurs directly after its implementation, instead of when the full system is implemented.
%        The goal of this step is to apportion the system into features.
%        Each feature must be small but independent, meaning that the feature can still be individually implemented and tested.
%
%        In some cases it is not possible to define a feature that can be implemented and tested independently.
%        This occurs when the feature is dependent on the implementation of other features.
%        This dependency can occur in requirements where, for example, strength of one feature limits the mass of another feature.
%        Such a dependency can work both ways and can be resolved by strengthening one feature, or reduce the mass of the other feature.
%        Another type of dependency is when the implementation influences other features.
%        In this case, if the implementation of one feature changes, it requires a change in the other features.
%        An example of this is a robot arm, where the type of actuation strongly influences the end-effector.
%        When the robot arm approaches an item horizontally, it requires a different end-effector than approaching the item vertically.
%
%    \subsubsection{Feature Hierarchy}
%        
%
%        
%
%
%        %%%%%%%% ->
%        There are two important responsibilities for the developer when the design encounters feature dependency.
%        The first one is that the developer must determine where to split the system.
%
%        In case of a dependency, the developer must evaluate the optimal order of implementation.
%        The developer must arrange the dependency of the features such that the influence on the dependent feature is as small as possible.
%        In other words, if feature A can be easily adapted to the implementation of feature B, but not the other way around, the developer must go for A dependent on B.
%        The second responsibility is organizing the feature requirements.
%        Due to these dependencies it is possible that the division of requirements changes, because the result of the implemented feature was not as expected.
%        This is not directly a problem, but a good administration of the requirements makes an update of these requirements easier. 
%
%        
%        
%
%
%
%
%    To achieve these short cycles, the features that are implemented in these cycles, are as small as possible. 
%    However, the features must still be implemented and tested individually during the implementation and can thus not be split indefinitely.
%    Together with the definition of the features, the requirements are divided along the features as well.
%    The optimal strategy on splitting features and requirements is strongly dependent on the type of system.
%    Therefore, the best engineering judgement of the developer the best tool available.
%
%    In some cases it is not possible to define a feature that can be implemented and tested independently.
%    This occurs when the feature is dependent on the implementation of other features.
%    This dependency can occur in requirements, where strength of one feature dictates the maximum mass of another feature.
%    Such a dependency can work both ways and can be resolved by strengthening the one feature, or reduce the weight of the other feature.
%    Another type of dependency is when the implementation influences other features.
%    In this case, if the implementation of one feature changes, it requires a change in the other features.
%    An example of this is a robot arm, where the type of actuation strongly influences the end-effector.
%    When the robot arm approaches an item horizontally, it requires a different end-effector than approaching the item vertically.
%
%    There are two important responsibilities for the developer when the design encounters feature dependency.
%    The first one is during the definition, where the developer has to decide on how to split the system and how the dependency is stacked.
%    For the requirement and the implementation dependency the developer must evaluate the optimal order of dependency.
%    The developer must arrange the dependency of the features such that the influence on the dependent feature is as small as possible.
%    In other words, if feature A can be easily adapted to the implementation of feature B, but not the other way around, the developer must go for A dependent on B.
%    The second responsibility is organizing the feature requirements.
%    Due to these dependencies it is possible that the division of requirements changes, because the result of the implemented feature was not as expected.
%    This is not directly a problem, but a good administration of the requirements makes an update of these requirements easier. 

\subsection{Test protocol}
    \label{sec:systemtesting}
    During the rapid development cycle and the variable detail approach, the system is tested constantly.
    This is to make sure that the design still performs as expected.
    The tests are based on the specifications.
    Each specification must be covered with at least one test.
    The tests consist of a description which specifies how to perform the test and what the result of the test must of must not be.
    Together with the description, there is a list of required features to perform the test and a list of specifications that are met if the test passes.

\subsection{Feature Selection}
    \label{sec:feature_selection}
    The rapid development cycle does not specify which feature is implemented first, even though the order of implementation does change the feasibility of the complete development.
    An example that shows the importance of the order of features is the development of a car.
    To have a critical damped suspension in a car, the weight distribution of the car must be known.
    If the suspension of the car is designed before all the features that determine the weight distribution, it is likely that the suspension design is not up to specifications.
    Resulting in a redesign of the suspension feature and thus increasing the overall development cost.
    This example is caused by the dependency between different features.
    \begin{marginfigure}
        \centering
        \includegraphics[width=2.9cm]{graphics/feature_dependency.pdf}
        \caption{Dependency graph for features.}
        \label{fig:feature_dependency}
    \end{marginfigure}
    \begin{table}[]
    \caption{Comparison of features with their corresponding risk and time. 
        The last column is the risk value divided by the number of days.}
    \label{tab:feature_selection}
    \begin{tabular}{l|r|r|r|r|r|}
    \cline{2-6}
                                  & \multicolumn{1}{l|}{Dependees} & \multicolumn{1}{l|}{Tests} & \multicolumn{1}{l|}{Risk} & \multicolumn{1}{l|}{Time} & \multicolumn{1}{l|}{Risk per time} \\ \hline
    \multicolumn{1}{|l|}{Feat. A} & 2 (2, 3)                       & 2                          & 15 \%                     & 3  days                   & 5                                  \\ \hline
    \multicolumn{1}{|l|}{Feat. B} & 0                              & 3                          & 40 \%                     & 5  days                   & 8                                  \\ \hline
    \multicolumn{1}{|l|}{Feat. C} & 1 (5)                          & 5                          & 25 \%                     & 2  days                   & 12.5                               \\ \hline
    \multicolumn{1}{|l|}{Feat. D} & 0                              & 4                          & 15 \%                     & 1  day                    & 15                                 \\ \hline
    \multicolumn{1}{|l|}{Feat. E} & 0                              & 4                          & 45 \%                     & 6  days                   & 7.5                                \\ \hline
    \end{tabular}
    \end{table}

    To determine the order of implementation of features, a dependency graph and a comparison table is made.
    The dependency graph and the comparison table for a theoretic system is shown in \autoref{fig:feature_dependency} and \autoref{tab:feature_selection} respectively.
    The comparison table has dependees column, that describes the number of features that are depending on that specific feature, and are derived from the dependency graph.
    The tests column describes the number of tests that are covered by implementing this feature.
    These tests are defined during the initial design and the feature definition, the number represents the amount of tests that pass after implementation of the feature.
    The risk per time score for third rule is calculated by dividing the risk score with the time score.
    The goal of this score is to eliminate as much risk as possible in the least amount of time.
    It seems logic to always implement the highest risk feature first, but it is possible to finish multiple features with a lower risk in the same time period.
    This is visible in \autoref{tab:feature_selection}: In a time span of 6 days it is possible to implement feature E or features A, C, and D.
    The risk that is cleared by E is 45 \% which is significantly less than the combined 65 \% of A, C and D.
    Due to the limited scope of this thesis, it is not possible to give a good metric for determining risk and time.
    Nevertheless, it is strongly advised that the developer defines some metric that fits his project best.

    With a completed table, the order of implementation of features is determined by the following rules:
    \begin{enumerate}
        \item Features that are dependencies of others must be implemented first.
        \item Features that complete more system test than other features when implemented have priority.
        \item Features with the higher \emph{risk per time} score than other features have priority.
    \end{enumerate}
    The rules are applied in order, if one rule reduces the set to a single feature to implement the rest of the rules are skipped.
    The third rule is a sorting rule, and the feature that fits best is implemented.
    In case of a draw or in special cases the developer decides what feature to implement next.

    Looking at an example of 5 features:
    As seen in \autoref{fig:feature_dependency}, Features B and C are dependent on feature A.
    Feature D does not have any dependency connections, and feature E is dependent on C.
    Together with the information in \autoref{tab:feature_selection}, the order of implementation is:
    \begin{description}
        \item[Feature A:] has two features that are dependent on this feature, more than any other.
        \item[Feature C:] has one feature that is dependent on this feature, most dependencies after A is implemented.
        \item[Feature D:] has the same number of tests as E, but D has a significant higher risk per time score than E
        \item[Feature E:] has the most number of tests.
        \item[Feature B:] only one left to be implemented.
    \end{description}
    Note that this example assumes that nothing changes.
    In case of a feature not being feasible during the implementation, the design has to be reviewed.
    This also means that the dependency graph and comparison table change, resulting in a different order of implementation.

\subsection{Rapid Development Cycle}
Each iteration of this rapid development cycle implements one complete feature.
The feature that is implemented is selected in the prior feature selection step.
The goal of this step is to lay the foundation for the development of the feature.
This foundation consists of a basic model, a set of detail elements and a list of tests.
The set of detail elements is a collection of design aspects that are added to increase the detail during the next design step.
These detail elements can represent behavior, parasitic elements, or components.
How these detail elements are implemented and what the basic model consists of is based on the initial design of the selected feature.

The initial design of the feature is similar to the system wide approach in \autoref{sec:se_initial_design}.
It consists of a design space exploration, but with more detail, which is possible as the feature is significantly smaller than the complete system.
From the design space exploration, the developer selects the optimal design choice for the current feature.
For this design choice, a design document is made that illustrates the rough shape and dynamics of the implementation.

The basic model and the detail elements are based on an initial design of the feature.
The basic model consists of only the most basic elements of the design.
As the basic elements that make the basic model differ strongly per system, there is not a specific approach.
In general, the basic elements should only represent dominant and essential behavior of the system.
A good starting point for the dominant behavior is to identify the interesting energy states of the system.
The energy states of interest can include the energy states that are dominant, but also the states that are chosen by the developer.
These last states could represent the output states or status that have to be measured.
In the end, the developer decides if states are required and implement them in the basic model.
All the elements that are part of the initial design but are not part of the basic model are the detail elements.

Lets take a motorized double inverted pendulum for example, which consists of two arms with motorized joints.
Both pendulum arms are dominant energy states.
The electrical motors have also internal states, but store significantly less energy than the pendulum arms.
An basic model would in this case only consists of the arms, possibly even without any dynamic behavior.
The dynamic behavior, motor characteristics, resistance, or gravitational force are examples of detail elements that can be added to increase the detail.

\subsection{Variable Detail Approach}
With the variable detail approach the basic model is developed into a refined model of the feature.
This is done by implementing the detail elements over the course of multiple iterations.
To determine the order of implementation of these elements the approach for the order of features from \autoref{sec:feature_selection}.
Each iteration produces a new model with more detail than the previous.
The newly added detail is evaluated by performing the tests that were defined during the rapid development cycle.
\begin{figure}
    \centering
    \includegraphics[width=8.5cm]{graphics/test_flow_graph.pdf}
    \caption{Decision flowchart to follow for failed tests on each detail level.}
    \label{fig:test_flow_graph}
\end{figure}
Not all tests are expected to succeed from the start, as not all details are implemented.
For example, if the internal resistance of a electric motor is not yet implemented in the model, the motor can draw an unlimited current, and this would exceed the current draw specifications of the motor.
The decision flowchart in \autoref{fig:test_flow_graph} in determines whether the design must be reviewed or can continue on a failed test.
The decisions are made with the following questions:
\begin{itemize}
    \item Passed Before? The current test of the current design failed, but was there a previous detail level where it passed?
    \item Expected to fail? Does the test fail as a direct result from the added detail and was that intentional?
    \item Expected to pass? Should the added detail to the model result in a pass of the test?
    \item Will pass in future? Is there an element that will be implemented that results in a pass of the test?
\end{itemize}
In the case that the implementation of a detail element fails multiple times, the developer has to investigate if implementing the feature is still feasible.
This could result in a redesign of the feature or system.
When and how this decision has to be made differs per situation and is outside the scope of this thesis.
The developer must evaluate if there are feasible alternatives left for this element, feature or system, and apply these alternative if possible.

When all detail elements are implemented and the basic model has evolved into a refined model of the feature, the design cycle moves back to the feature selection.
In the case that this is the last feature to implement, this concludes the development.

\section{Summary of Design Plan}
\begin{marginfigure}
    \centering
    \includegraphics[width=6cm]{graphics/design_flow_analysis.pdf}
    \caption{Combined design plan, based on the \ac{ridm} and the Waterfall model.}
    \label{fig:design_plan_analysis}
\end{marginfigure}
The waterfall model from \ac{se} and the \ac{ridm} \autocite{broenink_rapid_2019} are combined to create the design plan as shown in \autoref{fig:design_plan_analysis}.
The first five steps of the design process form the preparation phase: problem description, specifications, initial design, feature definition, and test protocol.
The initial design step creates a holistic design based on the prior problem description and specifications step.
The last step of the preparation is the feature definition, where the initial design is split into different features.
The resulting initial design and its features together form the design proposal for the development steps.
The last step of the preparation phase is the test protocol step, where the tests are defined to monitor the design process and validate that the system meets the specifications.
The development cycle consists of the feature selection, rapid development, and variable detail steps.
These three steps are applied to each feature in the initial design individually.

With each iteration of the development cycle a new feature is added to the complete system.
All the tests of the individual features are performed in the complete system as well.
This ensures that the one feature does not break a another feature.
The design is finished when all the features are implemented, tested and combined.

In the optimal situation the preparation phase is only performed once at the start of the design, and the development cycle is performed for each feature.
However, if features prove to be infeasible, some steps have to be revised.