fixup! Process feedback design method evaluation

před 4 roky · 221263a7e4
--- a/content/reflection.tex
+++ b/content/reflection.tex
@@ -1,6 +1,12 @@
 %&tex
 \chapter{Design Method Evaluation}
    \label{chap:reflection}
    This chapter evaluates the design method as described in \autoref{chap:analysis}.
    The first section is about the system complexity of \ac{cps}.
    The second section evaluates the elements of a feature.
    The third section discusses the difference between model and design.
    The preparation phase and the \ac{ridm} are discussed in the last two sections.


    \section{System Complexity}
        \autoref{sec:time_investment} explains the time resources required for the development of the software in the system.
@@ -10,7 +16,7 @@
        
        Although the focus was on complex hardware solution, this solution was only possible with the use of software.
        The interaction between the \ac{scara} and \ac{cdc} is only possible with software that can switch states.
        Furthermore, the path planning used to write characters on the board is completely software dependent as well.
        Furthermore, the path planning that writes characters on the board is completely dependent on software as well.
        \textcite{sheard_718_1998} discusses that pure-hardware solution are relatively simple in their problem space perspective.
        However, the hardware solution is often complex in the solution space perspective 
        And indeed, during the initial design in the case study, the choice was made for the most complex hardware solution.
@@ -19,58 +25,44 @@
        Another point on system complexity is prototyping.
        Because hardware tends to be relatively simple, building a hardware prototype such as the \ac{scara} is cheap and quick.
        An initial hardware prototype is easily constructed with \ac{ots} readily available.
        Because the hardware transfers power the interfacing between components is straight forward.
        Because the hardware transfers power the interfacing between components is trivial.
        For example, linear actuation can be achieved with a rack and pinion construction, linear motor, gear and chain link, or a connecting rod.
        This might not be part of the final product, but it is useful to investigate the feasibility of the project.

        Furthermore, the changes are also easily made to hardware.
        It is possible to weld or glue new parts on or remove them with the angle grinder.
        It is possible to weld or glue on new parts or remove them with the angle grinder.
        Adding components to software is tedious and can lead to unwanted behavior.
        However, this is difficult to test because the software is more complex.
        Moreover, unwanted behavior of the hardware is discoverable, and when it breaks it is often destructive.
        The software can run for multiple days before crashing, as a result of integer, stack or buffer overflows for example.

        As long as the development is still in progress, one hardware system is more malleable than the software in terms of resources.
        When the production of a product starts, changing multiple hardware systems becomes economically unviable.
        A design method for \ac{cps} must acknowledge that software has a high \emph{cost of change} and has also a high \emph{chance of failure}.
        As long as the development is still in progress, one hardware prototype is more malleable than the software in terms of resources.
        However, when the designed system is put into production, changing multiple hardware systems becomes economically unviable.
        A design method for \ac{cps} must acknowledge that the inherent complexity of software comes with a high \emph{cost of change} and a high \emph{chance of failure}.
        Additionally, the design method must use the hardware prototype low \emph{cost of change} to its advantage.


    \section{Elements of a Feature}
    % Om de ontwerpmethode toe te passen voor de ontwikkeling van een nieuw systeem is de definitie van een feature uitgebreid.
    % De methode kreeg meer structuur door een hierarchie aan te brengen.
    % Door specifiek features te onderscheiden als functie of component, werd het mogelijk om hardware toe te voegen.
    % De huidige feature definitie bestaat uit een hierarchische structuur die meerdere lagen aan functionaliteit beschrijft.
    % De onderste laag beschrijft een verdeling in hardware.
    In the design plan as described in \autoref{chap:analysis}, more structure was added to feature definition. 
    The design plan as described in \autoref{chap:analysis} improves the feature definition by adding more structure.
    The goal of this extra structure was to make a design from scratch possible.
    For features a distinction was made between functional features and component features.
    A distinction was made between functional features and component features.
    The functional features are obtained by splitting the functionality of the system, which are then organized in a hierarchical tree.
    The hardware, which provides a platform for the functionality, is split into component features.
    These component features form the bottom layer of the hierarchical tree.

    % Ondanks de aanpassingen bied de huidige methode te weinig structuur om een goede feature definition op te zetten.
    % De evaluatie van de feature definiton stipt aan dat de huidige methode geen ruimte bied voor het structureren van componenten.
    % Voor componenten is op dit moment maar een niveau en kan dus geen sub-componenten weergeven.
    % Een ander punt is dat het opdelen van de requirements over features is ondergesneeuwd.
    % Doordat features nu werden opgesplits in verschillende niveaus aan functies of componenten werd het opsplitsen van requirements een te grote opgave.
    Still, the current approach not provide sufficient structure to define the features of the system effectively.
    The evaluation of the feature definion (\autoref{sec:case_featuredefinition_evaluation}) points out that it does not provide any structure for compoments.
    Still, the current approach does not provide sufficient structure to define the features of the system effectively.
    The evaluation of the feature definition (\autoref{sec:case_featuredefinition_evaluation}) points out that it does not provide any structure for components.
    It is currently not possible to define sub-components for components.
    Furthermore, making connections between a task or mission and a (sub-)component would make the hierarchical structure unclear.

    Another point is that the current approach creates a set of requirements and a set of features.
    The original plan was to distribute the requirements allong the features.
    The original plan was to distribute the requirements along the features.
    However, this was more complex than expected and ended up in the background.

    % Kordon legt uit hoe JPL gebruikt maakt van systems engineering process that is known as functional decomposition.
    % Dit process beschrijft een methode om het systeem gestructureerd te splitsen.
    % Dit resulteerd in drie gescheiden hierarchise structuren voor de functies, fysieke componenten, en systeemeisen.
    % De relaties tussen functies, en componenten en eisen worden weergegeven doormiddel van verbindingen tussen de structuren.
    A more suitable approach for the definition of features is a \ac{se} process that is known as functional decomposition.
    \textcite{kordon_model-based_2007} describes this process as a method for structured decomposition of the functionality of a system.
    Instead of one hierachical structure that contains functions and components, the process results in three separate hierarchical structures.
    Each of these structures describe the elements and sub-elements for functions, physical components and system requirement separately.
    Instead of one hierarchical structure that contains functions and components, the process results in three separate hierarchical structures.
    Each of these structures describe the elements and sub-elements for functionality, physical components and system requirements separately.
    Between the elements in these structures are connections created that describe the relationships.
    These relationships describe the link between functions, components and the rationale for requirements.

@@ -89,7 +81,8 @@
    The current design plan as described in this thesis, considers the feature as a component or a function.
    As explained in the previous section, the hardware gets its function from the software.
    Implementing an individual function or component does not deliver a testable feature.
    By grouping the elements that are connected via the relationships, where both are a result of the functional decomposition process, form a feature (\autoref{fig:functional_relation}).
    
    With this new approach, a feature can be formed by grouping the elements that are connected via the defined relationships (\autoref{fig:functional_relation}).
    This feature describes a function that is performed by a component.
    Furthermore, the requirement specifies both the function and component, and the requirement defines the test of the feature.

@@ -97,15 +90,16 @@
    % Dit voorkomt dat eerst alle requirements vastgelegd worden om later nog eens alle functionaliteit op te splitsen.
    % In de case study werd tijdens het opsplitsen van de functionaliteit duidelijk dat er requirements gemist waren.
    % En later ook nog dat bij het splitsen van de functionaliteit geen order of operation was gespecificeerd.
    In contrary with the design plan in this thesis, the \ac{se} process decomposes the functionality of the system in multiple iterations.
    This is a significant improvement over the current approach where all requirements are determined before any feature is defined.
    Contrary to the design plan in this thesis, the \ac{se} process decomposes the functionality of the system over multiple iterations.
    This is a significant improvement compared to the current approach, in which all requirements were determined before any features was defined.
    The feature definition during the case study, showed that specific requirements were overlooked.
    Later, while defining the test protocol, it became clear that there was not order of operation was specified.
    Later, while defining the test protocol, it became clear that no order of operation was specified.

    % Functional decomposition of een ander gelijkwaardig SE process verbeterd niet alleen de feature definition, maar de preparation phase in het geheel.
    % Het beschrijft een beproefd process dat van een problem description een degelijke featureset kan opzetten.
    Functional decomposition, or a similar \ac{se} process, would not only improve the feature definition step, but the preparation phase as a whole.
    Future implementations of the \ac{ridm} must consider such a process, as it provides a method that creates a set of features from the problem description.
    Future implementations of the \ac{ridm} must consider such a process, as it provides a structured method to develop a solution for a problem.
    Whereby the solution is split in to a elaborate set of features.


 %        Over the course of this study, the definition of a feature evolved into requirements, components and functions.
@@ -142,7 +136,7 @@
        The \ac{ridm} as well as the design method in this study do not make an explicit distinction between the model and the design.
        This implicitly resulted in a model that represents the complete design.
        Over the course of the development the complexity of the design increased, resulting in more complex modelling as well.
        The model used in the case study was first implemented as a kinematics model, and as the design became more complex it was represented in 2D and 3D physics, and a CAD drawing.
        The model used in the case study was first implemented as a kinematics model, and as the design became more complex it was represented with 2D and 3D physics, and a CAD drawing.

        There are two issues with this approach:
        first, that the approach does not comply with the general model properties;
@@ -164,14 +158,14 @@

        \subsection{Design Parameters}
        The design of the \ac{scara} is currently represented by two types of models: a dynamics model and a CAD drawing.
        Both these modelling types have a different purpose and represent different aspects and parameters of the design.
        Both these modelling types have different purposes and represent different aspects and parameters of the design.
        However, both models share parameters of the design as well.

        For the \ac{scara} design, the dynamics represent mostly the motor and controller behavoir.
        The CAD drawing represents the shape of the components.
        But the kinematics play an important role for both models.
        A direct result from this is that it increases the \emph{cost of change}.
        When the design changes, these changes must be applied for both models, increasing the amount of work.
        When the design changes, the changes must be applied for both models, increasing the amount of work.

        This distribution of design parameters has more disadvantages, as copying the parameters between different models is error-prone and labor-intensive.
        The case study in this thesis is small, but did already involve 8 different models spread over 4 different modelling approaches.
@@ -186,29 +180,20 @@
            The three general properties must apply to every model made.
            Instead of creating a model of the complete design, only small parts of the design are modelled.
            
            Additionally, a method to organize all design parameters is needed reduce the \emph{cost of change}.
            Additionally, a method to organize all design parameters reduces the \emph{cost of change}.
            The goal is that all the models automatically use the design parameters from a centralized location.
            Any changes to the design are made at that centralized location, each model can than be tested automatically with the updated parameters.
            This eliminates copying of parameters and allows for automated testing.
            It removes the human factor and produces direct feedback about the design change.

    \section{Preparation Phase}
        % Doordat de focus op de RIDM lag heb ik de prep fase onderschat.
        % De prep fase viel initieel niet in de scope van de thesis.
        % Achteraf is duidelijk dat een prep fase cruciaal is voor het design en dus ook voor het evaluaren van RIDM.
        % De lineaire set aan stappen die is gekozen is triviaal toe te passen, maar niet geschikt voor een complex ontwerp.
        Initially adding a preparation phase to the \ac{ridm} was not within the scope of the thesis.
        Causing me to underestimate the role that the preparation phase had for the \ac{ridm}
        Causing me to underestimate the role that the preparation phase had for the \ac{ridm}.
        In hindsight it is clear that the preparation phase is crucial for the design process, and thus also for the evaluation of \ac{ridm}.
        The linear set of steps were chosen as it was trivial to put those in front of the \ac{ridm}.
        However, the linear set of steps proved to be inapt for the development of a complex \ac{cps}.

        % Ik verwacht dat het resultaat van het RIDM sterk afhangt van de tot stand gekomen initiele ontwerp en bijbehordende features.
        % Om een consistent resultaat te krijgen moet er dus een concrete methode worden aangeleverd om tot dat ontwerp en features te komen.
        % Zonder duidelijke methode om van een 'need' tot een functional design met features te komen is de RIDM niet toe te passen als design methode voor CPS.
        % Hierbij kan een process als de functional decomposition uitkomst bieden, maar ook state analysis \autocite{ingham_engineering_2005} of spiral model \autocite{boehm_spiral_1988}.
        % Moderne rapid prototyping technieken maken het mogelijk om in korte tijd een prototype te maken.
        Without a concrete approach\footnote{Here, I specifically use the term approach because preparation phase implies that it must be a phase prior to the \ac{ridm}.} to get from a \emph{problem} to a functional design with features, the \ac{ridm} is unsuitable for the development of a \ac{cps}.
        However, the linear set of steps proved to be inapt for the development of complex \ac{cps}.

        Without a concrete approach\footnote{Here, I specifically use the term approach because preparation phase implies that it must be a phase prior to the \ac{ridm}.} to get from a \emph{problem} to a functional design with features, the \ac{ridm} is unsuitable for the development of \ac{cps}.
        Describing such an approach is far outside of the scope of this thesis.
        Nonetheless, several \ac{se} processes offer a possible solution, such as functional decomposition, state analysis \autocite{ingham_engineering_2005} or spiral model \autocite{boehm_spiral_1988}.
        Furthermore, the advantages of modern techniques of rapid prototyping should also be considered to aid the design process.
@@ -219,39 +204,6 @@
        This research should use data and experience from existing design projects.
        Above all, the design of such an approach needs direct involvement of experienced systems engineers.
        




        


        % 
        % Hoe deze methoden toegepast kunnen worden in combinatie RIDM ligt buiten de scope van deze thesis.
        % Een spiral model als basis met daarin de technieken van RIDM, rapid prototyping en functional decomposition is goed mogelijk.
        % Een andere optie is om de development cycle van het RIDM uit te breiden met functional decomposition en rapid prototyping.
        % Verder onderzoek is nodig om hier een goede optie in te vinden.
        % Daarbij moet er naar bestaande ontwerp projecten en methoden gekeken worden.
        % Maar vooral moet iemand die zelf meerdere jaren ervaring heeft in werken met design methoden hier direct bij betrokken zijn.

 %        The start of this chapter explains the reason to prepend the preparation phase to the \ac{ridm}.
 %        Where the preparation phase aims to produce the requirements and features, based on the waterfall method.
 %        However, during the case study, the waterfall method proved to be problematic.
 %        Especially during the first steps, the amount of information was scarce, which made it tempting to work ahead.
 %        For example, a simple proof of concept during the requirement step would have resulted in valuable information.
 %        This was however, not possible as the goal was to follow the specified design method as close as possible.
 %
 %        Looking at the current case study where the system under design is relatively simple, more design experience is sufficient to overcome the information shortage.
 %        Unfortunately, it requires experienced developers, which are scarce by themself.
 %        As was pointed out in \autoref{sec:evaluation_reflection_protoype}, perceiving the current design as a prototype would also improve the information situation.
 %        Similarly, \textcite{royce_managing_1970} proposed to use a prototype in order to reduce the reliance on human judgment.
 %        A common denominator of these proposals is that they all deal with the dependency on human judgement, either by improving or reducing this judgement.
 %        Nonetheless, these proposal seem like a suppression of symptoms, instead of an actual improvement of the design method.
 %
 %        Interestingly, when the current design is regarded as prototype and the design method is repeated, the approach is comparable with the first cycle of the spiral model \autocite{boehm_spiral_1988}.
 %        \textcite{broenink_rapid_2019} state about the \ac{ridm} that the development cycle is based, among other methods, on the spiral model.
 %        It may be the case therefore that prepending the waterfall model was an attempt to reinvent the wheel.

    \section{Rapid Iterative Design Method}
        This chapter began by a breakdown of the elements of a feature, argued the importance of distinction between design and model, and explained the need for an integrated preparation phase.
        The commonality between these three issues is that they all stem from the rapid development cycle, which was introduced in \autoref{sec:background_rdc} as part of the \ac{ridm}.
@@ -282,9 +234,9 @@

            This does not alter the fact that to complete the design all tests have to pass.
            That all test have to pass is also the reason for this criterium in the first place: give priority to the feature that passes the most tests on completion.
            Even though it is difficult to draw concrete conclusions about the feature selection, a recommendation is to use the number and \ac{cof} of tests as a metric.
            Even though it is difficult to draw concrete conclusions about the feature selection, a recommendation is to use the number of tests and the change of failure for each test as a metric to calculate the \ac{cof}-time value.
            In addition, other metrics and approaches that can improve the \ac{cof}-time calculation are: number of dependees, the number of tests of those dependees, and planning poker.
            Further work is required to establish which metrics are suitable to improve the \ac{cof} calculation.
            Further work is required to establish which metrics are most suitable to calculate the \ac{cof} values.

        \subsection{Variable-Detail Approach}
            The variable-detail approach is a very practical development tool.
@@ -294,11 +246,11 @@
            Based on the test, the development continues or the model is rolled back to an earlier version.
            In addition, the models, independent of the level of detail, can be reused in other models.
            
            However, multiple difficulties were encountered during the case study, which hindered the variable-detail approach.
            However, multiple difficulties were encountered during the case study that hindered the variable-detail approach.
            As was mentioned in \autoref{sec:evaluation_reflection_development}, the lack of good version control made it difficult to work with multiple versions of a model.
            This made it difficult to switch or revert to other levels of detail.
            However, the greatest difficulty is due to the model representing the design, as discussed in \autoref{sec:evaluation_model_and_design}.
            Because the design contains a certain level of detail and the model is a full representation of the design, it is difficult to make a simple implementation or to switch back.
            Because the design contains a high level of detail and the model is a full representation of the design, it is difficult to make a simple implementation or to switch back.
            This strong relation between the model and the design, also caused the complete model to be switched to a different representation.
            
            Even though the variable-detail approach did not perform as planned, I expect this approach to be a very strong part of the design method, given that a solution is found to the problems described above.