From 3d6c4d57f96f27be313ec5dc0aa04d91c39ed1cf Mon Sep 17 00:00:00 2001 From: Wouter Horlings Date: Wed, 9 Dec 2020 17:38:29 +0100 Subject: [PATCH] Rework Test protocol --- content/case_experiment_test_protocol.tex | 177 +++++++++++++--------- 1 file changed, 104 insertions(+), 73 deletions(-) diff --git a/content/case_experiment_test_protocol.tex b/content/case_experiment_test_protocol.tex index 6325d5e..79759e1 100644 --- a/content/case_experiment_test_protocol.tex +++ b/content/case_experiment_test_protocol.tex @@ -1,82 +1,113 @@ %&tex - \subsection{Test Protocol} - The last step of the preliminary design is to design tests. - The tests are designed around the specifications. - However, during the creation of the tests it was found that the list of specifications were lacking. - Furthermore, the previous steps did not provide any order of operation. - During this step the order of operation and additional specifications are determined. +\subsection{Test Protocol} + The last step of the preparation phase is to design tests. + The tests are designed to validate if the system meets the specifications. + While defining the tests, it became clear that part of the specifications was not explicitly defined. + As the specifications were made before the design decissisions in the initial design and the feature definition, the specifications did not take that information into account. + Updating the specifications after the design decissions was overlooked during the specification of the design plan. + To ensure the repeatibility of the tests, this step starts with specifying the order of operation and improved specifications. - There are two modes of operation: writing and wiping. - The writing operation can be split in the following steps: - \begin{enumerate} - \item Move cable driven carriage to position of characters. - \item Write three characters with the SCARA. - \item Repeat step 1 and 2 till the Tweet is on the board. - \item Move carriage away from the text on the board. - \end{enumerate} + \subsubsection{Defining the Order of Operation} + There are two modes of operation: writing and wiping. + Defining the order of operation also distributes the responsibility between the different components. + The writing operation can be split in the following steps: + \begin{order}{Writing} + \emph{Precondition:} Marker as end-effector. + \begin{enumerate} + \item Move cable driven carriage to position of characters. + \item Write three characters with the SCARA. + \item Repeat step 1 and 2 till the Tweet is on the board. + \item Move carriage away from the text on the board. + \end{enumerate} + \end{order} - The other operation is wiping. - This is similar to the operation of writing with the following steps: - \begin{enumerate} - \item Move cable driven carriage to position of characters. - \item Clear the area in reach of the SCARA. - \item Repeat step 1 and 2 till the Tweet is removed from on the board. - \end{enumerate} - Furthermore, switching between the states requires the tool to be switched. - As this is a difficult operation there are not yet requirements or order of operation specified. + The other operation is wiping. + This is similar to the operation of writing with the following steps: + \begin{order}{Wiping} + \emph{Precondition:} Wiper as end-effector. + \begin{enumerate} + \item Move cable driven carriage to position of characters. + \item Clear the area in reach of the SCARA. + \item Repeat step 1 and 2 till the Tweet is removed from on the board. + \end{enumerate} + \end{order} + Furthermore, switching between the states requires the tool to be switched. + However, at this point, it is not known how tools will be switched. + Therefore, the order of operation is determined during the development of the end-effector. + Additionally, the missing order of operation for the end-effector did not result in difficulty while defining the tests. - Based on the order of operation, the following specifications were added to the list in \autoref{sec:specifications}: - \begin{enumerate} - \setcounter{enumi}{11} - \item While writing, the SCARA shall have a writing speed of at least 1.5 characters per second. - \item When the Carriage/base of the SCARA is at a static position, the SCARA shall be able to write at least 3 characters at that position. - \item When the SCARA finished writing at their current position, the Carriage shall move the SCARA to it's next position where it can write the subsequent characters. - \item When the SCARA has to be moved to a new position, the Carriage shall perform this movement within 1 second. - \end{enumerate} - These additional specifications are also based on the combined system decission that was made in section \autoref{sec:initialdesign}. - These specifications distribute responsibility between sub-components. + \subsubsection{Improving Specifications} + Based on the order of operation, the following specifications were added to the list in \autoref{sec:specifications}: + \begin{enumerate} + \setcounter{enumi}{11} + \item While writing, the SCARA shall have a writing speed of at least 1.5 characters per second. + \item When the Carriage/base of the SCARA is at a static position, the SCARA shall be able to write at least 3 characters at that position. + \item When the SCARA finished writing at their current position, the Carriage shall move the SCARA to it's next position where it can write the subsequent characters. + \item When the SCARA has to be moved to a new position, the Carriage shall perform this movement within 1 second. + \end{enumerate} + These additional specifications are also based on the combined system decission that was made in section \autoref{sec:initialdesign}. + These specifications distribute responsibility between sub-components. - With the updated specifications it was possible to create a number of test cases. - In total there are five small test cases and four large test cases. - The small tests cover a sub-system and the large tests apply on the complete systems. - Each tests has a list of specifications that are covered with the test and for smaller tests the subsystem under test is also determined. - With a short description it is described how the test should be performed. + \subsubsection{Setting up the tests} + With the updated specifications it was possible to create a number of test cases. + In total there are five small test cases and four large test cases. + The small tests cover a sub-system and the large tests apply on the complete systems. + Each tests has a list of specifications that are covered with the test and for smaller tests the subsystem under test is also determined. + With a short description it is described how the test should be performed. - One of the small tests is performed by drawing a square: - \begin{test} - \begin{description} - \item[Coverage] SCARA - \item[Specifications] 3, 7, 11, 13 - \item[Description] - The SCARA must draw a square of at least \SI{50}{\milli\meter} high and \SI{70}{\milli\meter} wide. - This box is large enough to draw at least 3 characters. - This square should be drawn within one second. - If it is slower than that, it is not able to achieve specification 7. - \end{description} - \end{test} - Repeatability is tested in one of the large system wide tests: - \begin{test} - \begin{description} - \item[Coverage] All features - \item[Specifications] 3, 4, 9, 11 - \item[Description] - To test the repeatability of the system must do four things: - \begin{itemize} - \item The system will be reset. - \item Draw multiple squares (\SI{60}{\milli\meter} x \SI{60}{\milli\meter}) at a random position within the drawing range (\SI{1000}{\milli\meter} x \SI{300}{\milli\meter}). - \item The system will be reset again. - \item Then in a random order, at least different from the order of squares, a circle with a \SI{55}{\milli\meter} diameter must to be drawn inside the squares. - \end{itemize} - The test is successful if the circles are not drawn outside of the squares. - \end{description} - \end{test} + One of the small tests focusses on the speed and range requirements of the SCARA: + \begin{test}{Speed and Range} + During this test, a rectangle will be drawn on the whiteboard using the SCARA. + This rectangle is will be at least \SI{50}{\milli\meter} high and \SI{70}{\milli\meter} wide, such that three characters fit within the rectangle. + To test the speed requirements, the rectangle should be drawn within one second. + \tcbline + \begin{description} + \item[Features:] SCARA + \item[Specifications:] 3, 7, 11, 13 + \item[Results:] The test passes when: + \begin{itemize} + \item Rectangle height is at least \SI{50}{\milli\meter} + \item Rectangle width is at least \SI{70}{\milli\meter} + \item Completion time is less than \SI{1}{\second} + \end{itemize} +% The SCARA must draw a square of at least \SI{50}{\milli\meter} high and \SI{70}{\milli\meter} wide. +% This box is large enough to draw at least 3 characters. +% This square should be drawn within one second. +% If it is slower than that, it is not able to achieve specification 7. + \end{description} + \end{test} + Repeatability is tested in one of the large system wide tests: + \begin{test}{Repeatability} + This tests if the Writer can reach draw repeatedly on the same position on the board. + The system will start with drawing multiple \SI{60}{\milli\meter} squares on the board in a random location. + To test the repeatability, a circle with a \SI{55}{\milli\meter} diameter must be drawn inside of the square. + This should be done with twenty squares in an area of at least \SI{1000}{\milli\meter} x \SI{300}{\milli\meter}. + The drawing order of each square must be different from the drawing order of circles, this ensures that the Cable bot makes a different approach path. + \tcbline + \begin{description} + \item[Features:] SCARA, Cable Bot + \item[Specifications:] 3, 4, 9, 11 + \item[Results:] + The test passes when: + \begin{itemize} + \item Each square has a circle drawn inside. + \item The squares and circles are within \SI{5}{\milli\meter} of their given dimensions. + \item All the circles are completely within their corresponding square. + \end{itemize} + \end{description} + \end{test} - \subsubsection{Evaluation} - This step was completed with not many difficulties. - The specific order of operation and extra specifications should not be part of this step. - It was already concluded that the steps in the preliminary design were not as expected. - The fact that this step resulted in an additional changes only adds to that conclusion. + \subsubsection{Evaluation} + This step was completed without many difficulties. + Even though it required a revision of the earlier specifications and defining order of operation. + However, this is an issue with the preparation phase in general, not with this step in particular. - The design of the test cases resulted in valuable information about the system. - This information would be very useful in an earlier stage of the preliminary design. + During the analysis, I expected more specific tests. + Each test could then be used as a milestone during the development of the system. + Every time detail is added, an additional test passes. + Or a test fails, notifying that something went wrong and should be investigated. + Creating such specific tests relies on the details in the design of the system. + The current design is basic and these details are added during the feature implementation. + Nevertheless, this step resulted in a set of tests that cover all specifications and features that are specified in this preparation phase. + When all the tests pass, the system should meet all the requirements.