All Data Analysis Procedures

In Vivo Data Analysis Workflow

The following list outlines the data analysis workflow for in vivo experimentation with corresponding specification pages that give further instruction/details of the procedures (see InVivoTesting).

Specifications/DataAnalysis/FileAssociation
- Associate the ultrasound files to the appropriate data (tdms) files and provide the time synchronization (dT) XML files.
Specifications/DataAnalysis/QualityAssurance
- Provides an overview of the data. The figures developed should be visually inspected for irregularities in the data and ultrasound images.
  - Supporting Specifications:
    Specifications/DataManagement: Data uploaded to internal storage (Midas)
Specifications/DataAnalysis/ThicknessAnalysis
- Output: XML document of thickness measurements with associated forces, force/thickness plots, first analyzed image for each trial, and thickness inclusion file
  - Supporting Specifications:
    Specifications/DataManagement: Thickness data uploaded to internal storage (Midas)

In Vitro Data Analysis Workflow

The following list outlines the data analysis workflow for in vitro experimentation with corresponding specification pages that give further instruction/details of the procedures (see InVitroTesting).

Specifications/DataAnalysis/FileAssociation
- Associate the ultrasound files to the appropriate data (tdms) files and provide the time synchronization (dT) XML files.
Combine associated data for arm and leg (use Linux computer)
- Download both zip folders from Midas
- Unzip both directories
- Use rsync to combine the folders
  - Command: rsync -azvh "Most recent arm/leg directory" "Initial arm/leg directory"
  - This will copy all data from the first directory into the second directory.
  - Zip and upload to Midas as CMULTIS00X-1_associated_combined.zip
Specifications/DataAnalysis/QualityAssurance
- Provides an overview of the data. The figures developed should be visually inspected for irregularities in the data and ultrasound images.
  - Supporting Specifications:
    Specifications/DataManagement: Data uploaded to internal storage (Midas)
Specifications/DataAnalysis/ThicknessAnalysis
- Output: XML document of thickness measurements with associated forces, force/thickness plots, first analyzed image for each trial, and thickness inclusion file.
  - Supporting Specifications:
    Specifications/DataManagement: Thickness data uploaded to internal storage (Midas)
Specifications/DataAnalysis/Registration
- Transform ultrasound probe tip positions into appropriate coordinate system for modeling (CT or MR).

Instrumented Surgical Tools Data Analysis Workflow

-  ⇤ ← Revision 27 as of 2016-09-19 15:42:54 → 
  Size: 8132
  Editor: 139
  Comment:
+   ← Revision 57 as of 2019-02-27 19:25:15 → ⇥
  Size: 3553
  Editor: owings
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 7:
-== Target Outcome ==
== Prerequisites ==
== Protocols ==
+= All Data Analysis Procedures =
-Line 11:
+Line 9:
-=== File Association ===
 1. Setting up the Directory
  1. Download and unpack the subject folder and place in MULTIS_trials folder
  1. The folder/file configuration should be the following
   * '''MULTIS_trials'''
    * '''MULTIS###-#''' (Subject folder containing all data)
     * '''Configuration''' - Configuration files and subject XML
     * '''Data''' - All "tdms" files
     * '''Ultrasound''' - All "dicom" files
   * '''PulseWidths300.csv''' - contains key to decode file association pulse
   * '''PythonScripts'''
    * '''FileAssociation.py'''
    * '''manualMatcherFA.py'''
    * '''plotterFA.py'''
    * '''XMLparser.py'''
    * '''dataAnalysis.py'''
    * '''tdsmParserMultis.py'''
 1. Running the File Association script
  1. Install libraries as needed
  1. Change Directory to match desired Subject ID
  1. Ensure Plot is True to save all verification plots
  1. Run script.
  1. Check directory in file browser under the subject folder for the following additions
   * FileAssociationPNG - folder 
   * TimeSynchronization.txt - file containing trial numbers and delta T's of accepted trials only
   * MULTIS###-#readme.txt - human readable file which summarizes the file matching
 1. Manually Matching
  1. Open the 'MULTIS###-#readme.txt' file 
   * Note the Unmatched and Accepted Trials, specifically the binary column
  1. Examine any files beginning with 'NMa', meaning no match found
   1. Locate the start and end pluses, each a burst of two peaks. 
   1. Compare the binary of the trial run number to each of the pulse sequences found in the 'NMa' png's. Note the file names of each. At this point, it is known which file each of the missing accepted trial numbers belong to. 
  1. Open the '''manualMatcherFA.py''' and '''plotterFA.py''' python scripts.
  1. Change the directory file names in the '''manualMatcherFA''' script to a pair of matching files found in step 2. Only one pair of files will be compared at a time. The files will be imported to the '''plotterFA''' script automatically.
  1. Run the plotterFA.py script, ensuring iteration is set to zero. 
  1. Examine the figure generated. 
   1. The pulse extracted from the dicom (bottom of figure) should match the pulse from the tdms file (top of figure). Note the dicom pulse will not always begin at the left most portion of the window as it is a continuous loop in the ultrasound machine. 
   1. As before, locate the beginning and ending double pulses. 
   1. Count the number of peaks present in the dicom pulse.
   1. Count the number of RWaves in the generated figure. 
   1. It is likely that one or more of the RWaves are missing, as that is why the files did not match. 
  1. Change the iteration from 0, to 1, to 2, and so on and re-plot each time until the RWaves appear to line up with the pulse peaks in the top figure. 
  1. Identify which peaks have RWaves, and which do not. 
   1. Each of the peaks in the upper figure has a number associated with it in the script. Starting from 0, instead of 1, the peaks are named accordingly. 
   1. Record which peaks have RWaves. For example. (1,2,5,6) may have RWaves.
 . {{attachment:sampleRWaveAnalysis.png|Sample RWave analysis|height=500}}
  9 . Return to '''manualMatcherFA.py'''
   1. Ensure iteration is set to 2.
   1. Change the chosenRwaves variable to reflect the the recorded RWaves i.e. (1,2,5,6)
   1. Run the script
   1. Check the following items
    1. The filename of the dicom and tdms changed appropriately
    1. The trial number and time synch were added to
     * '''MULTIS###-#readme.txt'''
     * '''TimeSynchronization.txt'''
    1. The updated PNG was added to the '''FileAssociationPNG''' folder
+. [[Specifications/DataAnalysis]]
  1. [[Specifications/DataAnalysis/FileAssociation]]
  1. [[Specifications/DataAnalysis/QualityAssurance]]
  1. [[Specifications/DataAnalysis/ThicknessAnalysis]]
  1. [[Specifications/DataAnalysis/Registration]]
-Line 68:
+Line 15:
-=== Quality Assurance ===
 1. Setting up the Directory
  * same as file association with the addition of folders created by file association
 1. Running the dataAnalysis.py script
  1. Change Directory to reflect subject ID
  1. Run script
  1. Check the following
   1. A folder named '''AnalysisPNG''' should appear under '''MULTIS###-#'''
   1. 56 png files should be found within this folder
 1. Analysis
  1. There are two types of figures
   * Anatomical 
    * 3 dicom images (at 1st pulse, minimum force, last pulse) over load cell data
   * Indentation
    * 4 dicom images (at pre, start, middle, max indentation) over load cell data
 . {{attachment:anatomical.png|Anatomical Analysis|height=500}}
 . {{attachment:indentation.png|Indentation Analysis|height=500}}
=== Thickness Measurements ===
 1. Setting up the Directory 
  1. Download and unpack the desired Subject ID folder, make sure you use the 'associated' data if you do not want to associate the files yourself. 
  Note: If you have just completed the file association, confirm that the dicom and tdms files within the '''Ultrasound''' and '''Data''' folders have been matched (should begin with a three digit number) and that the '''TimeSynchronization.txt''' file is located within the subject folder. 
 1. The folder/file configuration should be the following 
  * '''MULTIS###-#'''  (Subject folder containing all data - the required data for the thickness measurements are shown below)
   * '''Data'''
   * '''Ultrasound'''
   * '''TimeSynchronization.txt'''
 1. Required Python scripts
  * '''ThicknessAnalysis_simple.py'''
  * '''tdsmParserMultis.py'''
 1. Running the manual thickness measurement script
  1. Install libraries as needed
  1. Change the directory to match where the Python scripts are located
  1. Run script
   * The first dialog window will allow you to select the Subject folder (you may also choose to hard code the path to the Subject folder, found within the getFiles() function)
    * Browse through to find the location of this folder and click 'ok'
   * A list will appear in another window which will show all the accepted trial locations for that subject (you may have to scroll down to see the entire list)
   * Double-click on the location you would like to analyze
 {{attachment:TA_ListBox.png|Example List of Locations to be Analyzed for one Subject|height=300}}
   * After a couple of seconds, the first image will appear
    * The program behaves differently depending on if the trial is anatomical or indentation
     * Anatomical - The program begins with the frame corresponding to the lowest normal force
     * Indentation - The program begins with the frame that corresponds to the beginning of indentation
   * Drag the red dots so that the center of the dot is over each boundary 
    * Superficial skin
    * Skin/Fat boundary
    * Fat/Muscle boundary
    * Muscle/Bone boundary
   * When you are happy with your selection, press 'enter' on your keyboard and the results will be calculated and saved to an xml file
   * To analyze the next image, hit the 'Next Image' button
   * Anatomical trials will give you the image corresponding to the next lowest force, until all frames within the first pulse and last pulse range have been analyzed
   * Indentation trials will give you the next image in the sequence until reaching the maximum force
   * You may end the program at any time by clicking the 'Close Program' button 
   * The xml file that was saved will be re-formatted using pretty print
   * All windows will be closed and the script will stop running 
  {{attachment:TA_example.png|Manual Thickness Analysis|height=500}}
 5. Data storage
   * The xml file will be named using the tdms filename with all of the descriptors for the trial
   * A new 'Location' child is appended for each frame that is analyzed
    * The Location child includes the following information:
     * Name - Location name with frame number
     * Forces - Forces in the x, y, and z directions (in N)
     * Moments - Moments in the x, y, and z directions (in Nm)
     * Thicknesses - Skin, Fat, and Muscle layers (in mm)
  {{attachment:TA_xmlexample.png|XML results output|height=300}}
+= In Vivo Data Analysis Workflow =

The following list outlines the data analysis workflow for in vivo experimentation with corresponding specification pages that give further instruction/details of the procedures (see [[Specifications/InVivoTesting|InVivoTesting|]]). 

  1. [[Specifications/DataAnalysis/FileAssociation]]
     * Associate the ultrasound files to the appropriate data (tdms) files and provide the time synchronization (dT) XML files.
  1. [[Specifications/DataAnalysis/QualityAssurance]]
     * Provides an overview of the data. The figures developed should be visually inspected for irregularities in the data and ultrasound images. 
      Supporting Specifications:
      [[Specifications/DataManagement]]: Data uploaded to internal storage (Midas)
  1. [[Specifications/DataAnalysis/ThicknessAnalysis]]
     * Output: XML document of thickness measurements with associated forces, force/thickness plots, first analyzed image for each trial, and thickness inclusion file
      Supporting Specifications:
      [[Specifications/DataManagement]]: Thickness data uploaded to internal storage (Midas)

= In Vitro Data Analysis Workflow =

The following list outlines the data analysis workflow for in vitro experimentation with corresponding specification pages that give further instruction/details of the procedures (see [[Specifications/InVitroTesting|InVitroTesting|]]).

  1. [[Specifications/DataAnalysis/FileAssociation]]
     * Associate the ultrasound files to the appropriate data (tdms) files and provide the time synchronization (dT) XML files.
  1. Combine associated data for arm and leg (use Linux computer)
     * Download both zip folders from Midas
     * Unzip both directories 
     * Use rsync to combine the folders
       * Command: rsync -azvh "Most recent arm/leg directory" "Initial arm/leg directory"
       * This will copy all data from the first directory into the second directory.
       * Zip and upload to Midas as CMULTIS00X-1_associated_combined.zip
  1. [[Specifications/DataAnalysis/QualityAssurance]]
     * Provides an overview of the data. The figures developed should be visually inspected for irregularities in the data and ultrasound images. 
      Supporting Specifications:
      [[Specifications/DataManagement]]: Data uploaded to internal storage (Midas)
  1. [[Specifications/DataAnalysis/ThicknessAnalysis]]
     * Output: XML document of thickness measurements with associated forces, force/thickness plots, first analyzed image for each trial, and thickness inclusion file.
      Supporting Specifications:
      [[Specifications/DataManagement]]: Thickness data uploaded to internal storage (Midas)
  1. [[Specifications/DataAnalysis/Registration]]
     * Transform ultrasound probe tip positions into appropriate coordinate system for modeling (CT or MR). 

= Instrumented Surgical Tools Data Analysis Workflow =

The following list outlines the data analysis workflow for in vitro experimentation with corresponding specification pages that give further instruction/details of the procedures (see [[Specifications/InstrumentedSurgicalTools|InstrumentedSurgicalTools|]]).