All Data Analysis Procedures

In Vivo Data Analysis Workflow

The following list outlines the data analysis workflow for in vivo experimentation with corresponding specification pages that give further instruction/details of the procedures (see InVivoTesting).

Specifications/DataAnalysis/FileAssociation
- Associate the ultrasound files to the appropriate data (tdms) files and provide the time synchronization (dT) XML files.
Specifications/DataAnalysis/QualityAssurance
- Provides an overview of the data. The figures developed should be visually inspected for irregularities in the data and ultrasound images.
  - Supporting Specifications:
    Specifications/DataManagement: Data uploaded to internal storage (Midas)
Specifications/DataAnalysis/ThicknessAnalysis
- Output: XML document of thickness measurements with associated forces, force/thickness plots, first analyzed image for each trial, and thickness inclusion file
  - Supporting Specifications:
    Specifications/DataManagement: Thickness data uploaded to internal storage (Midas)

In Vitro Data Analysis Workflow

The following list outlines the data analysis workflow for in vitro experimentation with corresponding specification pages that give further instruction/details of the procedures (see InVitroTesting).

Specifications/DataAnalysis/FileAssociation
- Associate the ultrasound files to the appropriate data (tdms) files and provide the time synchronization (dT) XML files.
Combine associated data for arm and leg (use Linux computer)
- Download both zip folders from Midas
- Unzip both directories
- Use rsync to combine the folders
  - Command: rsync -azvh "Most recent arm/leg directory" "Initial arm/leg directory"
  - This will copy all data from the first directory into the second directory.
  - Zip and upload to Midas as CMULTIS00X-1_associated_combined.zip
Specifications/DataAnalysis/QualityAssurance
- Provides an overview of the data. The figures developed should be visually inspected for irregularities in the data and ultrasound images.
  - Supporting Specifications:
    Specifications/DataManagement: Data uploaded to internal storage (Midas)
Specifications/DataAnalysis/ThicknessAnalysis
- Output: XML document of thickness measurements with associated forces, force/thickness plots, first analyzed image for each trial, and thickness inclusion file.
  - Supporting Specifications:
    Specifications/DataManagement: Thickness data uploaded to internal storage (Midas)
Specifications/DataAnalysis/Registration
- Transform ultrasound probe tip positions into appropriate coordinate system for modeling (CT or MR).

Instrumented Surgical Tools Data Analysis Workflow

-  ⇤ ← Revision 21 as of 2016-09-13 17:40:29 → 
  Size: 4695
  Editor: 192
  Comment:
+   ← Revision 57 as of 2019-02-27 19:25:15 → ⇥
  Size: 3553
  Editor: owings
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 7:
-== Target Outcome ==
== Prerequisites ==
== Protocols ==
+= All Data Analysis Procedures =
-Line 11:
+Line 9:
-=== File Association ===
 1. Setting up the Directory
  1. Download and unpack the subject folder and place in MULTIS_trials folder
  1. The folder/file configuration should be the following
   * '''MULTIS_trials'''
    * '''MULTIS###-#''' (Subject folder containing all data)
     * '''Configuration''' - Configuration files and subject XML
     * '''Data''' - All "tdms" files
     * '''Ultrasound''' - All "dicom" files
   * '''PulseWidths300.csv''' - contains key to decode file association pulse
   * '''PythonScripts'''
    * '''FileAssociation.py'''
    * '''manualMatcherFA.py'''
    * '''plotterFA.py'''
    * '''XMLparser.py'''
    * '''dataAnalysis.py'''
    * '''tdsmParserMultis.py'''
 1. Running the File Association script
  1. Install libraries as needed
  1. Change Directory to match desired Subject ID
  1. Ensure Plot is True to save all verification plots
  1. Run script.
  1. Check directory in file browser under the subject folder for the following additions
   * FileAssociationPNG - folder 
   * TimeSynchronization.txt - file containing trial numbers and delta T's of accepted trials only
   * MULTIS###-#readme.txt - human readable file which summarizes the file matching
 1. Manually Matching
  1. Open the 'MULTIS###-#readme.txt' file 
   * Note the Unmatched and Accepted Trials, specifically the binary column
  1. Examine any files beginning with 'NMa', meaning no match found
   1. Locate the start and end pluses, each a burst of two peaks. 
   1. Compare the binary of the trial run number to each of the pulse sequences found in the 'NMa' png's. Note the file names of each. At this point, it is known which file each of the missing accepted trial numbers belong to. 
  1. Open the '''manualMatcherFA.py''' and '''plotterFA.py''' python scripts.
  1. Change the directory file names in the '''manualMatcherFA''' script to a pair of matching files found in step 2. Only one pair of files will be compared at a time. The files will be imported to the '''plotterFA''' script automatically.
  1. Run the plotterFA.py script, ensuring iteration is set to zero. 
  1. Examine the figure generated. 
   1. The pulse extracted from the dicom (bottom of figure) should match the pulse from the tdms file (top of figure). Note the dicom pulse will not always begin at the left most portion of the window as it is a continuous loop in the ultrasound machine. 
   1. As before, locate the beginning and ending double pulses. 
   1. Count the number of peaks present in the dicom pulse.
   1. Count the number of RWaves in the generated figure. 
   1. It is likely that one or more of the RWaves are missing, as that is why the files did not match. 
  1. Change the iteration from 0, to 1, to 2, and so on and re-plot each time until the RWaves appear to line up with the pulse peaks in the top figure. 
  1. Identify which peaks have RWaves, and which do not. 
   1. Each of the peaks in the upper figure has a number associated with it in the script. Starting from 0, instead of 1, the peaks are named accordingly. 
   1. Record which peaks have RWaves. For example. (1,2,5,6) may have RWaves.
 . {{attachment:sampleRWaveAnalysis.png|Sample RWave analysis|height=500}}
  7. Return to '''manualMatcherFA.py'''
   1. Ensure iteration is set to 2.
   1. Change the chosenRwaves variable to reflect the the recorded RWaves i.e. (1,2,5,6)
   1. Run the script
   1. Check the following items
    1. The filename of the dicom and tdms changed appropriately
    1. The trial number and time synch were added to
     * '''MULTIS###-#readme.txt'''
     * '''TimeSynchronization.txt'''
    1. The updated PNG was added to the '''FileAssociationPNG''' folder
+. [[Specifications/DataAnalysis]]
  1. [[Specifications/DataAnalysis/FileAssociation]]
  1. [[Specifications/DataAnalysis/QualityAssurance]]
  1. [[Specifications/DataAnalysis/ThicknessAnalysis]]
  1. [[Specifications/DataAnalysis/Registration]]
-Line 68:
+Line 15:
-=== Quality Assurance ===
 1. Setting up the Directory
  * same as file association with the addition of folders created by file association
 1. Running the dataAnalysis.py script
  1. Change Directory to reflect subject ID
  1. Run script
  1. Check the following
   1. A folder named '''AnalysisPNG''' should appear under '''MULTIS###-#'''
   1. 56 png files should be found within this folder
 1. Analysis
  1. There are two types of figures
   * Anatomical 
    * 3 dicom images (at 1st pulse, minimum force, last pulse) over load cell data
   * Indentation
    * 4 dicom images (at pre, start, middle, max indentation) over load cell data
 . {{attachment:anatomical.png|Anatomical Analysis|height=500}}
 . {{attachment:indentation.png|Indentation Analysis|height=500}}
=== Thickness Measurements ===
+= In Vivo Data Analysis Workflow =

The following list outlines the data analysis workflow for in vivo experimentation with corresponding specification pages that give further instruction/details of the procedures (see [[Specifications/InVivoTesting|InVivoTesting|]]). 

  1. [[Specifications/DataAnalysis/FileAssociation]]
     * Associate the ultrasound files to the appropriate data (tdms) files and provide the time synchronization (dT) XML files.
  1. [[Specifications/DataAnalysis/QualityAssurance]]
     * Provides an overview of the data. The figures developed should be visually inspected for irregularities in the data and ultrasound images. 
      Supporting Specifications:
      [[Specifications/DataManagement]]: Data uploaded to internal storage (Midas)
  1. [[Specifications/DataAnalysis/ThicknessAnalysis]]
     * Output: XML document of thickness measurements with associated forces, force/thickness plots, first analyzed image for each trial, and thickness inclusion file
      Supporting Specifications:
      [[Specifications/DataManagement]]: Thickness data uploaded to internal storage (Midas)

= In Vitro Data Analysis Workflow =

The following list outlines the data analysis workflow for in vitro experimentation with corresponding specification pages that give further instruction/details of the procedures (see [[Specifications/InVitroTesting|InVitroTesting|]]).

  1. [[Specifications/DataAnalysis/FileAssociation]]
     * Associate the ultrasound files to the appropriate data (tdms) files and provide the time synchronization (dT) XML files.
  1. Combine associated data for arm and leg (use Linux computer)
     * Download both zip folders from Midas
     * Unzip both directories 
     * Use rsync to combine the folders
       * Command: rsync -azvh "Most recent arm/leg directory" "Initial arm/leg directory"
       * This will copy all data from the first directory into the second directory.
       * Zip and upload to Midas as CMULTIS00X-1_associated_combined.zip
  1. [[Specifications/DataAnalysis/QualityAssurance]]
     * Provides an overview of the data. The figures developed should be visually inspected for irregularities in the data and ultrasound images. 
      Supporting Specifications:
      [[Specifications/DataManagement]]: Data uploaded to internal storage (Midas)
  1. [[Specifications/DataAnalysis/ThicknessAnalysis]]
     * Output: XML document of thickness measurements with associated forces, force/thickness plots, first analyzed image for each trial, and thickness inclusion file.
      Supporting Specifications:
      [[Specifications/DataManagement]]: Thickness data uploaded to internal storage (Midas)
  1. [[Specifications/DataAnalysis/Registration]]
     * Transform ultrasound probe tip positions into appropriate coordinate system for modeling (CT or MR). 

= Instrumented Surgical Tools Data Analysis Workflow =

The following list outlines the data analysis workflow for in vitro experimentation with corresponding specification pages that give further instruction/details of the procedures (see [[Specifications/InstrumentedSurgicalTools|InstrumentedSurgicalTools|]]).