Goals

The fundamental goal of the Model Calibration phase is the calibration of initial working knee models of each team, which would be the outcomes of Model Development, the previous modeling and simulation phase.

Specific goals of the Model Calibration phase are

Tentative Timeline

December 2018

Planning - determine earmarked data and deliverables (group)

January-February 2019

Development of documentation for M&S processes - write specifications (individual teams)

May 20191

Delivery of process documentation - specifications submitted to Ahmet Erdemir by individual teams; review for completeness (group)

May-November 2019

Execution of activities faithful to specifications (individual teams); continued documentation of M&S processes - document protocol deviations (individual teams)

December 2019

Delivery of M&S outputs and documentation (individual teams); delivery of protocol deviations (individual teams) - all material submitted to Ahmet Erdemir; review for completeness (group); packaging for comparative analysis (group)

Earmarked Data

oks003 from Open Knee(s):

DU02 from Natural Knee Data:

The modeling teams do not have to use all of the earmarked data. They can also utilize information in literature to define model components that may not have any associated specimen-specific data.

Simulation Cases

Each team will simulate the same simulation case used in ModelDevelopment - passive flexion from 0 to 90 degrees, using the calibrated models. The teams are expected to be faithful to their original interpretation of the term "passive flexion", and therefore its reflection as loading and boundary conditions. The simulations' primary aim will be the prediction of joint kinematics.

Naturally, each team will also simulate the load case(s) that they choose from the earmarked data for calibration. Selection and interpretation of the load case(s) for calibration will be left to individual teams. The simulation(s) will portray joint kinematics-kinetics response.

Each team will also simulate all load cases of the earmarked data for calibration with the calibrated models. Interpretation of these load cases, i.e., their reflection as loading and boundary conditions, will be left to individual teams. The simulation(s) will portray joint kinematics-kinetics response.

Deliverables

Modeling & Simulation Outputs

Intermediate and End-Point M&S Outputs

Two packages representing final and intermediate outcomes of Model Calibration phase will be the outputs of this phase. The focus is the complete curation of the processes and results of each individual team's Model Calibration activities.

Documentation of M&S Outputs

Two document sets should provide information to help navigate the M&S output package and to describe what M&S Outputs are (not how they are obtained).

Modeling & Simulation Processes

An important set of deliverables are the documentation of the M&S processes, i.e. the individualized workflows to transform earmarked data to M&S outputs. The documents will include Specifications, to provide each team's initial plans for processing data to obtain M&S outputs, and Protocol Deviations, to describe modifications to Specifications during the execution of the initial plans. Ideally, when a third-party modeler has the earmarked data, specifications, and protocol deviations, they can follow the steps of the modeling team to reproduce their disseminated M&S outputs.

Specifications

Specifications documents are aimed to describe individual steps to obtain M&S outputs from earmarked data. The detail should be provided in a manner that other teams can reproduce the decision making process of the modeler. Any subjective and arbitrary decisions should be noted and as much insight as possible should be provided for justification and reproduction these decisions. This document should be prepared and disseminated before M&S activities start.

Protocol Deviations

If specifications need to be updated during activities to go from earmarked data to M&S outputs, these should be documented as protocol deviations. As much insight as possible should be provided for justification of the deviation. This document should be prepared during M&S activities and disseminated along with M&S outputs.

Package for Comparative Analysis

Two packages will be prepared to curate M&S Outputs and process documentation of each team; one starting with data from Open Knee(s), the other utilizing Natural Knee data. The M&S Output files submitted by the individual teams will be organized in these packages and provided as is with the possible exception of renaming of master package files. The documents will be provided as is. Yet, additional organization of output description and process documentation may be necessary to meet third-party requirements or recommendations (see guidance from #USFDA, #ASME, #NASA, #Erdemir2015).

Guidance

Where to get the Earmarked Data?

The teams can download data in the downloads section of project site, https://simtk.org/projects/kneehub, check packages

Tell me more about joint mechanics data from Natural Knee Data

Mechanical testing of the Natural Knee Data is described in Harris et al. (2016). In relation to earmarked data, the loads representative of passive flexion and laxity were manually applied with live feedback on loading. The data were processed and described in a joint coordinate system. The data files are in human readable CSV format, including kinematics-kinetics of the tibiofemoral joint in an anatomical coordinate system representation. Files represent separate loading cases of laxity or passive flexion.

Known Limitations

This list provides identified limitations that needs to be managed by all teams while processing the data for modeling and simulation.

Tell me more about joint mechanics data from Open Knee(s)

Mechanical testing of Open Knee(s) is described in specifications at #OKS-SEJM. Notes related to experimentation on the specimen can be found at #OKS-oks003. The Open Knee(s) data is in a raw form, i.e. as collected using the robotics testing system. It may be useful to get familiar with the tools and experimentation setup to navigate the contents of the data files. A cohort of documents describing testing infrastructure, data collection and storage conventions is available at #OKS-IEM.

Coordinate system conventions used for data collection and storage are described in detail at #OKS-KCS. In a nutshell, kinematics-kinetics data are represented in anatomical joint coordinate systems. There are some caveats that the users of the data should pay attention to. First, there are two anatomical coordinate systems - an original based on anatomical landmarks, and an optimized to minimize off-axis motion representation during passive flexion. The latter only changes the original femur coordinate system to re-orient the flexion axis. Second, the joint coordinate systems are in right knee abstraction, i.e. essentially a left knee, if tested, is mirrored for convenience in testing. This situation needs to be accommodated when registering the data to physical coordinate systems, i.e that of imaging. Third, the movements (translations and rotations) are described along the axis of joint coordinate system; a femur fixed flexion axis, a tibia fixed internal-external rotation axis, and a floating axis. Fourth, the kinematics are represented relative to a reference testing state of the joint, which is recorded in configuration file as an offset. Last, the joint loads are represented in tibia coordinate system with a clinical convention, i.e. external loads applied to the tibia as in a physical exam.

Two types of files are provided with the Open Knee(s) data set. A configuration file (.cfg) contains settings and data necessary to perform mechanical testing. Contents of this file are described in detail at #OKS-SCF. Of importance to model calibration are content related to coordinate system transformations between motion tracking sensors (femur and tibia as rigid bodies), robot and motion tracking system (worlds), anatomical coordinate systems (original and optimized, in right knee abstraction); joint coordinate system offsets (representing reference kinematics of the joint) probed points (anatomical and registration markers) including the point in world coordinate system and rigid body pose and orientation (where the point rests) in world coordinate system.

The second type of file contains mechanical data in a National Instruments file format (.tdms). This file is binary but can be accessed with widely available tools for Python and Matlab. Passive flexion data are provided along with laxity data. For laxity testing, each file represents mechanics of the joint at a fixed flexion angle (based on optimized joint coordinate system) focusing on testing of three degrees of freedom one at a time: anterior-posterior, varus-valgus, and internal-external rotation. Raw motion data are stored in relevant channels, e.g. pose and orientation of femur and tibia motion sensors (Sensor.Femur sensor and Sensor.Tibia Sensor), loads in load cell coordinate system (Sensor.Static Load Cell). During data collection, the raw mechanics data are processed to facilitate control. The processed data is in a right knee abstraction, i.e. left knees are represented in a mirrored fashion for testing. The processed kinematics-kinetics data include joint kinematics: in original anatomical coordinate system (State.Knee JCS) or optimized one (State.JCS) (all relative to reference/offset state in configuration file); and joint kinetics: in tibia coordinate system (State.JCS Load) with loads on tibia. It should be noted that the tibia coordinate system of original and optimized anatomical coordinate systems are the same, only the femur coordinate system is reoriented (optimized) to minimize off-axis motions of passive flexion. For more details of file contents, one can refer to #OKS-TDMS.

Known Limitations

This list provides identified limitations that needs to be managed by all teams while processing the data for modeling and simulation.

How to prepare and submit Specifications?

Specifications are concerned about documentation of the processes to transform earmarked data to M&S outputs. This information is intended to be provided separately for processing of both data sets to capture nuances in response to variable data content and quality. This documentation will be provided before execution of Model Calibration activities to demonstrate the path each modeling team has chosen. Specifications should be in a detail such that other modeling teams can follow the steps to reproduce M&S outputs. When possible, justifications to support modeling & simulation decisions should be provided.

Recommended content for Specifications includes:

Each team will submit two sets of documents, one for Model Calibration using Open Knee(s) data and another for Model Calibration using Natural Knee data. Each set will include the source document, i.e., in preferred format that it is written, and as a PDF copy. The documents should be submitted by any convenient means - e-mail, file sharing, upload to project site, etc. Ahmet Erdemir, erdemira@ccf.org, will push the submitted documents to the project site in the Documents section with labeling indicative of modeling phase, modeling teams, and data source.

How to prepare and submit Protocol Deviations?

It is likely that initial specifications may need to be modified to accomplish Model Calibration activities. These modifications, i.e. Protocol Deviations, should be documented during Model Calibration activities. Each protocol deviation should include information to understand who did, what and when, and why. The documentation should be in a detail such that other modeling teams can utilize it along with specifications to reproduce M&S outputs. This document should also include the actual burden of M&S activities including changes in software and hardware requirements, and deviations from anticipated labor effort and computational cost. Protocol Deviations should be collated in two sets of documents (one for using Open Knee(s) data and another for Natural Knee data). The documents will be provided after execution of Model Calibration activities in source form, i.e., in preferred format that it is written, and as a PDF copy. The documents should be submitted by any convenient means - e-mail, file sharing, upload to project site, etc. Ahmet Erdemir, erdemira@ccf.org, will push the submitted documents to the project site in the Documents section with labeling indicative of modeling phase, modeling teams, and data source.

How to curate, document and submit Modeling and Simulation Outputs?

All M&S outputs (intermediate and final) should be provided in their preferred format, i.e., accommodating individual teams' workflows. Each team should provide two packages (one for Open Knee(s) data and another for Natural Knee data) as a compressed archive (e.g. *.zip). For each package, an accompanying document should describe M&S outputs - in source form, i.e., in preferred format that it is written, and as a PDF copy. This information should focus on i) file descriptors and formats, and folder organization to help navigate the provided content and ii) what actually each M&S output is (NOT how they are obtained). The latter should be reflective of the description of target intermediate and final M&S products of Specifications. It should also be indicative of necessary information for other modeling teams to judge the nature and extent of the provided content. The packages should be submitted by any convenient means - e-mail, file sharing, upload to project site, etc. Ahmet Erdemir, erdemira@ccf.org, will push the submitted packages to the project site in the Downloads section with labeling indicative of modeling phase, modeling teams, and data source.

Preparing a package for Comparative Analysis

All material will likely need to be reorganized and converted to respond to third-party requirements for comparative analysis (see guidance from #USFDA, #ASME, #NASA, #Erdemir2015). Nonetheless, this should not prevent individual M&S teams to personalize their workflows and documentation style. Preparation of a package for comparative analysis will collate M&S outputs and documents submitted by the individual teams. M&S outputs will be provided in formats as delivered by the teams. Documentation may need to be reorganized using source copies of documentation from individual teams. Ahmet Erdemir, erdemira@ccf.org, will submit the prepared package to the third-party by any convenient means - e-mail, file sharing, upload to project site, etc.

References

Erdemir, A., Mulugeta, L. and Lytton, W. W. Ten “not so” simple rules for credible practice of modeling and simulation in healthcare: a multidisciplinary committee perspective, 2015 Biomedical Engineering Society / Food and Drug Administration Frontiers in Medical Devices Conference: Innovations in Modeling and Simulation, May 18-20, 2015, Washington, DC. Available at: https://simtk.org/svn/cpms/doc/posters/fmd_2015.pdf.

Erdemir, A., Guess, T. M., Halloran, J. P., Tadepalli, S. C. and Morrison, T. M. (2012) Considerations for reporting finite element analysis studies in biomechanics, Journal of Biomechanics, 45, 625-633. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3278509/.

U.S. Food and Drug Administration. Reporting of Computational Modeling Studies in Medical Device Submissions. Available at: https://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM381813.

NASA-STD-7009: Standard for Models and Simulations. Available at: https://standards.nasa.gov/standard/nasa/nasa-std-7009.

ASME V&V American Society of Mechanical Engineers - Committee Pages - V&V 40 Verification and Validation in Computational Modeling of Medical Devices. Available at: https://cstools.asme.org/csconnect/CommitteePages.cfm?Committee=100108782.

Harris, M. D., Cyr, A. J., Ali, A. A., Fitzpatrick, C. K., Rullkoetter, P. J., Maletsky, L. P. and Shelburne, K. B. (2016) A combined experimental and computational approach to subject-specific analysis of knee joint laxity, Journal of Biomechanical Engineering, 138, 081004. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4967880/.

Open Knee(s): Virtual Biomechanical Representations of the Knee Joint - Specifications/ExperimentationJointMechanics. Available at: https://simtk.org/plugins/moinmoin/openknee/Specifications/ExperimentationJointMechanics.

Open Knee(s): Virtual Biomechanical Representations of the Knee Joint - oks003. Available at: https://simtk.org/plugins/moinmoin/openknee/oks003.

Open Knee(s): Virtual Biomechanical Representations of the Knee Joint - Infrastructure/ExperimentationMechanics. Available at: https://simtk.org/plugins/moinmoin/openknee/Infrastructure/ExperimentationMechanics.

Knee Joint Coordinate System v2.1. Available at: https://simtk.org/plugins/moinmoin/openknee/Infrastructure/ExperimentationMechanics?action=AttachFile&do=get&target=Knee+Coordinate+Systems.pdf.

bioRobotics, simVitro - State Configuration File Explanations, User Manual, 2013CB-031-001.A. Available at: https://simtk.org/plugins/moinmoin/openknee/Infrastructure/ExperimentationMechanics?action=AttachFile&do=get&target=2013CB-031-001.A_State+Configuration+File+Explanations.pdf.

bioRobotics, simVitro - simVITRO Data File Contents for Open Knee, User Manual, 2013CB-031-002.B. Available at: https://simtk.org/plugins/moinmoin/openknee/Infrastructure/ExperimentationMechanics?action=AttachFile&do=get&target=2013CB-031-002.B+simVITRO+Data+File+Contents_Open+Knee.pdf.

Footnotes

  1. Postponed from March 2019 to accommodate completion of the previous phase (1)

ModelCalibration (last edited 2019-04-03 12:41:21 by aerdemir)