Share 
Follow 
AboutDownloadsDocumentsForumsSource CodeIssuesNews
Date:
2019-03-25 22:29
Priority:
4
State:
Closed
Submitted by:
Joy Ku (joyku)
Assigned to:
Joy Ku (joyku)
Target Release:
4.2.1
Status:
Testing complete
Summary:
[data share] Provide metadata creation script

Detailed description
Potentially let user upload Excel or csv file that is parsed into the required metadata files

Add A Comment: Notepad

Message  ↓
Date: 2023-08-12 00:05
Sender: Joy Ku

Formatting updates confirmed.

Date: 2023-08-10 23:20
Sender: Joy Ku

Also on the ImportCSVFile.php, please make this text black and bold: It can be hard to clean up the automatically generated metadata files, so FIRST click “Verify” to confirm that there are no errors with the spreadsheet.

I just realized that we use orange/red to indicate links so it can be confusing to use that same color for regular text.

Date: 2023-08-10 23:18
Sender: Joy Ku

Thanks for all the updates. A couple other minor edits to the instructions:

1) In metadata.php: Most datasets use both a data directory structure template [1] AND one of the metadata file approaches [2 or 3]. --> Most datasets use both a data directory structure template [3] AND one of the metadata file approaches [1 or 2].

2) In metadata.php: Remove "Questions?" under "How Do I Provide Metadata?"

3) In the "Questions?" section, update "Check out the FAQ" --> "Check out the FAQ or Submit question to SimTK discussion forum" linking to the FAQ and forum respectively

Date: 2023-07-31 21:29
Sender: Joy Ku

Parsing is working great! However, if I upload a second .csv file and select it, the system still seems to read from the initial .csv file. Could you look into this?

Also some formatting/text updates:

1. Update "Enabling Query" section to read:

Provide Metadata & Enable Query Feature
To enable querying of your dataset, you need to provide metadata. Options to add metadata:
[bullet] SimTK can semi-automatically generate metadata files from a CSV file (See "Populate from Metatadata CSV File" section below).
[bullet] You can also explicitly provide metadata via files in each data folder or implicitly via your directory structure. More details

2. Add additional blank lines between "Import Status" section and "Populate from Metadata CSV File" section

3. Unbold "Populate from Metadata CSV File"

4. Add numbering to bolded instructions for "Populate from Metadata CSV File" section. Make step 1 "Import CSV file" - Tooltip should read "File must contain a header row. Import file to the same directory level as subject folders to which metadata files will be added." Remove tool tip for "Select CSV file..."

5. Change tooltip for "Row number" to read "Content in the row header"

6. Change tooltip for "Column number" to read "1 = Column A in Excel,..."

I will send text for metadata.php.

Date: 2023-04-19 02:52
Sender: Anthony Souza

- metadata parsing working as specified.

Date: 2023-04-18 22:27
Sender: Henry Kwong

- Updated tooltip.

Date: 2023-04-14 18:59
Sender: Henry Kwong

- The restriction on characters is by design.

Date: 2023-04-14 06:44
Sender: Anthony Souza

- some column header values are being displayed incorrectly. For example, [n-n/m] is displayed as [nn/m]

Date: 2023-04-06 19:42
Sender: Henry Kwong

- Updated per feedback comments.

Date: 2023-03-31 18:54
Sender: Anthony Souza

- when working with an empty project, after processing metadata, the query button is not displayed. This was tested on the "Disk Quta High Setting" project and the "new Study for Issue" data share study.

- sometimes when the metadata.csv file is uploaded, and then selected, clicking verify or process produces, an "invalid file" error is shown.

- some columns in the header row are not displayed correctly when viewing them in the query builder after metadata has been parsed.

Date: 2023-03-23 23:43
Sender: SimTK Admin

- When metadata is not present, the Query Data button is not visible.

- A project administrator can import study data without metadata (for browse only using "Browse Data") but cannot query the study data based on metadata to select subsets.

- Metadata file can be imported by a project administrator using a CSV file, which can be saved from an Excel spreadsheet or the Metadata CSV file can be generated using other tools, or manually.

- Import/Edit Data page has a new feature "Populate from Metadata CSV File". A project administorator can select a metadata file to process to populate metadata. The project administrator can first verify that the file is read and parsed propertly, based on the parameters specified: row number of header and column number of column with subject ID. The subject ID in the Metadata CSV file must match the subject folders in the study and the subject prefix setting of the study: case matters; hence "Subject" is different from "subject". The Metadata CSV file should reside in a folder directly above the subject folders.

- The project administrator can choose to "Verify", which test the Metadata CSV file only but does not use it for processing to populate metadata, or choose to "Process", which process the Metadata CSV file to populate metadata.

- As a result of processing the metadata, metadata.json files are put under each subject folder.

- Query Data button will be enabled if metadata has been populated.

Field Old Value Date By
status_idOpen2023-08-22 00:48hykwong2012
close_dateNone2023-08-22 00:48hykwong2012
StatusReady for testing2023-08-12 00:05joyku
assigned_toasouza882023-04-19 02:52asouza88
assigned_tohykwong20122023-04-14 19:00hykwong2012
StatusImplementing2023-04-14 18:59hykwong2012
assigned_toasouza882023-04-14 06:44asouza88
StatusReady for testing2023-04-14 06:44asouza88
assigned_tohykwong20122023-04-06 19:42hykwong2012
StatusImplementing2023-04-06 19:42hykwong2012
assigned_toasouza882023-03-31 18:54asouza88
StatusReady for testing2023-03-31 18:54asouza88
assigned_tohykwong20122023-03-23 23:43admin
StatusImplementing2023-03-23 23:43admin
Target ReleaseNone2022-11-22 19:09hykwong2012
StatusAccepted2022-05-31 19:21hykwong2012
assigned_tonone2022-04-26 19:28hykwong2012
StatusNone2022-04-26 19:28hykwong2012
Feedback