TC - JDI Train Set Test

I. Testing target:

Three tables are generated for TC database in the JDI train set. They are:

• word-Jdid-Wc-Dc table
• mh-Jdid-Dc table
• sh-Jdid-Dc table

These tables are tested (by comparing to the previous release) after they are generated. The source file details of each release are:

II. Testing Procedures:

• Convert to standard format:
  Convert WordJdidWcDcTable.txt to WordJdidWcTable.txt and WordJdidDcTable.txt with following standard format:

  entity name: word/Mh/Sh	JDID	count score

  	shell> flds 1,2,3 WordJdidWcDcTable.txt > WordJdidWcTable.txt
  	shell> flds 1,2,4 WordJdidWcDcTable.txt > WordJdidDcTable.txt
  	

• Compare four files:
  • wordJdidWcTable.txt (converted from WordJdidWcDcTable.txt)
  • wordJdidDcTable.txt (converted from WordJdidWcDcTable.txt)
  • MhJdidDcTable.txt
  • ShJdidDcTable.txt

• Compare procedures:
  • Get common JDs (GetCommonJds.java):
    Journal descriptors (JDs) can be different between versions due to different lsi${year}.xml. Different JDs results in different word count and document count for word, MeSH Main headings, and MeSH subheadings. To simplify the comparing procedures, only JD counts with common JDs in both releases are compared. So, the first step is to get the list of common Jds.
  • Compare JD vectors (CompareJdiVectors.java):
    Use similarity distance (Cosine coefficient) to compare common JD scores (vectors) for all words, MHs, SHs. The output is generated to ${TC_TEST}/TrainSetTest/data/Output/${SRC_YEAR}-${TEST_YEAR}/Jdi/ with following format:

    entity name: word/Mh/Sh	Similarity distance

    Please note that some entity does not have any common JD score in both/either release. In such case, the similarity distance will be a NaN and should not be compared. For examples, in the comparison of 2007 and 2007+ releases, four MeSH main headings falls in this category

    Main Heading	Not common JDs, 2007	Not common JDs, 2007+
    butirosin sulfate	JD007	JD136
    capreomycin sulfate	JD007	JD136
    certificate of need	JD027
    congenital, hereditary, and neonatal diseases and abnormalities	JD006

  • Analyze & summarize reports (AnalyzeSimilarityResults.java):
    Analyze comparison results from above files:
    • Total entity number
    • Total similarity distance
      Calculate the similarity distance between the similarity results and 100% similar result. 1.0 is completely similar. This value should be above 0.9.
    • Similarity distribution with increment by 0.05.

III. Testing Results:

Please refer to JDI similarity tests for the TC annual release.

2007 to 2008

Compare to 2007

Compare to 2008