TC Package - Annual Release Procedures

This page describes an annually release procedures for Text Categorization tools package with new set of training data.

1. Prepare tc${YEAR} baseline
   • Copy tc${PREV_YEAR} to tc${YEAR}
     shell> cp -rp ${TC}/tc${PREV_YEAR} ${TC}/tc${YEAR}
   • Change ${PREV_YEAR} to ${YEAR} in build.html files under ${TC}, ${TC}/examples, ${TC}/install
   • Change ${PREV_YEAR} to ${YEAR} in ${TC}/overview.html
   • Update ${TC}/data/Config/tc.properties, tc.properties.TEMPLATE
     => Try to build with shell> ant release (should be OK to build)
2. Update Lib/*.jar file
   • Update ${TC}/lib/Other/lvg${YEAR}api.jar
   • Update ${TC}/lvg${YEAR}lite
     => This is needed when run stWsd (unzip from lvg${YEAR}lite.tgz)
   • Update ${TC}/lib/jdbcDrivers/hsqldb.jar
3. Update Java source code
   • Modify prolog of java files
     • Remove all SCRs-XX from history tag
     • Modify V-${PREV_YEAR} from version tag

       		shell> cd ${LVG}/Components/BaselineCode/bin
       		shell> ModifyTcJavaCode
       		shell> YYYY (${YEAR})
       		shell> 1
       		shell> y
       		

       => build and test to make sure the result is same as last release

4. Update JDK/JRE
   • Download JDK from SUN
   • Install JDK to /usr/local/Applications/Java
   • Update symbolic link of /usr/bin/java
   • Update symbolic link of /usr/bin/javac
   • Update ${JAVA_HOME} in ~/.cshrc (for javadoc)

   • Update 2 JREs ${TC}/bin/jreDist/
     • Linux
     • windows
5. Update Installation Program
   • Update ${project.year} in ${TC}/install/build.xml
   • Update ${TC}/install/sources/gov/nih/nlm/nls/tc/install/Setup/Param.java
     • VERSION
     • JRE_DIR
     • DATABASE_NAME
   • Update scripts in ${TC}/install/bin/*
     • TC_YEAR
     • JRE_VERSION
     • CLASSPATH
6. Update DB
   • Download latest version of HyperSql Db, copy 3 files to ${TC}/lib/jdbcDrivers
     • hsqldb.jar
     • hsqldb_lic.txt
     • hypersonic_lic.txt
7. Reload data to Database (if it is upgraded)
   • cd /bin/loadDb/
   • Change ${PREV_YEAR} to ${YEAR} in ${TC}/bin/loadDb/1.CreateDb
   • Load to new Database
     • Create DB
     • Load data to database
       • Word-Jd Scores
       • Mh-Jd Scores
       • Sh-Jd Scores
       • Word-St Scores
     • Change readonly=true in tc${YEAR}.properties
     • Check is the result is the same
8. Integrate with New JDI Training Data
   • Generate JDI dataset, see TC Preprocess procedures
   • Load to new Database
     • Create DB
     • Change hsqldb.cache_file_scale=8 in tc${YEAR}.properties
     • Load data to database
       • Word-Jd Scores
       • Mh-Jd Scores
       • Sh-Jd Scores
     • Change readonly=true in tc${YEAR}.properties
   • Find and update Max. Signal
   • Test JDI similarity between ${YEAR} and ${PRE_YEAR}
9. Integrate with New STI Training Data
   • Generate STI dataset, see TC Preprocess procedures
   • Use STRI to refine StDocument
   • Load data to database
     • Word-St Scores
   • Test STI and STRI through WSD data collection set
10. Complete SCRs for ${YEAR} release
    • Update version ${YEAR}
      • {TC_SRC}/Tools/Jdi.java
      • {TC_SRC}/Tools/Sti.java
      • {TC_SRC}/Tools/Stri.java
      • {TC_SRC}/Tools/StWsd.java
      • {TC_SRC}/Tools/Mlt.java
    • Update default value for Max. normalized signal (from observation of file wordSignalWcDcGt1.txt)
      • MAX_SIGNAL in ${TC_SRC}/FilterApi/LegalWordsOption.java
    • Update -rv:YEAR option
      • ${TC_SRC}/Lib/TcSystemOption.java
    • Standardize Java source code
shell> cd ${LVG}/Components/BaselineCode/bin/
shell> ModifyTcJavaCode
${YEAR}
2
11. Update other software components in the package
    • ${TC}/bin
      • Update ${YEAR} in ${TC}/bin/runProg
      • Update SCR_NO in ${TC}/bin/genBuildInfo
    • ${TC}/data
      • Modify ROOT_DIR=AUTO_MODE in ${TC}/data/Config/tc.properties
    • ${TC}/docs
      • Modify ${YEAR} in ${TC}/docs/updateDoc
    • Example Codes
      • Update ${TC_YEAR} in {TC_EXAMPLE}/bin/runExample
12. Installation Test
    • Update ${TC}/install/Msg/jdiGold.txt (for the new results)
      => This needs to be done after the new database is reloaded.
13. Compile & Pack
    • shell> cd ${TC}
    • Update ${TC}/genBuildInfo

    • shell> ant clean
    • shell> ant release
    • shell> cd ..
    • shell> gtar -czvf tc${YEAR}.tgz tc${YEAR}
14. Update web site
    • Update documents
      • change ${PRE_YEAR} to ${YEAR} on ${TC_WEB}/${YEAR}/Home/topMenu*.html
      • change ${PRE_YEAR} to ${YEAR} on ${TC_WEB}/${YEAR}/web/interactiveTools.html
15. Test
    • Install TC.{YEAR} to ${PROJECTS}
      • shell> mv tc.{YEAR}.tgz to ${PROJECTS}/TC
      • shell> gtar -xzvf tc.{YEAR}.tgz
      • shell> cd ${PROJECTS}/TC/tc${YEAR}
      • shell> ./install/bin/install_linux
    • Add links to previous data sets
      • ln -sf /export/home/lu/Development/TC/tcData/data.2007 data.2007
      • ln -sf /export/home/lu/Development/TC/tcData/data.2008 data.2008
      • ln -sf /export/home/lu/Development/TC/tcData/data.2009 data.2009
      • ln -sf /export/home/lu/Development/TC/tcData/data.2010 data.2010
      • ln -sf ./data data.2011
    • Test all programs for all datasets
      • Test all programs: jdi, sti, stri, stWst, mlt
      • Test all versions: -rv:${YEAR}
    • Test JDI dataset similarity
    • Test STI by WSD collection data set
16. Update Web Applications
    • Web Tools:
    • TCAT: