Lexical Tools - CI-CD Processes

This pages describes the high-level CI-CD commands and processes on Lexical Tools annual releases

• Files
  • .gitlab-ci.yml
    => defines CI-CD pipeline process - build, packLite, packFull
  • Makefile
    => a file called by ./gitlab-ci to define tasks on all phases in the CI-CD pipeline
    • build: use maven to build lvg
    • packLite: pack lite (min.) Lvg to lvgLite${YEAR}.tgz
    • packFull: pack full Lvg to lvg${YEAR}.tgz
  • pom.xml
    => used by maven to build the software
  • build.xml
    => used by ant to build the software (used in the development build)

• Variables
  • pom.xml
    • version:
      • use ${YEAR}.0-SNAPSHOT for snapshot build
      • use ${YEAR}.0 for official release build
        => delete artifact (*.jar) of ${YEAR}.0 in Nexus -> lhc-lexicon-maven-release -> gov -> nih -> nlm -> lvg -> ${YEAR}.0 (before use mvn for release build)
        => for snapshot development build is ok
  • packLite & packFull:
    • ${VERSION}
      • updated in the Makefile
    • Other enviroment variables. For use Git and CiCd, they are deinfed in the environment variables (for linux) or in Git-> Lexicon -> Setting -> CI/CD -> Variables -> Reveal Values

      Variable name	function	Notes
      ${LVG_PACK}	pack lvg${YEAR}lite.tgz and lvg${YEAR}.tgz	TRUE: default
      ${LVG_UPLOAD}	upload to Nexus	FALSE: development (default) TRUE: release Used for Website download and WebLvg if the *.tgz files exist in the Nexus, the upload will be failed. delete artifact (*.tgz) of ${YEAR} in Nexus -> lhc-lexicon-raw -> www -> lvg -> ${YEAR} (before use mvn for release build) delete artifacts of the *.jar (release files) for deploying the same release version. for snapshot development build is ok
      ${LVG_MVN_SNAPSHOT}	snapshot build	TRUE: development (default) FALSE: release
      ${LVG_MVN_JAR}	Use mvn to build	TRUE: maven build FALSE: ant build (default)

  • Quick CiCd variable setup
    • used for packLvgFull and packLvgLite
    • Development: used in command line
    • Production: change in lhc-git -> Setting -> CD/CD -> Variables

    Variable name	Dev-Ant	Dev-Mvn	Dev-Make all	Snapshot Default	Release
    ${LVG_PACK}	FALSE	FALSE	TRUE	FALSE	TRUE
    ${LVG_UPLOAD}	FALSE	FALSE	FALSE	FALSE	TRUE
    ${LVG_MVN_SNAPSHOT}	TRUE	TRUE	TRUE	TRUE	FALSE
    ${LVG_MVN_JAR}	FASLE	TRUE	TRUE	TRUE	TRUE

• For the release:
  • build test (snapshot):
    • update version to ${YEAR}.o inpom.xml
    • shell> make build
      => ./target/lvg-${YEAR}.0-SNAPSHOT.jar
      => ./target/lvg-${YEAR}.0-SNAPSHOT-shaded.j
    • shell> packLvgFull ${YEAR} TRUE FALSE TURE TRUE
    • shell> make packFull
      => lvg${YEAR}.tgz
      • install to ${PROJECTS}
      • conduct unit tests
      • cp -p lvg2025.tgz lvg2025.tgz.mvn
    • shell> packLvgLite ${YEAR} TRUE FALSE TURE TRUE
    • shell> make packLite
      => lvg${YEAR}lite.tgz
      • conduct lite test
      • cp -p lvg2025lite.tgz lvg2025lite.tgz.mvn
  • Production build:
    • update version to ${YEAR}.o in pom.xml
    • update above variables in lhc-git -> Setting -> CD/CD -> Variables
    • delete the following assets in the Nexus before a new (final) Ci-Cd build
      • ${NEXUS_URL}/repository/lhc-lexicon-maven-releases/gov/nih/nlm/lvg/${YEAR}.0/
        • used for production release
        • lvg-${YEAR}.0.jar
        • ...
      • ${NEXUS_URL}/repository/lhc-lexicon-raw/www/lvg/${YEAR}/lvg${YEAR}
        • used for web applications
        • lvg${YEAR}.tgz
        • lvg${YEAR}lite.tgz
      • shell> git add -A
      • shell> git commit -m "LEX_.."
      • shell> git push origin

      • TBD .. donwload and test

• Git Porcesses
  • for both development and master branches in git
    • shell> git branch
      • check which branch is on
    • shell> git checkout develop
      • Use "develop" for development
      • development uses snapshot build
  • Built for snapshot: using commit and push
    • make sure version: ${YEAR}.0-SANPSHOT in pom.xml
    • make sure LVG_VAR is setup with no upload in Git.
    • shell> git add -A
    • shell> git commit -m "LEX_XXX, msg"
    • shell> git push

    • commit and push will trigger CI-CD.
      => check if the Git CI/CD pipeline pass
  • Merge the significant development and final offical release to master
    • shell> git checkout master
    • shell> git merge develop
  • Built for release: using commit and push
    • make sure version: ${YEAR}.0 in pom.xml
    • make sure LVG_VAR is setup with pack and upload in Git.
    • shell> git add -A
    • shell> git commit -m "LEX_XXX, msg"
    • shell> git push

    • commit and push will trigger CI-CD.
      => check if the Git CI/CD pipeline pass
      => check lhc-nexus for lhc-lexicon-maven-releases and lhc-lexicon-raw
  • Test
    • use script downloadFiles ${YEAR} TRUE to download:
      • lhc-nexus-lhc-lexicon-raw:lvg${YEAR}.tgz to ./download/lvg${YEAR}.tgz
        => unit test, platform test, performance test, etc.
      • lhc-nexus-lhc-lexicon-raw:lvgLite${YEAR}.tgz to ./download/lvgLite${YEAR}.tgz
        => lite test
  • Tag (Save) the significant development and final offical release to master
    • shell> git checkout master

    • shell> git tag -a v.${YEAR}.0.0.0 -m "msg" (tag the version)
      => tag for offical release (v.${YEAR}.0.0.0)
    • shell> git tag (show all tags)
    • shell> git tag -a v.${YEAR}.0.X -m "msg"
    • shell> git push origin tagName (push a tag to remote origin)
    • shell> git push origin -- tags (pushshow all tags to remote origin)