Database Update Processes

Liquibase

Liquibase Installation

  • As of KFS 6, the impex tool has been deprecated in favor of liquibase.
  • Configure the relevant properties in your local kfs-build.properties as desired.

If you are using Oracle XE and get the following error, "ORA-12519, TNS:no appropriate service handler found", or to avoid the error in the first place, run the following command and restart the listener:

alter system set PROCESSES=150 scope=SPFILE;

System Requirements

LiquiRelational 


LiquiRelational is a tool in the main Financials application, and can be used to load all KFS and Rice data or just incremental updates. The tool can be run automatically on application startup or from the command line. This tool reads and executes liquibase xml files from the classpath.

Running LiquiRelational

To have LiquiRelational run automatically on application startup, set the updateDatabaseOnStartup property to true, and set the other properties to the values you want (see below).

To run LiquiRelational from the command line, the updateDatabaseStartup property is not required, but the other properties should be set to the values you want. Also, the classpath needs to include the kfs-datatools classes as well as the liquibase xml to run (the easiest way to do this is to just include the classes needed to run KFS) as well as a jdbc driver. For example:


java -classpath (Financials Directory)/kfs-web/target/kfs-web/WEB-INF/lib/*:(user home)/java/drivers/mysql-connector-java-5.1.25-bin.jar -Dadditional.kfs.config.locations=(user home)/liquirelational-override-config.properties org.kuali.kfs.sys.datatools.liquirelational.LiquiRelational

Liquibase File Locations

Base structures and data are in the following locations in the financials project:

kfs-core/src/main/resources/org/kuali/kfs/core/db/phase1 - Core Tables and Sequences

kfs-core/src/main/resources/org/kuali/kfs/core/db/phase2 - Core Data

kfs-core/src/main/resources/org/kuali/kfs/core/db/phase3 - Core Primary Keys

kfs-core/src/main/resources/org/kuali/kfs/core/db/phase4 - Core Foreign Keys, Unique Constraints, Indexes and Views

kfs-core/src/main/resources/org/kuali/kfs/core/db/phase5 - Core database updates

Financial Optional module dliquibase files are located in each module.  For example: kfs-ar/src/main/resources/org/kuali/kfs/module/ar/db/phaseX 

kfs-core/src/main/resources/org/kuali/rice/db/phase1 - Rice Tables and Sequences 

kfs-core/src/main/resources/org/kuali/rice/db/phase2 - Rice Data

kfs-core/src/main/resources/org/kuali/rice/db/phase3 - Rice Primary Keys

kfs-core/src/main/resources/org/kuali/rice/db/phase4 - Rice Foreign Keys, Unique Constraints, Indexes and Views

 

kfs-core/src/main/resources/org/kuali/rice/db/phase5 - Rice database updates

 

LiquiRelational Properties

The following properties control how LiquiRelational operates:

Property
Default
Description

updateDatabaseOnStartup

falseDetermines if LiquiRelational will be run on application startup (true to run, false to not).
updateDatabaseFullRebuildfalseDetermines if LiquiRelational will process all liquibase files (true), or just updates (i.e. phase5) (false).
updateDatabaseContextbootstrapDetermines which context LiquiRelational will pass to liquibase to process (typically bootstrap for clean bootstrap database or demo to include demo data).
updateDatabasePackages

org/kuali/kfs/core

,org/kuali/kfs/module/ar

,org/kuali/kfs/module/cam

,org/kuali/kfs/module/cg

,org/kuali/kfs/module/ec

,org/kuali/kfs/module/kc

,org/kuali/kfs/module/ld

,org/kuali/kfs/module/purap

Determines which packages will be processed for Financials liquibase files. By default, all modules are included.

updateDatabasePackagesRice

org/kuali/riceDetermines which packages will be processed for Rice liquibase files. By default, all is included.

Since LiquiRelational needs a database to operate on, the following properties are also used to build a DataSource. Most of these properties are also described in Configuration Properties.

Property
Default
Notes
Property
Default
Notes
datasource.drivercom.mysql.jdbc.Driver
kfs.datasource.driver.name${datasource.driver}
kfs.datasource.validating.queryselect 1
kfs.datasource.pool.maxActive

50


kfs.datasource.pool.minIdle5
kfs.datasource.pool.initialSize5
kfs.datasource.pool.accessToUnderlyingConnectionAllowedtrue
kfs.datasource.ddl.usernamekfsSeparate from the normal kfs.datasource.username property in case that user doesn't have enough privileges to execute DDL.
kfs.datasource.ddl.password${kfs.datasource.username}Separate from the normal kfs.datasource.password property in case that user doesn't have enough privileges to execute DDL.
kfs.datasource.ddl.urljdbc:mysql://localhost/${kfs.datasource.username}Separate from the normal kfs.datasource.url property in case that user doesn't have enough privileges to execute DDL.
rice.datasource.driver.name${datasource.driver}
rice.datasource.validating.queryselect 1
rice.datasource.pool.maxActive50
rice.datasource.pool.minIdle5
rice.datasource.pool.initialSize5
rice.datasource.pool.accessToUnderlyingConnectionAllowedtrue

rice.datasource.ddl.url

${kfs.datasource.ddl.url}Separate from the normal rice.datasource.username property in case that user doesn't have enough privileges to execute DDL.
rice.datasource.ddl.username${kfs.datasource.username}Separate from the normal rice.datasource.password property in case that user doesn't have enough privileges to execute DDL.
rice.datasource.ddl.password${kfs.datasource.password}Separate from the normal rice.datasource.url property in case that user doesn't have enough privileges to execute DDL.


Workflow Ingestion

Ingest Workflow XML

  • Workflow xml is located in the following folders in the financials project:
    • kfs-core/src/main/resources/org/kuali/kfs/sys/workflow/parent

    • kfs-core/src/main/resources/org/kuali/kfs/sys/workflow/child

    • kfs-ar/src/main/resources/org/kuali/kfs/module/ar/workflow

    • kfs-cam/src/main/resources/org/kuali/kfs/module/cam/workflow

    • kfs-cg/src/main/resources/org/kuali/kfs/module/cg/workflow

    • kfs-ec/src/main/resources/org/kuali/kfs/module/ec/workflow

    • kfs-kc/src/main/resources/org/kuali/kfs/module/external/kc/workflow

    • kfs-ld/src/main/resources/org/kuali/kfs/module/ld/workflow

    • kfs-purap/src/main/resources/org/kuali/kfs/module/purap/workflow

  • Workflow xml can be ingested through the KFS Application by one of the following methods:
    • Archive the files in the above folders as a zip file.  Log into KFS as a user able to ingest workflow document types and go to the XML Ingester screen (with foundation data, backdoor in as user "admin" and go to the Admin tab, configuration channel, under "Workflow").  Choose the zip file you've created, and press the "upload xml data" button.
    • Alternatively, you can upload each of the XML files within the the above folders src/main/config/workflow individually.  If you choose this course of action, you should load them in the order above.
    • As a third option, you can turn on the XmlPollerService.  The XmlPollerService starts when KFS is started, and as long as the properties it depends upon on present, it will sit in the background and attempt to ingest any xml files put into the directory it listens to.  What are those properties?

      Property NameDescription
      xml.pipeline.lifecycle.enabled
      If set to "true", then the XmlPollerService will start to poll the data.xml.root.location for incoming XML files to ingest.
      data.xml.root.location
      The directory which the XmlPollerService should check to find new workflow documents.
      data.xml.pending.location
      The directory where XmlPollerService will place XML files which it finds, before it has a chance to process them.
      data.xml.completed.location
      The directory where XmlPollerService places XML files which were successfully ingested.
      data.xml.problem.location
      The directory where XmlPollerService places XML files which could not be successfully ingested.
      data.xml.pollIntervalSecs
      The number of seconds XmlPollerService will wait before checking data.xml.root.location for new XML files to ingest.
      initialDelaySecs
      The number of seconds XmlPollerService should wait before it starts processing files in the data.xml.root.location directory.  Setting this to some amount of time - say, 30 seconds - is likely a good idea, as it will give KFS time to fully start before polling begins.

      Just add these to your runtime properties file, and any files added to data.xml.root.location will be ingested automatically.

    • In order to automatically ingest workflow xml on application startup, set the updateWorkflowOnStartup property to true and populate the workflow.path property with the workflow xml you wish to have ingested. This property can be set to a comma-separated list of files or locations. It can contain files or paths to read from the classpath ( prefix with classpath: ) or the filesystem ( prefix with file: ) and paths can contain wildcards. The files or paths should end in .xml.

For example:


workflow.path=classpath:org/kuali/kfs/sys/workflow/parent/*.xml,file:/opt/workflow/org/kuali/kfs/module/ar/workflow/*.xml

To ingest all workflow xml (except KC):


workflow.path=classpath:org/kuali/kfs/sys/workflow/parent/*.xml,\
  classpath:org/kuali/kfs/sys/workflow/child/*.xml,\
  classpath:org/kuali/kfs/module/ar/workflow/*.xml,\
  classpath:org/kuali/kfs/module/cam/workflow/*.xml,\
  classpath:org/kuali/kfs/module/cg/workflow/*.xml,\
  classpath:org/kuali/kfs/module/ec/workflow/*.xml,\
  classpath:org/kuali/kfs/module/ld/workflow/*.xml,\
  classpath:org/kuali/kfs/module/purap/workflow/*.xml



kew.mode=LOCAL
dev.mode=true
    • Workflow ingestion can also be done via the command line by running the org.kuali.kfs.sys.context.WorkflowImporter class. The classpath needs to include the classes needed to run KFS.

For example:

java -classpath /Users/user/IdeaProjects/financials/kfs-web/target/kfs-web/WEB-INF/lib/*:/Users/user/java/servers/apache-tomcat-7.0.59/lib/* -Dadditional.kfs.config.locations=/Users/user/kfs-override-config.properties org.kuali.kfs.sys.context.WorkflowImporter
In Baseline Financials, this process only applies to Rule Attributes and Document Types which are the only types of workflow xml included in baseline.

When running WorkflowIngestion with dev.mode=false and kew.mode=EMBEDDED vs. LOCAL, it is possible to see a stacktrace indicating that Rule Attributes were attempted to be ingested without a transaction. While disconcerting, this error can be ignored as the Rule Attributes are in fact persisted.


2015-10-21 10:57:45,782 [main] u:/d: ERROR org.apache.ojb.broker.core.PersistenceBrokerImpl :: No running tx found, please only store in context of an PB-transaction, to avoid side-effects - e.g. when rollback of complex objects
java.lang.Exception: ** Try to store object without active PersistenceBroker transaction **
at org.apache.ojb.broker.core.PersistenceBrokerImpl.store(Unknown Source)
at org.apache.ojb.broker.core.PersistenceBrokerImpl.store(Unknown Source)
at org.apache.ojb.broker.core.PersistenceBrokerImpl.store(Unknown Source)
at org.apache.ojb.broker.core.DelegatingPersistenceBroker.store(Unknown Source)
at org.apache.ojb.broker.core.DelegatingPersistenceBroker.store(Unknown Source)
at org.springmodules.orm.ojb.PersistenceBrokerTemplate$9.doInPersistenceBroker(PersistenceBrokerTemplate.java:246)
at org.springmodules.orm.ojb.PersistenceBrokerTemplate.execute(PersistenceBrokerTemplate.java:141)
at org.springmodules.orm.ojb.PersistenceBrokerTemplate.store(PersistenceBrokerTemplate.java:244)
at org.kuali.rice.kew.rule.dao.impl.RuleAttributeDAOOjbImpl.save(RuleAttributeDAOOjbImpl.java:32)
at org.kuali.rice.kew.rule.service.impl.RuleAttributeServiceImpl.save(RuleAttributeServiceImpl.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.cache.interceptor.CacheInterceptor$1.invoke(CacheInterceptor.java:58)
at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:213)
at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:66)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy472.save(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.cache.interceptor.CacheInterceptor$1.invoke(CacheInterceptor.java:58)
at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:213)
at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:66)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy473.save(Unknown Source)
at org.kuali.rice.kew.xml.RuleAttributeXmlParser.parseRuleAttributes(RuleAttributeXmlParser.java:90)
at org.kuali.rice.kew.xml.RuleAttributeXmlParser.parseRuleAttributes(RuleAttributeXmlParser.java:66)
at org.kuali.rice.kew.rule.service.impl.RuleAttributeServiceImpl.loadXml(RuleAttributeServiceImpl.java:140)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:196)
at com.sun.proxy.$Proxy472.loadXml(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:196)
at com.sun.proxy.$Proxy473.loadXml(Unknown Source)
at org.kuali.kfs.sys.context.WorkflowImporter.importWorkflow(WorkflowImporter.java:98)
at org.kuali.kfs.sys.context.WorkflowImporter.main(WorkflowImporter.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

 

 

Kuali documentation is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License. 

Kuali software is licensed for use pursuant to the Affero General Public License, version 3.

 Copyright © 2014 Kuali, Inc. All rights reserved. 

Portions of Kuali are copyrighted by other parties as described in the Acknowledgments screen. 

Kuali ® is a registered trademark of the Trustees of Indiana University.