Database Update Processes
Liquibase
Liquibase Installation
- As of KFS 6, the impex tool has been deprecated in favor of liquibase.
- Configure the relevant properties in your local kfs-build.properties as desired.
If you are using Oracle XE and get the following error, "ORA-12519, TNS:no appropriate service handler found", or to avoid the error in the first place, run the following command and restart the listener:
alter system set PROCESSES=150 scope=SPFILE;
System Requirements
LiquiRelational
LiquiRelational is a tool in the main Financials application, and can be used to load all KFS and Rice data or just incremental updates. The tool can be run automatically on application startup or from the command line. This tool reads and executes liquibase xml files from the classpath.
Running LiquiRelational
To have LiquiRelational run automatically on application startup, set the updateDatabaseOnStartup
property to true
, and set the other properties to the values you want (see below).
To run LiquiRelational from the command line, the updateDatabaseStartup property is not required, but the other properties should be set to the values you want. Also, the classpath needs to include the kfs-datatools classes as well as the liquibase xml to run (the easiest way to do this is to just include the classes needed to run KFS) as well as a jdbc driver. For example:
java -classpath (Financials Directory)/kfs-web/target/kfs-web/WEB-INF/lib/*:(user home)/java/drivers/mysql-connector-java- 5.1 . 25 -bin.jar -Dadditional.kfs.config.locations=(user home)/liquirelational-override-config.properties org.kuali.kfs.sys.datatools.liquirelational.LiquiRelational |
Liquibase File Locations
Base structures and data are in the following locations in the financials project:
kfs-core/src/main/resources/org/kuali/kfs/core/db/phase1 - Core Tables and Sequences
kfs-core/src/main/resources/org/kuali/kfs/core/db/phase2 - Core Data
kfs-core/src/main/resources/org/kuali/kfs/core/db/phase3 - Core Primary Keys
kfs-core/src/main/resources/org/kuali/kfs/core/db/phase4 - Core Foreign Keys, Unique Constraints, Indexes and Views
kfs-core/src/main/resources/org/kuali/kfs/core/db/phase5 - Core database updates
Financial Optional module dliquibase files are located in each module. For example: kfs-ar/src/main/resources/org/kuali/kfs/module/ar/db/phaseX
kfs-core/src/main/resources/org/kuali/rice/db/phase1 - Rice Tables and Sequences
kfs-core/src/main/resources/org/kuali/rice/db/phase2 - Rice Data
kfs-core/src/main/resources/org/kuali/rice/db/phase3 - Rice Primary Keys
kfs-core/src/main/resources/org/kuali/rice/db/phase4 - Rice Foreign Keys, Unique Constraints, Indexes and Views
kfs-core/src/main/resources/org/kuali/rice/db/phase5 - Rice database updates
LiquiRelational Properties
The following properties control how LiquiRelational operates:
Property | Default | Description |
---|---|---|
updateDatabaseOnStartup | false | Determines if LiquiRelational will be run on application startup (true to run, false to not). |
updateDatabaseFullRebuild | false | Determines if LiquiRelational will process all liquibase files (true), or just updates (i.e. phase5) (false). |
updateDatabaseContext | bootstrap | Determines which context LiquiRelational will pass to liquibase to process (typically bootstrap for clean bootstrap database or demo to include demo data). |
updateDatabasePackages | org/kuali/kfs/core ,org/kuali/kfs/module/ar ,org/kuali/kfs/module/cam ,org/kuali/kfs/module/cg ,org/kuali/kfs/module/ec ,org/kuali/kfs/module/kc ,org/kuali/kfs/module/ld ,org/kuali/kfs/module/purap | Determines which packages will be processed for Financials liquibase files. By default, all modules are included. |
updateDatabasePackagesRice | org/kuali/rice | Determines which packages will be processed for Rice liquibase files. By default, all is included. |
Since LiquiRelational needs a database to operate on, the following properties are also used to build a DataSource. Most of these properties are also described in Configuration Properties.
Property | Default | Notes |
---|---|---|
Property | Default | Notes |
datasource.driver | com.mysql.jdbc.Driver | |
kfs.datasource.driver.name | ${datasource.driver} | |
kfs.datasource.validating.query | select 1 | |
kfs.datasource.pool.maxActive | 50 | |
kfs.datasource.pool.minIdle | 5 | |
kfs.datasource.pool.initialSize | 5 | |
kfs.datasource.pool.accessToUnderlyingConnectionAllowed | true | |
kfs.datasource.ddl.username | kfs | Separate from the normal kfs.datasource.username property in case that user doesn't have enough privileges to execute DDL. |
kfs.datasource.ddl.password | ${kfs.datasource.username} | Separate from the normal kfs.datasource.password property in case that user doesn't have enough privileges to execute DDL. |
kfs.datasource.ddl.url | jdbc:mysql://localhost/${kfs.datasource.username} | Separate from the normal kfs.datasource.url property in case that user doesn't have enough privileges to execute DDL. |
rice.datasource.driver.name | ${datasource.driver} | |
rice.datasource.validating.query | select 1 | |
rice.datasource.pool.maxActive | 50 | |
rice.datasource.pool.minIdle | 5 | |
rice.datasource.pool.initialSize | 5 | |
rice.datasource.pool.accessToUnderlyingConnectionAllowed | true | |
rice.datasource.ddl.url | ${kfs.datasource.ddl.url} | Separate from the normal rice.datasource.username property in case that user doesn't have enough privileges to execute DDL. |
rice.datasource.ddl.username | ${kfs.datasource.username} | Separate from the normal rice.datasource.password property in case that user doesn't have enough privileges to execute DDL. |
rice.datasource.ddl.password | ${kfs.datasource.password} | Separate from the normal rice.datasource.url property in case that user doesn't have enough privileges to execute DDL. |
Workflow Ingestion
Ingest Workflow XML
- Workflow xml is located in the following folders in the financials project:
kfs-core/src/main/resources/org/kuali/kfs/sys/workflow/parent
kfs-core/src/main/resources/org/kuali/kfs/sys/workflow/child
kfs-ar/src/main/resources/org/kuali/kfs/module/ar/workflow
kfs-cam/src/main/resources/org/kuali/kfs/module/cam/workflow
kfs-cg/src/main/resources/org/kuali/kfs/module/cg/workflow
kfs-ec/src/main/resources/org/kuali/kfs/module/ec/workflow
kfs-kc/src/main/resources/org/kuali/kfs/module/external/kc/workflow
kfs-ld/src/main/resources/org/kuali/kfs/module/ld/workflow
kfs-purap/src/main/resources/org/kuali/kfs/module/purap/workflow
- Workflow xml can be ingested through the KFS Application by one of the following methods:
- Archive the files in the above folders as a zip file. Log into KFS as a user able to ingest workflow document types and go to the XML Ingester screen (with foundation data, backdoor in as user "admin" and go to the Admin tab, configuration channel, under "Workflow"). Choose the zip file you've created, and press the "upload xml data" button.
- Alternatively, you can upload each of the XML files within the the above folders src/main/config/workflow individually. If you choose this course of action, you should load them in the order above.
As a third option, you can turn on the XmlPollerService. The XmlPollerService starts when KFS is started, and as long as the properties it depends upon on present, it will sit in the background and attempt to ingest any xml files put into the directory it listens to. What are those properties?
Property Name Description xml.pipeline.lifecycle.enabled
If set to "true", then the XmlPollerService will start to poll the data.xml.root.location for incoming XML files to ingest. data.xml.root.location
The directory which the XmlPollerService should check to find new workflow documents. data.xml.pending.location
The directory where XmlPollerService will place XML files which it finds, before it has a chance to process them. data.xml.completed.location
The directory where XmlPollerService places XML files which were successfully ingested. data.xml.problem.location
The directory where XmlPollerService places XML files which could not be successfully ingested. data.xml.pollIntervalSecs
The number of seconds XmlPollerService will wait before checking data.xml.root.location for new XML files to ingest. initialDelaySecs
The number of seconds XmlPollerService should wait before it starts processing files in the data.xml.root.location directory. Setting this to some amount of time - say, 30 seconds - is likely a good idea, as it will give KFS time to fully start before polling begins. Just add these to your runtime properties file, and any files added to data.xml.root.location will be ingested automatically.
In order to automatically ingest workflow xml on application startup, set the
updateWorkflowOnStartup
property to true and populate theworkflow.path
property with the workflow xml you wish to have ingested. This property can be set to a comma-separated list of files or locations. It can contain files or paths to read from the classpath ( prefix withclasspath:
) or the filesystem ( prefix withfile:
) and paths can contain wildcards. The files or paths should end in .xml.
For example:
workflow.path=classpath:org/kuali/kfs/sys/workflow/parent/*.xml,file:/opt/workflow/org/kuali/kfs/module/ar/workflow/*.xml |
To ingest all workflow xml (except KC):
workflow.path=classpath:org/kuali/kfs/sys/workflow/parent/*.xml,\ classpath:org/kuali/kfs/sys/workflow/child/*.xml,\ classpath:org/kuali/kfs/module/ar/workflow/*.xml,\ classpath:org/kuali/kfs/module/cam/workflow/*.xml,\ classpath:org/kuali/kfs/module/cg/workflow/*.xml,\ classpath:org/kuali/kfs/module/ec/workflow/*.xml,\ classpath:org/kuali/kfs/module/ld/workflow/*.xml,\ classpath:org/kuali/kfs/module/purap/workflow/*.xml kew.mode=LOCAL dev.mode=true
- Workflow ingestion can also be done via the command line by running the
org.kuali.kfs.sys.context.WorkflowImporter
class. The classpath needs to include the classes needed to run KFS.
- Workflow ingestion can also be done via the command line by running the
For example:
java -classpath /Users/user/IdeaProjects/financials/kfs-web/target/kfs-web/WEB-INF/lib/*:/Users/user/java/servers/apache-tomcat-7.0.59/lib/* -Dadditional.kfs.config.locations=/Users/user/kfs-override-config.properties org.kuali.kfs.sys.context.WorkflowImporter |
When running WorkflowIngestion with dev.mode=false and kew.mode=EMBEDDED vs. LOCAL, it is possible to see a stacktrace indicating that Rule Attributes were attempted to be ingested without a transaction. While disconcerting, this error can be ignored as the Rule Attributes are in fact persisted.
2015-10-21 10:57:45,782 [main] u:/d: ERROR org.apache.ojb.broker.core.PersistenceBrokerImpl :: No running tx found, please only store in context of an PB-transaction, to avoid side-effects - e.g. when rollback of complex objects
java.lang.Exception: ** Try to store object without active PersistenceBroker transaction **
at org.apache.ojb.broker.core.PersistenceBrokerImpl.store(Unknown Source)
at org.apache.ojb.broker.core.PersistenceBrokerImpl.store(Unknown Source)
at org.apache.ojb.broker.core.PersistenceBrokerImpl.store(Unknown Source)
at org.apache.ojb.broker.core.DelegatingPersistenceBroker.store(Unknown Source)
at org.apache.ojb.broker.core.DelegatingPersistenceBroker.store(Unknown Source)
at org.springmodules.orm.ojb.PersistenceBrokerTemplate$9.doInPersistenceBroker(PersistenceBrokerTemplate.java:246)
at org.springmodules.orm.ojb.PersistenceBrokerTemplate.execute(PersistenceBrokerTemplate.java:141)
at org.springmodules.orm.ojb.PersistenceBrokerTemplate.store(PersistenceBrokerTemplate.java:244)
at org.kuali.rice.kew.rule.dao.impl.RuleAttributeDAOOjbImpl.save(RuleAttributeDAOOjbImpl.java:32)
at org.kuali.rice.kew.rule.service.impl.RuleAttributeServiceImpl.save(RuleAttributeServiceImpl.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.cache.interceptor.CacheInterceptor$1.invoke(CacheInterceptor.java:58)
at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:213)
at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:66)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy472.save(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.cache.interceptor.CacheInterceptor$1.invoke(CacheInterceptor.java:58)
at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:213)
at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:66)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy473.save(Unknown Source)
at org.kuali.rice.kew.xml.RuleAttributeXmlParser.parseRuleAttributes(RuleAttributeXmlParser.java:90)
at org.kuali.rice.kew.xml.RuleAttributeXmlParser.parseRuleAttributes(RuleAttributeXmlParser.java:66)
at org.kuali.rice.kew.rule.service.impl.RuleAttributeServiceImpl.loadXml(RuleAttributeServiceImpl.java:140)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:196)
at com.sun.proxy.$Proxy472.loadXml(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:196)
at com.sun.proxy.$Proxy473.loadXml(Unknown Source)
at org.kuali.kfs.sys.context.WorkflowImporter.importWorkflow(WorkflowImporter.java:98)
at org.kuali.kfs.sys.context.WorkflowImporter.main(WorkflowImporter.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Kuali documentation is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License.
Kuali software is licensed for use pursuant to the Affero General Public License, version 3.
Copyright © 2014 Kuali, Inc. All rights reserved.
Portions of Kuali are copyrighted by other parties as described in the Acknowledgments screen.
Kuali ® is a registered trademark of the Trustees of Indiana University.