XMCCat Server Build and Installation

Back to the XMC Cat Table of Contents

Which Code to Use?

The source code is available from the the XMC Cat SourceForge project: https://sourceforge.net/projects/xmccat/

To download a released version, click on the Files tab in SourceForge and then click on the XMC Cat Source directory. You can download the tarball to the directory of your choice, untar it, and proceed to the instructions on building XMC Cat using Maven2. When you unzip the tarball there will be a single directory named xmccat-server. In the remainder of this documentation this directory is referred to as <source>.

If you want to use the latest source code, click on the Code tab instead of the Files tab on the XMC Cat page in SourceForge. If you want to checkout everything about XMC Cat, you can use the svn command shown on the code page. However, this will get you all of the prior versions in the tags directory, any on-going branches, and both the XMC Cat server and GUI code. If you really just want the latest source code, click on the trunk tag in the tree. You will see two subdirectories

* xmccat-server
* xmccat-gui

Although probably obvious, if you want to build the server, check out the xmccat-server branch. At the command line, you can checkout XMC Cat with the following svn command:

 svn checkout svn://svn.code.sf.net/p/xmccat/code/trunk/xmccat-server

The above command will create an xmccat-server folder in the directory where you execute the svn command. In the remainder of this documentation, we refer to this directory as <source>. Please do not rename this directory when you checkout the code. The POM used to build the code using Maven2 expects that it is being built in a directory named xmccat-server.

Prior versions of XMC Cat can also be downloaded from the Data to Insight web page for the XMC Cat project: http://pti.iu.edu/d2i/xmc-cat-downloads

Building XMC Cat Using Maven2

Once it is unzipped, XMC Cat can be compiled using Maven2 (please see the prerequisites page). With both Java and Maven2 in your classpath, switch to the <source> directory and type the following command to use Maven2 to build XMC Cat:

 mvn -e

The first time that you compile XMC Cat, you will need to be online so Maven2 can download the dependencies for the build. By default, Maven2 will download the necessary jars to the repository hierarchy of the .m2 directory contained within your home directory (Maven2 will automatically create the .m2 directory if it does not exist). Depending on the speed of your network connection, the first build will take much longer than subsequent builds on a slow internet connection.

Configuring Your Environment

When running the tomcat server for XMC Cat, the following environment variables should be set:

JAVA_HOME should be set to the root directory of your jdk installation.

MYSQL_HOME should be set to the root directory of your MySQL installation (the directory referred to as <mysql> in this guide).

CATALINA_HOME and CATALINA_BASE should both be set to the directory for the Tomcat installation you are using for XMC Cat (the directory referred to as <tomcat> in this guide).

Additionally, the following JAVA_OPTS environment variable should be set to enable logging in XMC Cat using Log4j (where "path" is replaced by the path to the log4j.properties file used for logging in XMC Cat):

   -Dcatalog.log4j=path/log4j.properties

You can define your own log4j.properties file (and you can use a different name, but be sure to also use that name in the above environment setting). A sample log4j properties file is available in the <source>/xmccat-service directory of the source tarball. This sample file can be used "as is" with default settings if you wish.

Creating the XMC Cat Database in MySQL

The following assumes that you have MySQL installed and that you are using the default MySQL port of 3306 (you are if you did not change it). Log into MySQL as the root user and create a database named xmccat using the following command:

  CREATE DATABASE xmccat;

The rest of these instructions will assume you have used the default name "xmccat" for the database, but you can use whatever name you wish. We will point out where you would need to change the name in the configuration files if you use a different name.

Create a user account on MySQL that will be used to access the database from XMC Cat. To create the user account from the command line, log into MySQL as the root user and execute the following commands (replacing "xmccatuser" with the name of the user wish to create for XMC Cat to access the database and "catpw" should be replaced by the password you wish to use for your xmccat user).

The third GRANT statement is needed to allow stored procedures to be run.

  GRANT ALL ON xmccat.* TO 'xmccatuser'@'%' IDENTIFIED BY 'catpw';
  GRANT ALL ON xmccat.* TO 'xmccatuser'@'localhost' IDENTIFIED BY 'catpw';
  GRANT SELECT ON mysql.proc TO 'xmccatuser'@'localhost' IDENTIFIED BY 'catpw';

The user account and password you use here will need to added to configuration files for XMC Cat, so be sure you do not use a personal password that you use for other systems.

The XMC Cat database is empty at this point (no tables have been defined). The script used to load the relational database schema for XMC Cat (xmccat_db.sql) is in the <source>/xmccat-service/database directory. to load the schema, type the following at the command line (replacing xmccatuser with the xmccat user you set up in the prior step).

  mysql -u xmccatuser -p xmccat < xmccat_db.sql

When you execute the above command, you will be prompted for the password for your xmccat user. This script installs the table definitions. in addition, there is some domain-independent data that must be loaded. This data is loaded by running the following command (the xmccat_domain_independent_data.sql file is also in the <source>/xmccat-service/database directory):

  mysql -u xmccatuser -p xmccat < xmccat_domain_independent_data.sql

To complete the XMC Cat database installation, the stored procedures must be loaded. The stored procedure definitions are contained in the <source>/xmccat-service/SP2Load folder in the source and are loaded by running the LoadStoredproc.sh script contained in that directory. That script has 3 required parameters: the database name, the database user name, and the database password for the user. In addition, if you need to specify the path to your MySQL installation when running from the command line, then a fourth parameter with that path is also required:

./LoadStoredProc.sh xmccat xmccatuser catpw /usr/local/mysql/bin

In the above, "xmccatuser" and "catpw" would be replaced by the username and password you specified above when creating the database. The first parameter is the name of the database (here we assume you used xmccat as the database name as specified above). the fourth parameter (/usr/local/mysql/bin) is generally NOT needed and should only be used if you to specify the path to your mysql installation - please note that the slash after the bin directory is omitted).

Setting up the Database Connection Pools:

XMC Cat uses Apache Tomcat's JNDI for pooling database connections, so the database resource needs to be defined for XMC Cat and the resource reference must be added to the web.xml for the Axis2 installation. To define the resource, create a text file named axis2.xml with the following content in the <tomcat>/conf/Catalina/localhost directory (the text below can be copy/pasted into a text file):

 <?xml version="1.0" encoding="UTF-8"?>
   <Context docbase="${catalina.home}/webapps/axis2">
     <Resource name="jdbc/XMCINSERT" 
              auth="Container" 
              type="javax.sql.DataSource" 
              maxActive="20"
              maxIdle="5"
              validationQuery="SELECT 1"
              testOnBorrow="true"
              testWhileIdle="true"
              timeBetweenEvictionRunsMillis="60000"
              minEvictableIdleTimeMillis="300000"
              username="xmccatuser"
              password="catpw"
              driverClassName="com.mysql.jdbc.Driver"
              url="jdbc:mysql://localhost:3306/xmccat"/>
     <Resource name="jdbc/XMCQUERY" 
              auth="Container" 
              type="javax.sql.DataSource" 
              maxActive="20"
              maxIdle="5"
              validationQuery="SELECT 1"
              testOnBorrow="true"
              testWhileIdle="true"
              timeBetweenEvictionRunsMillis="60000"
              minEvictableIdleTimeMillis="120000"
              username="xmccatuser"
              password="catpw"
              driverClassName="com.mysql.jdbc.Driver"
              url="jdbc:mysql://localhost:3306/xmccat"/>
   </Context>

The username and password parameters in both resource definitions must be changed from "xmccatuser" and "catpw" to the database user name and password you defined above for the XMC Cat user above. A copy of the above text file is in the <source>/xmccat-service/database directory and can be copied to your <tomcat>/conf/Catalina/localhost directory (but be sure to update the username and password settings for your xmccat user).

If you named your database something other than "xmccat", then you will also need to replace "xmccat" in the url parameter in both resource definitions. Likewise, if your MySQL database is not running on the MySQL default port 3306, then you will also need to replace "3306" in the url parameters.

Add the following resource reference to the <axis2>/WEB-INF/web.xml file (just after the display-name near the top of the web.xml file, but before the first servlet element):

  <resource-ref>
      <description>DB Connection</description>
      <res-ref-name>jdbc/XMCINSERT</res-ref-name>
      <res-type>javax.sql.DataSource</res-type>
      <res-auth>Container</res-auth>
  </resource-ref>
  <resource-ref>
      <description>DB Connection</description>
      <res-ref-name>jdbc/XMCQUERY</res-ref-name>
      <res-type>javax.sql.DataSource</res-type>
      <res-auth>Container</res-auth>
  </resource-ref>

The JNDI database connection pooling also requires that the JDBC connector jar (mysql-connector-java-3.1.11-bin.jar) be copied to the <tomcat>/common/lib/ directory. This jar can be copied from the <source>/xmccat-service/target/aar/lib directory after the XMC Cat build is run. In version 6 and later of tomcat the <tomcat>/common/lib/ directory is instead the <tomcat>/lib/ directory.

Installing The Binary

The XMC Cat build process creates an Axis2 archive file (.aar) for theXMC Cat service, and such archives can be deployed in the <axis2>/WEB-INF/services folder. However, to be able to adjust some of the configuration settings for XMC Cat, the archive needs to be expanded. First create a directory named CatalogService within the <axis2>/WEB-INF/services folder. In the build process, the Axis2 archive file for the XMC Cat server (xmccat-service-2.x.aar) is created in the <source>/xmccat-service/target folder. Copy this archive file into the <axis2>/WEB-INF/services/CatalogService folder you just created and then unzip the archive:

  unzip xmccat-service-2.x.aar

After unzipping the archive, the CatalogService directory will contain three directories: edu, lib, and META-INF. The xmccat-service-1.x.x.aar archive file should then be deleted from the CatalogService directory. In the remainder of this install guide, the <axis2>/WEB-INF/services/CatalogService directory will be referred to as <xmccat>.

Enabling Streaming Responses from XMC Cat:

Query results can be returned by XMC Cat as a stream instead of in the body of the web service response (at the discretion of the user calling the XMC Cat web service). To be able to stream back results requires installing the XMC Cat streaming delivery servlet. When the XMC Cat archive file was compiled using Maven2, the jar for the delivery servlet was automatically built in the <source>/xmccat-catalog-delivery-servlet/target directory as a jar named xmccat-catalog-delivery-servlet-1.X.X.jar. This jar and the JDBC connector jar for MySQL (mysql-connector-java-3.1.11-bin.jar) must be copied to the <axis2>/WEB-INF/lib directory. The JDBC connector jar can be copied from the <xmccat>/lib directory of the server installation. Additionally, Axis2 must be told about this servlet by adding the following two sections to the <axis2>/WEB-INF/web.xml file.

The following servlet should be added after the last existing servlet in the web.xml file:

 <servlet>
     <servlet-name>XMCCatServlet</servlet-name>
     <display-name>XMCCat Streaming Servlet</display-name>
     <servlet-class>edu.indiana.dde.catalog.delivery.CatalogDeliveryServlet</servlet-class>
     <init-param>
         <param-name>catalog.log4j</param-name>
         <param-value>/usr/XMCCat/CatalogService/log4j.properties</param-value>
     </init-param>
     <init-param>
          <param-name>xmccat</param-name>
          <param-value>1</param-value>
     </init-param>
     <init-param>
         <param-name>catalog.db</param-name>
         <param-value>jdbc:mysql://localhost:3306/xmccat</param-value>
     </init-param>
     <init-param>
         <param-name>user.name</param-name>
         <param-value>xmccatuser</param-value>
     </init-param>
     <init-param>
         <param-name>user.pw</param-name>
         <param-value>catpw</param-value>
     </init-param>
     <load-on-startup>2</load-on-startup>
 </servlet>

A few modifications need to be made to the above servlet snippet to reflect your configuration:

The catalog.log4j parameter's value must be modified to reflect the path to your log4j.properties file instead of the /usr/XMCCat/CatalogService/log4j.properties path included in the above snippet.

The user.name parameter's value must be changed from "xmccatuser" to the user name created in your MySQL database as the user for the xmccat database. We strongly discourage using the root MySQL user.

The user.pw parameter's value must be changed from "catpw" to the password of the MySQL database user specified by the user.name parameter.

If your database is not named "xmccat", the value for the catalog.db parameter must be updated to your database's name instead of "xmccat". Likewise, if your MySQL database is not running on the default port of 3306, then you also need to replace the port number in the value for the catalog.db.

The following servlet-mapping should be added after the last existing servlet-mapping entry in the web.xml file:

 <servlet-mapping>
     <servlet-name>XMCCatServlet</servlet-name>
     <url-pattern>/servlet/CatalogStreamDeliveryServlet</url-pattern>
 </servlet-mapping>

Configuring XMC Cat

In the <xmccat>/META-INF directory there is a file named services.xml that contains parameters that are used to configure XMC Cat. Most of the parameters can be left at their default values. There are comments in the file that describe the purpose of each parameter. However, the following 7 parameters must be updated when you install XMC Cat:

host: The default setting is "localhost", which is fine if you are running on your laptop or desktop to try out XMC Cat. Otherwise it should be set to the name of your server (e.g., myserver.myschool.edu).

port: The default setting is 8080 which is the default port for a Tomcat server. If you have configured your server to run on a different port, enter that port number.

serviceUrl: This is the full URL to the XMC Cat web service. The host and port in this URL need to be updated for the values you set for the host and port parameters. If you followed the standard XMC Cat installation, the rest of the URL should not be modified.

deliveryUrl: Similar to the serviceUrl parameter, the host and port need to be updated. The rest of the URL should not be modified.

dbuser: The name of the database user you defined for your xmccat database in MySQL should be entered here.

dbpw: The MySQL password for the xmccat user specified for the dbuser parameter.

log4jDefaultPath: This should be set to the path where your log4j.properties file is located. There is also log4jPathProperty parameter, but if you set the JAVA_OPTS to "catalog.log4j" in your environment settings as specified above for setting up the XMC Cat environment, then that parameter does not need to be modified.

Database not named xmccat? If you did not name your database "xmccat", or your MySQL installation is not using the default port 3306, then you will also need to update the dburl parameter.

Configuring XMC Cat for Your Metadata Schema

To configure XMC Cat for your schema, you need the following:
1. An XML Bean jar for your metadata schema (could be multiple jars)
2. A database script to load to load definitions for concepts in your schema and other configuration settings.
3. XSLT templates for shredding the metadata as XML into concepts and elements.

To assist in creating the above files, there is a beta version of a wizard for walking through this process based on XML metadata schemata:

  https://bitternut.cs.indiana.edu:10081/builder

The XMC Cat builder for creating the metadata schema configuration files is currently being updated for XMC Cat version 2.0.

The XML Bean jar(s) for your schema should be put in the <xmccat>/lib folder. Additionally, you can create a jar that is called to configure a new user's workspace. This was used in the LEAD project, and the jars for that optional feature also need to be put in the <xmccat>/lib folder.

If you are implementing XMC Cat to handle mutiple schemata simultaneously, we recommend not using the default workspace option since it would be tied to one of the schemas.

The XSLT templates (which have a .xsl file extension) should be put in the following directory:

  <xmccat>/edu/indiana/dde/catalog/catalogservice/

In the XMC Cat database, each metadata schema registered is assigned an integer ID. If you are creating a configuration for your specific schema, please use an ID greater than 1000. We will be posting sets of pre-configured files for setting up metadata schemata in XMC Cat.

You can setup your XMC Cat to handle multiple schemata simultaneously (version 2.0 and later).

Pre-Configured Schemas

Currently we are making two schemata available pre-configured for XMC Cat, the metadata schema based on the FGDC standard defined by the Federal Geographic Data Committee. This is based on an implementation of the standard that is available from NOAA as a set of XML schemas. The second schema for which we currently make configuration files available is the LEAD Metadata Schema (LMS) that is a profile and extension of the FGDC schema that was defined by the Linked Environments for Atmospheric Discovery scientific gateway.

LEAD Schema

To register and use the LEAD schema with your XMC Cat installation, you only need to:
1) Download and install the jars for the LEAD metadata schema, and
2) Load the LEAD-specific database script.

Installing the LEAD jars:
From the "LEAD-Specific Jars and Files for XMC Cat Version 2.X" section of the XMC Cat download page download the following files and put them in the <xmccat>/lib folder:

  lead-metadata-1.8.1.jar (the XML Bean for the LEAD Metadata Schema)
  lead-initial-workspace-1.2.4.jar(the LEAD-specific default project created for each new user)
  lead-metadata-util-1.6.jar (used by the initial workspace jar)

Installing the Database Script and Registering the Schema:
The LEAD Metadata Schema has been assigned the ID "1" and the following file contains all of the database configuration data including the entry to register the LEAD metadata schema in the xmccat database: LEAD schema database script (All_LEAD_metadata_concepts.txt)

Download this script and install it from the command line (outside of MySQL) using the following command:

  mysql -u xmccatuser -p xmccat < All_LEAD_metadata_concepts.txt

In the above command, the "xmccatuser" should be replaced by the database user name you created for your XMC Cat installation. If your XMC Cat database is not named "xmccat", then replace "xmccat" in the above command with the name of your database. If you are not executing this command from the directory containing the All_LEAD_metadata_concepts.txt script, you will also need to include the path to the file.

XSLT Templates for Shredding Metadata:
The XSLT templates for the LEAD schema (myLeadCreateObject.xsl, myLeadAddProperty.xsl, and myLeadUpdateProperty.xsl), are available on the XMC Cat downlad page but there is no need to copy them to XMC Cat in that they are already included in the default XMC Cat server installation.

FGDC Schema

To register and use the schema with your XMC Cat installation, you only need to:
1) Download and install the jar for the FGDC metadata schema, and
2) Load the FGDC-specific database script.

Installing the FGDC Schema XML Bean jar:
From the "FGDC-Specific Jars and Files for XMC Cat Version 2.X" section of the XMC Cat download page download the following jar file and put it in the <xmccat>/lib folder:

  d2i-fgdc-schema-1.0.0.jar (the XML Bean for the FGDC Metadata Schema)

This XML Bean jar is based on a NOAA version of the FGDC schema that does not use namespaces.

Installing the Database Script and Registering the Schema:
The FGDC Metadata Schema has been assigned the ID "2" and the following file contains all of the database configuration data and the entry to register the FGDC metadata schema in the xmccat database: FGDC schema database script

Download this script and install it from the command line (outside of MySQL) using the following command:

  mysql -u xmccatuser -p xmccat < fgdc_db_script.txt

In the above command, the "xmccatuser" should be replaced by the database user name you created for your XMC Cat installation. If your XMC Cat database is not named "xmccat", then replace "xmccat" in the above command with the name of your database. If you are not executing this command from the directory containing the fgdc_db_script.txt script, you will also need to include the path to the file.

XSLT Templates for Shredding Metadata:
The XSLT templates for the FGDC schema (fgdcCreate.xsl, fgdcAdd.xsl, and fgdcUpdate.xsl), are available on the XMC Cat downlad page but there is no need to copy them to XMC Cat in that they are already included in the default installation.