Refactor transformation of OQL to SQL queries (GSoC)

Table of contents

Project description

Student-programmer: Johannes Venzke

Mentor: Ralf Joachim

Uncommenting not working download links (C-3064)

First of all I uncommented some dead links at the download page of castor.

Testing and making oneself familiar with the application

I had an extensive session of testing. I ran through the tests in cpactf package and compared the results with the ODS Matrix (C-3075). Doing so I got a first expression about how the application works. Also I took a closer look at TestDefaultQueryVisitor, which deals with the persistence.sql.query classes to understand how they work. That is, what I'm going to include in the SQLRelationLoader. Tests 73 and 74 are also important for that.

Increase of test coverage and improvement of tests

Some tests missed CREATE or DROP scripts and I looked after it. Amongst other things I had the opportunity to get to run test39.TestStoredProcedure for the first time. The tests give validation for functionality of the whole program and I helped to increase coverage. 

Refactoring of SQLRelationLoader (C-3086)

Use of persistence.sql.query classes to represent constructed queries

Introduce CastorStatement and CastorConnection classes to handle query construction

Use ColumnInfos instead of SQLColumnInfo to bind parameters

Refactoring of ClassMolder and SQLEngine

Move SQL-related function of ClassMolder to SQLEngine. The creation of SQLRelationLoader has already been moved to SQLEngine with CASTOR-3087.

As a first step the move to SQLEngine was completed and improved by also moving "if statement" for many table decision and saving a parameter.

After that SQLRelationLoader retrieves all information in a newly created class called RelationTableInfo. This class is used as it represents the relation table and holds information about left and right foreign keys needed by SQLRelationLoader (CASTOR-3096). No other parameter is needed and no other procedure in SQLEngine. This is a way more elegant way to bind parameters. I had this in mind with CASTOR-3089, but the TableLink class has disappeared in the development process. It hasn't displayed n-to-n relation tables well.

Improve build-up of table and column hierarchy in InfoFactory

InfoFactory didn't create ColumnInfos well. Types and convertors were missing in some cases. This was fixed with C-3095 and C-3100.

Add field information to ColumnInfo class

After a lot of time has elapsed trying to replace the old column retrieval procedure for SQLRelationLoader in SQLEngine, it has turned out, that the information that was presently stored in ColumnInfo wasn't sufficient to find proper RelationTableInfo. That is to say we needed to add an additional field that stores the name of field in mapping to match correctly the field name of the stored foreign reference with the field name of the field descriptor.

This progress is included in the patch of CASTOR-3096

New QueryObjects for parsers

Objects are used to represent a SQL tree and real queries are generated automatically out of the tree structure. Refactorings of old code are being done to make process more extensible and easier to understand.

CastorParameter objects were added, that deal with parameters of the form "... WHERE $1 = 1000" and "$(int)2 = 5" to make the new CastorQLParser backward compatible (C-3117). These objects plus the following are the foundation of parsers to work well.

As a next step new QueryObjects should get created, that could represent queries with COUNT aggregation functions and translate it to simple SQL. We canceled this job in favor of the more important GSoC projects.


Old Lexer and Parser should be replaced with new ones. This new Parser, CastorQLParser, is in the starting blocks.

There were only problems of two kinds, that hindered them to be used:

There is one problem left, which has to be solved before the new parser can be activcated: 

To approach this, I first added some Tests to TestCastorQLParseTreeWalker and  to be able to view the current state of success.


The EJBQLParser should be brought to run. There is only one desired functionality that isn't supported at the moment. This is the simple COUNT aggregation function. 

A test was added to TestEJBQLParseTreeWalker to show current state of query support. We didn't continue the development.

Code cleanup

Meanwhile I'm going to resolve Checkstyle warnings in the cpa packages. Did it with CASTOR-3110. Also with CASTOR-3112

Add test for all required features to TestCastorQLTreeWalker

Tests are needed to validate all existing functionality of existing parser. As recently as the CastorQLParser supports all constructs that the old one supported, it will be used instead of the old one. 

I first created graphics that visualise the possible queries. Then I created tests for LIMIT and ORDER BY clause in C-3129

Ralf did the remaining tests.

Author tag and license information in cpa package (C-3146)

I had a look at all classes in package (except the classes in driver package) and created an result table as an overview what has to be done.

I created the JDO documentation in DocBook format. 

First I took the old pages and replaced tags that are functional same with the new ones.

Examples are: 














<emphasis role="bold">



The sections needed further adaptation and needed all an "id" for links and so.




<section title="Introduction">

<section id="jdo.howto.introduction">

Sometimes I also needed to rearrange the structure of the document to fit for DocBook.

Some tags also disappeared as there is no DocBook correspondent like


<font color="...">



I had to change some tags as the functionality isn't supported by DocBook. 




<api package="package">class</api>


The old api tag produced a link to the javaDoc reference. The new one only highlights the class.







Create Test case for property changes in extends hierarchy

There was a bug, that no fields of an extended table got stored. I created a test case in C-3183 that showed the bug.