Refactor transformation of OQL to SQL queries (GSoC)
Table of contents:
|Table of Contents|
Student-programmer: Johannes Venzke
Mentor: Ralf Joachim
Uncommenting not working download links (C-3064)
First of all I uncommented some dead links at the download page of castor.
Testing and making oneself familiar with the application
I had an extensive session of testing. I ran through the tests in cpactf package and compared the results with the ODS Matrix (C-3075). Doing so I got a first expression about how the application works. Also I took a closer look at TestDefaultQueryVisitor, which deals with the persistence.sql.query classes to understand how they work. That is, what I'm going to include in the SQLRelationLoader. Tests 73 and 74 are also important for that.
Increase of test coverage and improvement of tests
Some tests missed CREATE or DROP scripts and I looked after it. Amongst other things I had the opportunity to get to run test39.TestStoredProcedure for the first time. The tests give validation for functionality of the whole program and I helped to increase coverage.
Refactoring of SQLRelationLoader (C-3086)
Use of persistence.sql.query classes to represent constructed queries
- This was a refactoring to use objects instead of strings, resulting in a more automated and easier to understand query.
Introduce CastorStatement and CastorConnection classes to handle query construction
- This refactoring had to do with the upper. The persistence.sql.query classes yielded a query, that is only compatible with Derby database. With the use of query visitors, the resulted queries get independant from the database used.
Use ColumnInfos instead of SQLColumnInfo to bind parameters
- The SQLEngine and SQLRelationLoader classes had a complex mechanism to retrieve information about column names, types and convertors, used to serve operations on many to many relations. The TableInfo class already holded the table hierarchy with all information about columns (ColumnInfo). This functionality is now used by SQLEngine and SQLRelationLoader.
Refactoring of ClassMolder and SQLEngine
Move SQL-related function of ClassMolder to SQLEngine. The creation of SQLRelationLoader has already been moved to SQLEngine with CASTOR-3087.
As a first step the move to SQLEngine was completed and improved by also moving "if statement" for many table decision and saving a parameter.
After that SQLRelationLoader retrieves all information in a newly created class called RelationTableInfo. This class is used as it represents the relation table and holds information about left and right foreign keys needed by SQLRelationLoader (CASTOR-3096). No other parameter is needed and no other procedure in SQLEngine. This is a way more elegant way to bind parameters. I had this in mind with CASTOR-3089, but the TableLink class has disappeared in the development process. It hasn't displayed n-to-n relation tables well.
Improve build-up of table and column hierarchy in InfoFactory
Add field information to ColumnInfo class
After a lot of time has elapsed trying to replace the old column retrieval procedure for SQLRelationLoader in SQLEngine, it has turned out, that the information that was presently stored in ColumnInfo wasn't sufficient to find proper RelationTableInfo. That is to say we needed to add an additional field that stores the name of field in mapping to match correctly the field name of the stored foreign reference with the field name of the field descriptor.
This progress is included in the patch of CASTOR-3096.
New QueryObjects for parsers
Objects are used to represent a SQL tree and real queries are generated automatically out of the tree structure. Refactorings of old code are being done to make process more extensible and easier to understand.
CastorParameter objects were added, that deal with parameters of the form "... WHERE $1 = 1000" and "$(int)2 = 5" to make the new CastorQLParser backward compatible (C-3117). These objects plus the following are the foundation of parsers to work well.
As a next step new QueryObjects should get created, that could represent queries with COUNT aggregation functions and translate it to simple SQL. We canceled this job in favor of the more important GSoC projects.
Old Lexer and Parser should be replaced with new ones. This new Parser, CastorQLParser, is in the starting blocks.
There were only problems of two kinds, that hindered them to be used:
- CastorParameters Parameters of the form "... WHERE $(int)1 = 1000" in conditions and even "... WHERE $1 = 1000" leaded to a ParseException or respectively a ArrayOutOfBoundsException. Ralf and I disabled the use of identifiers with "$" in it - as it is in the old parser - to allow the parser to detect CastorParameters well. After that I improved the logic of the ParseTreeWalker, that different parameter statements don't lead to no Exception any more (C-3117).
- ORDER BY and LIMIT I fixed bugs at parsing of ORDER BY and LIMIT statements with C-3153
There is one problem left, which has to be solved before the new parser can be activcated:
- COUNT All kinds of COUNT aggregation functions were unsupported by the Parsers before. Castor-3118 deals with that problem. We didn't finish the development for the favor of GSoC projects that were more important.
To approach this, I first added some Tests to TestCastorQLParseTreeWalker and to be able to view the current state of success.
The EJBQLParser should be brought to run. There is only one desired functionality that isn't supported at the moment. This is the simple COUNT aggregation function.
A test was added to TestEJBQLParseTreeWalker to show current state of query support. We didn't continue the development.
Add test for all required features to TestCastorQLTreeWalker
Tests are needed to validate all existing functionality of existing parser. As recently as the CastorQLParser supports all constructs that the old one supported, it will be used instead of the old one.
I first created graphics that visualise the possible queries. Then I created tests for LIMIT and ORDER BY clause in C-3129.
Ralf did the remaining tests.
Author tag and license information in cpa package (C-3146)
I had a look at all classes in org.castor.cpa package (except the classes in driver package) and created an result table as an overview what has to be done.
- I checked if they have an author tag given. Then I added author tags where authors were given in license part.
- And I checked if they are declared under Apache license. After that I added license information to classes where information was missing. I also added authors so that authors given in license part and author tags match.
- I added delimiters. This is just to improve the visual appearance of the classes.
- I finally added version tags where they were missing.
Recreate JDO and Advanced JDO documentation in DocBook format
I created the JDO documentation in DocBook format.
First I took the old pages and replaced tags that are functional same with the new ones.
The sections needed further adaptation and needed all an "id" for links and so.
Sometimes I also needed to rearrange the structure of the document to fit for DocBook.
Some tags also disappeared as there is no DocBook correspondent like
I had to change some tags as the functionality isn't supported by DocBook.
The old api tag produced a link to the javaDoc reference. The new one only highlights the class.
Create Test case for property changes in extends hierarchy
There was a bug, that no fields of an extended table got stored. I created a test case in C-3183 that showed the bug.