KDb TODO

Legend: [ ] TODO [x] done [p] in progress

General hints:
*Use explicit ctors where possible
*check other==this in operator=() overloads
*look at http://ebn.kde.org/krazy/reports/playground/libs/kdb/
*Use prepared SQL statements everywhere and cache them instead of constructing SQL strings by hand
*Move members of public classes to d-pointers
*Remove boolean traps everywhere [http://ariya.ofilabs.com/2011/08/hall-of-api-shame-boolean-trap.html]
**Connection: databaseNames, databaseExists, useDatabase, tableNames, commitTransaction, rollbackTransaction, querySingleRecord, querySingleString, querySingleNumber, resultExists, createTable, alterTableName, dropTable, drv_databaseExists, updateRecord, insertRecord, deleteRecord, querySingleRecordInternal, storeObjectDataInternal
**replace Connection::dropDatabase() with Connection::dropCurrentDatabase()
**make Connection::drv_useDatabase return tristate instead of using bool *cancelled
**Driver: addLimitTo1
**Cursor: updateRecord, insertRecord, deleteRecord,
**FieldList: 2 ctors
**IndexSchema: attachRelationship
**QueryColumnInfo: ctor
**OrderByColumn: 2 ctors, toSQLString
**OrderByColumnList: appendFields, appendColumn x 2, appendField x 2, toSQLString
**QuerySchema: insertField x 2, addField x 2, addExpression, addAsterisk, field, columnInfo, 
**RecordEditBuffer: ctor, at
**TableSchema: ctor 
**TableOrQuerySchema: ctor

TODO: add explanation of expressions equality with examples, similar to
      bool QDomNode::operator== ( const QDomNode & n ) const
TODO: port KEXI_DEBUG_GUI; merge 3fa9e1a75c5fae1c from calligra/master
TODO: port 'Field list: return bool from removeFiled(), add moveField()' 9ab089d7a1306d45be from calligra/master
TODO: port 'Force lowercase letters for object ids; fix for not-all-lowercase namespace' 5d39c992354a from calligra/master
TODO: port 'Fix possible crash when saving changes to design of table used elsewhere' e410a4f4fc9da from calligra/master

UNIT TESTS TODO:
*FunctionExpression
*ExpresionData*
*QueryParameterExpression
*VariableExpression
*Pass expression string to parser, parse it and compare with the original string

Buildsystem:
[ ] find replacement for -DKDE_DEFAULT_DEBUG_AREA=44000
[ ] move drivers/ to the main dir
[ ] move *Validator classes from tools to a separate KDbGui lib
[x] version macros like KDB_VERSION are generated using cmake

Drivers:
[x] add SQLite driver for 3.6.1+ version (use FindSQLite.cmake);
    Compatibility with Kexi 1.x (SQlite 3.2.8) is acheived because we use the 1st
    file format of SQLite, which is the default for 3.3.6+.
    (see http://www.sqlite.org/formatchng.html)
[ ] TODO: let's add this note:
     In June 2006 we have migrated to 3.2.8 from 3.0.7
     (http://websvn.kde.org/trunk/koffice/kexi/3rdparty/kexisql3/version?view=log)
     Thus, any future use autovacuum will make the db read-only for 3.0.7-based drivers
     and ALTER TABLE ADD COLUMN will make it unreadable for 3.0.7-based drivers 
     unless VACUUM is executed. We hope it will not be a problem by the time
     we decide to use these features in the final, since 3.0.7-based Kexi will be already
     3 years old. It will be always possible to perform simple migration using the following command
     "sqlite-old old.db .dump | sqlite-new new.db"
[ ] SQLite: (security) Add connection flags to the driver and global setting to control the "secure delete" pragma
            for now secure delete is always enabled
[x] MySQL driver builds
[ ] replace the Pqxx driver with a new one using libpq C API
[ ] Port the Sybase driver (untouched for now)
[ ] Port the ODBC driver (untouched for now)
[ ] Add string escpaing to connection, call driver's escpaing by default
[23:12] <jstaniek> piggz: what's the reason for escape in connection?
[23:13] <piggz> jstaniek: short answer, the connection knows the database encoding, and can do the right think
[23:14] <jstaniek> piggz: but this would be just a helper function calling driver anyway, right?
[23:14] <piggz> no, the actual connection to the database is needed to know the encoding
[23:15] <piggz> the driver could have a simple fallback like i have put in today
[23:16] <piggz> jstaniek: difference between pqescapestring and pqescapestringconn here http://books.google.co.uk/books?id=EwSt1gAjRjgC&pg=PA273&lpg=PA273&dq=pqescapestringconn+documentation&source=bl&ots=dVAChqjY5b&sig=mUWYdPqS0Xzz0n30etYg2EjLjpQ&hl=en&ei=8LaCTJqaGIyA4Ab7143UCw&sa=X&oi=book_result&ct=result&resnum=7&ved=0CCwQ6AEwBg#v=onepage&q&f=false
[23:16] <piggz> pqescapestring, the non-connection version is deprecated as a security risk
[23:17] <piggz> becuase, in a multi-byte char set, \ can be part of a character, the postgres guys told me
[23:18] <jstaniek> I see...
[23:18] <jstaniek> so indeed good idea about the fall back
[23:28] <piggz> it may require escapeString and valueToSQl functions in Connection class, as driver::valueToSql calls escapeString
[23:28] <piggz> that was where i traced todays bug
[ ] PostgreSQL driver: define USE_SSL and add support for SSL connections (http://www.postgresql.org/docs/8.4/static/libpq-status.html)
[ ] PostgreSQL driver: Use binary mode with PQexecParams() instead of PQexec(), this will reduce number
    of conversions from string.
    See http://www.postgresql.org/docs/8.4/interactive/libpq-example.html#LIBPQ-EXAMPLE-3
    Passing numeric values requires knowledge of the server storage format, as implemented in
    src/backend/utils/adt/numeric.c::numeric_send() and src/backend/utils/adt/numeric.c::numeric_recv().

Naming, API:
[ ] what to do with kexi__ prefix for db objects (backward compat.)?
[ ] apply rules from http://techbase.kde.org/Policies/Library_Code_Policy
[x] driver's .desktop file format changed:
"X-Kexi-DriverName" -> "Name"
"X-KDE-Library" -> "FileName"
"X-Kexi-DriverType" -> "Type"
"X-Kexi-FileDBDriverMime" -> "MimeTypes"
[ ] change the above in kexidb-qt3 too
[ ] make Driver::Features use Q_FLAGS
[p] remove usage of QObject in Driver, DriverManager, etc.
[p] make classes implicitly shared
[p] replace usage of 'row' term with 'record' in most places, e.g. Cursor::storeCurrentRow -> Cursor::storeCurrentRecord
[p] port debug methods to uniform "QDebug operator<<()"
[p] make certain methods of *Expr classes const if possible
[ ] Add MONEY/DECIMAL data type; for sqlite use INTEGER
    (see http://www.mail-archive.com/sqlite-users@sqlite.org/msg26965.html
         http://www.mail-archive.com/sqlite-users@sqlite.org/msg27247.html)
    Currency in MS Access: Currency values and numeric data used in mathematical calculations involving data
                           with one to four decimal places. Accurate to 15 digits on the left side of the
                           decimal separator and to 4 digits on the right side.
    Decimal Number (NUMERIC?) in MS Access: Decimal Stores numbers from -10^381 through 10^381 (.adp),
                                            from -10^281 through 10^281 (.mdb); decimal precision: 28 B,
                                            storage size: 12 B
[ ] Add extension-functions.c from http://sqlite.org/contrib: mathematical and string extension functions
    for SQL queries using the loadable extensions mechanism
[ ] add Connection::reset(), e.g. for pgsql that would call PQreset():
    "This function will close the connection to the server and attempt to reestablish a new connection
     to the same server, using all the same parameters previously used. This may be useful for error recovery
     if a working connection is lost."

Other:
[ ] store config in ~/.config/kdb/, e.g. *rc files
[x] load plugins from QT_PLUGINS_DIR and similar locations
[ ] add more Q_INTERFACES than just Driver: add Import, Export (replaces KexiDB migration) so driver plugin can support more or less features

Optimization & Cleanup
[ ] use QLatin1String where possible

TODO: port files (Legend: *-compiles, no warnings; **-cleanup and naming notes also applied)
  Admin.h
  Admin.cpp
  Alter.h
  Alter.cpp
*  Connection.h
*  Connection_p.h
*  Connection.cpp
** prepareStatementInternal() has now no arguments and has PreparedStatementInterface* return type
TODO: move to Connection::prepareStatement() and all the drv_*() to ConnectionInterface just like we moved execute() and prepare() to PreparedStatementInterface...
*  ConnectionData.h
*  ConnectionData.cpp
*  Cursor.h
*  Cursor_p.h
*  Cursor.cpp
  DbObjectNameValidator.h
  DbObjectNameValidator.cpp
  DbProperties.h
  DbProperties.cpp
*  Driver.h
*  Driver.cpp
** Driver::Info is implicitly shared class now
** Driver::fileDBDriverMimeType() moved to Driver::Info::fileDBMimeType()
** Driver::isFileDriver() renamed to Driver::isFileBased(), also available as Driver::Info::isFileBased()
** Driver no longer inherits QObject
*  DriverManager.h
*  DriverManager_p.h
*  DriverManager.cpp
** DriverManager no longer inherits QObject
*  Driver_p.h
*  Driver_p.cpp
*  Expression.h
*  Expression.cpp
** Expresion classes are now is ExpressionClass enum
TODO: change KexiDBExpr_Unknown to UnknownType, and ints to enums, etc.
*  Error.h
*  Field.h
*  Field.cpp
*  FieldList.h
*  FieldList.cpp
** added isEmpty()
  FieldValidator.h
  FieldValidator.cpp
*  Global.h
*  Global.cpp
*  IndexSchema.h
*  IndexSchema.cpp
*  keywords.cpp
TODO: enable generation of this file as custom target using kexi/tools/sql_keywords/sql_keywords.sh and kexi/kexidb/parser/sqlscanner.l and koffice/kexi/tools/sql_keywords/kexi_reserved
  LookupFieldSchema.h
  LookupFieldSchema.cpp
  MessageHandler.h
  MessageHandler.cpp
*  Object.h
*  Object.cpp
** made implicitly shared, used in fewer number of classed than before
*  PreparedStatement.h
*  PreparedStatement.cpp
** it's implicitly shared class now
** arguments are no longer provided using operator << but instead in execute(), so PreparedStatement is more generic now. Operator << and clearArguments() is thus removed.
** StatementType -> Type, added Arguments typedef
** execute() aborts now if at least one field provided on whereFieldNames list is not found (before it was ignored)
** internal: the statement is now created using utf8, not latin1 functions
** PreparedStatement::Arguments renamed to PreparedStatementParameters
** internal: to implement PreparedStatement in a driver, one has to write these methods:
*** virtual bool executeInternal( const Arguments& args );
*** virtual bool prepare(QByteArray& statement) = 0;
TODO: more changes like these related to WHERE expression
*  QuerySchema.h
*  QuerySchema.cpp
*  QuerySchemaParameter.h
*  QuerySchemaParameter.cpp
*  RecordData.h
*  RecordData.cpp
  Relationship.h
  Relationship.cpp
*  RowEditBuffer.h
*  RowEditBuffer.cpp
** renamed to RecordEditBuffer
*  SchemaData.h
*  SchemaData.cpp
*  TableSchema.h
*  TableSchema.cpp
  tools/SimpleCommandLineApp.cpp
*  Transaction.h
*  Transaction.cpp
  Utils.h
  Utils_p.h
  Utils.cpp
  kdb_export.h
  kdb_global.h

parser:
* parser/extract_tokens.sh
* parser/generate_parser_code.sh
  parser/Parser.h
  parser/Parser.cpp
  parser/Parser_p.h
  parser/Parser_p.cpp
* parser/SqlParser.y
  parser/SqlTypes.h
* parser/SqlScanner.l
  parser/tokens.cpp

tools:
  tools/SimpleCommandLineApp.h
* tools/Tristate.h

drivers:
*  drivers/mysql/MysqlConnection.h
*  drivers/mysql/MysqlConnection.cpp
*  drivers/mysql/MysqlConnection_p.h
*  drivers/mysql/MysqlConnection_p.cpp
*  drivers/mysql/MysqlCursor.h
*  drivers/mysql/MysqlCursor.cpp
*  drivers/mysql/MysqlDriver.h
*  drivers/mysql/MysqlDriver.cpp
*  drivers/mysql/MysqlKeywords.cpp
*  drivers/mysql/MysqlPreparedStatement.h
*  drivers/mysql/MysqlPreparedStatement.cpp
** for now it's nonfunctional; port and improve code within KDB_USE_MYSQL_STMT

  drivers/odbc/odbcconnection.h
  drivers/odbc/odbcconnection.cpp
  drivers/odbc/odbcdriver.h
  drivers/odbc/odbcdriver.cpp
  
  drivers/oracle/oracleconnection.h
  drivers/oracle/oracleconnection.cpp
  drivers/oracle/oracleconnection_p.h
  drivers/oracle/oracleconnection_p.cpp
  drivers/oracle/oraclekeywords.cpp
  drivers/oracle/oraclecursor.h
  drivers/oracle/oraclecursor.cpp
  drivers/oracle/oracledriver.h
  drivers/oracle/oracledriver.cpp
  drivers/oracle/oraclepreparedstatement.h
  drivers/oracle/oraclepreparedstatement.cpp
  drivers/oracle/oracle_keywords.sh
  
*  drivers/pqxx/PqxxConnection.h
*  drivers/pqxx/PqxxConnection.cpp
** forked to postgresql driver
* drivers/pqxx/PqxxConnection_p.h
* drivers/pqxx/PqxxConnection_p.cpp
** forked to postgresql driver
* drivers/pqxx/PqxxCursor.h
* drivers/pqxx/PqxxCursor.cpp
** forked to postgresql driver
* drivers/pqxx/PqxxDriver.h
* drivers/pqxx/PqxxDriver.cpp
** forked to postgresql driver
* drivers/pqxx/PqxxPreparedStatement.h
* drivers/pqxx/PqxxPreparedStatement.cpp
** forked to postgresql driver
* drivers/pqxx/PqxxKeywords.cpp

* drivers/sqlite/driver/sqlite.h
** removed, system SQLite is used
*  drivers/sqlite/SqliteAdmin.h
*  drivers/sqlite/SqliteAdmin.cpp
*  drivers/sqlite/SqliteConnection.h
*  drivers/sqlite/SqliteConnection_p.h
*  drivers/sqlite/SqliteConnection.cpp
*  drivers/sqlite/SqliteCursor.h
*  drivers/sqlite/SqliteCursor.cpp
*  drivers/sqlite/SqliteDriver.h
*  drivers/sqlite/SqliteDriver.cpp
*  drivers/sqlite/SqlitePreparedStatement.h
*  drivers/sqlite/SqlitePreparedStatement.cpp
** refactored: internal bindValue() added
*  drivers/sqlite/SqliteVacuum.h
*  drivers/sqlite/SqliteVacuum.cpp
*  drivers/sqlite/SqliteKeywords.cpp

  drivers/sybase/SybaseConnection.h
  drivers/sybase/SybaseConnection.cpp
  drivers/sybase/SybaseConnection_p.h
  drivers/sybase/SybaseConnection_p.cpp
  drivers/sybase/SybaseCursor.h
  drivers/sybase/SybaseCursor.cpp
  drivers/sybase/SybaseDriver.h
  drivers/sybase/SybaseDriver.cpp
  drivers/sybase/SybasePreparedStatement.h
  drivers/sybase/SybasePreparedStatement.cpp
  drivers/sybase/SybaseKeywords.cpp

  drivers/xbase/XbaseConnection.h
  drivers/xbase/XbaseConnection.cpp
  drivers/xbase/XbaseConnection.cpp
  drivers/xbase/XbaseConnection_p.h
  drivers/xbase/XbaseConnection_p.cpp
  drivers/xbase/XbaseCursor.h
  drivers/xbase/XbaseCursor.cpp
  drivers/xbase/XbaseDriver.h
  drivers/xbase/XbaseDriver.cpp
  drivers/xbase/XbaseExport.h
  drivers/xbase/XbaseExport.cpp
  drivers/xbase/XbaseKeywords.cpp

kexiutils:
  debuggui.cpp - port and re-enable KEXI_DEBUG_GUI