Roadmap RDBMS Decoupling

DataMapper’s core system dm-core has very few concepts that bind it tightly to RDBMS, aside from the adapters included in dm-core for MySQL, PostgreSQL, SQLite 3, Oracle, SQL Server. There are also libs containing migrations and transaction related code for each of those databases.

Early in DataMapper’s lifecycle it was beneficial to have these adapters included in dm-core. It allowed easy testing without having to install any other gems, and the short distance from the dm-core source was quite convenient. Now that these adapters are quite mature, it is less beneficial to have them included with DM. For one, it adds alot of cognitive overhead to someone trying to understand DM’s source code. It also provides an (invalid) hint that perhaps DM does not work with anything other than RDBMS. Also there is logic in some of the migration code that is duplicated (poorly) in dm-migrations, and it would be better to merge the code together.

Another problem with this organization is that when you install dm-core, and you have the SQLite3 adapter, it’s unclear that you also need to install the do_sqlite3 adapter. We used to have hard dependencies on data_objects from within dm-core, but not only did that result in some weird circular dependencies, it gave the false impression that DM was only for RDBMS’.

The goal with this change is to remove all RDBMS specific code from dm-core, and move it to the correct locations. More specifically we will be:

• Moving all the DataObjects adapters to dm-more as separate adapters:
  • dm-mysql-adapter
  • dm-postgres-adapter
  • dm-sqlite-adapter
  • dm-oracle-adapter
  • dm-sqlserver-adapter
• Adding adapters for JRuby drivers:
  • dm-h2-adapter
  • dm-derby-adapter
  • dm-hsql-adapter
• Moving the migration specific code into dm-migrations.
• Moving the transaction specific code into a new gem in dm-more called dm-transactions.

Moving the DO adapters into separate DM adapters provides a nice benefit to installing DM. Instead of referring new users to install do_sqlite3, and dm-core, we can tell them to install dm-core, and dm-sqlite-adapter. The dm-sqlite-adapter will have a gem dependency on the correct database driver and version, but the new user does not have to be exposed to these details up-front. No doubt over time they will become aware of the relationship between DM and DO, but I think this is an unnecessary detail for the first time user.

The JRuby DataObjects drivers have support for all the standard DataObjects adapters, plus 3 java databases: h2, derby and hsql. DM adapters are desperately needed for these and will not only all new usage in JRuby, but it really helps the JRuby DO developers better test their driver because they can run the adapters using the DM specs.

The removal of the migration code from dm-core does introduce a few interesting changes to the DM API. For one if someone uses dm-core, with dm-sqlite-adapter, but not the dm-migrations plugin they may get an error when calling DataMapper.auto_migrate!. In the interim we will need to come up with a way to perhaps require dm-migrations, or have it as a dependency of dm-sqlite3-adapter, or defer requiring it until auto_migrate! or auto_upgrade! is called, perhaps with some deprecation warning. This will need some more thought on how to handle this so it is not that painful for existing users.

The transaction code in dm-core is functional and has a decent public API, but I think it needs a rewrite or a pretty major refactoring. The style of the code doesn’t really match anything else in dm-core, and it is overly clever at times. I would like to see a new transaction API with only a few simple operations, perhaps even a simplification of the existing API (with some sort of deprecation warnings to allow people to move to the new API). It would be nice if the API was generic enough from the user’s point of view that it could be applied to a Unit of Work API later on to be useful to non-RDBMS systems.