DataFaucet PersistenceService Update - Tested Many-to-Many

I just committed some updates to the SVN repository that includes testing for many-to-many relationships with the PersistenceService. And yes there were a handful of issues that needed to be addressed to get these working. ;)

The object code for a reciprocal many-to-many relationshiup looks like this:

<cfcomponent displayname="department">
<cfproperty name="departmentID" type="uuid" required="true" key="1" />
<cfproperty name="departmentName" type="string" required="true" />
<cfproperty name="facultyArray" type="array" required="false" />

...
</cfcomponent>

<cfcomponent displayname="faculty">
<cfproperty name="facultyID" type="uuid" required="true" key="1" />
<cfproperty name="firstname" type="string" required="false" />
<cfproperty name="lastname" type="string" required="false" />
<cfproperty name="departmentArray" type="array" required="false" xref="tblDepartmentFaculty" />

...
</cfcomponent>

Once you've got these CFCs ready, then you would set up your PersistenceService with the appropriate IoC factory (or multiple factories if you prefer) and call the install method to install the tables to hold the data for these components. (The below code would normally be configured in your IoC factory - it's shown here for reference.) :)

<cfscript>
cacheFactory = CreateObject("component","datafaucet.system.classcachefactory").init();
cacheManager = CreateObject("component","datafaucet.system.classcachemanager").init(cacheFactory);
daoFactory = CreateObject("component","datafaucet.system.daofactory").init(accessorPattern="get*",mutatorPattern="set*");
factory = CreateObject("component","datafaucet.system.classfactory").init("datafaucet.demo");
service = CreateObject("component","datafaucet.system.persistenceservice").init(factory,daoFactory,cacheManager);

service.install("department");
service.install("faculty");
</cfscript>

And viola! You've got tables. Then all you need to do is get and save a few departments and faculty. As you create and save them it will both cache the objects and relationships and it will update the cross-reference table indicated in the xref attribute. Restart your ColdFusion server and fetch any department or faculty record via service.get("department",departmentid) and it will automatically load the associated objects.

As previously mentioned, this is a deceptively simple feature and it can be very easy to swamp your server with hundreds or even thousands of objects that aren't being used, simply because you declared a couple properties of type "array". So BE CAREFUL if you choose to use this. Make sure these are objects that really are frequently used and really should be cached in memory.

ORM PersistenceService BER

Okay so as of today, the bleading-edge release of DataFaucet includes a reasonably complete INITIAL PROTOTYPE of the PersistenceService. It now includes reciprocal one-to-many/many-to-one relationships and the purge operation has been given some preliminary testing. Aggregation (many-to-many relationships) have not been tested and cache purging needs some additional testing.

What does this mean?

For those of you who are advanced users and may have been interested in an "external ORM" where you could use your own business objects that are ignorant of the persistence engine, you can now have components like this:

<cfcomponent displayname="conference">
<cfproperty name="conferenceid" type="uuid" required="true" key="1" />
<cfproperty name="conferencename" type="string" required="true" />
<cfproperty name="forumArray" type="array" required="false" />

...
</cfcomponent>

<cfcomponent displayname="forum">
<cfproperty name="forumid" type="uuid" required="true" key="1" />
<cfproperty name="forumname" type="string" required="true" />
<cfproperty name="conference" type="conference" required="true" />

...
</cfcomponent>

Now where I put those elipses, you would have your getters and setters (however you want to write them) and in the PersistenceService you can set a pattern for them, so you can use either getValue/setValue the way I mostly use DataFaucet or you can use getConference/setConference. And then you can simply request your objects from the service in much the same way that Transfer works. In this above example when you request a conference object via PersistenceService.get("conference",conferenceid) it will automatically load that conference object with an array of its cached forums and similarly, each forum object will be loaded with a reference to the cached conference object. So if you've got one conference and 5 forums, you'll have 6 objects in the cache.

Can't wait?

For more information about how to configure the service, see the previous blog entry. There is NOT any documentation in the distribution yet. To get the code, export from the SVN repository (it's NOT in the public download yet and won't be until there's documentation).

In my mind there are specific use cases in which this is beneficial over using the existing active records and imo vice versa as well.

On the plus side:

Using the PersistenceService means that your individual cached objects won't contain all the data access methods as is the case with active record objects. That means they'll have a smaller memory footprint.

For some of you OO enthusiasts you'll be happy that the object is nearly independent of the ORM (aside from a few special attributes in the cfproperty tags).

Also there's just one DAO class, instead of having a separate DAO CFC for each object, woohoo! The DAO is loaded with some information about the class it's being used to persist, but generally speaking you shouldn't need to extend the DAO, it should just work as is to persist most business classes.

On the minus side:

Because of the seamless, recursive nature of the links between these objects, it could be really easy to code yourself into a situation where you've got dozens or even hundreds of objects loading when you only asked for ONE. As an example, given a product catalog of any size, you wouldn't want to have an array of Products in each Category obect. And that's really not the way online shopping is done anyway, so hopefully that won't be a problem for you. ;)

As of right now there are no solutions to lazy-loading property arrays. I have a few ideas about ways to handle that, but no code yet. Doing that in ColdFusion will probably mean subclassing your business objects to add hooks into the ORM for fetching those properties, although there will need to be a way of informing the DAO that those properties will be lazy-loaded.

Race conditions... race conditions... and more race conditions... To be honest there are race conditions with the active record implementation also, however, the addition of caching for any collection of objects increases the number and type of race conditions that will occur in your application. Unfortunately beyond "in your business objects", I can't tell you where you'll find them, I just know that you will. If any given method in your business objects sets or gets more than one instance variable, there's a potential race condition in it.

Use Cases

You might want to use the PersistenceService instead of active records if you have a behavior-rich application in which a number of objects need to remain resident in memory for frequent access. A good example of this would be application security / authentication. User objects, Role objects and Permission objects (if you have them) are apt to be accessed rather frequently and may include somewhat rich behaviors for determining which users have access to what features of your application. In this scenario it makes a lot of sense to retain these objects in memory to reduce the amount of database access and keep the individual objects readily accessible to each other to handle their exchange of information.

As always, I'm available to answer whatever questions you may have. ;)

New PersistenceService option in DataFaucet ORM

So exactly one week ago today on the 3rd of November, Joe Rinehart posted this blog entry in which he suggested that none of the existing ORM solutions for ColdFusuion are "true ORMs" because they function in ways that are a bit different from the way "traditional" ORM tools work in other languages... or maybe just in Java, I'm not sure.

Anyway as you might expect Joe's article has been fairly popular. MangoBlog doesn't show a view count, but there are 34 comments! In a follow-up he listed a handful of more specific problems he believes need to be solved in order to call a tool an "ORM".

Now I actually disagree with the notion that we should just avoid calling these tools ORMs unless they work in this specific way, in part because the name "object relational mapping" doesn't include any words that actually describe several of the problems listed. Caching as an example isn't described by the name although it's part of the problems outlined. I am however a fan of options.

Until recently the only caching in DataFaucet was for queries. It didn't have any caching utilities for objects. Part of the reason for an object cache being needed in Joe's description is because of the "identity" problem he wants an ORM tool to solve. The basic idea is that for instance X of a given class, there should only ever be one object in memory. So you couldn't have say Product #5103 and have Joe have one instance of 5103 and Ike have another instance of 5103 and they be different objects, because there's only ever supposed to be one Product 5103 object. This of course gives rise to a variety of new race conditions.

The active record objects in DataFaucet on the other hand are designed on a somewhat different concept of identity, one that handles race conditions related to referential integrity in a different way. Actually most of the potential race conditions that occur when everything is cached as is the case in Joe's description (and in Transfer), don't occur when you're using DataFaucet's active record objects, and so there are only a much smaller handful of race conditions to consider, those being largely related to deleted records and referential integrity.

Now I'm not saying that the active record model I developed in DataFaucet is better, I'm merely pointing out that it's different and so it has different strengths and weaknesses. If you're like me you may consider that there are a lot of applications for the web where the kind of very complicated caching done to solve the identity problem in Transfer isn't really necessary and so a somewhat simpler approach may work just fine.

On the other hand there certainly may be cases in which knowing that those objects are atomic may be very important to your business model, in which case you would need something that handles caching. In the past I just handled the cache on a case-by-case basis. The onTap framework's member plugin does this, having a manager object that maintains cache for the individual member objects and resolves the identity problem for just those objects. They actually are active record objects but in that case I felt it was important that the identity problem be solved the way Joe described in order to facilitate a more bulletproof security model. ;)

I realize I'm getting kind of long-winded here (as usual). What I'm getting at is that I've started working on a PersistenceService object in the DataFaucet core that provides a generic service/factory for working with objects in the way Joe described in his article. I have no intention for this to replace any of what already exists in DataFaucet. The existing features will stay right where they are, I'm merely extending the framework to include some new features in addition for those who prefer to work with their data in a manner more like Transfer.

One thing that you will find different from Transfer with the PersistenceService is that it's not going to create the individual objects itself. Instead it's going to defer to an IoC framework like ColdSpring or LightWire (or the simplified IoC factory in the onTap framework). And so you won't be creating any decorators like you would with Transfer because it's going to use your objects from the word go.

Also as Joe described in his article, this will make it possible to create your objects as "pure" objects, which have no internal knowledge of the ORM. Instead the ORM will analyze the object's properties and build the data schema from that - or you can provide a schema file for a given class if you need more control. I'm still working out the details of how to configure the PersistenceService and its related cache manager and DAO objects. The DAO objects and object-cache objects can also come from your IoC factory if you prefer to configure them that way (which if you're using IoC already, I rather expect you will prefer that).

I also made a particular point of designing the DAO objects to support not only the generic get/set methods that I prefer but to also support explicit getters and setters as well if you prefer that. Of course they default to getValue and setValue because the onTap framework's "duck" object has those method names, but if you've got existing objects with explicit getters and setters (from a snippet maybe?) then you can use those too.

Here's an example of a CFC I'm using for testing to show how you can use it with objects that are rather unlike the kind of objects I usually create:

<cfcomponent displayname="vendor">
   <cfproperty name="vendorid" type="uuid" required="true" key="1" />
   <cfproperty name="vendorname" type="string" required="true" />
   <cfproperty name="vendornotes" type="string" required="false" length="250*" />
   
   <cffunction name="init" access="public" output="false">
      <cfset variables.instance = newInstance() />
      <cfreturn this />
   </cffunction>
   
   <cffunction name="newInstance" access="private" output="false">
      <cfset var result = structNew() />
      <cfset result.vendorid = "" />
      <cfset result.vendorname = "" />
      <cfset result.vendornotes = "" />
      <cfreturn result />
   </cffunction>
   
   <cffunction name="getVendorID" access="public" output="false">
      <cfreturn instance.vendorID />
   </cffunction>
   
   <cffunction name="setVendorID" access="public" output="false">
      <cfargument name="vendorid" type="string" required="true" />
      <cfset instance.vendorid = arguments.vendorid />
   </cffunction>
   
   <cffunction name="getVendorName" access="public" output="false">
      <cfreturn instance.vendorName />
   </cffunction>
   
   <cffunction name="setVendorName" access="public" output="false">
      <cfargument name="VendorName" type="string" required="true" />
      <cfset instance.VendorName = arguments.VendorName />
   </cffunction>
   
   <cffunction name="getVendorNotes" access="public" output="false">
      <cfreturn instance.vendorNotes />
   </cffunction>
   
   <cffunction name="setVendorNotes" access="public" output="false">
      <cfargument name="VendorNotes" type="string" required="true" />
      <cfset instance.VendorNotes = arguments.VendorNotes />
   </cffunction>
   
</cfcomponent>

You might notice that, as Joe described, this CFC doesn't extend anything. It doesn't have to. The persistenceService is able to get all the information it needs either from the metadata of the object or from its configs, but the object itself doesn't need to know anything at all about the ORM. It doesn't even need to know you're using an ORM at all.

So having said that, the setup for my test page to configure the service looks like this:

<cfscript>
   cacheFactory = CreateObject("component","datafaucet.system.classcachefactory").init();
   cacheManager = CreateObject("component","datafaucet.system.classcachemanager").init(cacheFactory);
   daoFactory = CreateObject("component","datafaucet.system.daofactory").init(accessorPattern="get*",mutatorPattern="set*");
   factory = CreateObject("component","datafaucet.system.classfactory").init("datafaucet.demo");
   service = CreateObject("component","datafaucet.system.persistenceservice").init(factory,daoFactory,cacheManager);
   
   service.install("vendor");
   
   vendor = service.get("vendor");
   vendor.setVendorName("sony");
   service.save(vendor);
   
   vendor2 = service.get("vendor",vendor.getVendorID());
   vendor3 = service.get("vendor",vendor.getVendorID());
</cfscript>

If this were a working application I wouldn't have those createObject calls at the top. Instead I would have those objects configured in my IoC framework and then I would fetch the PersistenceService from the IoC factory and call service.get() as you see here. I would probably also use just the one IoC factory for all three types of objects (cache, dao and business objects). The init method of the service object even defaults the daoFactory to the business object factory.

The function call service.install("vendor") analyzes the object and configuration and creates the necessary tables to hold the data for the vendor object.

Then my get() call fetches a new vendor object from my IoC factory (in this case "datafaucet.system.classfactory" is just a naive tool that creates an object and spits it back, rather than an actual IoC factory - it's a stub really, existing only to support the model for testing - and the daoFactory here is the same).

The call to service.save() then checks to see if the object is new and performs either an insert or an update of the database depending on its status, just like Transfer.

Subsequent calls to service.get() where I've included the id of the record I want then return the already created object from the cache.

So far I've tested this on the single object. It creates one table and there are no composed objects yet. There is some code in SVN for handling the composition cache, but I haven't tested that part yet. There are a bunch of other things I expect don't work yet. Many of them are things that are already working with the active record objects. For example it won't handle autonumber ids yet or storing data for a single object in multiple tables. Heck, I haven't even tested the functions for purging the cache!

I wanted to post a note to let everyone know about roughly two days work on this. The code is in SVN and none of it is documented yet because of course it's not released yet. But if you're interested in playing around with it, you're more than welcome to get the BER from the SVN repository and check it out. :)

I will say that one particularly keen drawback of the PersistenceService is that it fails one of the primary objectives of the DataFaucet framework and that is making data access easy. I've said it on several occasions that it's called DataFaucet because I wanted to make getting data as easy as getting a drink of water from your kitchen sink! ;) In many ways I feel the gateway and active record objects really do live up to that ideal, particularly with features like and/or keyword searching. The PersistenceService not so much.

There's not a whole lot I think I can do about that because it needs an IoC framework to function and of course that introduces a new dependency that's beyond the scope of the ORM. But I'm certainly not opposed to the framework having "advanced techniques" for people who need or want them. As long as I can maintain that gentle learning curve, I'll be happy. :)

BlogCFC was created by Raymond Camden. This blog is running version 5.5.006.