Enjoy!

Easy enough. With one little line of code, you can very simply convert ORM entities to a ColdFusion query…just like so:

// get the orm entities
ormstuff = EntityLoad( "Art" );
// convert the orm goodness to a CF query
anicequery = EntityToQuery( ormstuff );

Cake, right? If I were to hand this query off to CF’s spreadsheet functions, I could easily prepare and export a nice Excel file full of the data I just converted. But of course, nothing is that simple…and that’s where the potential bug announced itself.

A Dark Foreboding

You see, instead of simply dumping all the data for all the columns of the query-which-was-once-an-entity, I wanted to pare down the list of columns that would be output in the Excel export. Since I already had a query handy, I figured passing a list of column names to a Query-of-Queries would make short work of this. However, the moment I tried this…

BOOM!

I received the following error:

Query Of Queries runtime error. Unsupported SQL type java.sql.Types.UNKNOWN

The Weirdness Begins

So what happened? Turns out that one of my columns (a BIT column on the database) was being converted to an “UNKNOWN” data type in the query result. You can see this easily by viewing the metadata of the query object (notice the “ISSOLD” column):

writedump( getMetaData( anicequery ) );

The Weirdness Grows

As strange as this is, it gets weirder.

• If I do a non-entity query of the database and the dump the result, the data type for “ISSOLD” is not UNKNOWN, but the correct data type.
• If I pass a “*” in the Q-of-Q instead of a list containing the offending column’s name, no error occurs
• CAST() the column, you say? Yep, tried that. No dice.
• Changing the “type” and “ormtype” attributes of the BIT column’s properties in the ORM mapping don’t help. I tried a number of combinations, but could not get the “TypeName” to ever NOT be “UNKNOWN”
• Ok, that’s not 100% true. if I leave those attributes off, the TypeName is set to “JAVA_OBJECT”, and the QofQ is able to execute without error. Of course, this is only for MySQL…in SQL Server, I couldn’t get it to change, even with the same settings as what “worked” for MySQL. Ugh.

The Weirdness Ends

After several unfruitful experiments and dozens of minutes Googling to no avail, I put an end to the madness by simply writing my own version of EntitytoQuery(). It’s ugly and still a work in progress, but it at least by-passes the issue I encountered.

If you’re interested in seeing this issue in action, check out the demo pages I put up (each page has a dump of the code for the entities):

• SQL Server 2008
• MySQL

I’ve also submitted a bug for this, so if you think it’s crazy, feel free to vote!

Some Context

Let’s say that you have two entities that are related via a one-to-many relationship (Artists and Art, for example). You can easily create this relationship in your model by doing something like the following:

property name="art" fieldtype="one-to-many" fkcolumn="artistid" cfc="Art" remotingfetch="true";

If we were to dump the result of an EntityLoad() of “Artists”, we’d see a nice structured relationship: First, an array of all Artist entities, and then an array of all Art entities related to the particular ArtistID. Additionally, I’ve specified remotingfetch=true, which will force CF to load the Art entities for each Artist.

The Bug

The bug, however, comes into play when you try to serialize the result of EntityLoad(). While it properly composes the relationships, it represents the first two entities in the “art” array; any subsequent entities in the given array are returned simply as “{}”.  Whether you have 3 “Art” entities for an artists, or 20, only the first 2 will ever be serialized properly.

See this happening in CF10:

• Regular, correct composition of entities via straight CFDUMP – LINK
• “First two” problem via dump of serializeJSON() – LINK

It Gets Worse

The worst part about this is that it is a relatively old bug. It was first submitted for CF9 back in October 2010, was deferred, and was apparently not fixed for the public beta of CF10.

I think this is a huge issue, and should be a priority to be fixed. Until it is (or a suitable workaround is devised), it will be incredibly difficult to create applications of any complexity that depend upon serialization of entities.

Vote, Vote, Vote

The best way to get it fixed, of course, is to make your voice heard and vote up the bug. Here are some links for that:

Bug Report – CF 9: https://bugbase.adobe.com/index.cfm?event=bug&id=3042650

Bug Report – CF 10: https://bugbase.adobe.com/index.cfm?event=bug&id=3175667

Anyway, with that in place, I’ve uploaded my examples of ArrayEach(), StructEach(), and ArrayFilter(). I’ve also put them out on GitHub, so feel free to take a look if it suits you.

Enjoy!

Like ArrayEach() and StructEach(), the new filter methods have a “function” parameter (which can also be either an inline function or a named function). However, unlike ArrayEach() and StructEach() which return nothing, the new filter functions return a new array, list, or struct, respectively.

To accomplish this, you need to return a boolean from the function parameter. If the current array element, list item, or structure value matches some criteria and returns “true”, it will be added to the new returned array/list/structure. Conversely, if it fails the criteria and returns “false,” it will be excluded from what is returned.

NOTE:  The array/list/struct filter() functions return NEW arrays/lists/structures. The original arrays/lists/structs against which the filter() functions are called are unaffected.

An Example

Let’s say that we have a simple set of “circles” for a Google+ account which includes a circle for friends, one for co-workers, and another for family. We might model this data like so:

'circles' = {
    'friends': ['Lucas','Phil','Dave','Brian','Dillon'],
    'coworkers': ['Tyler','Mike','Doug'],
    'family': ['Dean','Jared','Jason']
};

Now, let’s imagine that for some reason, we want to filter all of the people in our circles, and return all the contacts (regardless of circle) whose names start with the letter “D”.  One approach might be as so:

// create an array to hold all the names
allDnames = [];
// use structeach() to evaluate each "circle" 
structeach(circles,function(key,value) {
     // "matched" names will be the new array returned from the arrayfilter()
     // which is applied to each circle in the structeach() loop
     matchednames = arrayfilter(value,function(obj) {
         // if the first letter is a "D", return true; otherwise, return false;
         return left(obj,1)=='D';
     });
     // now let's add each filtered name to our master array of "D" names 
     arrayeach(matchednames,function(obj) {
          arrayappend(allDnames,obj);
     });
});

See the live example or grab the code from GitHub

Wrapping Up

As you can see, using CF10′s new ArrayFilter() function is pretty simple, and the ListFilter() and StructFilter() functions work exactly the same, just with slightly different parameters and return types.

But perhaps more importantly, this example shows how functions like StructEach() and ArrayFilter() can make processing of complex data structures a snap. Instead of bothering with messy for-loops in which it would be incredibly easy to lose our place, this approach feels much more sane and easy to follow.

NOTE: The ArrayEach() and StructEach() functions act on the original array/struct. Therefore, if you make changes to the array/struct within these functions (as in the ArrayEach() example below), the original array/struct will be affected (thanks Andy!).

ArrayEach()

So, for example, let’s say you have an array of friends’ names, and you want to make the list all uppercase. Pre-CF10, you might approach this by looping over the array, and executing a function at each iteration. Fine, easy enough, and it works. However, with ArrayEach(), you can ditch the clunkiness of the for-loop, and keep everything self-contained to the array. And, you can execute either an “inline” function, or call a named function. The two examples (which produce the same results) are below:

names = ['John','mary','JOseph','Glenda','Mike'];

// named function
// notice that the function takes one argument, the value of the current iteration of the array
function makeLowercase(obj) {
    names[arrayfind(names,obj)] = lcase(obj);
}

//  run arrayeach() with an inline function
arrayeach(names,function(obj) {
    names[arrayfind(names,obj)] = lcase(obj);
});
writedump(names);

// run arrayeach() with a named function
arrayeach(myarray,makeLowercase);
writedump(myarray);

See the live example or grab the code from GitHub

StructEach()

Not to be outdone, structures get some each() lovin’ in CF10 as well. But instead of iterating over an array, the StructEach() function executes for every key-value pair in the structure. As with ArrayEach(), you can use either use an inline function or a named function.

In this example, we have a simple collection of “favorites,” which we want to print to the screen.

favs = {color: 'blue', food: 'pizza', sport: 'basketball'};

// named function
// notice that the function takes two arguments, the key and value pair of the current iteration of the structure's key-value pairs
function getFavorites(key, value) {
    writeoutput('My favorite ' & key & ' is ' & value);
} 
// run structeach() with a named function
structeach(favs,getFavorites);
// run structeach() with an inline function
structeach(favs, function(key,value) { 
    writeoutput('My favorite ' & key & ' is ' & value);
});

See the live example or grab the code from GitHub

Wrapping Up

While these two functions probably won’t garner the attention that other additions will, I think they’re pretty dang awesome. Not only are they a long time coming (honestly, how often do you loop over an array…like 20 times every hour!!), but they help to make developing with these basic data types just a bit easier as they more closely align methodologies in CF with those you’d find elsewhere.

As expected, this is pretty straightforward, and has the same capabilities as the <cfoutput> corollary. Here’s a quick example:

<cfquery name="qproducts" datasource="somedatasource">
     select *
     from   products
     order by category,subcategory,product
</cfquery>

<cfoutput>
     <cfloop query="qproducts" group="category">
          <h1>#category#</h1>
          <cfloop group="subcategory" groupcasesensitive="true" >
               <h2>#subcategory#</h2>
               <cfloop>
                    #product#<br />
              </cfloop>
          </cfloop>
     </cfloop>
</cfoutput>

Pretty simple, but something which I know a lot of people have wanted for quite a while. Well, now you have it!

A big part of the appeal of ExtJS data Models is their flexibility. While some data “plugins” in other JS frameworks certainly allow you mimic some of the behaviors of a data model, ExtJS’ data Model wins hands down because of its extreme flexibility. With a ExtJS Model, you can quickly and easily create powerful definitions of data objects (fields, data types, associations, validations, etc.) in your application. These models can then in turn be used to craft a data Store, which can themselves be plugged into many of ExtJS’ data components (DataView, Grid, Chart). What you end up with is a super-powerful, deeply-data-driven application in a ridiculously small number of lines of code.

Of course, the perfect complement to the simplicity and grace of ExtJS’s data Model is to “feed” it using ColdFusion. Below, I’ll outline how to retrieve data from CF, as well an extremely easy way (there are obviously more!) to format data from a ColdFusion query directly into an ExtJS data Model.

Note: For these examples, I am using ColdFusion 9 and ExtJS 4.0.7

ExtJS Models

As mentioned above, the ExtJS data Model allows you to create pretty powerful–and flexible–definitions for data objects in your application. The beautiful part about this is that you can create a Model out of pretty much any data structure that is presented to your application. This allows you to spend your time making your JavaScript application work the way you want it to, without being enslaved to a particular requirement for data formatting (or type, for that matter). So whether you’re consuming XML, JSON, JSONP, etc., the ExtJS Model is flexible and configurable enough to normalize the incoming data structure into what you ultimately want (and need) your data objects to be for your application.

ColdFusion Query to JSON

How does this relate to ColdFusion? Well, let’s say that you have an existing, remotely-accessible component that returns a query result. In ColdFusion 8 and above, you can specify the “returnformat=json” argument in your method invocation, and ColdFusion wil serialize the query object into a JSON-formatted string, and return it to your JavaScript application.

remote any function getqueryarrays() {
    fauxquery = querynew('id,name,age');
    queryaddrow(fauxquery,4);
    querysetcell(fauxquery,"id","1",1);
    querysetcell(fauxquery,"name","Joel",1);
    querysetcell(fauxquery,"age","31",1);
    querysetcell(fauxquery,"id","2",2);
    querysetcell(fauxquery,"name","Jason",2);
    querysetcell(fauxquery,"age","31",2);
    querysetcell(fauxquery,"id","3",3);
    querysetcell(fauxquery,"name","Godzilla",3);
    querysetcell(fauxquery,"age","100000",3);
    querysetcell(fauxquery,"id","4",4);
    querysetcell(fauxquery,"name","Batman",4);
    querysetcell(fauxquery,"age","0",4);
    return fauxquery;
}

The one downside of this approach is that you don’t have a lot of control over how ColdFusion formats the JSON that it returns. It looks something like this:

As you can see, ColdFusion’s JSON-encoding has done a few things:

• To represent the data, it has created two arrays: one for the column definitions, and one for the data. I would prefer an array of objects, but this is CF’s default behavior…
• Perhaps worse, all of the “keys” have been uppercased, regardless of the original case of the columns

In other JS libraries, you might have to do some pre-processing of the JSON in order to get the data into something more like an array of structures, and/or even re-work the casing on the keys (or modify the requirements in your app to account for the casing). This all, of course, assumes that you have control over the source generating the JSON. Good luck if you don’t!

Making the Model

With ExtJS’s data Model, the JSON formatting–and even the ultimate structure of the source data–just doesn’t matter. The Model is flexible enough to adapt, and you can easily configure it to consume just about any incoming structure/format that you can think of.

Here’s what we can do for our example data:

Ext.define('Person', {
    extend: 'Ext.data.Model',
    fields: [
        {name:"ID", type:'int'},
        {name:"NAME", type:"string"},
        {name:"AGE", type:"number"}
    ],
    proxy: {
        type: 'ajax',
        url: 'remote.cfc?method=getqueryarrays&returnformat=json',
        reader: {
             type: 'array',
             root: 'DATA'
        }
    }
})

Seriously, it’s that simple. We just tell the Model that we’d like to define a data object with a certain number of fields, and it’s smart enough to automatically find and create an association with the incoming data. Here’s how we did it:

• The “fields” config option allows us to tell ExtJS what we want our data object to look like. Notice that in this example, the “name” attribute is upper-case. This matches the upper-case keys being returned from CF.
• The “proxy” config allows us to tell the Model how we’ll retrieve the data that will build the Model. In this case, we’re using the AjaxProxy.
• In the proxy, we’ve also defined a “reader.” This is where the flexibility really comes into play. For the reader type, “array” has been specified. This tells ExtJS, at the very least, that our data is in an array format. Next, we’ve specified a “root” of “DATA.” This tell ExtJS that the “DATA” array will contain the values for the field definitions that we’ve already created.

And with that, our Model is complete. We can now hook this up to a data store, and we’ll be able to use the store on any number of data-driven ExtJS components.

A Bit of Tweaking

Even though that’s pretty simple, we’re still a bit tied to the formatting that ColdFusion has returned–after all, we used the uppercase column names in our field definitions. But what if we don’t want uppercase field definitions in our Model?

While we can certainly leave it as is, the ExtJS Model allows us to alternatively map field definitions using the “mapping” attribute. For the ArrayReader, the mapping can be the position in the data structure, rather than the explicit field name. So with just a slight modification, we can easily map our fields as before, but also create arbitrary field names in our Model field definitions:

Ext.define('Person', {   
     extend: 'Ext.data.Model',   
     fields: [       
        {name:"theid", type:'int',mapping:0},       
        {name:"myname", type:"string",mapping:1},       
        {name:"myage", type:"number",mapping:2}   
     ]
})

Wrapping Up

As we’ve, ExtJS’s data Model is an extremely powerful and versatile way of creating robust data objects for JavaScript applications. Not only do they serve as a great foundation for ExtJS’ data-driven components, but they (perhaps more importantly) are flexible enough to suit just about any requirement for incoming data. So instead of having to craft your application around the arbitrary data definitions which come from systems over which you may not have any control, you can easily leverage ExtJS data Models to take back the reins and define data objects based on the needs of your application alone. Pretty awesome

Ah, but that’s the neurosis of the Western mind; what can be done about it? Nothing

But even though I’m reticent about making resolutions, I have been thinking about the year that lies ahead, particularly from a professional standpoint. Over the last year, I’ve been very productive in my professional development. Particularly in relation to ColdFusion, JavaScript and SQL, I have tremendously expanded my competency, and I feel quite comfortable with where I am as a developer.

However, as we all know, complacency can be a silent killer. While it may not immediately harm a person in relation to their employment prospects, it can be detrimental in the long-term, if nothing else than by souring them to the prospects of learning new skills, exploring new technologies, etc.

I don’t want this to ever happen. I don’t feel like it’s a particular danger right now; I’m a naturally inquisitive person, so exploring new technologies, improving skills, etc. is just something I like to do. However, I do see this problem manifest in others, and I don’t like what it looks like. So, in my pre-emptive attempts to ward off such a fate, I’ve come up with a resolution (or two) of sorts for 2012.

Becoming “Expert”

When it comes to ColdFusion and JavaScript development, I feel very confident in my abilities. For any task, no matter the complexity, I am more than capable of crafting a solution that utilizes these technologies. I don’t say that in arrogance; it’s simply a fact. Ater so many years of using each, I can accomplish many things…and very quickly at that.

But while I am fully capable of using these technologies, I am also keenly aware of the extent (and limitations) of my competency. I have no delusions about my understanding of CF and JS. Yes, I can use them quite capably to do just about anything that is proposed. However, in regard to a more fundamental knowledge about the technology qua technology, I find many gaps in my understanding.

Therefore, one of the things that I want to pursue in 2012 is to begin a journey of becoming “expert” in ColdFusion and JavaScript. To me, the “expert” part that I’m missing is precisely this deeper understanding of how each technology *works* apart from the day-to-day grind of *using* them to accomplish discreet tasks. So this will mean that I will probably be doing a lot more reading about the technologies than actually using them. And you know what? I think that’s okay. The school of “learning by doing” has gotten me a VERY long way in my career. However, I think the next step wil come only by a very deep commitment to learning the bits that won’t necessarily come by trial and error.

You’ll notice I said “begin a journey.” I don’t expect to become “expert” by the end of 2012. However, I do want to determine a path that will get me where I intend to go. If I can accomplish just that, 2012 will have been a success.

Explore New Frontiers

One of the biggest keys to professional development, especially when it comes to web development, is to have an expansive portfolio of skills. This doesn’t mean, of course, that one needs to be “expert” in 7 different languages; that would probably be impossible for most people. However, it does mean that one is open to actively pursuing an understanding of how different technologies work and how each can be leveraged in various situations to accomplish particular goals.

In the coming year, there are a lot of technologies that I want to explore. In order to not get completely overwhelmed, I’m going to take a dual approach.

First, I’ve selected a handful that I want to do more of a “deep dive” into; these will require several weeks/months of exploration each. By the end of the year, I’d like to be able to say that I have a base-level competency with each technology–not something on the basis of which you’d hire me, but a working knowledge expansive enough to be able to accomplish something, and a solid foundation upon which I can build in the future. Here are the targets:

• Ruby
• Node.js
• jQuery
• Python
• ORM
• Git

Second, I want to build time into each week to take an hour or two to make a surface exploration of new technologies/snippets/etc. that I come across in the course of my daily blog trolling. So for example, if a new JavaScript framework is highlighted in one of the blogs I read, I want to explore that. My intention is, at the very most, to devote enough time to make a surface exploration. While this will not by any means make me competent in this or that technology, I think the practice will help to expand my skill set, simply by seeing how other people do things. I don’t have a discreet list for these, as I hope to leave time open for exploration as opportunities present themselves.

Avoiding Blackhole Projects

Over the last year, I found myself buried in blackhole projects. These were projects that I came up with on my own that didn’t have particular goals, or defined execution plans. With each of them, I started with the *intention* of learning/doing something new. However, I dreamed way too big, never accomplished whatever I had set out to do, and wasted a ton of time and effort for nothing except frustration.

In 2012, I’m going to make a concerted effort to avoid projects altogether. I’ve found that projects tend to devour an inordinate amount of time, simply because I end up spending a lot of effort on things that aren’t even central to the project itself (like getting the design pixel perfect, abstracting the 20 lines of JavaScript being used, etc.). So instead of using projects as a way of exploring this or that technology, I’m going to try to focus on simply executing very small, manageable development tasks. Whether this is simply creating a nice “Hello world” or developing some extension/module/whatever, keeping the scope tiny and defined will hopefully help me be much more productive in my learning and exploration.

Blogging

My blogging output in 2011 was a little underwhelming. While I did add some fairly in-depth posts, the number of posts was a bit disappointing. While quantity doesn’t really mean anything, it does reiterate to me the project blackhole problem I had during the year. So one of the things I really want to focus on in 2012 is becoming a better blogger. For me, I think this means being more intentional about regularly positing my experiences, while also not being quite so concerned with feeling like I have to write a novel each time.

Wrapping Up

So, it’s an ambitious plan for 2012. However, I’ve already made one important step: today, I officially abandoned a blackhole project that I had started last week. I’m beginning with a clean slate, and am looking forward to a great year of leaning, improving, and expansion of my development skills.

If anyone has any constructive thoughts/suggestions, I’d love to hear them!

<cfswitch expression="#img.extension#">
	<cfcase value=".gif,.jpg">
		Not transparent
	</cfcase>
	<cfcase value=".png">
		Transparent
	</cfcase>
</cfswitch>

No rocket-science here, just a simple switch statement which just so happens to have a “case” which evaluates a list of values. Honestly, I’m not sure I had ever done a list of values in a <case>, but it just works.

Thinking nothing of it, I quickly converted it to <cfscript> syntax:

<cfscript>
	switch(img.extension) {
		case ".gif,.jpg":
			writeoutput("Not transparent");
			break;
		case ".png":
			writeoutput("Transparent");
			break;
	}
</cfscript>

At first, I thought it was working. But as I tested a bit more, I noticed that it wasn’t. No error, per se, but the “case” was not evaluating as it did in the tag-based equivalent.

Turns out you have to do something a bit different with <cfscript>. Here’s an alternative that will replicate the tag-based functionality:

<cfscript>
	switch(img.extension) {
		case ".gif": case ".jpg":
			writeoutput("Not transparent");
			break;
		case ".png":
			writeoutput("Transparent");
			break;
	}
</cfscript>

According to the docs,

Multiple case constant: statements can precede the statement or statements to execute if any of the cases are true. This lets you specify several matches for one code block.

Ah, so it’s a “feature.” Hmm, personally, I think this feels kind of clunky. I imagine there is a good reason for why it’s done this way, but I’d love to know why–anyone?