Displaying PDF Form Fields Using iText and ColdFusion

One of my most recently developed web application is a new health care system for a number of laborites that perform various testing. The initial workflow has the central office receiving the lab test order and sample, entering it into the system, and then sending the sample and five or six (depending on the test ordered) worksheets along to the lab techs. Currently the worksheets that have to be completed are a variety of MS Excel, MS Word and an Adobe PDF file. This can take anywhere between 3 to 8 minutes to complete, depending on which forms are needed and what information is available on the test order. They asked for an easier way to do this.

I decided to explore generating a single PDF file that would be generated after the test order is entered in the system. The fields on the various documents that normally the administrative staff have to fill in would be populated by the web application. Normally this would be easy to do using cfpdfform, however for this project we’re using Railo instead of Adobe ColdFusion and the cfpdfform tag isn’t available. I was hoping that Railo 4 would include this tag as a feature, but there was no mention of it being included at cf.Objective() 2012 by the Railo team when I asked. I’ve asked on the Railo list twice, initially they said yes but most recently they haven’t responded. Therefore I needed to find an alternative solution, which I did, and here’s what I ended up doing.

Create a single Adobe Acrobat PDF Form

The first step was to use Adobe Acrobat Professional to combine the MS Excel, MS Word and Adobe PDF files into a single PDF file. The next step was to create the form fields within the PDF for the web application to populate from the database. Since every page had a “name” field, we just made that field name the same on each page.

Get iText

iText is a Java component that is designed to work with PDF files. It can create, read and update PDF files. The technical documentation on the website is pretty thin, they really encourage you to buy a copy of their “iText in Action” book. I’d have no problem with this, but they don’t currently offer a Kindle version yet so I’m holding off on that. I did find the API documentation is available online that you can use, and they do have the samples from the book available for download that may help.

Download the iText PDF community version from here. I decompressed these files and placed them in the same directory as I had my Railo JAR files, then I restarted Apache Tomcat. I’m pretty sure I could put these in my ColdFusion on Wheels application directory /lib directory and load them using the javaLoader for CFWheels, but I haven’t tried it yet.

CFDump the form fields

The next thing I wanted to do was to dump the form fields from the PDF file to make sure I was able to read the file correctly. I placed the single PDF file that we had created with all of the form fields with my ColdFusion on Wheels application directory. I then created a CFML page and used the following code to dump the form fields that are found within the entire file.

readPDF = expandpath("the_file_name_here.pdf");
writePDF = expandpath("#createUUID()#.pdf");
fileIO = createObject("java","java.io.FileOutputStream").init(writePDF);
reader = createObject("java","com.itextpdf.text.pdf.PdfReader").init(readPDF);
stamper = createObject("java","com.itextpdf.text.pdf.PdfStamper").init(reader, fileIO);
pdfForm = stamper.getAcroFields();

This dumped out a list of all of the form fields within the PDF so I can make sure that iText was able to read them, and that I spelled them correctly.

Populate the Form Fields

The next step was to populate these fields with data from my database. I also wanted to display this PDF in the users browser. I was able to to this pretty much as I did above, but just with a minor changes.

readPDF = expandpath("the_file_name_here.pdf");
writePDF = expandpath("#createUUID()#.pdf");
fileIO = createObject("java","java.io.FileOutputStream").init(writePDF);
reader = createObject("java","com.itextpdf.text.pdf.PdfReader").init(readPDF);
stamper = createObject("java","com.itextpdf.text.pdf.PdfStamper").init(reader, fileIO);
pdfForm.setField("date_sample_collected", "#date_sample_collected#");
pdfForm.setField("patient_name", "#patient_name#");
pdfForm.setField("date_of_birth", "#date_of_birth#");
pdfForm.setField("order_number", "#order_number#");
stamper.setFormFlattening(true);
stamper.close();
reader.close();
fileIO.close();

Summary

Using this code I was able to satisfy the initial request that the administrative staff, to save them from creating these additional forms manually with the data they had just entered into the web application.

I know I can make this better though, since not every page of that PDF is needed, it depends on the test selected. I plan to re-factor this application code in the future to to determine which test was selected, then only grab those worksheets. This means I’ll need to separate each of the worksheets out into a separate PDF file, then stitch them together, depending on what the user has entered for the order, into a single file.

Thanks To

Thanks to this cfSearching blog post that helped, as well as rip747’s Stackoverflow posting that helped me complete this task.

Beating the Flying Spaghetti Code Monster with CFWheels - Part 2

Recap

As I mentioned in part 1 of this post, when I started writing web applications with ColdFusion I quickly amassed a large collection of autonomous files that would often break if I changed a variable or parameter that should be passed into the next. Adding or removing a column to a form / database table could take 25 minutes to update all of the SQL files. I had become enslaved to the “Flying Spaghetti Code Monster”, as I like to call it, and knew there was a better way and wanted to find it.

The Application

As I also mentioned in part 1, I had been re-hired to maintain and update a web application I wrote 7 years prior. This application had well over 350 individual files and the folder size was near 80MB. Although I had started to use the Fusebox framework, which helped start to organize my files, I didn’t follow-through on converting everything to Fusebox. This meant that my directory structure looked something like this

Conversion

As also mentioned in part 1 of this post that I decided to slowly convert the application to the Coldfusion on Wheels framework. I decided to do this cutting off one “tenticle” at a time, or one section/feature at a time.
First I identified a small section of the application that I could replace with the CFWheels framework. I selected one of the “system options” that allows users to add new values to a drop-down list of patient types. The code for this was located off the root of my application in a /system folder along 15 other similiar system options that were almost identifcal. I also needed a place to put my CFWheels core files.
I decided to put the CFWheels framework files in a directory named “hla”, which is the web application name as referred to by the staff, under the /system folder, so /system/hla/

I setup my database connection info in the /conf/settings.cfm file so that my database connection would be the same as the spaghetti code portion. Next, I didn’t want to have two sets of Javascript, CSS or other asset files like images, so I had to modify my CFWheels layout.cfm file to reference the ones that the spaghetti code files were using.

At this point I could browse to the CFWheels portion, by manually entering the URL, and receive the default CFWheels welcome page. I then added a new entry on my system options menu to point to the new CFWheels location where the system option I was going to convert would live.
At this point I created a model file, Patiettype.cfc, and in the init function I added code to specify my database table names that didn’t follow the Wheels convention, but were used by the Spaghetti Code application
<cfset table("tbl_authors")>
Next I followed with each database table column as a property, using the current database column name for the column attribute but a CFWheels convention name for the name attribute, as outlined in the CFWheels documentation.
<cfset property(name="firstName", column="tbl_auth_f_name")>
These system options required that I setup some associations, and this was a great place to learn about that as I’d need it for every other part of the application.
At this point I could follow the CFWheels conventions and develop the CRUD for the Patienttype just like I would a brand new CFWheels app. After I have the CRUD working for this system option, I’d then remove the link to the old files and leave just the new one. Then I’d have the users work with that section for about a week. Once we were sure everything worked correctly, I’d slowly remove the old files (that I now had in Git version control).
Using this same method I slowly worked through all of the 16 system options, as if cutting off one “tenticle” of the Spaghetti Monster at a time. After I finished with those small, simple CRUD actions, I moved onto more complicated sections. The whole Contacts portion, then Hospitals portion, etc.

Progress

I found the Coldfusion on Wheels framework in February 2011, then started implementing it in March 2011, working part-time and slowly through each of the tenticles I mentioned. If new features were requested, or something needed to be changed, I fought the urge to do a “quick and easy” one or two file solution and forced myself to do it the CFWheels way. At first this was challenging, but once I got use to it, the changes happened faster.
At this point, December 2011, I’ve removed at least 23 of these “tenticles” with two medium sized tenticles and one large tenticle left to remove from the monster. I’m excited to finish these sections off and sever the head of this monster once and for all. In hindsight I’m glad I started with smaller tenticles as it allowed me to learn CFWheels with smaller sections of code, but also to find “gotchas” early on.

Gotchas

One of the things I didn’t expect was the Spaghetti Monsters session scope being overwritten when I moved went to a CFWheels framework page. This made it a little tricky because I’d use the session scope to retain the currently logged in user ID and if they were a valid user or not (to prevent unauthorized access). After I’d leave a CFWheels page and go back to the Spaghetti Monster code I’d be kicked out of my application. My “fix” for this, at least for the time being, was to use client variables for the Spaghetti code portion of the application and session variables for the CFWheels portion of the application.
The next thing I ran into, since my session scope was being overwritten, was once I moved to a CFWheels portion of the application I’d didn’t have my session variable with the user ID from the Spaghetti Monster code. To fix this I added an obtuse URL variable on the menu links in the Spaghetti Monster code that CFWheels would then use to identify the user and create the session scope variables I wanted.

Summary

I feel converting this application to the CFWheels framework has helped with the following:

1. Brought some sanity back to how this application is structured and maintained (huge benefit)
2. Saved gobs of time not having to maintain query files with endless CFIF statements and cfqueryparam tags (CFWheels ORM does that all for me, huge benefit)
3. Made the application run faster, I mean noticably faster to my users
4. Decreased the code folder from 80MB to 33MB
5. Allows me to add new features or make changes in less time then before because things break less when changes are made
6. Allows anyone that knows about the MVC pattern to help or take over this application
7. Made Coldfusion programming fun again (most important of course)

While converting this application from the Spaghetti code to the CFWheels framework from “the inside out” has been a slow process (because of my, not the framework), I don’t regret doing it at all for the reasons stated above. It’s been a great way to learn CFWheels and to work with the CFWheels community. At this point I can’t imagine developing Coldfusion applications without using the CFWheels framework.

Beating the Flying Spaghetti Code Monster with CFWheels - Part 1

Background

I started writing web application using Coldfusion in 2000 with version 4.5. Like probably most of us back in those early days, we had one hand on the keyboard and a copy of Ben Forta’s Web Application Contrution Kit (WACK) in the other. My development of applications consisted of creating a single autonomous file, with both database queries and conditional display code. This of course was tied to yet another autonomous which did the same, etc, etc, etc. Thus the “Flying Spaghetti Code Monster” Even after I started to use the Application.cfm “framework” within ColdFusion, I still had all of these autonomous files that would often break if I changed a variable or parameter that should be passed into the next.
After developing a few applications like this I realized there had to be a better way. Somehow I came across the Fusebox 2 framework and started to look into it. This framework encouraged you to break the application up into separate files, query file, action file, display file, and then “chain” these together like fuses in an electrical circuit.
Shortly after Fusebox version 3 was released and I begin using this framework to develop my applications. I especially loved the formurl2attributes function that put all of your url and form variables into a single scope. I started to convert small pieces of my existing applications from my spaghetti code to organized code following the Fusebox methodology, however I ended up sticking more with the file naming then actually using the framework after a while. When Fusebox version 4 was released I found it difficult to grasp, as well as the CFC concept that was brand new. My two and a half years of part-time web application programming was coming to a close though as I left to work full-time as a systems administrator.
Over the next 7 years the largest of the web applications I wrote, a Laboratory Information System (LIS), continued to work for the University lab I wrote it for with minimal adjustments to it.

Back to Developing

At the end of those 7 years I was hired back, part-time, for this same lab, and part-time for another lab (so it’s a full-time position) on campus. When I looked at this old source code I saw the “Flying Spaghetti Code Monster” glaring back at me. I had well over 350 individual files and the folder size was near 80MB, not to mention no source control software to help manage it. I knew there had to be a better way to handle this, I didn’t want to continue to develop this way.
The first thing I did was get the software into a source control system. Subversion was the choice as it was the easiest for me to grasp initially. I went back to look at the Fusebox framework and saw it was still around, but it didn’t seem like many were using it any longer. There was the Mach-II framework, but I couldn’t seem to understand that either. In the meantime I continued to serve the hard master known as the “Flying Spaghetti Code Monster” and maintain the code as it was, even adding new features, without much difference in my development pattern.
After five months of developing this way I came across the ColdFusion on Wheels framework website, not sure how (probably Twitter). After greedily consuming all of the available screencasts I knew this was the way I needed to develop the application. The question was, how do I learn the CFWheels framework and use it to power this application? I saw two choices:

1. start from scratch in a parallel environment using CFWheels and recreate all of the functionality
2. slowly, one “tenticle” at a time, battle the “Flying Spaghetti Code Monster” by converting a section to the CFWheels framework

After considering it I decided choice 2 was for me, to slowly convert the application. In the following post I’ll explain how I’m doing this.

Coda book for CFWheels // Stofke on wheels

via wheels.onebuttonapps.net

This is a great reference tool for looking at the ColdFusion on Wheels framework documentation within the Coda web editor from Panic