Tim Dietrich

Custom Software Developer

Home Services Portfolio Blog About Contact Newsletter

Fireball: A Technique For Integrating Hosted FileMaker Databases with Native Apps

FileMaker is an absolutely fantastic database platform. With it, you can easily develop custom database solutions that run on Macs, Windows-based PCs, iOS devices. And with the platform's Custom Web Publishing (CWP) capabilities, as well as the new WebDirect technology, you can also extend FileMaker solutions to the Web.

However, FileMaker is a database platform. It is not a full-blown software development environment. As a result, it is not an ideal tool for developing native applications or SaaS offerings.

Why? Let’s look at FileMaker’s desktop, browser-based, and mobile options.

Desktop Options

For SaaS, you could require that customers purchase FileMaker Pro licenses. But that can be extremely expensive, and a somewhat odd (and messy) way to provide your service. Explaining to a prospective client that, in order to use our service, they will first need to purchase FileMaker licenses, can be very awkward.

With FileMaker Pro Advanced, you could create runtime solutions for use on Macs and Windows-based PCs. This is a potentially good solution if your goal is to provide native apps only for those operating systems. However, it is not an option if you need to provide native apps for use on iOS or other mobile operating systems (such as Android). Also, FileMaker runtime solutions have limitations. For example, you cannot use them to access hosted FileMaker databases. (Well, sort of. More on that in a minute.)

Browser-Based Options

You could provide browser-based SaaS solutions using FileMaker as the backend database, and either a CWP-based solution or FileMaker WebDirect. However, CWP solutions - especially complex solutions - can take a considerable amount of time to develop, and come with all of the limitations and challenges or browser-based solutions. And FileMaker WebDirect, while extremely impressive as a browser-based FileMaker client, isn’t as full-featured as its desktop counterparts. In addition, WebDirect sessions count as concurrent connections, which can increase the cost and potentially reduce the scalability of your service.

Mobile Options

As far as mobile solutions go, there’s FileMaker Go. It is an amazing app, providing an excellent mobile FileMaker client experience. With it, users can access local FileMaker databases, as well as hosted solutions. However, Go is iOS-only, and there’s no support for other mobile operating systems in sight.

For SaaS needs, FileMaker Go might be an option. You would need to explain to clients that they will need to download the free Go app. In addition, just like WebDirect users, FileMaker Go users who are connecting to a hosted database count as concurrent connections. (There is a way to avoid this, which I’ll describe later.)

Rolling Our Own Clients

I’ve been discussing these challenges in great depth with Hal Gumbert, who has similar interests. Like me, he’s interested in providing native apps and possibly SaaS offerings.

Over the course of several lengthy phone conversations and chats, we realized that what we really want to be able to do is develop native apps, some of which (for the SaaS offerings) would serve as clients that would connect to FileMaker on the backend. In other words, we want to be able to use our knowledge of FileMaker as leverage as much as possible, while working through the issues that I described above.

To create the native apps and clients, we would need to use a development environment. Here is what we are looking for in a development environment:
• Support for creating native (compiled) apps.
• A single environment that can handle all of our targets (OS X, Windows, iOS, and possibly Android).
• Possibly support for generating Web solutions. (Xojo can do this in an interesting way.)

Hal is using Xojo as his development environment. He has quite a bit of experience with it, and with Xojo's new support for iOS, it seems like a great fit.

As for me, I’m still evaluating both Xojo and LiveCode. At the moment, I'm leaning towards using LiveCode.

Integrating With FileMaker

Regardless of what development environment(s) that we end up using, we’re going to need a way for our applications to communicate with FileMaker Server on the backend. This needs to be a method that is both affordable and scalable.

And that’s where the idea for Fireball came from...

Introducing Fireball

At some point, I had one of those “Eureka!” moments. I realized that we already have everything that we need to handle the integration. In another series of chats with Hal, the idea and technique became fully formed.

We’re calling it Fireball. (Geeky, I know. But whatever. No apologies!)

Here's Fireball in a nutshell:
• “Fireballs" are XML-like payloads. (They are similar to the payloads that EasySync uses.) See the end of this post for a few examples.
• They are sent directly to FileMaker Server via CWP XML calls.
• Fireballs can be used to create new records, update existing records, delete records, and query a database (using a SQL statement).

So, the native, compiled apps would “cast” a fireball at FMS. FMS would receive it, process it, and return a fireball as a response. Easy!

Here are some of Fireball’s benefits:
• It works with any native app that can issue an HTTP request. (We’re using HTTPS POST.)
• It uses CWP XML calls, which do not count as concurrent connections.
• It can be easily extended with custom fireballs.
• It is extremely fast and reliable.

Earlier in this post, I mentioned that we have found a way for FileMaker runtimes to communicate with hosted databases, and for FileMaker Go-based solutions to communicate with FMS without using concurrent connections. You can do so using the Fireball technique.

Challenges

Of course, there are some challenges with this technique, and using it will take some work.

For instance, this is the ultimate separation model, in that it is entirely stateless. So you cannot take anything for granted, especially things that we do tend to take for granted when we use FileMaker clients. That means that you’ll have to put effort into handling found sets, processing finds (queries), identifying and handling collisions, dealing with record locking, and so on. In addition, a single fireball isn’t transactional - and the technique “out of the box” isn’t transactional. (We have come up with a way, using multiple fireballs, to resolve this.)

The Possibilities Are Intriguing

The Fireball technique opens up a world of possibilities.

As I mentioned above, it can be used with any client that has the ability to send HTTP requests. That includes FileMaker runtimes, as well as FileMaker Go.

While the intent is to use the technique to integrate with a FileMaker database on the backend, it could also be easily modified to work with other types of databases (MySQL, SQL Server Oracle, etc).

Security can be greatly enhanced. I’m not going to get into the details in this post, but it is entirely possible to fully secure the fireballs.

For solutions that require transactional processing, multiple fireballs (or custom fireballs) can be used to prepare a transaction, execute it, and evaluate it. For example, you could easily use fireballs to push an order (with a parent header record and potentially multiple child lineitem records). And you could also use fireballs for syncing data.

Progress and Next Steps

Hal and I have been working with fireballs for a few weeks now. We’ve successfully used them in FileMaker Go and runtime solutions. Hal has successfully developed Xojo-based clients that use them. And just yesterday, I was able to get the first LiveCode-based client to use fireballs to push new records back to FMS (complete with standard fields and an image). We’re taking baby steps, but it feels as though we’ll be sprinting soon.

At this point, most of our effort seems to be going into learning our new development environments. For Hal, even though he has quite a bit of experience with Xojo, its support for iOS is very new, so he is getting familiar with that. As for me, I’m getting familiar with LiveCode, especially its English-like programming language.

Our next step is to create full-featured, native apps that take full advantage of this Fireball technique. We’ll keep you posted as things progress.

In future posts, we'll share more about how the fireballs work on a technical level. Stay tuned...

An example of a "record" Fireball, which can be used to add a new record, or update an existing record.

<fireball>
<record>
<table_name>Contacts</table_name>
<pk>
<name>Contact_ID</name>
<value>BCC0F93B-E0EF-4E73-8551-0681ACA0D4A6</value>
</pk>
<field>
<name>First_Name</name>
<value>Jane</value>
</field>
<field>
<name>Last_Name</name>
<value>Smith</value>
</field>
</record>
</fireball>

An example of a "SQL" Fireball, which can be used to issue SQL commands.

<fireball>
<sql>
<statement>SELECT Company, Full_Name FROM Contacts</statement>
<field_delimiter>|</field_delimiter>
<record_delimiter>¶</record_delimiter>
</sql>
</fireball>