Making Knockout.js Support External Templates

The last month has been a blast for me!  Contributing to my “kid in candy store” syndrome was my discovery of Steve Sanderson’s Knockout.js framework.  In a nutshell – Knockout.js provides the basis for using an MVVM (Model View ViewModel) or, if you’re familiar with Martin Fowler’sPatterns of Enterprise Application Architecture”, a “Presentation Model” approach to building web applications in JavaScript and HTML.  Beyond that, I’m going to assume you are either familiar with Knockout.js (if not, go here, read a bit, enjoy yourself and then come back and finish my post), OR you like throwing yourself to the wolves by skipping introductions and diving right into extending unfamiliar frameworks.

Knockout.js comes with an out-of-the-box template engine (using jQuery.tmpl).  Using this template engine requires you to place your templates inside script elements.  While I could probably guess at the reasons that led Steve Sanderson to set it up this way (all good reasons, by the way) – I think it’s less than ideal for anything but the smallest of projects.  So, I set out to tackle the following:

  • Allow for templates to be created and edited in separate files that don’t require script tags, with the added benefit of your IDE-of-choice being able to provide proper syntax highlighting (since many break once you place HTML inside a script tag).
  • Allow for templates to be loaded from the server on an as-needed basis, and once they’ve been downloaded, cache them so that they don’t have to be downloaded again.

As I began to look over Steve Sanderson’s default jQuery template engine plugin for Knockout, I realized I could take advantage of his work and simply tweak a few things to allow for external templates.  The primary change relates to the behavior of the “getTemplateNode” function.  Let’s look at the original:

this.getTemplateNode = function (template) {
        var templateNode = document.getElementById(template);
        if (templateNode == null)
            throw new Error("Cannot find template with ID=" + template);
        return templateNode;
    }

As you can see, it’s very straightforward. The function looks for an item in the body with an id that matches the template name passed to it. If it cannot find it, an error is thrown, otherwise, it returns the node. Now let’s look at my version:

this.getTemplateNode = function (templateId) {
        var node = document.getElementById(templateId);
        if(node == null)
        {
            var templatePath = this.getTemplatePath(templateId);
            var templateHtml = null;
            $.ajax({
                "url":templatePath,
                "async": false,
                "dataType": "html",
                "type": "GET",
                "timeout": this.timeout,
                "success": function(response) { templateHtml = response;},
                "error": function(exception) {
                    if(this.useDefaultErrorTemplate)
                        templateHtml = this.defaultErrorTemplateHtml.replace('{STATUSCODE}', exception.status);
                }.bind(this)

            })

            if(templateHtml === null)
                throw new Error("Cannot find template with ID=" + templateId);

            var node = $("<script/>", {
                                         "type": "text/html",
                                         "id":templateId,
                                         "text":templateHtml
                                      }).appendTo("body");


        }
        return node;
    }

So what's different?

  • Like the original jQuery template engine plugin, my version checks to see if the template is already in the DOM, and if so, it simply returns it.  However, if the template is not found in the DOM, an ajax call is made retrieving the template from the server. 
  • You’ll notice we’re calling “this.getTemplatePath” – it’s here that we’re utilizing the configuration values that may have been provided to specify where and how to access the template on the server (more on that in a moment).
  • If the call to the server fails – and if the value of “useDefaultErrorTemplate” is true – then a default error template is provided instead. 
  • The ajax call is synchronous, and the result of the call is assigned to the templateHtml variable (you can set an optional timeout value to prevent the call from blocking the browser indefinitely!).  Once we have a template, we add a script element to the document, set the attributes appropriately, and set the contents equal to what we just retrieved from the server.  We’re now at a state identical to if we had simply included the template in the document from the start (i.e. – Knockout’s functionality is intact).

My external template plugin allows you to set the following options:

  • templateUrl: the directory on the server where the templates reside.  For example, “/Templates” for a sub-directory relative to where the document originated.
  • templatePrefix: a standard prefix that is pre-pended to all template file names.  For example, if you want a naming convention of “template_”, then you’d set this member equal to that value.
  • templateSuffix: just like templatePrefix, except it’s appended to the template name.  For example, you might have naming convention where all template files end in “.tpl.html” instead of “.html”.
  • useDefaultErrorTemplate: defaults to true.  Setting this to false means that your application will display the error response you receive for any template that cannot be found.  This could get messy (ever seen an IIS 404?)
  • defaultErrorTemplateHtml: allows you to set the actual html content of the default error template.

The script file containing this plugin takes care of auto-wiring itself to knockout.js so that all you have to do is simply reference the script in your html document.  It does this by executing the following:

ko.ExternaljQueryTemplateEngine.prototype = new ko.templateEngine();
// Giving you an easy handle to set member values like templateUrl, templatePrefix and templateSuffix.
ko.externaljQueryTemplateEngine = new ko.ExternaljQueryTemplateEngine();
// overrides the default template engine KO normally wires up.
ko.setTemplateEngine(ko.externaljQueryTemplateEngine);

The full source for the “Knockout.js External Template Engine” can be found on github.  Let’s take a quick peek at a very simple example showing nested external templates being pulled down as they are needed (this example is included in the git repo):

The Templates:

The following three templates are all in their own .html files.

The ‘Master’ template (below) is simply a div container.  In a real world scenario, it might contain several other elements that should appear at the highest (“app”) level. In our example, it simply provides a top level container within which "companies" are displayed.

<div data-bind="template: {name: 'Company', foreach: Companies }"></div>

The ‘Company’ template (below) is displayed for each company in the collection. It contains a list of Employees, and iterates over each one, using another ‘Employee’ template.

<fieldset>
	<legend>
		<span data-bind="text: CompanyName"></span>
	</legend>
	<div data-bind="template: {name: 'Employee', data: Employees}"></div>
</fieldset>

The ‘Employee’ template (below) is self-referencing. If an employee has children, then each child is rendered using the ‘Employee’ template.

<ul>
    <li>
        <span data-bind="text: Name"></span>
        <span data-bind="template: {name: 'Employee', foreach: Children}"></span>
    </li>
</ul>

The Main Page:


<html>
<head>
    <link rel="stylesheet" href="style.css" />
    <script type="text/javascript" src="jquery-1.5.js"></script>
    <script type="text/javascript" src="jquery.tmpl.js"></script>
    <script type="text/javascript" src="knockout-latest.debug.js"></script>
    <script type="text/javascript" src="koExternalTemplateEngine.js"></script>
    <script type="text/javascript">
        // Simple view model.  This is a one-way binding example (i.e. - none of the members are observable)
        var viewModel = {
                Companies: [
                {
                    CompanyName: "ACME",
                    Employees: {
                                Name: "Bugs Bunny",
                                Children: [
                                    {
                                        Name: "Peeps",
                                        Children: [
                                            {
                                                Name: "Daffy Duck",
                                                Children: []
                                            },
                                            {
                                                Name: "Tweety Bird",
                                                Children: []
                                            },
                                            {
                                                Name: "Road Runner",
                                                Children: []
                                            }
                                        ]
                                    },
                                    {
                                        Name: "Always Getting the Short End of the Stick",
                                        Children: [
                                            {
                                                Name: "Yosemite Sam",
                                                Children: []
                                            },
                                            {
                                                Name: "Wyle E. Coyote",
                                                Children: []
                                            }
                                        ]
                                    }
                                ]
                    }
                },
                {
                    CompanyName: "Superfriends",
                    Employees: {
                                Name: "Batman",
                                Children: [
                                    {
                                        Name: "Lesser Peeps",
                                        Children: [
                                            {
                                                Name: "Superman",
                                                Children: [
                                                    {
                                                        Name: "Aquaman",
                                                        Children: []
                                                    }
                                                ]
                                            }
                                        ]
                                    },
                                    {
                                        Name: "Wonder Woman",
                                        Children:   [
                                            {
                                                Name: "Robin",
                                                Children: []
                                            }
                                        ]
                                    }
                                ]
                    }
                }
            ]
        };
        $(function() {
            // demonstrating that templates can be called from a different path
            // than this html file was delivered from (same server, of course)
            // and also that special template file naming conventions can still
            // be used without cluttering up the name of the template itself.
            ko.externaljQueryTemplateEngine.setOptions({

                                                            templateUrl: "Templates",

                                                            templatePrefix: "tmpl",

                                                            templateSuffix: ".tpl.html"

                                                       });

            ko.applyBindings(viewModel);
        })
    </script>


    <!-- Thanks to the prefix, suffix and url settings above, the "Master" template will be pulled down from Templates/tmplMaster.tpl.html -->
    <div data-bind="template: {name: 'Master';, data: viewModel}"></div>


In the code above, we’re creating a view model that contains a simple – but ragged – hierarchy (note that this is a one-way binding example, since the members are strings and arrays, not observables).  Then, in our DOM-ready function, we’re setting the templateUrl, templatePrefix and templateSuffix.  (Since we’ve already included a reference to “koExternalTemplateEngine.js”, the plugin has replaced the default Knockout template engine.)  Lastly, we call ko.ApplyBindings() on our view model to tell Knockout to wire everything up.

Here’s a screen shot of the requests (using Chrome).  Note that the last three requests are for the templates, and they occur after the page is loaded (click on the image for a larger view):

image

If you’re working with Knockout.js, I encourage you to try this plugin out and give me your feedback!

 

Tags: , , ,

  • http://knockmeout.net Ryan Niemeyer

    Great work! Had a couple of quick questions/comments for you:

    1. I think that it is great how you just reference the script and it works out of the box. Also, options that you provide are helpful. I think that it is a little less desirable that it has to be so much of a fork of the current jQuery templating engine. I was wondering if you considered ways to minimize the duplication of code.

    For example, the current 'template' binding does not have an 'init' function. You could add an 'init' function for the template binding at run-time and do your template loading there. You could still set/read your options off of the templateEngine object and I suppose even support people passing overrides through the template binding itself. Otherwise, you could create a custom binding (like 'xtemplate') that wraps the current template binding and but does the loading, if necessary. Just a thought, since there were some fairly recent fixes to the jQuery templating engine for memory leaks (your code has the fixes) and this would ensure that your plug-in would never be out-of-sync and reduce the need to duplicate code.

    Another thought is that maybe you could take a look at what it would take to add a hook for this type of functionality to Knockout core. Steve seems receptive to hooks that allow plug-ins to be developed externally. Maybe something like a "preGetTemplateCallback".

    2. I have always stayed away from forcing AJAX requests to be synchronous (or at least been told to avoid it), because of the possibility of locking up the browser if the web server is slow to return the request. I completely understand why you want to use a sync request and see how it would be complicated to load the templates on demand and asynchronously. I would love to hear someone say that it is not a big deal. Just wondering if you have any thoughts on it.

    Again, nice work on this project. I look forward to seeing other thoughts that you have about Knockout.

    • http://ifandelse.com Jim Cowart

      Ryan – thanks for the input. I *love* the idea of just adding a hook to KO core and passing a callback. I'm going to mull that over this weekend and try to get something submitted early this week. It's funny you mentioned the idea of adding an 'init' function – I started to go that route twice and stopped myself (and can't for the life of me remember why I stopped). I hear you on forcing the ajax call to be synchronous! It left a queasy feeling in my stomach – but I didn't see a way in the existing code base to handle it asynchronously (without significant forking). I think a great thing for all of us KO devs to consider long term would be re-factoring key pipelines to operate async – it would probably provide an easier way to extend the core via injected callbacks (like your idea above). Also – just wanted to say that when I first stumbled onto Knockout.js, I read through a lot of your discussion board posts, and some of your site and I've seen the commits you've made on github – GREAT work man!

  • http://knockmeout.net Ryan Niemeyer

    I do know of one case that would fail in the 'init function', but would work in your engine. That is when you are using the new "template polymorphism" feature where you can pass in a function to decide the name of the template. When using 'foreach' the function is passed the item in your array and is evaluated in the engine. A binding couldn't really handle that part. It is a less common use case, but maybe you noticed that when you initially tried to use the 'init' function.

    Adding a hook in the core for this functionality would be a valuable addition. I hope that you keep posting about your experiences with Knockout. Cheers!

  • Toby

    Hey there. I appreciate the article. It's one of the first I hit on when looking to do this kind of thing. However, maybe I'm missing something, but I don't really think you need to overwrite any of the ko functions to use external templates. Maybe I'm missing something. I stole some of your work and put together this sample to kind of prove out the use of dynamic views/view models:
    https://github.com/LittleUmbrella/single-page–dy

    Let me know what you think – if I'm off the mark, or what. Thanks again for the article and bits that sped me along.

    • http://ifandelse.com Jim Cowart

      Toby – my apologies for taking so long to respond to your comment (things have been a bit hectic schedule wise for me). I checked out your git repo and have a couple of thoughts. Sure, it's certainly possible to do things the way you are, but I believe you're making some assumptions and trade-offs that aren't worth the risk. First, if you were to build a Knockoutjs app like this, you now effectively have two entry points for templates – the manual route you're taking, plus what KO provides. KO is by no means the perfect framework, but the point of a framework is to provide a consistent abstraction over the problems it's attempting to solve. By handling templating outside KO core, you are forced to apply bindings each time you load a template. You've also ruled out the option for nested templates – unless you manually recurse them, and apply the bindings on each one. Ultimately, by then time you write the code to handle all these issues, you will have solved the same issue Steve Sanderson has already solved in context of working with templates inside KO. While I'm not crazy about the sync call to get an external template, it's a one-time hit per template, and you have the option to time it out – I consider those far less of a disadvantage as having to solve the same problems in my own code that KO already solves for me, and still lose the advantages that templates have working through a template engine that has been implemented for KO specifically (like the stock one, or the one I created, or any of the others out there). It's also a reasonable course of action to fork KO and submit a pull request with changes that allow for a callback to be invoked prior to retrieving a template, that takes a 'continuation callback' as an argument. I've voiced this frustration before – if that were possible, then a sync call wouldn't be necessary. As it stands, though, KO was written with the assumption that template retrieval would be synchronous. Anyway – I think it's good that you're exploring this, but I don't think I would go the route you're recommending. If you're going to work with a framework like KO, I would either fork and modify to extend it to do what I want it to do, or abandon it and roll my own (or go with another) – rather than code a large of part of a separate framework and run it along side one that already has the facilities to do what I've coded as well. Hopefully that makes sense! Thanks a ton for giving me your feedback and for visiting the blog…

  • http://www.facebook.com/profile.php?id=668916410 Jayme Edwards

    Just started looking at this yesterday and it looks great but I see one problem with it. The main reason I see to use this is not to request the templates until you need them, but Knockout grabs templates for any div with a data binding expression that evaluates to a remote template even if it’s wrapped by a div that’s not visible. What approach are you using for dynamically adding divs to the page with data binding expressions and applying those bindings on an as-needed basis?

    • Anonymous

      Jayme – thanks for the comment. My plugin is effectively monkey-patching Sanderson’s template engine plugin that comes as part of KO core. It intercepts the getTemplateNode call, and enables the engine to fetch the template from a remote endpoint. Once it gets the response, it places the contents in a Script block (just like normal KO templates) and hands control back to the normal KO process – so it’s using Sanderson’s approach to cache the templates in local memory, and to render them where they are referenced. This means that anything you can do in a normal out-of-the-box template should be possible also with external templates – since they use the exact same processing pipeline other than my engine’s patch to allow external fetching.

      I have a question though – you said “Knockout grabs templates for any div with a data binding expression that evaluates to a remote template”. That is not my understand of KO core’s functionality, unless the ability to retrieve external templates has recently been added to it. Do you have an example/gist of your code doing this now that I could check out?

      Thanks!

      • http://www.facebook.com/profile.php?id=668916410 Jayme Edwards

        Yeah I didn’t explain real well. I figured this out since, it was a basic knockout knowledge misunderstanding. I don’t want my remote templates loaded until I show them, and any div on the page with a template that’s not local (using your helper) goes to the server so I have to dynamically add a div with the right binding and re-call applyBindings when I show a new view.

        • Anonymous

          Jayme – one thing you could try is to make the template name a function call that returns an observable of the template name you want to render. You can effectively have a placeholder div that renders what template the function returns, and by setting the observable’s value, it causes the template to be rendered/bound, etc. It’s probably easier to “see” rather than explain….a while back I was working on troubleshooting the ko.namespaces plugin with Hunter Loftis, and I threw up this git hub repository: https://github.com/ashbylane/ko.namespace.test. Check out the code in the Index.html file, along with the code in js/main.js. You’ll see that the main div’s template binding is to “templateOption” – which is a function on the view model. Bring this up in Chrome and open the Developer/JavaScript panel and watch the requests happen on the “Network” tab – and you’ll see that the templates are pulled down later, only as they are needed, since the template name isn’t available for KO to resolve until you take an action that causes the templateOption function to evaluate again….ping me if you have any questions! (@ifandelse on twitter, or jim at ifandelse dot com)

          • http://www.facebook.com/profile.php?id=668916410 Jayme Edwards

            Makes good sense. I’ll have to try that approach.

        • Anonymous

          Meant to say – forgive the rough nature of the repo I mentioned in my first reply. :-) It was literally something I threw together to test out some issues with Hunter Loftis’s namespaces plugin.

  • Anonymous

    Thanks so much for the kind words. I, too, hope frameworks like Knockoutjs help make web development more attractive and productive for .NET developers (who, in my opinion, have suffered too long under horrible web abstractions like WebForms)….

  • http://www.facebook.com/profile.php?id=668916410 Jayme Edwards

    One problem I seem to be having, if you have a remote template that uses another template that’s a remote template, it seems to retrieve the “nested” one but it doesn’t apply it (show it on the page).

    • http://www.facebook.com/profile.php?id=668916410 Jayme Edwards

      I stand corrected – a visible binding in my nested template was evaluating to false with a recent model change. Works great now!

  • http://dynamic-tools.net Peter

    This is a great technique. I changed it slightly so I could have template names like my.template.name and have that map to my/template/name.html.

    Works a treat!

    • http://ifandelse.com Jim Cowart

      Peter – great idea! Feel free to submit a pull req on github – (or send me a gist to how you implemented that) and I can see if it would make sense to incorporate it.

  • John Fields

    Hey Jim, I was just trying to get to the source on github, but I’m getting a 404. I’ll check back later, but if the repository is moving please let us know!

    Thanks.

    John

  • http://twitter.com/gudmundurh GuðmundurHreið..

    Thanks for a nice article! It inspired me when writing my own solution.

    It struck me that you use async=false when fetching templates. As getTemplateNode is itself synchrounous, it seems to be the only way, but doesn’t it bother you blocking the page – possibly multiple times for complex scenarios? It does bother me, so I resorted to preloading all templates through a single JS file now, to be free of blocking issues while rendering.

    Another thing: I had problems with appending the template script tag with jQuery in IE<=8, so I resorted to using old fashion DOM code for doing that. Does the jQuery method work for you?

  • Anonymous

    Hey, finally got this working after some frustration… FYI – it doesn’t seem to work with Knockout 1.30beta but is fine with 1.2.1 ;)

    • http://ifandelse.com Jim Cowart

      Hey – unfortunately I haven’t had time to update it to work against the 1.30 beta, but I do hope to find some time in the near future to take care of that…

  • https://buthrakaur.myopenid.com/ Buthrakaur

    Is it possible to specify external template name otherwise than using data-bind=”template..” attribute? I’d prefer to supply it in ko.applyBindings() call like ko.applyBindings(viewModel, element, ‘templateX’). My application screens are divided into many small KO view models/js files and I like to keep HTML views clean of KO code. My usual view looks like


    new CustomerSelector($(‘#customerSelector’));
    new ListOfProjects($(‘#listOfProjects’));

    The KO ligic and bindings are kept separated inside of JS objects.

  • Srabonti

    Hi Jim, we are using your template rendering for our new jquery mobile project, and it has been working wonderfully so far, however I came across a bit of an issue when rendering multiple questions, it seems to render all but the first template type, so in other words if I have 6 questions, 4 radio and 2 dropdowns, it renders the last 3 radio and 1 dropdown question, I am not sure why it would omit this, if I removed the script tags from the template and place them directly in the html page, it works fine. Furthermore, we were initially using div tags within the templates but end up having issues with radiobutton rendering, once we switched to script tags it seemed to work fine. Is this an issue that has been reported before?

    Thanks
    Srabonti

  • Itnoabrs Ganguly

    Hi, I tried posting my comment here before, but somehow it didn’t go through.
    We have been working your framework for templating out different question types we wish to display. It has been working really well for us except when it came to binding to a list of radio buttons, I am having issues where for some reason every question after the first radio button question of the page has its options repeated 5 times, till I was forced to change to script tags instead of div tags and the options stopped getting duplicated. However I still have an issue, in that all but the first question now being shown…

    here is the main htm

    Previous
    Next

    and here is the Radiobutton template

    • http://ifandelse.com Jim Cowart

      Hi – sorry for the trouble you’re experiencing. I will try to take a look at this over the weekend. Bear with me – currently buried in projects!

  • Epstone

    something is wrong with your pre tag / formatting?!