And in case you're curious, this is how the spotlight looked in practice :)

When it makes sense: self-contained components.

When using external components, you have to decide whether the external component has responsibility for managing the DOM or ember does. If the responsibility can be handed off entirely to the external component, it can be a good choice. For example, imagine a calendar popup component that is a jquery plugin, triggered with this imaginary api:

var calendar = $(someElement).calendar({  
  onChange: function(date) {
    alert(date);
  }
});

calendar.setDate(new Date());  

In this case, the internals of the calendar are opaque and we don't want Ember to manage any of the dom. This can be trivially wrapped in an ember component like so:

App.SomeCalendarComponent = Em.Component.extend({  
  currentDate: null,
  initCalendar: (function() {
    var _this = this;
    // this.$() returns a jQuery object scoped to the component's DOM element
    this.calendar = this.$().calendar({
      onChange: function(date) {
        _this.set('currentDate', date);
      }
    });
  }).on('didInsertElement'),

  updateCalendarDate: (function() {
    // the date changed elsewhere, so update the displayed date
    this.calendar.setDate(this.get('currentDate'));
  }).observes('currentDate')
});

This component can now be easily used in your templates like so, binding the date property to a variable of your choosing:

{{some-calendar currentDate=someVariable}}

EvilTrout has a great tutorial that goes into this in more detail.

But can you do better?

Maybe. There are two cases when the above may not be the best approach.

1. Has someone already made an ember specific version of what you want? For example, http://emberui.com/documentation/calendar. This can both save you hassle and maintenance and integrate better out of the box, including better variable binding and runloop integration.
2. Do you need lots of customization? Ember provides massively powerful tools for building complex UIs. If you want something that you can customise to your precise specifications, then rolling your own will probably pay off long term.

When it very rarely makes sense: components that wrap a lot of DOM.

For example, a modal component. Treating external components as a "black box" is quite trivial to integrate. However if you want your component to have DOM managed by Ember inside of it you will almost always regret trying to shoehorn external libraries into this role. The reason for this is that there is now an unclear seperation of concerns between which library is responsible for which part of the DOM. You need to manually keep track of the rendering flow of e.g.

  Ember renders container
    → Plugin renders modal container and animates
      → Ember renders modal contents

The reverse must be handled for teardown, and it can get very complex when trying to square the expectations of the external library with the expectations of the ember runloop and rendering process. Whilst it can be done, and is sometimes unavoidable, in my experience you will almost always write less, and simpler code just using ember's built in powertools to build what you need, rather than trying to bash something into shape to work in this role.

In addition, for the major use-cases there are often pre-built ember-specific components that will be easier to adapt for your use case than trying to make the non-ember library do what you need. This is doubly the case as soon as you have very specific custom requirements for your UX interactions - customising ember specific code will make your life a lot easier, give you more flexibility, and take less time in the long run.

Two examples of common components that are relevant to this discussion are:

• ic-tabs for tabbed navigation
• ic-modal for modal dialogs

In short, I would argue that rolling your own components often corresponds to the red line on this graph, whist non-ember components corresponds to the blue line:

[1] https://www.quora.com/What-are-the-most-profound-life-lessons-from-Stanford-Professor-John-Ousterhout?share=1

The real issue is that the documentation is bad and the API is worse! In this talk I will show how to start with nothing but a ruby interpreter, and generate sine waves, notes, scales, modes, chords, arpeggios and songs, in a way that will be understandable to those who have never touched an instrument before and will (hopefully) offer an interesting new perspective even to those who are already well versed in music theory.

However, client side validations can be complex to implement, and you will always need to do server side validations anyway, both because you can't trust anything done on the client side, and some validations like uniqueness can't be implemented correctly without the transactional semantics only available on the server side.

It's a lot easier to start out doing server-side only validations, and this post shows how to take advantage of Ember's built in error handling.

Shameless plug: This post explains some techniques that are used in Emberkit, my Ember/Rails SaaS kit. If you find this post useful, have a look, as you'll find many more useful techniques demonstrated there:

https://emberkit.com

Starting out with a form

I'm going to start out with a basic form, using the form components I build in my previous post - if there's anything in this post that's not making sense you might want to read that one first.

A you can see, nothing works yet. If you click on the Save button, you will get an error in the console:

Error: The backend rejected the commit because it was invalid: {name: must be present, email: Is not an email,Is blank, base: This person is a generally unsavoury character and is not allowed to sign up}  

I'm going to be using mockjax to emulate the response needed from your server, which should have a response code of 422 Unprocessable Entity, and a body that looks like this:

{
  "errors": {
    "field_one": ["error 1", "error 2"],
    "field_two": ["error 1"]
  }
}

Intro to DS.Errors

So how do we get ember to handle the errors more gracefully? It turns out that all we need to do is to handle the save() promise rejection, by providing a function as the second argument to the then handler:

model.save().then ->  
  alert 'saved'
, ->
  #empty function

Just by providing that empty function, the error is considered handled and no console error is thrown. Your model also gets it's errors property populated with an instance of DS.Errors, which you can find out more about in the API docs.

For now, we're just going to use the length property to see if there are any validation errors and display a message to the user:

Using form components to display errors

So now we have an error message, but it's quite useless because we're not telling the user what errors actually occurred. To improve this a bit, I'm going to extend the form components from my previous post to show the errors on each field, along with in the alert above the form:

The key parts here are:

• Extend App.FormFieldComponent to know what errors are on each field so inline messages can be shown:

hasError: (->  
  @get('object.errors')?.has @get('for')
).property 'object.errors.[]'

errors: (->  
  return Em.A() unless @get('object.errors')
  @get('object.errors').errorsFor(@get('for')).mapBy('message').join(', ')
).property 'object.errors.[]'

• Create a new error-messages component to display the error messages nicely
• Add a titleize helper to convert "form_field" to "Form field", so that you can display the error messages in human-readable form.
• Use the new component in the template to display the errors.

Handling arbitrary errors

If you've been paying very close attention, you may have noticed that we have a problem - not all error messages we returned are displayed!

The reason for this is that DS.Errors is currently only populated for attributes that you have defined on your model with DS.attr (or DS.belongsTo / DS.hasMany). But we have returned an error for base in our (fake) json response. Currently it is just ignored. It's quite easy to hack around this, by changing how errors are applied to your models:

DS.Model.reopen  
  adapterDidInvalidate: (errors) ->
    recordErrors = @get 'errors'
    for own key, errorValue of errors
      recordErrors.add key, errorValue

In this example, you can return arbitrary errors and they will be displayed in the alert above the form. I also add special handling for an error called base, allowing you to have errors that don't apply to a specific field but the whole record in general:

I currently have a pull request open in ember data to make this the default behaviour - hopefully it will be merged and you won't have to worry about this in the future.

Handling Errors with the RESTAdapter

All the previous examples use DS.ActiveModelAdapter. However many ember applications use DS.RESTAdapter, which doesn't include this error handling by default. If you are using the RESTAdapter, you can add support for automatic error handling like this:

App.ApplicationAdapter = DS.RESTAdapter.extend  
  ajaxError: (jqXHR) ->
    error = @_super(jqXHR)

    if jqXHR and jqXHR.status is 422
      response = Ember.$.parseJSON(jqXHR.responseText)
      errors = {}
      if response.errors?
        jsonErrors = response.errors
        Ember.keys(jsonErrors).forEach (key) ->
          errors[Ember.String.camelize(key)] = jsonErrors[key]

      new DS.InvalidError(errors)
    else
      error

Bonus: Generating the correct responses in Rails

The format of the json required is quite simple, you shouldn't have any problems generating it in any backend language. However, if you happen to be using Rails on the backend, it's really easy to generate this automatically. If you already have validations on your model, you can just render the errors object to get the exact format you need:

def create  
  user = User.new sanitized_user_params
  if user.save
    render json: user
  else
    render json: {errors: user.errors}, status: 422
  end
end  

In fact, it can be even easier than this, because the respond_with helper will do just that automatically:

def create  
  user = User.create sanitized_user_params
  respond_with user
end  

Now you have implemented server-side validations, have a look at Ember Validations to see how you can add client side validations that can validate your form on the client side in real-time, for a better UX for your visitors.

Shameless plug: This post builds a simplified example of the forms that are used in Emberkit, my Ember/Rails SaaS kit. If you find this post useful, have a look, as you'll find many more useful techniques demonstrated there:

https://emberkit.com

Starting out

Let's start out with a basic form implemented using built–in ember primitives:

This is much like a normal HTML form. One thing that may be different to how you would usually use buttons in ember is that the button itself is a plain <button type="submit"> with no Ember code whatsoever, and the {{action "updateUser" on="submit"}} helper is attached to the form tag.

This ensures that the form can be submitted both by clicking on the button, and by pressing "Enter" when one of the fields is focussed. It's important to preserve the UX that your users are used to and forms that don't submit properly with the usual keyboard interactions can be particularly frustrating.

Introducing a component template

This is a good start, and it works fine. However it's a bit verbose and repetitive. Each form input needs a lot of code to add a label and assign a unique ID so that the label is associated with the input. This is annoying and error-prone. Let's add in a component to reduce some of the duplication:

This is already a big improvement. The code duplication is reduced and it's simpler to look at. However I'm not completely happy with the API, and currently there's the potential for a bug - the form-field component uses the label that you pass in as its ID. This is not ideal as IDs are supposed to be unique, and this component would only let you have one field with a given label on the page at a time.

Adding a component object

Currently there's only a component template, but no object. Components can be template–only, object–only, or have both a template and an object. For an example of an object-only component, have a look at my discuss post about Bootstrap, active links and LIs.

Let's add a component object and give it a smart default for the field type. It will default to "text" unless the label contains the string "password":

Nesting components

Of course, components can be nested. If we have lots of forms in our app, we probably want to make it simpler to write the surrounding form tag as well:

This is looking really good. Note the use of sendAction in App.ObjectFormComponent to send an action from the component that is otherwise isolated from the surrounding scope.

Adding some magic

Up to this point I don't think anything I've done is remotely controversial. However this next bit might be. Components are supposed to be isolated from their surrounding scope. However, I have found that it makes sense to break this isolation in the circumstance where you want to nest components and give them specific behaviour based on how they are nested. If you want to read more or have an opinion on this topic, have a look at my discourse post. For now, to improve this api even further and also solve the id problem, I'll show an example of what component nesting and dynamic binding can do:

There's quite a bit going on here so I'll break down the last changes one at a time.

The end result is that we now have a terser API for specifying form fields – {{form-field for="name"}} instead of {{form-field label="Name" value=name}} – and the form fields each have access to an object property, which is what is passed into the parent object-form component as the for=this argument. Remember that in a template {{this}} refers to the controller, which in this case is an ObjectController that proxies to the current User created in the model hook.

Having access to the object that the form refers to allows solving the ID problem mentioned earlier, by creating a field id that is composed of a guid unique to the object, and also the field name:

  fieldId: (->
    "#{Em.guidFor @get('object')}-input-#{@get('for')}"
  ).property 'object', 'for'

The most complex part of this final version is the dynamic binding setup:

  setupBindings: (->
    @binding?.disconnect @ # Disconnect old binding if present

    # Create a binding between the value property of the component,
    # and the correct field name on the model object.
    @binding = Em.Binding.from("object.#{@get 'for'}").to('value')

    # Activate the binding
    @binding.connect @

  ).on('init').observes 'for', 'object'

  # Ensure the bindings are cleaned up when the component is removed
  tearDownBindings: (->
    @binding?.disconnect @
  ).on 'willDestroyElement'

The reason this is necessary is to allow for passing in a string to the component in the template, and getting access to both that string, and the value of the property referred to by the string. Usually it would be one or the other. Because we have access to the "object" that the form is referring to, through an alias to the parent component (object: Em.computed.alias 'parentView.for'), we can use the string to create a binding to the correct property on the object.

First any existing dynamic binding is disconnected if it is present. Then a manual binding is created that connects the correct property on the form's object to a property called value on the form-field component, and then that binding is activated by calling the connect method. A little bit of binding magic can often help adding polish to APIs!

Finally, we destroy the binding when the component is removed from the DOM using the disconnect method.

Bonus - Optional block component

So there's one more feature of components that's not that often used but is really useful. Our form-field component only handles text or password inputs at the moment, but what if we want to handle more input types? Sure, we can keep adding more automatic behaviour (and in the version of this code I use, textareas are automatically handled as well). But you can never plan for everything. What if you want some custom input that you only use in one place in your app?

This is when an optional block will come in handy. The idea is to make the component behave differently depending on whether it's used in block form or inline form:

Inline form - use default input field:

{{form-field for="email"}}


Block form - provide custom input but still use wrapper and label etc:

{{#form-field for="somethingElse"}}
  {{my-crazy-custom-custom-input-field value=somethingElse}}
{{/form-field}}

This is easy enough to achieve with {{#if template}} {{yield}} {{/if}} in your component's handlebars template. If a block was provided, the template property will be present. If no block is provided, it will be absent:

Wrapup

Hopefully this has been a useful walk-through of some advanced uses of components. If you want a much more comprehensive form library that helps with building forms in ember, you might want to check out Ember EasyForm which inspired the API demonstrated here.

Ember.debouncedObserver = function(func, key, time) {  
    return Em.observer(function() {
        Em.run.debounce(this, func, time);
    }, key)
};

Ember.throttledObserver = function(func, key, time) {  
    return Em.observer(function() {
        Em.run.throttle(this, func, time);
    }, key)
};

Using them is easy, just wrap your observer function with them like this:

App.IndexController = Em.ArrayController.extend({  
    selectObserver: Ember.debouncedObserver(function() {
        this.set('debounceSelected', this.get('selectedValue.firstName'));
    }, 'selectedValue', 1000),

    selectObserver2: Ember.throttledObserver(function() {
        this.set('throttleSelected', this.get('selectedValue.firstName'));
    }, 'selectedValue', 1000)

});

If you are using coffeescript, these have a slightly nicer argument order and support multiple dependant keys:

Ember.debouncedObserver = (keys..., time, func) ->  
  Em.observer ->
    Em.run.debounce @, func, time
  , keys...

Ember.throttledObserver = (keys..., time, func) ->  
  Em.observer ->
    Em.run.throttle @, func, time
  , keys...

You need to decide whether throttling or debouncing is right for you. Ben Alman has a good blog post that goes into more detail.

Here's a JSFiddle showing the code in action.

If you haven't seen this in action before, the idea is incredibly simple but brilliant. Instead of simply searching for a substring in a list of items, the string you type is scored against each item in the list. If you simply type the full string, you will get a high score the same as a naaive search implementation. However if you type the first character of each word, e.g. “uoo” for “University of Oxford”, then that will be one of the top results. If this isn't making much sense, a live demo might help.

The problem at hand

So, now I need to provide a way to select from a large number of university names in a web application, it would be remiss of me not to give my users the same great user experience of this search algorithm. In fact, it is an especially appropriate use case because universities are frequently referred to by abbreviations, for example UCL instead of University College London. If you just tested if the name contained the search string, a user typing “ucl” would get no results, however with the right fuzzy matching algorithm University College London will be the first result.

I started off by using LiquidMetal, which has been working brilliantly for the roughly 300 universities in our database. Late last week, I discovered the number of organisations is about to rise to 15,000 - surely this approach simply won't be able to scale to that number of strings, and we'll have to revert back to simple string matching, perhaps even on the server to keep clients responsive?

Time to break out the benchmarks

First I needed a large list of university names. DBpedia to the rescue! I used the following query to get me a nice list of 15639 university names, which was perfectly serendipitous.

PREFIX dbo: <http://dbpedia.org/ontology/>

SELECT DISTINCT ?name WHERE {  
     ?university rdf:type dbo:University.
     ?university dbpprop:name ?name
}
ORDER BY ?university  

Then I scoured the interwebs looking for various implementations of this fuzzy matching algorithm, and set up some benchmarking code. I wanted to rank all of the strings against a set of likely queries, hopefully some of which would be pathological, i.e. “This string is definitely not in the dataset at all and is also quite long”. I also compared against more simple matching using String#indexOf and String#match.

Conclusions

It turns out that the times are pretty fast even for the intelligent scoring algorithms. LiquidMetal seems to be vulnerable to a few pathological cases but the difference isn't really that big. I think from these results I will be using string_score.js, due to the more advanced featureset and better matching it provides.

I was really struck by how short these times are though - less than 100 milliseconds to rank 16,000 strings? Amazing. Even more amazing is the average of less than 3 milliseconds for the fastest method, with a simple indexOf. This suggests that even for a far larger dataset, searching on the client side should be relatively responsive and probably limited by memory more than anything else, even if you do have to sacrifice the fuzzy matching.

Instructions to get this working with rails 3.2 and Chrome Canary:

1. Follow Ryan Florence's instructions to get a build of coffeescript in your path that can create source maps
2. Put the code below in config/initializers/source_maps.rb
3. Run rake tmp:cache:clear and restart your server. Perhaps several times for luck.
4. Fix any errors the new compiler gives about your scripts. It has a great one, indentation error, with no line number. Try coffeelint if you're having problems as it will show any indentation errors.
5. Get a taste of the future! An example would be to use console.log('hello world') in your coffeescript file. You will see that on the right of the log line in the web inspector, the link will point to a coffee file! Click the link, and the correct line is highlighted in the source view above.

Important Note: this rather brutal hack replaces the normal coffeescript compiler by shelling out to the CoffeeScriptRedux compiler, which is not in fact finished. This is just a proof of concept, you probably shouldn't use it.

The Code

View as gist

# config/initializers/source_maps.rb

if Rails.env.development?  
  require 'open3'
  module CoffeeScript
    class SourceMapError < StandardError; end;

    class << self

      def map_dir
        return @map_dir if @map_dir
        # Make the directory the maps are served from
        @map_dir = Rails.root.join("public/source_maps")
        @map_dir.mkpath
        @map_dir
      end

      def check_coffeescript_version
        version = `coffee --version`

        unless version.match(/(\d+)\.\d+\./)[1].to_i >= 2
          raise "You must have coffeescript version 2.0.0-dev or higher to use source maps - see http://ryanflorence.com/2012/coffeescript-source-maps/"
        end
      end


      def compile script, options
        script = script.read if script.respond_to?(:read)

        if options.key?(:no_wrap) and !options.key?(:bare)
          options[:bare] = options[:no_wrap]
        else
          options[:bare] = false
        end

        check_coffeescript_version
        compile_with_source_map script, options
      end

      def compile_with_source_map script, options
        flags = %w(--js)
        flags << "--bare" if options[:bare]

        javascript, stderr, status = Open3.capture3("coffee #{flags.join(' ')}", stdin_data: script)

        raise SourceMapError, stderr unless status.success?

        source_map_comment = generate_source_map options[:pathname], script if options[:pathname]

        return javascript << source_map_comment

      rescue SourceMapError => e
        message = e.message.lines.first.chomp.gsub('"', '\"')
        relative_path_name = options[:pathname].to_s.gsub(Rails.root.to_s, '') if options[:pathname]
        relative_path_name ||= '<unknown path>'
        %Q{throw Error("Coffeescript compile error: #{relative_path_name}: #{message}")}
      end


      def generate_source_map pathname, script
        basename    = pathname.basename('.coffee')
        coffee_file = map_dir.join("#{basename}.coffee")
        coffee_file.open('w') {|f| f.puts script }


        # This command actually generates the source map, and saves it to a file
        source_map, status = nil
        Dir.chdir Rails.root.join('public/source_maps').to_s do
          source_map, stderr, status = Open3.capture3("coffee --source-map -i #{pathname.basename}")
        end

        raise SourceMapError, "Error while generating source map for file #{pathname}: #{stderr}" unless status.success?

        # I couldn't figure out how to control the 'file' and 'sources' values in the output,
        # so parse the map to JSON and rewrite these to ones that will work here
        data = JSON.parse(source_map)
        data['file'] = pathname.basename.to_s.gsub(/\.coffee/, '')
        data['sources'] = [pathname.basename.to_s]


        map_file = map_dir.join "#{basename}.map"
        map_file.open('w') { |f| f.puts data.to_json }
        Rails.logger.info "Compiled source map #{map_file}"

        return "\n//@ sourceMappingURL=/source_maps/#{map_file.basename}\n"
      end

    end
  end



  # Monkeypatch this method to include the scripts' pathname
  require 'tilt/template'

  module Tilt
    class CoffeeScriptTemplate < Template
      def evaluate(scope, locals, &block)
        @output ||= CoffeeScript.compile(data, options.merge(pathname: scope.pathname))
      end
    end
  end
end  

I worked on DevTest on a total of 42 seperate days, over a period of around 6 months. This was all in my spare time during evenings and weekends, with the exception of a week of vacation to finish everything up and finally launch. And the only money I've spent so far has been $40 on hosting charges. A lot of those 42 days I only did a small amount of work; I think that if I'd been working on it full time, without taking long breaks between pieces of work, I'd have been able to do it in 2 months.

In this series of blog posts I'm going to explain some tips and tricks I used to make this possible. A large part of being able to pull off a feat like this is experience and practice, and having been programming for around 20 years in one form or another has made me able to get things done quickly, and a sense of what will work and what won't. But here I will try to share some of that knowledge and hopefully some will be useful for others.

Micro agile

In the past I've worked almost entirely in agile development environments with a focus on SCRUM. I'm wouldn't say I'm a "true agile believer", but it's a good tool to know and understand. For this project, I've been using what I call micro-agile. It's based on SCRUM development but cuts out almost everything because it's irrelevant when you're working solo. Doing this has helped me to keep focussed and stick the minimum viable product.

Trello

I've used Trello to manage my todo list, it's amazing and free and above all simple. Before finding this tool I'd have said anything more than a pen and paper was overkill. Pen and paper is still fine, but Trello is simple enough to not get in the way and slightly more convenient than carrying around your list everywhere with you.

Planning a sprint

You should have one "product backlog" board. Use this as a dumping ground for new ideas. I had three columns, "Essential", "Post Launch" and "Inbox". New ideas go into the inbox. Simple.

Example product backlog

Now you need to plan your first sprint, this is what you'll need to do every 2 weeks or so:

1. Brain dump into the inbox. Just put every little thing you can think of in there.
2. Sort into Essential and Post Launch.
3. Prioritize the "Essential" column by what's most important.
4. Create a new board for your first sprint, with these columns: "Sprint backlog", "Specced", "In progress", "Done". Call this board "Sprint 1"
5. Move an item from the top of the prioritized "Essential" column into the "Sprint Backlog" column of your sprint 1 board column. Repeat until you've got as much work as you think can be done in 2 weeks.

Example sprint board

Executing a sprint

Now you have a prioritized, realistic list of tasks that you can plug on with one by one. Recall you have 4 columns on your sprint board, "Sprint backlog", "Specced", "In progress", "Done". At the moment everything should be in the backlog column. Now it's time to spec it out. Add a checklist to the first card in your sprint backlog, and try to flesh it out in more detail. Here's an example of a checklist for the candidate login feature:

You can move that card into the "Specced" column when you're happy with the requirements. You should try to think of all the requirements for this part up front, although this isn't always possible. The idea is that once you've specced an item, actually building it is just automatically following a list of checkboxes.

Not rocket surgery right? It's just common sense. But it really helps to have a simple process. One of the problems I find as a developer is that I easily get distracted and procrastinate. Having a mechanical process to get myself back on track when I've lost focus is essential.

Standups

Standups in a full size development team are great for getting everyone on track. I found I missed this on my own, so I faked it. Whenever I had a block of time to work on DevTest, I'd start by looking at the current sprint board, and doing it in a different location I was working, and looking at what the next task to do was. Usually this would be downstairs in my living room. I'd do this until I got excited about building the next feature and couldn't wait to get programming, and found this helped again with motivation.

What not to build

The most important thing I can tell you is that you must be really, really ruthless in throwing out ideas, or at least pushing to post launch. There's so many more features, countless enhancements that I really want to add, and wanted to add before I launched. You have to ask yourself if it really matters. Really? Are you sure? I threw out a tonne of features that I though I needed. Not just big ideas but simple, basic things.

Do you really need to implement pagination in your backend? Not until you have customers with unwieldy lists of stuff, and that's not going to be for a while after launch most likely! Do you need an admin backend for yourself? Not if you're a programmer, just use the database console. If it starts to piss you off you can write something at that point to fix it. All of the assumptions that you have if you've been building software for other people for most of your career are now open to questioning.