Patterns for Asynchronous Programming with Promises

Patterns for Asynchronous Programming with PromisesPromises are currently the best tool we have for asynchronous programming and they appear to be our best hope for the forseeable future, even if they'll be hiding behind generators or async functions. For now, we'll need to use promises directly, so we should learn some good techniques for using them right now, especially when dealing with multiple asynchronous operations, whether they happen in parallel or sequentially.

Before We Start

Before I get into details on the patterns, I'm going to have to fill you in on some points. First off, I'm using Q for my promises implementation. I'm also using Underscore or Lodash (pick your favorite; personally I like Lodash) for things like map, each, and reduce. In the code, asyncOperation just represents a function that takes a single number parameter, performs an asynchronous operation according to that number, and returns a promise, while // ... represents whatever code is specific to your application that operates on the values returned from asyncOperation.

Parallel Asynchronous Operations

First we'll take a look at parallel operations. This refers to getting multiple asynchronous operations queued up and running at the same time. By running them in parallel, you can significantly increase your performance. Sadly, this isn't always possible. You may be required to run the operations in sequential order, which what we'll be talking about in the next section.

Anyway, we'll first look at running the asynchronous operations in parallel, but then performing synchronous operations on them in a specific order after all of the asynchronous operations have finished. This gives you a performance boost from the parallel operations, but then brings everything back together to do things in the right order when you need to.

function parallelAsyncSequentialSync (){
    var values = [1,2,3,4];

    // Use `map` to create an array of promises by performing
    // `asyncOperation` on each element in the original array.
    // They should happen in parallel.
    var operations = _.map(values, asyncOperation);

    // Return a promise so outside code can wait for this code.
    return Q.all(operations).then(function(newValues) {
        // Once all of the operations are finished, we can loop
        // through the results and do something with them
        _.each(newValues, function(value) {
            // ...
        });

        // Make sure we return the values we want outside to see
        return newValues;
    });
}

We use map to get all of our asynchronous operations fired off right away, but then use Q.all to wait for them all to finish, and then we just run a loop over the new values and do whatever operations we need to do in the original order.

Sometimes, the order that our synchronous operations run in don't matter. In this case, we can run each of our synchronous operations immediately after their respective asynchronous operations have finished.

function parallelAsyncUnorderedSync (){
    var values = [1,2,3,4];

    // Use `map` to create an array of promises
    var operations = _.map(values, function(value) {
        // return the promise so `operations` is an array of promises.
        return asyncOperation(value).then(function(newValue) {
            // ...

            // we want the new values to pass to the outside
            return newValue;
        });
    });

    // return a promise so the outside can wait for all operations to finish.
    return Q.all(operations);
}

For this, we use map again, but instead of waiting for all of the operations to finish, we provide our own callback to map and do more inside of it. Inside we invoke our asynchronous function and then call then on it immediately to set up our synchronous operation to run immediately after the asynchronous one has finished.

Sequential Asynchronous Operations

Let's take a look at some patterns for sequential asynchronous operations. In this case, the first asynchronous operation should finish before moving on to the next asynchronous operation. I have two solutions for doing this, one uses each and one uses reduce. They are quite similar, but the version with each needs to store a reference to the promise chain, whereas the version with reduce passes it through as the memo. Essentially, the version with each is just more explicit and verbose, but they both accomplish the same thing.

function sequentialAsyncWithEach (){
    var values = [1,2,3,4];
    var newValues = [];
    var dfd = Q.defer();
    var promise;

    dfd.resolve();
    promise = dfd.promise;

    _.each(values, function(value) {
        promise = promise.then(function() {
            return asyncOperation(value);
        }).then(function(newValue) {
            // ...
            newValues.push(newValue);
        });
    });

    return promise.then(function() {
        return newValues;
    });
}
function sequentialAsyncWithReduce (){
    var values = [1,2,3,4];
    var newValues = [];
    var dfd = Q.defer();

    dfd.resolve();

    return _.reduce(values, function(memo, value) {
        return memo.then(function() {
            return asyncOperation(value);
        }).then(function(newValue) {
            // ...
            newValues.push(newValue);
        });
    }, dfd.promise).then(function() {
        return newValues;
    });
}

In each version we just chain each asynchronous operation off of the previous one. It's annoying that we need to create a "blank" promise that is simply used to start the chain, but it's a necessary evil. Also, we need to explicitly assign values to the newValues array (assuming you want to return those), which is another necessary evil, though maybe not quite as evil. I personally think the version with each is slightly easier to read thanks to its explicit nature, but it's a stylistic choice and reduce works perfectly for this situation.

Conclusion

I used to think promises were very straight-forward and even had a hard time finding a reason to use them over standard callbacks, but the more I need them, the more useful I find them to be, but I also find them to be more complicated with numerous ways they can be used, as shown above. Understanding your options and keeping a list of patterns you can follow great helps when the time comes to use them. If you don't already have these patterns embedded in your brain, you may want to save them somewhere so you have them handy when you need them.

Well, that's all for today. God bless! Happy Coding!

About the Author

Author: Joe Zim

Joe Zim

Joe Zimmerman has been doing web development ever since he found an HTML book on his dad's shelf when he was 12. Since then, JavaScript has grown in popularity and he has become passionate about it. He also loves to teach others though his blog and other popular blogs. When he's not writing code, he's spending time with his wife and children and leading them in God's Word.


  • farshad

    I did enjoy the article..such a useful subject..thx

  • jostylr

    Does this mean you have settled on Promises over Events?

    I recently wrote an event-hub library: https://github.com/jostylr/event-when which handles waiting for multiple events to happen before triggering (that’s the when part) and has scopes and named actions. It is less about attaching listeners to different objects and more about managing the control flow of the whole program.

    But I have been wondering about promises in comparison. I feel like the same flow-control can happen in both, but with events one also gets a “paper trail” with being able to log the events and action responses.

    So I was wondering if you have had any must-have features of promises that events either don’t have or it seems more awkward using.

    Also, why do you use a library (lo-dash) rather than the (now) native methods (map, forEach, reduce) on arrays? Just compatibility with old browsers and not wanting to use a shim?

    • http://www.joezimjs.com Joe Zimmerman

      I haven’t settled on either one. Events and Promises each have their own purposes. I’m assuming you’re referring to my old article: http://www.joezimjs.com/javascript/javascript-asynchronous-architectures-events-vs-promises/

      That article was more of a debate on a specific feature and which should be used for it. It might not sound quite that way in the article, but that’s what it was supposed to be.

      The thing with events is that something can happen multiple times and you have no idea if it’ll ever happen at all. Promises are actually a promise that something will happen, whether it’s successful or fails, and it can only happen once before you’re required to get another promise.

      As for using Lodash:
      1) I prefer to support old browsers
      2) the native methods are somewhat inconsistent and are actually slower (hence why Lodash is faster than Underscore in many cases)
      3) I use Backbone a lot, which requires Underscore or Lodash anyway, so why not use them?

      • jostylr

        Yes, I was referring to that article. I thought it was a more general debate.

        For multiple times, there is always once. I don’t see how events can handle not happening, so that may be something. But I am unsure when that might actually happen in a control-flow kind of way. For example, if a database call returns, you emit that return event or an error event. If the database call never returns, that could be a problem, but does a promise actually execute in that case? Doesn’t it just wait forever?

        I do admit that the events might require a bit more conscious effort in triggering the next event in sequence rather than just the functions in sequence ending, if I understand promises enough. Maybe I need a concrete example.

        Thanks for the insight on your pick of Lodash. None of them applies to my uses, but its good to know when it might.

        • http://www.joezimjs.com Joe Zimmerman

          In regards to your question about the db call: if the call doesn’t return, then there is generally a timeout that would cause an error, in which case you’d fire an error event or fail the promise.

          If “nothing” happens, neither events or promises or standard callbacks would work.

          Events are best used for what they’re callled: events. The let you know when something happened so you can respond accordingly. Promises are best used when you specifically ask for something to be done, and then you are informed when it is done and probably given a value back as the “return” value.

          Food for thought: If you use events when trying to request data from a database, what happens if you make multiple requests before any of them are finished? Then you have several handlers listening for the return but only one handler will match any one of the returns. There are certainly ways around this (putting request identifiers in the “return value”; being given an object back when you make the request that you can register the event callback on instead of registering the event callback on the entire db [this is essentially a promise btw], etc).

          There are two things to consider when choosing between events and promises:
          1) semantics: events don’t make sense to represent a return value, and promises don’t make sense for something that won’t neccessarily be fulfilled, can fulfill multiple times, or is an event
          2) simplicity: ultimately simplicity and maintainability are key. If it’s too dang hard to understand, or requires too much workaround to prevent bugs, etc. don’t use it. Your teammates will appreciate the simplicity when they have to maintain your code.

          • jostylr

            I see. I guess I just think of a db return as an event. But you are suggesting that that is more happenstance and that, at its core, a db return is a function call that just happens to finish out of sync. That makes sense.

            In my event-when library, I use scopes. So the event might be “db returned:userjack” and the callback would be registered on “db returned”, but use the “user jack” object to store and act on the data.

            For me, events make the flow clear in terms of logging. For example, “db returned:userjack” would automatically appear in the listing of events and one could then see what responded to it.

            But I can see now how promises are a natural fit for async function calling. Thanks!

            • http://www.joezimjs.com Joe Zimmerman

              I was thinking about what you’re saying about using events for things like database calls, and then I realized that IndexedDB does that, which I absolutely abhor. When I first saw the API for IndexedDB I was so confused and decided I’d never work with it. I’d much rather use an abstracted API for it because that event architecture just didn’t make sense to me for something like that. I would have greatly preferred promises (not really an option till ES6 for native APIs, though) or even Node.js’s callback structure.

              Just goes to show how strongly I believe that events and promises have their specific places. :)

  • http://blog.piotrnalepa.pl/ Piotr Nalepa

    Nice article about promises. I noticed that using jQuery’s Deferred object has similar effect as using other Promises libraries. Am I wrong?
    Some time ago I wrote an article about using promises and lazy initialization of data in the modules: http://blog.piotrnalepa.pl/2014/04/17/js-leniwa-inicjalizacja-danych-w-module-za-pomoca-js-promises/

    • http://www.joezimjs.com Joe Zimmerman

      Yup, jQuery does promises with a very similar syntax, but they neglected to implement one of the key features of Promises: error catching. In a true promise implementation, if an error occurs during an asynchronous operation, rather than the error being thrown, it should be caught and cause the promise to be rejected with the error being the value it was rejected with. jQuery’s implementation just lets the error be thrown. I often use jQuery’s promises in projects just so I don’t need to load an additional library, but for examples on this blog, I’m going to try to show people how to do things with libraries that are implemented correctly.

      As for the post you linked, it’s not a whole lot of help since I only know English. :)

      • http://blog.piotrnalepa.pl/ Piotr Nalepa

        Thanks for explanation. Good stuff :)
        About the article of mine – yep, unfortunately it’s in Polish.

  • http://geekswithlogs.net/rgupta Rohit Gupta

    Is there a downloadable code sample available for this blog post?

    • http://www.joezimjs.com Joe Zimmerman

      Just copy and paste. There isn’t all that much code.