Scale using your users

I’ll use an example scenario to show what i mean.

As part of your site functionality, you allow users to create, update, search for, and view “thingies”.  This thingie has an address.

When people look at your thingie (er, that doesn’t really sound right), you want to display a nice dynamic map of the address with a cute little red marker.  You don’t have the coordinates for that address and therefore want to do some geocoding.

Having those coordinates would allow you to support all types of fancy location-based searches to tell if someone is near your thingie.  Even better, your cute little red marker would be in the right place.

So how do you go about doing this?  Well, some possible solutions could be:

1) Send a geocode request when saving/updating your thingie

A good first start, but probably not the best.  This has a few disadvantages, but for the sake of simplicity, let’s say the main reason  is you don’t want users waiting for unnecessary synchronous steps to complete.  Factor in network uncertainly, timeouts, etc. and you have a nice set of disasters waiting as your site grows.

You also realize that these geocoding services have limits on daily usage!  Too many thingie’s a day will be great for you, but you’d be tied to the limitations on your geocoding daily allowance and throttling requirements.  What to do?

So you come up with:

2) Create a queue to batch your geocoding requests

Better idea! When you create new thingies – and assuming you don’t have the addresses cached somewhere – you queue them in your uber geocoding async job.  It chugs away in the background sending nicely throttled requests and stops after reaching the daily limit.  Unfortunately, unless you want to pay lots of cash to up your daily limit, your queue will start getting longer and longer as it can’t keep up with new thingies.  Also, the geocoding might not even be done for thingies that your users are viewing right now.

Fortunately, this is a great scenario of how you can take advantage of your users to help you grow.  And you do it by leveraging client-side JavaScript.  Each user using your site is a potential resource which you can use to offload some of your work.  Help your users help you!

So why not have the users geocode for you?

3) Have the first user who views your map geocode it for you

I personally like this solution best because it’s being lazy.  When you have to display your map and that thingie has no coordinates yet, have the user send out a geocoding request from their client machine and use the results to show the map.  More importantly, send another async request afterwards to your server to save that value for future users!  Now you both get what you want at the right time.

Now assuming the geocoding API you’re using allows anonymous requests (like Google),  you don’t have to worry about daily limits, throttling issues, and other headaches.

The point of it all

Now this was all a hypothetical scenario to stress a point.  Lots of services today have an HTTP API accessible via JavaScript and this will probably grow and grow.  Align that with the increased horsepower of modern machines and improved browser JavaScript engines and you have an army of resources waiting to be tapped if you can figure out how to do so.  The great part is this resource scales linearly with demand.

The tricky part though is changing your frame of mind when writing features to consider this new option.  Why use server bandwidth and resources to do it when the client can do it just as well?

Have you had any scenarios like this?


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: