Idea: Asynchronous Cache

The Problem

What if the database is slow? Can I build an app so that a user never has to wait on the database again? This is a common question that our engineers at Sweetwater are facing every day. With millions of database queries a day, you can imagine the load our databases take.

We leverage the Laravel Cache::remember() method quite a bit. If you are unfamiliar with how it works, it’s simple. It takes 3 arguments. A $key, $ttl, and $callback. The $key parameter is the key that the cached value is stored in. The $ttl parameter is the how long the key should be valid for. For example, a ttl of 30 would mean that the cache value is valid for 30 seconds. Finally, the $callback parameter. If your cache key expires, this is the function that is called to cache the value again.

Now, without the wonderful Cache::remember method, we’d have to do something like the following to get and cache a list of users:

if (Cache::has('all-users')) {
  $allUsers = Cache::get('all-users');
} else {
  $allUsers = User::get();
  Cache::put('all-users', $allUsers, 30);
}

The above code could easily be replaced with the following:

$allUsers = Cache::remember('all-users', 30, function() {
  return User::get();
});

So what’s the problem with this? If the all-users cache key expired, we would call User::get(). What if that User::get() method call took 45 seconds to resolve? Users would see that delay, and if the app is getting a lot of traffic, the site could go down due to hanging requests. What if we could call that User::get() method in a background worker?

A Solution

Enter my proposal: Cache::rememberAsync(). This method would be called similarly to the Cache::remember method from before. However, when the cache key expires, it would call the $callback in a background process with a queue worker. At the time of a cache key expiring, we would return the old value and refresh it in the background. What would this code look like?

function rememberAsync($key, $ttl, $callback, $queue = "default") {
    $currentValue = Cache::get($key);
    if (!is_null($currentValue)) {
        return $currentValue;
    }
    $fallbackKey = "{$key}/fallback";
    dispatch(function () use ($key, $callback, $ttl, $fallbackKey) {
        Cache::put($key, $callback(), $ttl);
        Cache::forever($fallbackKey, $callback());
    })->onQueue($queue);
    if ($fallbackValue = Cache::get($fallbackKey)) {
        return $fallbackValue;
    }
    return null;
}

Let me break down what this function is doing:

• We look for a valid value with the given cache key
• if the key is a non-null value, we return it (no need to do any further steps)
• if the key is null, we assume it’s expired
• we dispatch a job that runs the $callback, caching it’s value in the cache key plus an extra fallback cache key
• we return the fallback cache key value which never expires and will be updated by $callback in the queue worker process
• if there is no value in the fallback key, return null 🙁

In closing, this way of caching is very beneficial in two main ways:

1. Customers don’t have to wait on long running process when cache expires
2. If your long running process fails (database dies), then you can still return data to the user

A quick note: This technique should be used on data that doesn’t absolutely have to be real time. If your app depends on data that is up to date by the second, you may research other caching techniques.