Using Redis Hashes for Caching in Ruby on Rails
Note: We created a Ruby gem to easily share the lessons we’ve learned in this article.
We’ve been using Redis with Ruby on Rails’s caching methods for a couple of years at Mrsool and it’s been a great experience. We have particularly been using a function called #delete_matched to delete keys starting with a certain prefix. Let's take a look at an example:
Let’s imagine we have SQL table called stores and each store has many branches, and we need to show store branches near a certain user using their latitude/longitude (with varying degrees of precision), but we have millions of users in many places around the world, and we don’t want to overload our database with the same query for users in the same area, so let's cache this value per area.
def nearest_branches(store_id:, latitude:, longitude:)
Rails.cache.fetch("nearest_branches_#{store_id}_#{latitude}_#{longitude}") do
# Logic to get nearest open branches
end
endWhenever certain kinds of updates happen to stores or branches or a certain amount of time passes, we cleared out all the caches for a specific store using #delete_matched.
This worked well for a long time but as the amount of content we had stored in Redis grew, this function kept performing noticeably slower. At some point we also had to move from a single Redis node to a cluster, and we found out that #delete_matched is only operating on a single node, and not the entire cluster.
After doing some research, we found that we weren’t the only ones who were facing these issues at scale, and we found a data type in Redis that’s a great solution to our problems: Redis Hashes. Compared to #delete_matched, deleting a hash is much faster and there is no risk of leaving out undeleted data across multiple nodes.
It’s very similar to Ruby hashes:
{ hash_key => { sub_key => value } }Unlike Ruby, hashes in Redis are flat, i.e. we can only have one level of sub-keys and it is not possible to create deeply nested structures.
From the structure above we can see the answer to our question of how to solve the problem of #delete_matched:
store_nearest_branches_hash_key = "nearest_branches_#{store_id}"
branches_in_area_a_key = "#{latitude}_#{longitude}"
branches_in_area_b_key = "#{latitude2}_#{longitude2}"Which can be cached like so:
redis = Rails.cache.redisredis.hset(store_nearest_branches_hash_key, branches_in_area_a_key, branches_in_area_a)redis.hset(store_nearest_branches_hash_key, branches_in_area_b_key, branches_in_area_b)
To clear the cache for some store, all we need to do is delete its hash:
redis.del(store_nearest_branches_hash_key)but it’s not that simple: How can we store objects like ActiveRecord models in the Hash? (by default it’s only possible to store strings)
I will try to show you how to do it. First of all, let’s create a wrapper class/module for the Redis client that will operate on hashes. I decided to put it in lib, but it can be a service object instead — it’s up to you.
# lib/redis_hash_store.rb
module RedisHashStore
extend self def write(hash_key, sub_key, value)
redis.hset(hash_key, sub_key, value)
end def read(hash_key, sub_key)
redis.hget(hash_key, sub_key)
end def delete(hash_key, sub_key)
redis.hdel(hash_key, sub_key)
end def delete_hash(key)
redis.del(key)
end private def redis
Rails.cache.redis
end
end
Now it’s a bit easier to work with hashes:
store_nearest_branches_key = "nearest_branches_#{store_id}"
branches_in_area_a_key = "#{latitude}_#{longitude}"RedisHashStore.write(store_nearest_branches_key, branches_in_area_a_key, value)RedisHashStore.read(store_nearest_branches_key, branches_in_area_a_key)RedisHashStore.delete(store_nearest_branches_key, branches_in_area_a_key)RedisHashStore.delete_hash(store_nearest_branches_key)
We still can’t store objects yet, let’s change that! After some investigations of ActiveSupport source I decided to use a lighter version of their implementation:
class Entry
attr_reader :value
def initialize(value, expires_in:)
@value = value
@created_at = Time.now.to_f
@expires_in = expires_in
end def expired?
@expires_in && @created_at + @expires_in <= Time.now.to_f
end
end
Unfortunately, Redis doesn’t have expire for Redis Hashes, so we have to implement our own expired? function.
Let’s move back to our RedisHashStore, which now looks like this:
module RedisHashStore
extend self
class Entry
attr_reader :value
def initialize(value, expires_in:)
@value = value
@created_at = Time.now.to_f
@expires_in = expires_in
end def expired?
@expires_in && @created_at + @expires_in <= Time.now.to_f
end
end def write(hash_key, sub_key, value)
redis.hset(hash_key, sub_key, value)
end def read(hash_key, sub_key)
redis.hget(hash_key, sub_key)
end def delete(hash_key, sub_key)
redis.hdel(hash_key, sub_key)
end def delete_hash(key)
redis.del(key)
end private
def redis
Rails.cache.redis
end
end
Wait a minute, we missed something here! Now we need to somehow convert Entry to a String and then safely move it back to the object. For this purpose we can use Marshal.
In our case we need 2 methods: #dump and #load, let’s add it to our RedisHashStore:
What about Benchmarks?
indexes = 1..1_000_000
indexes.each do |index|
Rails.cache.write("some_data_#{index}", index)
RedisHashStore.write("some_data", index, index)
endBenchmark.bm do |x|
x.report("delete_matched") {
Rails.cache.delete_matched("some_data_*")
}
x.report("delete_hash") { RedisHashStore.delete_hash('some_data') }
enduser system total real
delete_matched 0.571040 0.244962 0.816002 (3.791056)
delete_hash 0.000000 0.000225 0.000225 (0.677891)# Machine info
# OS: macOS Big Sur
# Processor: 2,6 GHz 6-Core Intel Core i7
# Memory: 16 GB 2667 MHz DDR4
#
# Runned on
# OS: Docker Desktop (linux)
# Architecture: x86_64
# CPUs: 4
# Total Memory: 4.827GiB
Thanks for reading through this article, and don’t forget to check out our Ruby gem which includes all of the problems we’ve solved in this article and more.
