With some help of our friend "Google Translate" :)
As we all know that if you want a web application goes faster there is a secret cache as much as you can. Avoid to generate the page each time and search for the content in the database.
To archieve this the today standard is Memcached. Memcached allows us to scale our application a simple way. We can think it as a big hash table, written in C, very fast and with libraries to access to it in almost any language. But thre is e a new competitor in this kind of applications: Redis, and I've already talked about it in the Celery post.
Memcached is an application aimed at dealing with cache, Redis is a noSQL general purpose key-value database in a similar way as Memcached, but with possibilities that go far beyond a cache. Let's see a few differences:
We can not see the keys we have in Memcached. With Redis can do a search for keys, or see all the command KEYS *
Redis has persitence.
Memcached is limited to memory you allocate, Redis can also swapt to disk and just put the keys in memory.
In Memcached we have no true replication. Redis replication is real and configurable with a simple line.
Redis is quite configurable, we can define the page size of cache, store to disk, have a password protected database...
With Redis we can define as many databases as you want. We can clean all the keys in a database without affecting the others.
So the next step is to ask if we could use Redis instead of Memcached for our web applications. Redis plus Django could partially solve one of the biggest problems we have: cache invalidation. As we can have an independent database for each application or for each purpose, we can FLUSHDB the database our application to invalidate the whole cache, or just delete the keys with a single Redis command.
But in Science hypotheses have to be confirmed. So what I did was to build a little sandbox application to see how if Redis was as good as it seems.
The machines availables to us for the experiment follows:
Dell Laptop Core 2 at 2:16 GH and 2 GB of RAM with Ubuntu 11:04 This is the Web application server and has the address 192.168.1.35
Virtual Server Ubuntu 10.04 with 512 MB of RAM on Virtualbox running on the laptop at 192.168.1.38. This server is 32 bit and will host Redis as Memcached.
Apple PPC 64 Dual Core with 3 GB of RAM, it will launch the tests.
The application is the one I created for creantbits. That is, to run it has to read BD for the last events and presents them in the page.
We will use two of Gunicorn workers to start the application, and we'll test the performance from the PPC with Apache Benchmark
ab -n 1000 -c 5 http://192.168.1.35:8000/
For each test case two consecutive tests are thrown and we discarded the first.
To test the Redis cache we have installed the application django-redis-cache
First we have to determine the starting point. So what we did is to clear our application's cache and see how many requests we get.
no-cache: 120 req / s Mean: 41.4 ms
We set up cache site for Django configured as locmem. Locmem is not recommended in production environments as it can't share the cache, but as before, we used to set the starting point:
locmem: 1380 req / s Mean: 3.6 ms
We configure the the cache as memcached
memcached: 626 req / s Mean: 8.0 ms
Cache Redis configured with persistence to disk
Redis: 623 req / s Mean: 8.0 ms
Cache Redis without persistence. Is the nearest Memcached equivalent.
Redis: 632 req / s Mean: 7.8 ms
We install the hiredis package
Redis: 650 req / s Mean: 7.7 ms
I think the results speak for themselves. If we use persistence Redis is comparable in speed to Memcached, and for the same price we have a NoSQL database at our disposal.
If we don't need persistence Redis shows a 4% improvement over Memcached. This percentage is not significative, but at least we can see that Redis is in the same leage as Memcached.
For me Redis it too good to not use it!
Traducciones/Translations by apertium
1. Redis vs Memcached 2. Redis 3. Un creant bits d'estiu 4. Avaluant Gondor 5. Celery, Redis and Django
1 Comentari de Andy a les 11:08 del Friday 5 Aug de 2011
It should be noted that Memcached is horizontally scalable while Redis isn't.
With Memcached if you run out of RAM in 1 server you just add more servers and Memcached automatically spreads out the data to all available machines.
Redis isn't distributed - you'd need to write your own code to manually perform any scaling out.
2 Comentari de aaloy a les 12:08 del Saturday 6 Aug de 2011
@Andy It's a good point. Sharding in Redis is on development as far as I know. But for web applications that just uses one Memcached instance to share a common cache memory this is not a problem.
If needed you could emulate Memcached feature, of course you have to write your own cache client, http://pypi.python.org/pypi/redis-shard gives some ideas about it.
Actually we don't have applications that would require more than the 24 Gb of RAM that our ISP provider could give us. We're running a single Memcached instance shared between applications.
For this kind of configuration Redis, given is as fast as Memcached, give us a lot of value. I can choose just to remove the cache keys without affecting the session keys for one application, ... I we store the session in Redis instead of Memcached we can avoid also the size limit of Memcached.
Redis is not "the solution" but in our web development (as we also use Celery) using it instead of Mencached give us lots of advantages.
3 Comentari de Davide Callegari a les 09:08 del Saturday 6 Aug de 2011
Google translate sucks.
Nice article by the way.
4 Comentari de aaloy a les 12:08 del Saturday 6 Aug de 2011
I think I need an English editor! :D