Normally, we use commands like GETBIT
and SETBIT
to read or modify a bitmap in redis. We also have corresponding APIs in python redis library (redis-py).
The problem is that, when reading or writing a big bitmap, if you use the native commands, it may cost too much time.
The solution is to read or write the bitmap in batch. That is to say, read and write the whole bitmap as a string!
Before we write the code, install related python libraries:
pip install redis bitarray
Here's the code to write a whole bitmap to redis:
import redis
from bitarray import bitarray
redis_ins = redis.Redis()
# the indexes that we want to save to the bitmap
indexes = [1, 2, 3, 13, 14, 15]
# use bitarray to convert indexes to a bitmap
arr = bitarray(max(indexes) + 1)
arr.setall(0) # clear the bitmap
for idx in indexes:
arr[idx] = 1 # set bit
print(arr) # bitarray('0111000000000111')
data = arr.tobytes()
# save generated bitmap to redis
redis_ins.set('mykey', data)
In the code above, we utilize bitarray
to covent indexes (offsets of bits) to a bitmap, then push it to the redis server once and for all. Now, we no longer need to execute SETBIT
again and again.
Here's the code to read and parse a whole bitmap from redis:
import redis
from bitarray import bitarray
redis_ins = redis.Redis()
# read the whole bitmap from the redis server
data = redis_ins.get('mykey')
# parse the bitmap to bit offsets
arr = bitarray()
arr.frombytes(data)
indexes = arr.search(1) # find bits
print(indexes) # [1, 2, 3, 13, 14, 15]
In the code above, we read the whole bitmap from the redis server, and utilize bitarray
to covent the bitmap to indexes (offsets of bits). Now, we no longer need to execute GETBIT
again and again.