Memcache python decorators

Using memcache on google app engine was always recommended but now it’s a must. Since the GAE pricing changed it became a lot more important to keep the resources required to run your application as low as possible since you don’t want to throw your money away. Using memcache is a simple way to increase the performance of your application while lowering the daily costs of running your application. One particular use case of caching expensive computations is when running queries against the datastore. It’s always faster to get a value from memcache than it is from the datastore. I have used memcache on one of my GAE projects, texd.

Memcache usage can be simplified down to getting and removing values using a unique key. When I tried integrating memcache into texd it became obvious that I will need a simple way to cache results from methods and also invalidate cache entries before running some methods. For example, the result of an expensive query will be cached but that result might need to deleted from the cache when a part of the result is modified in the datastore. What follows is a code listing that can be found here in the texd project on github.

 1 from functools import wraps
 2 from google.appengine.api import memcache
 3 
 4 def get_args(f, args, kwargs):
 5     """Returns all passed arguments to a certain function."""
 6     
 7     import inspect
 8     args_names, _, _, defaults = inspect.getargspec(f)
 9     passed_args = {}
10         
11     if defaults is not None:
12         passed_args = dict(zip(args_names[-len(defaults):], defaults))
13 
14     passed_args.update(dict(zip(args_names, args)))
15     passed_args.update(kwargs)
16 
17     passed_varargs = args[len(args_names):]
18 
19     return passed_args, passed_varargs
20     
21     
22 class cache_entry(object):
23     """Decorator for functions that return cacheable results."""
24     
25     def __init__(self, name_format):
26         self.name_format = name_format
27 
28     def __call__(self, f):
29         def wrapped(*args, **kwargs):
30             args_values, _ = get_args(f, args, kwargs)
31             entry_name = self.name_format.format(**args_values)
32             value = memcache.get(entry_name)
33             
34             if value is None:
35                 value = f(*args, **kwargs)
36                 memcache.set(entry_name, value)
37 
38             return value
39 
40         return wrapped
41 
42 class cache_invalidate(object):
43     """Decorator for functions that need to invalidate some cache
44     entries before they are executed."""
45     
46     def __init__(self, name_formats):
47         self.name_formats = name_formats
48 
49     def __call__(self, f):
50         def wrapped(*args, **kwargs):
51             args_values, _ = get_args(f, args, kwargs)
52 
53             names = [ name.format(**args_values) for name in self.name_formats ]
54             memcache.delete_multi(names)
55 
56             return f(*args, **kwargs)
57 
58         return wrapped

Here are some examples of using these decorators:

1     @cache_entry("{param}_expensive_function")
2     def expensive_function(param):
3         pass
4     
5     @cache_invalidate(["{param}_expensive_function"])
6     def another_function(param):
7         pass

As you can see we are using the cache_entry decorator where we need to cache method return value(s) and the cache_invalidate decorator where we need to delete cache entries before running the method. Key names are formed with function arguments that can be used in the key name using python standard string formatting options. For more concrete examples take a look at texd models.

Android IceCream is out!

Android IceCream source code has been finally released into the wild! IceCream (or Android 4.0) has a lot of new features that will refresh your Samsung Galaxy S or Galaxy S II. Here’s to hoping that custom RAM’s will be released soon and we will be able to upgrade both Galaxy S and Galaxy S II models.

The Quest For A Silent PC

Lately it’s been very uncomfortable to sit in front of my PC for longer periods of time simply because the CPU fan gets extremely loud. The fact that there are three additional fans on the case (Thermaltake V9) doesn’t help either. A simple google query or two and I found two candidates for my upgrade: Thermaltake Jing and Thermaltake Frio. The price difference was not big and I ultimately decided to go with Jing because Frio was not in stock.

Putting it all together didn’t go as easy as I planned it would. I’v spent more hours replacing the fan than I feel comfortable to admit. Basically the problem was that one of the parts was not drilled properly during the production and after a lot of frustration I had to drill a bigger hole and use different screws to hold it in place. It would have been a better idea to replace the fan altogether because it had a defect but I thought that I would waste more time going that route. This made me moderately frustrated.

I made another mistake while replacing the fan. I placed the CPU together with the fan on the motherboard and then tried to attach the motherboard back into the case. Word of advice: do not do this. Simply attach the CPU holder that goes on the back of the motherboard and then attach the motherboard into the case without mounting the fan. The problem with the first approach is that the Jing fan is huge. You will not have a lot of space to tighten the screws that hold the motherboard in place and you will ultimately end up removing the fan. That’s what I did anyway. While trying to place the fan (and the CPU) back to the motherboard I didn’t align the CPU correctly. At this point I was very frustrated and should have gone to bed way before doing something stupid like this. Playing around with an expensive CPU is simply wrong. CPUs in general are very sensitive.

You can probably guess what happened at this point. I wasn’t able to boot my PC. That’s when I decided to go to sleep. The next day I adjusted the CPU, attached the fan again and everything was fine. Well, everything except the fact that Windows is not able to boot anymore. That’s fine though, I’m not in the mood to play games anyway. I’ll stick with archlinux.

The good news is that the Jing fan is completely silent. The only audible sound coming from my PC is the top fan which is probably going to get replaced too.

Effective Java Second Edition

I haven’t published a post in a while so I decided to start writing book reviews in hope to force myself to write more posts in the future. The first review will be for the Effective Java Second Edition book by Joshua Bloch since it’s the last book that I read. In fact, I just finished it few minutes ago.

Even though this book is not intended to be read from cover to cover it’s still interesting enough that you will probably end up doing just that. I started reading it as part of getting more familiar with Java since I started working full time at a company that uses it exclusively. I don’t remember where I found out about it in the first place (probably on StackOverflow) but the book itself has a very good reputation and a lot of promising reviews. Even the inventor of the Java language is recommending it.

“I sure wish I had this book ten years ago. Some might think that I don’t need any Java books, but I need this one.” - James Gosling, inventor of the Java programming language

Others agree with James and recommend the book. The book is structured in “items” where each item describes potential perfomance and/or security improvements. Some of the items (almost all of them in the general programming section) are not even tied to the Java programming language such as:

• Item 13: Minimize the accessibility of classes and members
• Item 15: Minimize mutability
• Item 16: Favor composition over inheritance
• Item 57: Use exceptions only for exceptional conditions
• Item 65: Don’t ignore exceptions
• Item 55: Optimize judiciously
• Item 56: Adhere to generally accepted naming conventions

There are many more interesting items in the book covering topics from managing objects to generics over general programming, concurrency and serialization. To summarize, this book is a must read for anyone working with Java. You can get the book at Amazon.

Uedit markdown editor

While working on the new, improved version of Tutorijali.net we have decided to use the WMD markdown editor. While using it we have discovered that it has many annoying bugs and isn’t very easy to modify. Searching for a better markdown editor which is small and doesn’t have external dependicies yielded no results. So I decided to write a small, easy to modify, markdown editor.

The result is Uedit, a markdown editor that Just Works in IE6+, Opera, Firefox, Chromium and Safari. Uedit supports automatic insertion of list items and a custom state manager for fine control of undo/redo operations. Note that I haven’t tested it in IE9 since that means installing an instance of Windows 7 in a virtual machine just for that. Someone will test it eventually in IE9 (if you do, please let me know of any bugs you find).

How to get it

You can download the upstream version of uedit packaged in a Zip file here. Or you can clone the repository at github.

Bugs, suggestions, patches

Please let me know if you find any bugs in uedit. I’m sure that I have forgot about something. Suggestions are welcome, patches even more so. Just drop me a pull request on github if you have a patch ready.

Thank you

Goran Zdjelar has helped with the design of uedit. He’s responsible for the nice forms design. Thanks!