Tested with Python2.7

Python manifesto

• import this
• Python manifesto by example

“I hate the code and I want as little of it as possible”

• Talk from PyCon2012
• We should add a lot of features not a lot of code
• We have customers because we have features, not because we have tons of code

Logging

Documentation

Examples

Simplest logger usage example

• log_basic.py
• How to configure log output format log_format.py
• How to read logger configuration from file
  • log_file.py
  • log_file.conf
• Logging in many modules
  • log_main.py
  • log_1.py
  • log_2.py

Documentation

• docstring.py
• based on google style guide

Data types

• data_type_dict.py
• data_type_lists.py
• data_type_tuples.py

Introspection

introspection.py

dir built-in function

• dir: abbrev ‘directory’ (list of things, not necessarily files)
• dir in docs
• returns list of attributes of given object
• dir is intended for shell use, so it doesn’t return all attributes (for example metaclass attrs are missing)
• you can override what dir returns for given object by supplying __dir__ method for it. Note that your class must inherit object to do that
• funny thing is that print is a statement (not a function) in Python2.x, hence you can’t say dir(print)

vars, __dict__

• instance variables of object obj are stored in its __dict__ dictionary. You can fetch it using function vars(obj)

using dot operator

• when you use dot operator on an instance of a class A to get a method foo Python first checks for foo in the instance (in its __dict__)

• it won’t find it as there are only instance variables
• then it looks for foo in the type of the instances - A.
• foo is in A.__dict__, so Python knows which method to call

getting list of methods in class / functions in module

• [for m in dir(C) if callable(getattr(m, C))]

type

• you can get type of the object using type function
  • import types
  • type(“hello world”) == types.StringType
• use isinstance if you want to include subclasses as well
  • isinstance(“hello world”, types.StringType) will also work for objects subclassing string
  • isinstance accepts tuples: isinstance(“hello world”, types.StringTypes) can be used to detect unicode strings as well (types.StringTypes is equal to (types.StringType, types.UnicodeType))
  • alternatively, use isinstance(“hello world”, basestring)
• In Python types and classes are often constructed on the fly and there’s no strict class hierarchy. Instead of checking type() or isinstance() a common programming idiom is to try to do whatever you want with the object and handle errors gracefully. This is known as EAFP (Easier to Ask Forgiveness than Permission) style.
  • Explanation on Wikipedia
  • Python glossary

getattr

• returns None in case of failure
• you can provide default value using three argument getattr

pitfalls of using and / or used as conditional expression

• and / or does not return True/False but one of it’s arguments
  • or returns first argument that evaluates to True or last argument
  • and returns first argument that evaluates to False or last argument
• you can simulate ?: from C with and or:
  • B ? a : b (python: if B then a else b) is equivalent to
  • B and a or b - well it’s not
  • but (B and [a] or [b])[0] is
  • the thing is that when a is False then the expression does not work as expected (always b is returned). We can make sure a is never False by putting it into list

Classes

multiple inheritance

• mutliple_inheritance.py

mro (Method Resolution Order)

• documentation
• mro resides in __mro__
• classes are in __mro__ before their base classes
• no duplicates in __mro__
• order of parents of all classes is preserved in __mro__
• there are cases for which valid mro does not exist (nomro.py)

super

• super solves diamond
• super let’s you avoid stating base class explicitly when calling methods
• Python super considered super
• Python super considered harmful
  • super DOES NOT call method in super class. You say WTF? and I say it calls method in class that is next in mro
  • you don’t know what arguments should be passed to next method. You need to pass all arguments you got and always implement methods to accept all arguments
  • you must always use super in all methods (say you forgot to call super in class that inherits only object - in method __init__. objects __init__ will never be called)
  • …

overall

This applies if you have complex class hierarchies and so on which is unlikely unless you produce code of really bad quality. Common case is “I don’t care it doesn’t affect me”.

virtual_methods

• virtual.py
• in Python all methods are effectively virtual
• I couldn’t find a way to use non-virtual interface idiom (c++) - anyone?

subclassing built-in data types

• cnt_dict.py

built-in data types sizes

• bidt_dict.py

you can pass classes/functions as arguments/return values treating them as objects

• class_func_val.py

Regexps

• Documentation
• string methods find/replace string_find.py are good for simple cases
• same as above but using regexps re_find.py
• re.MatchObject documentation (re_match.py - examples). Few functions:
  • group() is the same as group(0) -> return entire match string
  • group(id) returns subgroup number id of the match (0 - everything)
  • group(id1, id2, …) returns tuple of subgroups for each group id
  • groups() returns tuple of all subgroups but 0th
    • if you ask - yes, there is empty tuple () in python
  • groupdict() - dict mapping all named groups to values
  • start(id), end(id) - returns indices where the group matched
  • span(id) - returns (start(id), end(id))
  • expand(template) - see re_match.py for an example
• instead of escaping backslash you can use raw strings
  • “\b” can be replaced with r”\b”. (\b is boundary of word in re)
  • \d works differently than \b (\b has special meaning)
  • “\d” is the same as r”\d” and “\d” - creepy :-)
• re.RegexObject methods are surprisingly different than re module functions
  • re.RegexObject.search(string[, pos[, endpos]])
  • re.search(pattern, string, flags=0)
  • this is just one example - see other functions for more differences
• argument of res.sub - ‘repl’ can be a function!
  • re_sub_repl_func.py
• regexps in Python are not regular - you can describe language that is not regular using Python regexps. For example r”(\w+)\1” describes language of all words composed of two identical parts glued together. For example words “HelloHello”, “WorldWorld” are in this language but “Hello” or “World” are not. This language is obviously not regular - you can’t build DFA that would recognize it. See description on Wikipedia
• troubleshooting
  • print regexp tree representation (re.compile(regexp, flags=re.DEBUG))
  • comments r”\w+(?# python comment in regexp)\d+” is r”\w+\d+”
  • using flags=re.X (re.VERBOSE) you can write regexps that don’t look like result of MD5 function. You can write regexp in multiple lines and add comments using # as usually. re.X documentation

Subclassing

• Talk from PyCon2012
  • classes are dictionaries of functions
  • subclasses point to other dictionaries to reuse their code
  • subclasses are in complete control of what happens
• dynamic_dispatch.py
• LSP (Liskov Substitution Principle)
  • “If S is a subtype of T, then objects of type T may be replaced with objects of type S”
  • LSP is usefull. but it’s ok (and common) to violate it (list can’t be substituted by array due to constructor args)
• OCP (Open-Closed Principle)
  • “Objects have internal invariants and subclasses shouldn’t be able to break those invariants”
  • In other words, the classes capabilities can be extended but the underlying class shouldn’t get broken
    • open_closed_dict.py

Decorators

• TechTalk from PyCon2012 (first part)
• decorators.py
  • decorator is a function that takes function/class and returns new function/class. Can modify the object however it wants
  • doesn’t preserve __doc__ etc.
  • you can have decorators with arguments by creating function that returns decorator

Context Managers - ‘with’ (like RAII)

• TechTalk from PyCon2012 (second part)
• You can define them using classes or @contextlib.contextmanager decorator
• You can use multiple context managers in the same ‘with’ statement (2.7+)
• contextlib.closing will call close for you
• You can handle exceptions in __exit__ (if you return True python thinks you handled the exception)
• context_manageres.py

Metaclasses

• TechTalk from 2012
• metaclasses.py
• Blog post by Ian Bicking
• Stack Overflow question
• Implementing singleton /w metaclass - Stack Overflow
• meta classes to classes are as classes to instances
• type of metaclasses is ‘type’
• you use __metaclass__ = MyMetaClass to use metaclass
• when to use?
  • when some information about the class is known only at run-time
• you can tinker with base classes, name of the class and it’s attriutes. You can even create instances of different class!