Why Python Pickle is Insecure

Python pickle is a powerful serialization module. It is the most common method to serialize and deserialize Python object structures. The pickle module has an optimized cousin calledcPickle that is written in C. In this post I’m going to refer to both modules by the namepickle unless I mention otherwise. The security issues I’m going to discuss apply to both of them.

What This is All About

Pickle was never claimed to be secure. In the pickle documentation there is a warning in red that says:

Warning The pickle module is not intended to be secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.

This clearly states that pickle is insecure. Many think this is because it can load classes other than what you expect and may trick you to run their functions. But the actual security risk is far more dangerous. Unpickling can be exploited to execute arbitrary commands on your machine!
Take this little example:

import pickle
pickle.loads("cos\nsystem\n(S'ls ~'\ntR.") # This will run: ls ~

Or of you are running windows try this instead:

import pickle
pickle.loads("cos\nsystem\n(S'dir'\ntR.") # This will run: dir

You can replace ls and dir with any other command.

I will use pickletools.dis to disassemble the pickle and show you how this is working:

import pickletools
print pickletools.dis("cos\nsystem\n(S'ls ~'\ntR.")

Output:

    0: c    GLOBAL     'os system'
   11: (    MARK
   12: S        STRING     'ls ~'
   20: t        TUPLE      (MARK at 11)
   21: R    REDUCE
   22: .    STOP

Pickle uses a simple stack-based virtual machine that records the instructions used to reconstruct the object. In other words the pickled instructions in our example are:

  1. Push self.find_class(module_name, class_name) i.e. push os.system
  2. Push the string 'ls ~'
  3. Build tuple from topmost stack items
  4. Apply callable to argtuple, both on stack. i.e. os.system(*(‘ls ~’,))

The example is not exploiting a bug in pickle. Reduce is a vital step to instantiate objects from their classes. Take this example where I am unpickling an instance of the built-in objectclass:

import pickletools
import pickle
print pickletools.dis(pickle.dumps(object()))

Output:

    0: c    GLOBAL     'copy_reg _reconstructor'
   25: p    PUT        0
   28: (    MARK
   29: c        GLOBAL     '__builtin__ object'
   49: p        PUT        1
   52: g        GET        1
   55: N        NONE
   56: t        TUPLE      (MARK at 28)
   57: p    PUT        2
   60: R    REDUCE
   61: p    PUT        3
   64: .    STOP

Note the REDUCE step. To create an instance of the class objectpickle has to get the__builtin__.object class and then apply it to the given arguments.

As of 2.3 Python abandoned any pretense that it might be safe to load pickles received from untrusted parties. Because no sufficient security analysis has been done to guarantee this and there isn’t a use case that warrants the expense of such an analysis. As a result all tests for __safe_for_unpickling__ or for copy_reg.safe_constructors were removed from the unpickling code.Source: pickletools.py source code comments

How to Make Unpickling Safer

To make unpickling saferThere is no 100% safety guarantee. pickle was never intended to be secure., you need to control exactly which classes will get created. In pickle this can be done by overriding the find_class method. For example:

import sys
import pickle
import StringIO

class SafeUnpickler(pickle.Unpickler):
    PICKLE_SAFE = {
        'copy_reg': set(['_reconstructor']),
        '__builtin__': set(['object'])
    }
    def find_class(self, module, name):
        if not module in self.PICKLE_SAFE:
            raise pickle.UnpicklingError(
                'Attempting to unpickle unsafe module %s' % module
            )
        __import__(module)
        mod = sys.modules[module]
        if not name in self.PICKLE_SAFE[module]:
            raise pickle.UnpicklingError(
                'Attempting to unpickle unsafe class %s' % name
            )
        klass = getattr(mod, name)
        return klass

    @classmethod
    def loads(cls, pickle_string):
        return cls(StringIO.StringIO(pickle_string)).load()

SafeUnpickler.loads("cos\nsystem\n(S'ls ~'\ntR.") # UnpicklingError: Attempting to unpickle unsafe module os

To extend the PICKLE_SAFE dictionary with your pickle safe classes and modules:

SafeUnpickler.PICKLE_SAFE.update({'__main__': set(['MyClass1', 'MyClass2']), 'MyModule': set(['MyClass3'])})

You need to be really careful with what you include in the PICKLE_SAFE dictionary. The__builtin__ module contains the eval method. Which can be as dangerous as the os.systemmethod.

In cPickle this has to be implemented a bit differently. There is a special attribute calledfind_global that needs to be set to a function that accepts a module name and a class name, and returns the corresponding class object. cPickle.Unpickler can’t be subclassed directly, instead we are going to wrap it in another class:

import sys
import cPickle
import StringIO

class SafeUnpickler(object):
    PICKLE_SAFE = {
        'copy_reg': set(['_reconstructor']),
        '__builtin__': set(['object'])
    }

    @classmethod
    def find_class(cls, module, name):
        if not module in cls.PICKLE_SAFE:
            raise cPickle.UnpicklingError(
                'Attempting to unpickle unsafe module %s' % module
            )
        __import__(module)
        mod = sys.modules[module]
        if not name in cls.PICKLE_SAFE[module]:
            raise cPickle.UnpicklingError(
                'Attempting to unpickle unsafe class %s' % name
            )
        klass = getattr(mod, name)
        return klass

    @classmethod
    def loads(cls, pickle_string):
        pickle_obj = cPickle.Unpickler(StringIO.StringIO(pickle_string))
        pickle_obj.find_global = cls.find_class
        return pickle_obj.load()

SafeUnpickler.loads("cos\nsystem\n(S'ls ~'\ntR.") # UnpicklingError: Attempting to unpickle unsafe module os

As you can see, this solution works. But it is hardly practical for many cases. You need to tell pickle what you want in advance and specifically. The moral of the story according to thepickle documentation

You should be really careful about the source of the strings your application unpickles.

Safer Alternatives

Fortunately, there are alternatives to pickle. They may not be as powerful when it comes to serializing python objects and classes. But for most cases all we need to serialize is basic types and simple data structures.

JSON

JSON is a lightweight computer data interchange format. Its human-readable format gives it an advantage over pickle. The json.org website provides a comprehensive listing of existing JSON bindings, including Python. The json module is now a standard part of python since 2.6.

YAML

YAML is a human-readable data serialization format. YAML has additional features lacking in JSON such as extensible data types, relational anchors, strings without quotation marks, and mapping types preserving key order. PyYAML is a Python binding for YAML. PyYAML allows sophisticated object instantiation to be executed which opens the potential for an injection attack. According to the PyYAML documentation, you need to use yaml.safe_load function to load data from untrusted sources.

Others

Depending on your application there are many other alternatives like: XMLProtocol Buffers,Thrift…There is a useful comparison of data serialization formats in wikipedia

Tagged , , | Leave a comment

PIL Tutorial: From Basic to Advanced Drawing

The ImageDraw module provides simple 2D graphics for Image objects. You can use it to create new images, annotate or retouch existing images, and generate graphics on the fly for usage on the web. In this tutorial I’m going to show you how to use this module to draw 2D graphics. I’ll start with basic shapes and then move on to fairly complicated ones.

The Draw Object

To draw on an image, you need to create an ImageDraw object first. This is what the Drawmethod is for:

Draw(image) => Draw instance

This object will enable you to draw on the given image. You can use it to draw as many shapes as needed, and the image will be modified in place.

Drawing a Rectangle

draw.rectangle(box, fill=None, outline=None)

The box can be any sequence object containing either 2-tuples [(x1, y1), (x2, y2)] or numeric values [x1, y1, x2, y2]. It should contain two pairs of coordinates. The outline is the color to use for the rectangle’s outline. The fill is the color to use for the rectangle’s interior.

Example:

from PIL import Image, ImageDraw
im = Image.new('RGBA', (100, 100), (0, 0, 0, 0)) # Create a blank image
draw = ImageDraw.Draw(im) # Create a draw object
draw.rectangle((10, 10, 90, 90), fill="yellow", outline="red")

Here I used color names to specify fill and outline colors. You can also use tuples or hexadecimal color specifiers as #rrggbb. Refer to ImageDraw documentation for the full list of supported color formats.

Tip: To draw a rectangle with a specific outline width that is greater than 1px, you can draw two rectangles on top of each other with the inner box slightly smaller. Fill the outer rectangle with the outline color and the inner one with the fill color:

from PIL import Image, ImageDraw

def rectangle(input, box, fill, outline, width):
    draw = ImageDraw.Draw(input)
    draw.rectangle(box, fill=outline) # The outer rectangle
    draw.rectangle( # The inner rectangle
        (box[0] + width, box[1] + width, box[2] - width, box[3] - width),
        fill=fill
    )

input = Image.new('RGBA', (40, 40), (0, 0, 0, 0)) 
rectangle(input, (0, 0, 39, 39), "lightblue", "blue", 5)

The result will look like this: 

Drawing a Circle

draw.ellipse(box, fill=None, outline=None)

Draws an ellipse inside the given bounding box. If width is equal to height, the result will be a circle:

im = Image.new('RGBA', (100, 100), (0, 0, 0, 0)) # Create a blank image
draw = ImageDraw.Draw(im) # Create a draw object
draw.ellipse((0, 0, 100, 100), fill=(255, 255, 255)) # Draw a circle

To specify the outline width you can use the same tip as in the rectangle example.

Drawing a Polygon

draw.polygon(coordinates, fill=None, outline=None)

The polygon outline consists of straight lines between the given coordinates, plus a straight line between the last and the first coordinate.
The coordinate list can be any sequence object containing either 2-tuples [ (x, y), ... ] or numeric values [ x, y, ... ]. It should contain at least three coordinates.
The following will draw a regular pentagon:

im = Image.new('RGBA', (100, 100), (0, 0, 0, 0)) # Create a blank image
draw = ImageDraw.Draw(im)
lines = [(50, 0), (0, 40), (20, 100), (80, 100), (100, 40)]
draw.polygon(lines, fill="black")

Drawing Text

text(position, text, fill=None, font=None)

Draws the string at the given position. The position gives the upper left corner of the text. The font option is used to specify which font to use. It should be an instance of the ImageFont class, typically loaded from file using the load or truetype method in theImageFont module. The fill option specifies the color to use for the text:

im = Image.new('RGBA', (100, 100), (0, 0, 0, 0)) # Create a blank image
draw = ImageDraw.Draw(im)
draw.text((10, 50), 'Hello World!')

You can use the ImageFont.truetype method to create a font object for drawing text.

ImageFont.truetype(font_path, size) => Font instance

This will load a TrueType or OpenType font file, and create a font object. This function loads a font object from the given file, and creates a font object for a font of the given size. On Windows, if the given file name does not exist, the loader also looks in Windows fonts directory.

Here are the default font directory paths:

Linux: /usr/share/fonts/
Max OS: /Library/Fonts/
Windows: C:\Windows\fonts

Example:

from PIL import ImageDraw, ImageFont
font = ImageFont.truetype(
    'path/to/font.ttf', 30
)
draw.text((10, 50), 'Logo', font=font, fill="blue")

If you want to draw text that fits exactly inside an image, then you can use the handyfont.getsize method:

def draw_text(text, size, fill=None):
    font = ImageFont.truetype(
        'path/to/font.ttf', size
    )
    size = font.getsize(text) # Returns the width and height of the given text, as a 2-tuple.
    im = Image.new('RGBA', size, (0, 0, 0, 0)) # Create a blank image with the given size
    draw = ImageDraw.Draw(im)
    draw.text((0, 0), text, font=font, fill=fill) #Draw text
    return im

img = draw_text('Google', 30, (82, 124, 178))

Output:

Drawing Text at an Angle

The text method doesn’t support writing at an angle. But we can achieve rotation by simply rotating the resulting image. Here is the modified version:

def draw_text(text, size, angle=0, fill=None):
    font = ImageFont.truetype(
        'path/to/font.ttf', size
    )
    size = font.getsize(text) # Returns the width and height of the given text, as a 2-tuple.
    im = Image.new('RGBA', size, (0, 0, 0, 0)) # Create a blank image with the given size
    draw = ImageDraw.Draw(im)
    draw.text((0, 0), text, font=font, fill=fill) #Draw text
    return im.rotate(angle, expand=True)

img = draw_text('Google', 30, 45, (82, 124, 178))

Output:

Drawing Rounded Corners Rectangle

The idea is simple enough: draw a rectangle then paste a rounded corner on top of each corner.

def round_corner(radius, fill):
    """Draw a round corner"""
    corner = Image.new('RGBA', (radius, radius), (0, 0, 0, 0))
    draw = ImageDraw.Draw(corner)
    draw.pieslice((0, 0, radius * 2, radius * 2), 180, 270, fill=fill)
    return corner

def round_rectangle(size, radius, fill):
    """Draw a rounded rectangle"""
    width, height = size
    rectangle = Image.new('RGBA', size, fill)
    corner = round_corner(radius, fill)
    rectangle.paste(corner, (0, 0))
    rectangle.paste(corner.rotate(90), (0, height - radius)) # Rotate the corner and paste it
    rectangle.paste(corner.rotate(180), (width - radius, height - radius))
    rectangle.paste(corner.rotate(270), (width - radius, 0))
    return rectangle

img = round_rectangle((50, 50), 10, "yellow")

The result: 

Drawing a Heart

The heart shape can be a little tricky. You need to get the calculations right. The following method draws a polygon and two ellipses, like this:
 +  => 

def heart(size, fill):
    width, height = size
    im = Image.new('RGBA', size, (0, 0, 0, 0))
    draw = ImageDraw.Draw(im)
    polygon = [
        (width / 10, height / 3),
        (width / 10, 81 * height / 120),
        (width / 2, height),
        (width - width / 10, 81 * height / 120),
        (width - width / 10, height / 3),
    ]
    draw.polygon(polygon, fill=fill)
    draw.ellipse((0, 0,  width / 2, 3 * height / 4), fill=fill)
    draw.ellipse((width / 2, 0,  width, 3 * height / 4), fill=fill)
    return im

im = heart((50, 40), "red")
Tagged , , , | Leave a comment

Animated Terminal Progress Bar in Python

In my previous article python terminal controller, I explained a simple way to control display in the terminal and print in colors. In this article I’m going to use the terminal module to draw a fancy animated progress bar.

progressbar

The following module simply draws an animated progress bar in the terminal along with a message. The progress bar will auto adjust itself to fit. And the message can spread over multiple lines.

# -*- coding: utf-8 -*-
# Copyright: 2009 Nadia Alramli
# License: BSD
"""Draws an animated terminal progress bar
Usage:
    p = ProgressBar("blue")
    p.render(percentage, message)
"""

import terminal
import sys

class ProgressBar(object):
    """Terminal progress bar class"""
    TEMPLATE = (
     '%(percent)-2s%% %(color)s%(progress)s%(normal)s%(empty)s %(message)s\n'
    )
    PADDING = 7

    def __init__(self, color=None, width=None, block='█', empty=' '):
        """
        color -- color name (BLUE GREEN CYAN RED MAGENTA YELLOW WHITE BLACK)
        width -- bar width (optinal)
        block -- progress display character (default '█')
        empty -- bar display character (default ' ')
        """
        if color:
            self.color = getattr(terminal, color.upper())
        else:
            self.color = ''
        if width and width < terminal.COLUMNS - self.PADDING:
            self.width = width
        else:
            # Adjust to the width of the terminal
            self.width = terminal.COLUMNS - self.PADDING
        self.block = block
        self.empty = empty
        self.progress = None
        self.lines = 0

    def render(self, percent, message = ''):
        """Print the progress bar
        percent -- the progress percentage %
        message -- message string (optional)
        """
        inline_msg_len = 0
        if message:
            # The length of the first line in the message
            inline_msg_len = len(message.splitlines()[0])
        if inline_msg_len + self.width + self.PADDING > terminal.COLUMNS:
            # The message is too long to fit in one line.
            # Adjust the bar width to fit.
            bar_width = terminal.COLUMNS - inline_msg_len -self.PADDING
        else:
            bar_width = self.width

        # Check if render is called for the first time
        if self.progress != None:
            self.clear()
        self.progress = (bar_width * percent) / 100
        data = self.TEMPLATE % {
            'percent': percent,
            'color': self.color,
            'progress': self.block * self.progress,
            'normal': terminal.NORMAL,
            'empty': self.empty * (bar_width - self.progress),
            'message': message
        }
        sys.stdout.write(data)
        sys.stdout.flush()
        # The number of lines printed
        self.lines = len(data.splitlines())

    def clear(self):
        """Clear all printed lines"""
        sys.stdout.write(
            self.lines * (terminal.UP + terminal.BOL + terminal.CLEAR_EOL)
        )

This module is very easy to use, for example:

from progressbar import ProgressBar
import time
p = ProgressBar()
for i in range(101):
    p.render(i, 'step %s' % i)
    time.sleep(0.1)

default progressbar

You can pass in any character to be used as a progress display. You can change the color and width as well. Here is another fancy example:

from progressbar import ProgressBar
import time
p = ProgressBar('green', width=20, block='▣', empty='□')
for i in range(101):
    p.render(i, 'step %s\nProcessing...\nDescription: write something.' % i)
    time.sleep(0.1)

fancy progressbar

Here is a useful list of unicode characters. Any of these characters can be used to draw the progress bar if needed.

Attachment Size
Progress Bar Module 2.49 KB
Tagged , , , | Leave a comment

Terminal Controller for Python

I was looking for an easy way to control the terminal and print colored text in Python. I found this useful recipe. But I wanted something simpler yet powerful, so I decided to write my own little terminal controller.

Most computer terminals and terminal emulators support color and cursor control through a system of special escape sequences. For example, printing the sequence \x1b[H\x1b[2J will result in clearing the screen. Likewise, printing \x1b[31m changes the current terminal foreground color to red. You can go ahead and try:

# Clear the screen.
print '\x1b[H\x1b[2J'

This lightweight module uses curses to do the job. Curses Programming with Python is a great guide if you want to learn more about it.

The module supports the following colors:
supported colors
It also support various capabilities like going up, down, to beginning of the line and clearing the screen.

# Copyright: 2008 Nadia Alramli
# License: BSD

"""Terminal controller module
Example of usage:
    print BG_BLUE + 'Text on blue background' + NORMAL
    print BLUE + UNDERLINE + 'Blue underlined text' + NORMAL
    print BLUE + BG_YELLOW + BOLD + 'text' + NORMAL
"""

import sys

# The current module
MODULE = sys.modules[__name__]

COLORS = "BLUE GREEN CYAN RED MAGENTA YELLOW WHITE BLACK".split()
# List of terminal controls, you can add more to the list.
CONTROLS = {
    'BOL':'cr', 'UP':'cuu1', 'DOWN':'cud1', 'LEFT':'cub1', 'RIGHT':'cuf1',
    'CLEAR_SCREEN':'clear', 'CLEAR_EOL':'el', 'CLEAR_BOL':'el1',
    'CLEAR_EOS':'ed', 'BOLD':'bold', 'BLINK':'blink', 'DIM':'dim',
    'REVERSE':'rev', 'UNDERLINE':'smul', 'NORMAL':'sgr0',
    'HIDE_CURSOR':'cinvis', 'SHOW_CURSOR':'cnorm'
}

# List of numeric capabilities
VALUES = {
    'COLUMNS':'cols', # Width of the terminal (None for unknown)
    'LINES':'lines',  # Height of the terminal (None for unknown)
    'MAX_COLORS': 'colors',
}

def default():
    """Set the default attribute values"""
    for color in COLORS:
        setattr(MODULE, color, '')
        setattr(MODULE, 'BG_%s' % color, '')
    for control in CONTROLS:
        setattr(MODULE, control, '')
    for value in VALUES:
        setattr(MODULE, value, None)

def setup():
    """Set the terminal control strings"""
    # Initializing the terminal
    curses.setupterm()
    # Get the color escape sequence template or '' if not supported
    # setab and setaf are for ANSI escape sequences
    bgColorSeq = curses.tigetstr('setab') or curses.tigetstr('setb') or ''
    fgColorSeq = curses.tigetstr('setaf') or curses.tigetstr('setf') or ''

    for color in COLORS:
        # Get the color index from curses
        colorIndex = getattr(curses, 'COLOR_%s' % color)
        # Set the color escape sequence after filling the template with index
        setattr(MODULE, color, curses.tparm(fgColorSeq, colorIndex))
        # Set background escape sequence
        setattr(
            MODULE, 'BG_%s' % color, curses.tparm(bgColorSeq, colorIndex)
        )
    for control in CONTROLS:
        # Set the control escape sequence
        setattr(MODULE, control, curses.tigetstr(CONTROLS[control]) or '')
    for value in VALUES:
        # Set terminal related values
        setattr(MODULE, value, curses.tigetnum(VALUES[value]))

def render(text):
    """Helper function to render text easily
    Example:
    render("%(GREEN)s%(BOLD)stext%(NORMAL)s") -> a bold green text
    """
    return text % MODULE.__dict__

try:
    import curses
    setup()
except Exception, e:
    # There is a failure; set all attributes to default
    print 'Warning: %s' % e
    default()

The render function provides an easy way to render text using Python string formatting syntax. Example of usage:

from terminal import render
print render('%(BG_YELLOW)s%(RED)s%(BOLD)sHey this is a test%(NORMAL)s')
print render('%(BG_GREEN)s%(RED)s%(UNDERLINE)sAnother test%(NORMAL)s')

The result will look like this:
terminal example
terminal example 2
Note that terminal.NORMAL escape sequence is used to turn off all attributes. You can also access escape sequences directly by calling terminal.BG_GREEN, terminal.GREEN, terminal.BOLD, terminal.UP, etc.

The setup function is where the logic resides. I'm using curses.tigetstr(capability_name)to get the escaping sequence corresponding to the capability name, andcurses.tparm(string, *params) to instantiate the string with the supplied parameters. For example, to get the escape sequence for the foreground color blue, I need to do the following steps:

  • Call curses.tigetstr('setaf') which returns '\x1b[3%p1%dm'. Where 'setaf' is the capability that set foreground color using ANSI escape sequences. Note that %p1%d is a parameter to be filled with the color index later using curses.tparm.
  • Then call curses.tparm('\x1b[3%p1%dm', curses.COLOR_BLUE) that returns '\x1b[34m'which corresponds to change the current foreground color to blue.

Unfortunately, those escaping sequences are not supported on all platforms. Therefore, any module that uses them should have a backup plan. The default function is only intended to give a default value to all attributes in case of a failure. This way, if the terminal doesn't support colors or if curses doesn't exist, all attributes will default to '' and won't have any effects.

This guide contains a comprehensive list of all terminfo string and numeric capabilities supported by curses. Any of theses capabilities can be added to the module if needed. Note that some capabilities may not be supported by some terminals.

Related:

Animated terminal progress bar
progressbar

Attachment Size
Terminal controller module 2.45 KB
Tagged , , , | Leave a comment

PIL Tutorial: Converting Between PNG and GIF

PIL supports both PNG and GIF image formats, but converting between the two while keeping transparency can be tricky. Inside is a list of tips for dealing with transparency and dithering issues when processing GIF image formats in PIL. If you are only interested in the final solution you can skip this tutorial and go directly to the code

Saving GIF with transparency

Support for writing GIF transparency was added in PIL1.1.4. Yet the latest PIL version 1.1.6 doesn’t not automatically write the transparency index for you. In order to keep the transparency, you need to explicitly specify the transparent color index as an option when saving the image in GIF format.

im = Image.open('icon.gif')
transparency = im.info['transparency'] 
im.save('icon.gif', transparency=transparency)

im.info is a dictionary that contains a set of properties defined by the Image.open method.

Converting from GIF into PNG

Luckily PNG supports palette-based images. This will save us from a lot of work. The only thing you have to do is setting the transparency option.

im = Image.open('icon.gif')
transparency = im.info['transparency'] 
im .save('icon.png', transparency=transparency)

Converting from PNG into GIF

This can be tricky for two reasons:

  • PNG gives a much wider range of color depths than GIF, truecolor up to 48-bit compared to 8-bit 256-color.
  • PNG gives a much wider range of transparency options than GIF, including alpha channel transparency.

PNG image is palette-based

If the PNG image is palette-based, i.e. P mode, then the conversion is straightforward. The only thing you have to do is setting the transparency option as before:

im = Image.open('icon.png')
assert im.mode == 'P'
transparency = im.info['transparency'] 
im .save('icon.gif', transparency=transparency)

PNG image is in RGBA mode

When converting form RGBA into P, we need to reduce the number of distinct colors used in the image to a maximum of 256 colors through a process called quantization.

PIL supports two types of palette quantizers, WEB and ADAPTIVE. The default quantizer in PIL1.1.6 is Image.WEB which reduces the color space to web safe colors and dithers the rest. While Image.ADAPTIVE quantizer is intended to keep the converted image as visually similar as possible to the original image.

The following table explains the difference in output between WEB and ADAPTIVE quantizers in PIL:

PNG: Original Image GIF: WEB quantizer GIF: ADAPTIVE quantizer
mouse mouse bad mouse

So to avoid dithering, we should use the ADAPTIVE quantizer like this:

im = Image.open('mouse.png')
im = im.convert('RGB').convert('P', palette=Image.ADAPTIVE)
im.save('mouse.gif')

Now that we know how to solve the first issue, we need to find a solution for the transparency issue.

Solid background workaround

Sometimes you don’t need to keep the transparency; all what you want is a solid background. This will do the trick for you:

im = Image.open('mouse.png')
# Create a new image with a solid color
background = Image.new('RGBA', im.size, (255, 255, 255))
# Paste the image on top of the background
background.paste(im, im)
im = background.convert('RGB').convert('P', palette=Image.ADAPTIVE)
im.save('mouse.gif')

mouse with white background

Solution for keeping transparency

GIF allows you to set one of the colors in the palette as fully transparent, whereas PNG images in RGBA mode allow you to have different levels of transparency through the alpha band. An alpha value of 255 means the pixel is fully opaque, 0 means fully transparent and Intermediate values indicate partially transparent pixels.

Let’s try to set the black background to transparent:

im = Image.open('mouse.png')
im = im.convert('RGB').convert('P', palette=Image.ADAPTIVE)
# The black index in the palette of this image is 255
im.save('mouse.gif', transparency=255)

bad transparent mouse
In addition to the background, black is also used in other places in the image, such as the eyes. That’s why the result is bad. A solution to this is to fill the transparent parts with a unique color that doesn’t exist in other opaque parts.

The solution

The trick is to use the colors keyword argument in im.convert. This argument tells converthow many colors it should use in the palette. The default is 256 which is the maximum possible number of colors in the palette. To reserve a color in the palette for transparency we can tell convert to only use 255 out of 256 and then use the last index 255 to fill the transparent parts.

from PIL import Image

im = Image.open('mouse.png')
# Get the alpha band
alpha = im.split()[3]
# Convert the image into P mode but only use 255 colors in the palette out of 256
im = im.convert('RGB').convert('P', palette=Image.ADAPTIVE, colors=255)
# Set all pixel values below 128 to 255,
# and the rest to 0
mask = Image.eval(alpha, lambda a: 255 if a <=128 else 0)
# Paste the color of index 255 and use alpha as a mask
im.paste(255, mask)
# The transparency index is 255
im.save('mouse.gif', transparency=255)

We used the paste function to fill the transparent parts with the last reserved index. For more examples on using the paste function please refer to PIL Tutorial: How to Create a Button Generator.

And here is the final result:
background mouse

 

Tagged , , , , | Leave a comment

PIL Tutorial: How to Create a Button Generator

Python Imaging Library (PIL) is an external library for Python. It supports opening, manipulating, and saving different image file formats. PIL is well documented and easy to install.

In this tutorial I’m focusing on the paste function. At the end of the tutorial, I’ll explain a useful script that converts any icon into a web 2.0 style button, just like these:

delicious button feed button blogger button

The paste function

im.paste(im, box=None, mask=None)

The first argument can either be an Image object or a color. If it is an Image, the image will be pasted into this image. If it is a color, the region specified by the box argument will be filled with a single color.
The box argument defines the size and position of the pasting region. It can be a 2-tuple that defines the left upper corner, or a 4-tuple that defines the left, upper, right and lower pixel coordinate. It defaults to None, which translates to (0, 0, self.width, self.height)PIL’s documentation says the default is (0, 0), but I checked the source code of version 1.1.6 and found this not to be the case..
When providing a mask argument, it only pastes the regions indicated by the mask. You can either use "1""L" or "RGBA" images. In the latter case, the alpha band is used as mask. Where the mask is 255, the given source is copied as is. Where the mask is 0, the current value is preserved. Intermediate values can be used for transparency effects.

Examples

Please note I’m using transparency as a background for illustrative purposes only; it’s not actually a part of the image itself.Those unfortunate to use IE6 (or older) won’t be able to view the transparent parts, but I guess this won’t affect the readability of the tutorial.

We have two images, a flower and a butterfly.The flower and butterfly icons were downloaded from iconspedia

flower    butterfly

We want to paste the butterfly image on top of the flower. Since the butterfly image is smaller in size, we need to set the box argument. Otherwise, PIL will complain saying “Images do not match”. I will set the box to (40, 10), which is roughly the top edge of the flower.

flower = Image.open('flower.png')
butterfly = Image.open('butterfly.png')
flower.paste(butterfly, (40, 10))

Now let’s see how the result looks:

butterfly on flower bad

Oops, transparency was pasted on the flower along with the butterfly’s image, definitely not what we want!
But that is to be expected; we asked to paste the image as it is. We should have specified a mask argument to tell PIL what to paste and what to leave.
If we specify a mask that is the same as the source then we will get the effect we want.

flower = Image.open('flower.png')
butterfly = Image.open('butterfly.png')
flower.paste(butterfly, (40, 10), butterfly)
butterfly on flower

Where the mask’s alpha is 255 (opaque), the source is copied as it is. Where the alpha is 0 (transparent), the source is not copied.

Very simple button effect

Now let’s say we want to create a very simple button effect. For this purpose we need an image with a transparent border effect.I created the border image with GIMP.

border effect
border = Image.open('border.png')
flower.paste(border, mask=border)
flower with border bad

No, this is not what we want. Part of the transparency is being copied along with the border, but why?
Because as we learned in the previous example, transparency is not ignored by the pastefunction. In fact transparency in the source image is being treated as any other color band.This is actually a personal conclusion from numerous tests. Therefore, even when using the same image as the source and mask, the intermediate transparency levels will be copied from the source into the result. To work around this, we should get rid of the alpha band in the source image, while keeping it in the mask. And what is easier to achieve this than converting the image into ‘RGB’.

border = Image.open('border.png')
source = border.convert('RGB')
flower.paste(source, mask=border)
flower with border

Now this worked!

Putting it all together

We’ll put everything we learned together to create a web 2.0 style button.
First we need a mask image to crop the icon in the shape and size we want. Then, we need a highlight image to create the button effect.Both the mask and the round button effect were created with GIMP.

round mask    round highlight

The mask and highlight images need to be of the same size for our example to work.
Let’s convert this Blogger icon into a button

blogger icon
highlight = Image.open('round.png')
mask = Image.open('round-mask.png')
# Read the icon and convert it into 'RGBA' in case it wasn't
icon = Image.open('blogger.png').convert('RGBA')
button = Image.new('RGBA', mask.size)

# Resize Icon
icon = ImageOps.fit(
  icon, highlight.size, method=Image.ANTIALIAS, centering=(0.5, 0.5)
)

# Create a helper image that will hold the icon after the reshape
helper = button.copy()
# Cut the icon by the shape of the mask
helper.paste(icon, mask=mask)

# Fill with a solid color by the mask's shape
button.paste((255, 255, 255), mask=mask)
# Get rid of the icon's alpha band
icon = icon.convert('RGB')
# Paste the icon on the solid background
# Note we are using the reshaped icon as a mask
button.paste(icon, mask=helper)

# Get a copy of the highlight image without the alpha band
overlay = highlight.copy().convert('RGB')
button.paste(overlay, mask=highlight)

button.save('button.png')

And the result is:

blogger button

You may wonder why ImageOps.fit was used instead of Image.resizeImage.resize will work perfectly fine with images that have the same aspect ratio as the new size, but will distort the image otherwise. ImageOps.fit will resize and crop the image to match the requested aspect ratio and size. You have the option to control the cropping position, which is the center (0.5, 0.5) in our case.

You have to keep in mind that if we want our script to handle icons with transparent backgrounds, we need to specify a background color that fills the transparent spaces, or we’ll end up with buttons that look like this:

button bad

This is why I fill the button’s background with a solid color before pasting the icon.

Web 2.0 style buttons generator

Now here is the complete script with command line arguments support using getopt module
Example of usage:

./make-button.py feed.png
./make-button.py delicious.gif blogger.png butterfly.png --shape round
./make-button.py butterfly.png --shape heart --color 200 120 22

I’ve also attached the full script with helper images in a zip file for convenience. The helper images include round, heart and square shaped button effects.

#!/usr/bin/python

# Button generator script
# Author: Nadia Alramli http://nadiana.com

from PIL import Image
import ImageOps

from optparse import OptionParser
import os.path

def create_button(iconPath, shape, bgColor):
  highlight = Image.open('helper/%s.png' % shape)
  mask = Image.open('helper/%s-mask.png' % shape)
  # Read the icon and convert it into 'RGBA' in case it wasn't
  icon = Image.open(iconPath).convert('RGBA')
  button = Image.new('RGBA', mask.size)

  # Resize Icon
  icon = ImageOps.fit(
    icon, highlight.size, method=Image.ANTIALIAS, centering=(0.5, 0.5)
  )

  # Create a helper image that will hold the icon after the reshape
  helper = button.copy()
  # Cut the icon by the shape of the mask
  helper.paste(icon, mask=mask)

  # Fill with a solid color by the mask's shape
  button.paste((255, 255, 255), mask=mask)
  # Get rid of the icon's alpha band
  icon = icon.convert('RGB')
  # Paste the icon on the solid background
  # Note we are using the reshaped icon as a mask
  button.paste(icon, mask=helper)

  # Get a copy of the highlight image without the alpha band
  overlay = highlight.copy().convert('RGB')
  button.paste(overlay, mask=highlight)

  iconPath, ext = os.path.splitext(iconPath)
  button.save('%s-%s.png' % (iconPath, shape))

def main():
  usage = 'usage: %prog file [file] [option]'
  parser = OptionParser(usage)

  parser.add_option('-s', '--shape', dest='shape', help='button shape')
  parser.add_option(
    '-c', '--color', dest='color', help='background color', 
    nargs=3, type= int
  )
  parser.set_defaults(shape='square', color=(255, 255, 255))
  options, args = parser.parse_args()

  if len(args) < 1:
    parser.error('Missing file operand')

  shape = options.shape
  bgColor = options.color

  for arg in args:
    iconPath = arg
    create_button(iconPath, shape, bgColor)

if __name__ == '__main__':
    main()

With this script you can add as many button effects as you want. All you need is two images: a mask and a highlight.
Let’s say the style you want is a “star” then name the highlight image as star.png and the mask image as star-mask.png and place both of them in the helper folder.
Then run:

./make-button.py icon.png --shape star
Attachment Size
button_generator.zip 60.1 KB
Tagged , , , | Leave a comment

jQuery Confirm Plugin

This plugin displays a confirmation message in place before doing an action. It does not require adding any extra code apart from a call to the plugin itself. One call to $(element).confirm() will do the magic. Also, this plugin doesn’t require you to provide a callback function; it figures it out on its own.
You can check the examples for more info.

How does it work?

To put it simply, it saves a copy of all event handlers bound to the element, unbinds them and binds itself instead. A confirmation dialog is displayed when the user triggers the action. If the user chooses to proceed with the action, it rebinds the handlers again and triggers the event. If the user chooses to cancel the action, the dialog disappears.

Docs

$(element).confirm(options)
options
Name Type Optional Description Default Value
msg String Optional Confirmation message ‘Are you sure?’
stopAfter string Optional Determines when the confirmation dialog will stop being promoted.
Possible values (‘never’, ‘ok’, ‘cancel’, ‘once’)
‘never’
wrapper String Optional Dialog wrapper ‘<span></span>’
eventType String Optional The event that triggers the confirmation ‘click’
dialogShow String Optional Dialog displaying effect ‘show’
dialogSpeed String Optional Dialog displaying speed ‘fast’
timeout Integer Optional Time in milliseconds to hide the confirmation dialog 0
options.buttons
ok String Optional Ok caption ‘Yes’
cancel String Optional Cancel caption ‘No’
wrapper String Optional Button wrapper ‘<a href=”#”></a>’
separator String Optional Separator between buttons ‘/’
cls String Optional Button class undefined

Examples

First Example

// The action.
$('a').click(function() {
  alert('click');
  return false;
});

// The most simple use.
$('a').confirm();

click me!

Second Example

// The action.
$('input[type=button]').click(function() {
  $(this).remove();
});

$('input[type=button]').confirm({
  msg:'Do you really want to delete this button?',
  timeout:3000
});

Third Example

// The action.
$('span').mouseover(function() {
  $(this).html('Here is the offer');
});

$('span').confirm({
  msg:'See my interesting offer?',
  stopAfter:'ok',
  eventType:'mouseover',
  timeout:3000,
  buttons: {
    ok:'Sure',
    cancel:'No thanks',
    separator:'  '
  }
});

confirmation on mouseover.

Fourth Example

$('a').confirm({
  timeout:3000,
  dialogShow:'fadeIn',
  dialogSpeed:'slow',
  buttons: {
    wrapper:'<button></button>',
    separator:'  '
  }  
});

click me!

Tagged , , , | Leave a comment

SEO Services provided by SEOExplode.us