How to get a Belkin N+ Router to work with the IPhone and XBox Live

I had a bit of a time trying to get my new Belkin N+ router (F5D8236uk4) to replace my beloved Linksys with custom DD-WRT firmware. My biggest issues were getting it to work smoothly with my iPhone and to work at all with XBox Live on my XBox 360. After searching through many forum posts I found at that many people have had similar issues. So although this is about the Belkin router I am sure that many of the same issues resolved here can apply to other routers, especially newer models.

iPhone issues

This was a bit of a long running and frustrating issue. Basically the iPhone would connect to the router and have lightening fast internet access thanks to the improved wifi capablilites of the Belkin, but after going to sleep and reconnecting, it would either take ages to connect and when it did there would be LAN access but no internet access OR would ask for the network password each time. The short term fixes seemed to be forgetting the network settings on the iPhone, rebooting the rooter or clicking on the renew lease button on the iPhone. So, this indicated to me that the router was experiencing DHCP issues, however none of our other devices were having problems! To rule it out I set a static IP address and hardcoded in the DNS address into the iPhone (the actual broadband provider's DNS not just the IP of the router). If you're not following this, don't worry it's not the solution unlike many forum post were seeming to indicate!

Now I had ruled this out the next natural thing to try was changing the broadcast mode of the router from b, g and n to b only as suggested by other posters. The rationale being that maybe the iPhone could not handle more advanced brodcast modes. Again as for the previous 'fix' this worked for a short period but sporadic connection faliures crept back in. So what could it be??

I had set up my router, naturally, to use the best for of wifi security available WPA2 as of course we all want the best security possible. All our laptops and even the XBox had no problem with this but it turns out that iOS4 cannot handle it. The clue here was the constant need to re-input the newtork password. So changing from WPA2 to the slighlty older but still pretty strong WPA encryption did the trick. You don't, as some people suggest, need to resort to the ancient WEP protocol. The iPhone now works like a dream.

XBox 360

For some reason, unknown to me I never had to port forward with my old Linksys, but after a bit of searching on the unusually good Microsoft support site, I learnt that I needed to set up port forwarding to allow connections to XBox Live. Actually, you don't need to do this. On your XBox go to configure network and set up a static IP address. If your DHCP server on the router assigns address up to xxx.xxx.xxx.200 for example then set the static address on your XBox to be xxx.xxx.xxx.201 to avoid conflicts. Next go to the router and set up a Demilitarised Zone (DMZ), setting the local address to be the one you choose on your XBox. Hey presto! Your XBox should now work fine without needing to to mess about with port forwarding, just don't forget to enable uPNP (it should be enabled by default).

Read more »

Singletons and the Zend Registry

Preface

Hi all, I know it's been an eternity since I have last posted on my blog, but life sometimes has a habit of getting in the way. I am going to be writing a series of posts on optimising code within the Zend Framework starting, rather illogically I guess with using the singleton design pattern and the Zend Registry system.

Singleton patterns?

First of all, what are singleton patterns and why do they help you write more efficient code? Basically a singleton is a class that can only ever have one instance in memory at one time, or in object-orientated programming parlance, only allows one object to be instantiated at a time. Generally singelton patterns are useful where an object is used to co-ordinate many parts of a system (from Singleton pattern on wikipedia). In the context of optimising our PHP code, it can be very useful to ensure that a class that is expensive, in terms of CPU cycles or memory to run is only run once and the instance is stored somewhere until it is needed.

Creating singletons

Good examples of a case where using the singleton pattern is advantageous, are classes that perform multiple database queries on instantiation. Examples where I have come across this are classes that dynamically generate Zend Navigation objects from the database and cache objects. Let's look at one of these two examples in more detail.

class productNavigation
{
    // Holds the Zend_Navigation object
    protected $_container;

    public function __construct()
    {
        $model = new Models_Some_Model();
        $items = $model->getAll();

        // Instantiate a Zend_Navigation
        $this->_container = new Zend_Navigation();

        if($items)
        {
            // Loop through items and add to container
            foreach($items as $item)
            {
                $this->getContainer()>addPage($item);
            }
        }
    }

    // Getter function
    public function getContainer()
    {
        return $this->_container;
    }
}

In this rather silly example you can see that every time we need to get the navigation container, for instance when building breadcrumbs, a sitemap or a menu system the code has to perform a number of database queries, create a new Zend Navigation object and add pages to it. A better way of doing it would be to instantiate the object once and store it. To do this we have two methods, the 'traditional' way and using the Zend Registry.

The traditional way to create a singleton class is to use static methods. Declare the constructor as private (or protected) to prevent the object being instantiated normally and create a public static method to retrieve an instance of the object, creating it if it does not already exist.

class productNavigation
{
    // Holds the Zend_Navigation object
    protected $_container;
    // Holds the instance
    private static $instance;

    // Set to be private to stop normal instantiation
    private function __construct()
    {
        $model = new Models_Some_Model();
        $items = $model->getAll();

        // Instantiate a Zend_Navigation
        $this->_container = new Zend_Navigation();

        if($items)
        {
            // Loop through items and add to container
            foreach($items as $item)
            {
                $this->getContainer()>addPage($item);
            }
        }
    }

    // Getter function
    public function getContainer()
    {
        return $this->_container;
    }
    
    //  Singleton method
    public static function get()
    {
        if(!isset(self::$instance))
        {
            // Create new instance if it is not already instantiated
            self::$instance = new productNavigation();
        }
        
        return self::$instance;
    }
}

Using Zend_Registry

The second way is by using the Zend Registry. The Zend Registry is basically a store of objects that can be retrieved at will by using an associated key. In this example you would not need to modify the class in any way, simply create and store an instance in your bootstrap or using a controller plugin and retrieve the instance as and when needed as the following example rather crudely demonstrates.

// Bootstrap code
$navigation = new productNavigation();
Zend_Registry::set('navigation', $navigation);

// In a controller, for example
$navigation = Zend_Registry::get('navigation')->getContainer();

There are other strategies for doing all of this, but these are the most common and can help streamline your bloated code. In another post I will describe processes for identifying areas of your code that are creating bottlenecks.

Read more »

Using Flask and ToscaWidgets

What is Flask?

Flask is yet another web framework for Python. It is based on Werkzeug for its controller functions, Jinja2 for the views and can use the excellent SQLAlchemy ORM for the M part of the MVC triad. Above all, Flask is a lightweight and fun framework to develop with. Although it lacks some of the features that Django and Turbogears have out-of-the-box, the real beauty of Flask is that it is simple to add extra functionality as needed, without it interfering with pre-existing components.

Introducing ToscaWidgets

ToscaWidgets is a Python widget library and has many easy to use widgets such as date pickers, captcha forms and trees. The most compelling reason to use ToscaWidgets for me is to use the Sprox form generation library. As a web developer you find half your working life is spent with forms, it's such an important yet undeniably tedious part of the job. Sprox introspects your SQLAlchemy models and automatically generates your forms, making a dreary task much more manageable.

A Load of CRUD

The Flask website has a great quickstart tutorial you can use to get you started so I am not going to re-invent the wheel here, it also details the Flask-SQLALchemy package which you will obviously need here.

Before we can use Sprox we need to wrap the ToscaWidgets middleware into the main Flask WSGI app. In the file where you define your Flask application do something similar to this:

from tw.api import make_middleware

# Create the WSGI app
app = Flask(__name__)
app.debug = True

# Insert ToscaWidgets Middleware
app.wsgi_app = make_middleware(app.wsgi_app, stack_registry=True)

There are several configuration options here for the middleware detailed on the ToscaWidgets website but the important one here is stack_registry=True without this, the middleware will not stack correctly and ToscaWidgets will not run correctly.

Once this runs correctly we can start using Sprox to rapidly knock together some CRUD (Create Records, Update, Delete) functionality for our little app.

I created a separate module for each of my models and I am going to use an image model for this example so I can show you how easy it is to upload images into a database using this setup.

The forms module:

from myapp.models.items import Image
from sprox.dojo.formbase import DojoEditableForm,  DojoAddRecordForm
from flask import url_for
from myapp.database import db_session
from formencode import validators
from sprox.tablebase import TableBase
from sprox.fillerbase import TableFiller,EditFormFiller
import formencode

class UniqueSlug(formencode.FancyValidator):

    model = None

    def _to_python(self, value, state):
        if not self.model is None:
            values = db_session.query(self.model.slug).first()
            if values is None:
                return value
        else:
            raise NameError('model is not defined')
        if unicode(value) in values:
            raise formencode.Invalid('That slug already exists', value, state)
        return value

class FormDefaults(DojoAddRecordForm):
    __omit_fields__ = ['id', 'discriminator', 'create_date', 'update_date']

class ImageForm(FormDefaults):
    __model__ = Image
    slug = UniqueSlug(model=Image)
    image = validators.FieldStorageUploadConverter(if_missing=None)

class EditImageForm(DojoEditableForm):
    __model__ = Image
    __omit_fields__ = ['id', 'slug']
    image = validators.FieldStorageUploadConverter(if_missing=None)

class EditImageFormFiller(EditFormFiller):
    __model__ = Image

class ImageList(TableBase):
    __model__ = Image
    __xml_fields__ = ['image']
    __omit_fields__ = ['alt_text']

class ImageListFiller(TableFiller):
    __model__ = Image

    def __actions__(self, obj):
        """Override this function to define how action links should be displayed for the given record."""
        primary_fields = self.__provider__.get_primary_fields(self.__entity__)
        pklist = '/'.join(map(lambda x: str(getattr(obj, x)), primary_fields))
        value = '<div><div><a class="edit_link" href="'+str(url_for('edit_image', id=pklist))+'" style="text-decoration:none">edit</a>'\
              '</div><div>'\
              '<form method="POST" action="'+str(url_for('delete_image', id=pklist))+'" class="button-to">'\
            '<input type="hidden" name="_method" value="DELETE" />'\
            '<input class="delete-button" onclick="return confirm(\'Are you sure?\');" value="delete" type="submit" '\
            'style="background-color: transparent; float:left; border:0; color: #286571; display: inline; margin: 0; padding: 0;"/>'\
        '</form>'\
        '</div></div>'
        return value

    def image(self, obj):
        image = ', '.join(['<img width="100px" height="75px" src="/image/'+str(obj.slug)+'" alt="'+str(obj.alt_text)+'" />'])
        return image.join(('<div>', '</div>'))

create_image_form = ImageForm(db_session)
edit_image_form = EditImageForm(db_session)
edit_image_form_filler = EditImageFormFiller(db_session)
image_list = ImageList(db_session)
image_list_filler = ImageListFiller(db_session) 

This should be fairly self explanatory to anybody who has read the sometimes sparse Sprox documentation. Note here that I have overridden the __actions__() method, this method creates the edit/delete links in the table. Sprox is used mainly with TurboGears so the default URLs work with TurboGears but need modifying to work with Flask. Also I have created a custom validator to ensure that the slug is always unique. This functionality probably exists out-of-the-box but it serves a purpose here as a demonstration of how to do it.

Also note that we are setting the flag if_missing=None on the image field validators. This is because in the controller we are passing request.form to the form's validate method, file uploads are stored in the request.files structure. We will always get a validation error about the image filed being empty. If you need to validate file uploads, and I strongly recommend you do, then do it separately from Sprox and Formencode using the Flask-Upload package.

Next create some controllers that handle listing, creating, editing and deleting.

### Image CRUD controllers
@admin.route('/images')
def list_image():
    value = image_list_filler.get_value()
    return render_template('admin_list.html', list=image_list, name='Images', value=value, create_url=url_for('create_image'))

@admin.route('/edit_image/<id>', methods=['GET', 'POST'])
def edit_image(id):

    if request.method == 'POST':
        try:
            edit_image_form.validate(request.form)
        except Invalid as error:
            flash(error.msg)
            return redirect(url_for('create_image'))
        file = request.files['image']
        image = db_session.query(Image).get(id)
        image.alt_text = request.form['alt_text']

        if not file.filename is None:
            image.image = file.read()

        db_session.commit()
        return redirect(url_for('list_image'))

    value = edit_image_form_filler.get_value(values={'id': id})
    return render_template('admin_edit.html', form=edit_image_form, name='Edit Image', value=value)

@admin.route('/create_image', methods=['GET', 'POST'])
def create_image():
    
    if request.method == 'POST':
        try:
            create_image_form.validate(request.form)
        except Invalid as error:
            flash(error.msg)
            return redirect(url_for('create_image'))
        file = request.files['image']
        image = Image(slug=request.form['slug'], alt_text=request.form['alt_text'], image=file.read())
        db_session.add(image)
        db_session.commit()
        flash('Image saved')
        return redirect(url_for('list_image'))
    return render_template('admin_form.html', form=create_image_form, name='Create Image')

@admin.route('/delete_image/<id>', methods=['GET', 'POST'])
def delete_image(id):
    image = db_session.query(Image).get(id)

    if image is None:
        flash('Image not found')
    else:
        db_session.delete(image)
        db_session.commit()
        flash('Image deleted')

    return redirect(url_for('list_image'))

Just create some templates that display the forms and you are done!

Read more »

Improving the Cart

Updating the Model

In my second update I will make some improvements to the shopping cart model, notably allowing multiple products to be added to the cart and fixing the remove() method so it actually works.

class Cart(object):
    """
    Shopping cart stores products in the session and persists to a database
    """
    # Stores the items
    __items = dict()

    # Constructor
    def __init__(self):
        # Get cartID from the session if it exists
        cartID = session.get('cartID', None)
        # If cartID is set then load an instance of the database model
        # populated with existing items
        if cartID:
            cart = CartModel()
            self.cartModel = cart.get(cartID)
        # If not then create a new instance and save self to the database
        # and set cartID in the session
        else:
            cart = CartModel(object=self, status=0)
            DBSession.add(cart)
            transaction.commit()
            session['cartID'] = cart.id
            self.cartModel = cart

    # Main methods
    def add(self, product, qty):
        # Save dictionary of product attributes
        items = dict({'id': product.id,
          'qty': qty,
          'description': product.name,
          'price': product.price})
        self.set_items(items)
        self.save()

    def remove(self, productID):
        # Remove an item
        items = self.get_items()
        if int(productID) in items:
            del items[int(productID)]

        self.save()

    def clear(self):
        # Clear items from session, but NOT database as we will need
        # a record of transactions
        session.delete()

    def save(self):
        # Persist changes
        session.save()
        self.cartModel.object = self
        transaction.commit()

    # Getters/Setters
    def set_items(self, items):
        self.__items[items['id']] = items
        session['items'] = self.__items

    def get_items(self):
        if session.get('items', None):
            self.__items = session['items']
        return self.__items

All I have done here is change the set_items() method to properly append the items dict to the self.__items dict. The remove method was puzzling to me coming from a background in PHP until I remembered that the keys in a Python dictionary can be any type (except another dictionary) and that the productID passed into it from the controller was a string and the key is an integer. In PHP this would not matter, but Python was throwing a KeyError, but explicitly casting productID to an integer solved this little issue.

Now the cart properly handles adding and removing products from it.

The Controller

I will briefly go over the controller functions in root.py. I think in my finished application I will move all cart related actions into a separate controller but for now when I am mainly just debugging my models I shall leave them in the root controller.

@expose()
    def add_to_cart(self, productID, quantity):
        # Get cart instance
        cart = self.get_cart()
        # create a new product object
        product = Product()
        # add product to cart
        cart.add(product.get(productID), quantity)
        # Feedback and redirect
        flash('Product added')
        redirect('/')

    @expose()
    def remove_from_cart(self, productID):
        # Get cart
        cart = self.get_cart()
        # Remove productID from cart
        cart.remove(productID)
        # Feedback and redirect
        flash('Product removed')
        redirect('/')

These are fairly self-explanatory, we simply call the instance of the cart add/remove and then set a flash message and redirect back to the home page.

In my next update I will bring in an address model and start creating some checkout logic so that the cart can do something useful.

Read more »

Building a Shopping Cart using TurboGears

Overview

This is my first foray into the world of Python and I am trying to build an e-commerce website. I have chosen TurboGears as my framework after messing about with Django and Web2Py it fits my needs well as a professional PHP developer with a lot of experience coding in Zend. The primary reason is that is uses the excellent SQLAlchemy ORM and unlike Django, follows a more traditional MVC design pattern.

After hours of fruitless searching for a decent tutorial that covers building a shopping cart I decided to take my experience as a Zend developer and try to do it in Python, like I would do it in PHP. I am documenting my progress to aid others, and also for the developer community to help me with my efforts. So here goes…

After setting up my models and tables, which I won't go into too much here, the main effort is building a shopping cart class that performs the following functions:

• Add products to the cart
• Remove products from the cart
• Persist changes to both the web session and also to a database 

Cart Database Model

class CartModel(DeclarativeBase, Defaults):
  # Table name
  __tablename__ = 'carts'
  # Columns
  id = Column('id', Integer, primary_key=True, autoincrement=True)
  user_id = Column('user_id', Integer, ForeignKey(User.user_id))
  object = Column('object', PickleType)
  create_date = Column('create_date', DateTime, default=func.now())
  update_date = Column('update_date', DateTime, onupdate=func.now())
  status = Column('status', String)

  def get(self, cartID):
    query = DBSession.query(CartModel).get(cartID)
    return query

# Set relationships
CartModel.user = relation(User, primaryjoin=CartModel.user_id==User.user_id)

Basically this class saves a serialised copy of the cart object (which I will detail next) in the object field. SQLALchemy is great for this, the PickleType field will automatically serialise/unserialise Python objects. Doing this in PHP would be a pain in the backside. We also relate carts to users and define some metadata about the cart.


Shopping Cart Model

class Cart(object):
  __items = dict()

  def __init__(self):
    cartID = session.get('cartID', None)

    if cartID:
      cart = CartModel()
      self.cartModel = cart.get(cartID)
    else:
      cart = CartModel(object=self, status=0)
      DBSession.add(cart)
      transaction.commit()
      session['cartID'] = cart.id
      self.cartModel = cart

  def add(self, product, qty):
    items = dict(id=product.id, qty=qty, description=product.name, price=product.price)
    self.set_items(items)
    self.save()

  def remove(self, product):
    items = self.get_items()
    del items[product.id]
    self.set_items(items)
    self.save()

  def clear(self):
    session.delete()

  def save(self):
    session.save()
    self.cartModel.object = self
    transaction.commit()

  def set_items(self, items):
    self.__items[items['id']] = items['qty']
    session['items'] = items

  def get_items(self):
    if session.get('items', None):
      self.__items = session['items']
    return self.__items  

Here I have chosen to represent my cart items as a dict, self.__items the add and remove methods basically take attributes from the Product object that's passed to it and saves them to this dict.

The __init__() method is responsible for ensuring that the database and the session are synchronised. The cartID which is the primary key is saved in the session and if it exists then we create a CartModel object from this primary key, if not we create a new row and save the primary key in the session so subsequent actions all act on the same database row.

This is really in it infancy at the moment and I am not really sure if I can refactor certain functions to be more pythonic.

Your comments and improvements are more than welcome.

Read more »