Feed Entries and New Module Layout¶

In this section we will be adding a new content type for feed entries, corresponding to the content that is stored in a feed folder. We use this step to re-organize our package into the general structure seen in repoze.bfg applications:

feedstool/models/ has a module per “type”, along with modules that support the types.
feedstool/views/ also has a module per type, along with the templates subdirectory.
Each directory has a configure.zcml included into the higher-level ZCML.

Note

Starting with this tutorial, working (and versioned for each section) subtrees of the feedstool module source are included in the repository with the documentation sources for the tutorials.

Tasks¶

Re-layout the models, including tests and a new content type (feed entry.)
Re-layout the views, including tests.
Update corollary files.
Add WSGI middleware to nicely format error pages.

Re-organized `feedstool/models` Directory¶

To better reflect the naming conventions in KARL3, we start by re-organizing the models. Instead of a single models.py and single tests.py, we make files for each content type and a tests subdirectory for each type. We also make a models/configure.zcml (empty for now) that manages the models, plus a models/interfaces.py that has all the interfaces.

Visually, the directory hierarchy will look like this:

feedstool/models/
  __init__.py (empty file)
  configure.zcml
  feed.py
  feedentry.py
  interfaces.py
  site.py
  tests/
    __init__.py (empty file)
    test_feed.py
    test_site.py

The models/configure.zcml is just a placeholder:

<configure xmlns="http://namespaces.repoze.org/bfg">

  <!-- Nothing to configure in models yet. -->

</configure>

The models/feed.py is terse, but includes some more information about a feed for later use by the synchronization script:

from repoze.folder import Folder
from zope.interface import implements

from feedstool.models.interfaces import IFeed

class Feed(Folder):
    implements(IFeed)
    persist_entries = True
    fetch_enclosures = False
    subtitle = None
    etag = None
    last_modified = None
    id = None
    updated = None

    def __init__(self, title, url):

        super(Feed, self).__init__()
        self.title = unicode(title)
        self.url = unicode(url)

Line 8-18. These are class attributes that serve as default values for Feed instances. We store them on the class to avoid unnecessary bloat in the database. More specifically, this information is provided with the feed XML is retrieved, not when the feed is added in the FeedsContainer.

A feed stores one or more feed entries (per the Atom specification). Thus we introduce the idea of a feedentry.py module:

from repoze.folder import Folder
from zope.interface import implements

from feedstool.models.interfaces import IFeedEntry

class FeedEntry(Folder):
    implements(IFeedEntry)
    title = None
    link = None
    id = None
    published = None
    updated = None
    summary = None
    content = None
    
    def __init__(self, title, link):

        super(FeedEntry, self).__init__()
        self.title = title
        self.link = link

This also makes liberal use of class attributes. In theory, all required feed entry element in the Atom specification would be in the constructor. All other data could be optionally added after the instance was made.

Note

Our philosophy is that the classes that are baked into the pickle should be as simple as possible. They certainly shouldn’t be expected to pick apart request data. For the most part, we try to use adaptation to inject extra methods, computed properties, and other behavior (in the interest of keeping our model classes simple.)

The models/interfaces.py file is very simple:

from repoze.folder.interfaces import IFolder

class IFeedsContainer(IFolder):
    pass

class IFeed(IFolder):
    pass

class IFeedEntry(IFolder):
    pass

The most complicated is the models/site.py:

from repoze.folder import Folder
from zope.interface import implements

from feedstool.models.interfaces import IFeedsContainer

class Site(Folder):
    implements(IFeedsContainer)

def appmaker(zodb_root):
    if not 'app_root' in zodb_root:
        app_root = Site()
        zodb_root['app_root'] = app_root
        import transaction
        transaction.commit()
    return zodb_root['app_root']

Lines 9-15. We moved the appmaker function out of feedstool/run.py. Its job is to make an instance of the root, which is a Site, so models/site.py is a logical place to put it.

The two modules holding the tests are relatively unchanged. First, models/tests/test_feed.py:

import unittest

from zope.testing.cleanup import cleanUp

class FeedTests(unittest.TestCase):

    def setUp(self):
        cleanUp()

    def tearDown(self):
        cleanUp()

    def _getTargetClass(self):
        from feedstool.models.feed import Feed
        return Feed

    def _makeOne(self, title=u'title', url=u'url',):
        tc = self._getTargetClass()
        return tc(title, url)

    def test_class_conforms_to_IFeed(self):
        from zope.interface.verify import verifyClass
        from feedstool.models.interfaces import IFeed
        verifyClass(IFeed, self._getTargetClass())

    def test_instance_conforms_to_IFeed(self):
        from zope.interface.verify import verifyObject
        from feedstool.models.interfaces import IFeed
        verifyObject(IFeed, self._makeOne())

And models/tests/test_site.py:

import unittest

from zope.testing.cleanup import cleanUp

class SiteTests(unittest.TestCase):

    def setUp(self):
        cleanUp()

    def tearDown(self):
        cleanUp()

    def _getTargetClass(self):
        from feedstool.models.site import Site
        return Site

    def _makeOne(self):
        tc = self._getTargetClass()
        return tc()

    def test_class_conforms_to_IFeedsContainer(self):
        from zope.interface.verify import verifyClass
        from feedstool.models.interfaces import IFeedsContainer
        verifyClass(IFeedsContainer, self._getTargetClass())

    def test_instance_conforms_to_IFeedsContainer(self):
        from zope.interface.verify import verifyObject
        from feedstool.models.interfaces import IFeedsContainer
        verifyObject(IFeedsContainer, self._makeOne())

Because we moved the appmaker bootstrapping function to models/site.py, we have to update the startup module in feedstool/run.py:

from repoze.bfg.router import make_app
from repoze.zodbconn.finder import PersistentApplicationFinder

def app(global_config, **kw):
    """ This function returns a repoze.bfg.router.Router object.
    
    It is usually called by the PasteDeploy framework during ``paster serve``.
    """
    # paster app config callback
    import feedstool
    from feedstool.models.site import appmaker
    zodb_uri = kw.get('zodb_uri')
    if zodb_uri is None:
        raise ValueError("No 'zodb_uri' in application configuration.")

    get_root = PersistentApplicationFinder(zodb_uri, appmaker)
    return make_app(get_root, feedstool, options=kw)

We import the appmaker function from feedstool.models.site.

Using `nose` For Test Suites¶

We now have our tests spread across multiple directories. KARL3 uses the nose package for improved unit test running, so let’s install it:

$ easy_install nose

Now edit FeedsTool/setup.py and say that this package uses nose as its test runner. Do this by replacing the existing test_suite line with the following:

test_suite="nose.collector",

KARL3 uses nose as its test suite runner.

Re-organized `feedstool/views` Directory¶

The structure of the feedstool/views directory follows a similar pattern:

feedstool/views/
  __init__.py (empty file)
  configure.zcml
  feed.py
  site.py
  templates/
    add_feed.pt
    layout.pt
    list_feeds.pt
    show_feed.pt
    static/
      (static resources, same as before)
  tests/
    __init__.py (empty file)
    test_feed.py
    test_site.py

Because we put the interfaces into feedstool/models/interfaces.py, the views/configure.zcml has changes on each directive:

<configure xmlns="http://namespaces.repoze.org/bfg">

  <!-- this must be included for the view declarations to work -->
  <include package="repoze.bfg.includes" />

  <view
     for="feedstool.models.interfaces.IFeedsContainer"
     view=".site.list_feeds_view"
     />

  <view
     for="feedstool.models.interfaces.IFeedsContainer"
     view=".site.static_view"
     name="static"
     />

  <view
     for="feedstool.models.interfaces.IFeedsContainer"
     view=".site.add_feed_view"
     name="add_feed.html"
     />

  <view
     for="feedstool.models.interfaces.IFeed"
     view=".feed.show_feed_view"
     />

</configure>

The views/feed.py module holds the (currently one) view for showing the contents of a feed:

from repoze.bfg.chameleon_zpt import render_template_to_response
from repoze.bfg.chameleon_zpt import get_template

def show_feed_view(context, request):
    layout = get_template('templates/layout.pt')

    return render_template_to_response(
        'templates/show_feed.pt',
        request=request,
        layout=layout,
        page_title = 'Show Feed ' + context.title,
        feed_url=context.url)

The views/site.py gets most of the code:

from repoze.bfg.chameleon_zpt import render_template_to_response
from repoze.bfg.chameleon_zpt import get_template

from webob.exc import HTTPFound
from repoze.bfg.url import model_url
from repoze.bfg.view import static

from feedstool.models.feed import Feed

static_view = static('templates/static')

def list_feeds_view(context, request):
    layout = get_template('templates/layout.pt')

    feeds = []
    for feed in context.values():
        feeds.append({
                'title': feed.title,
                'model_url': model_url(feed, request),
                })

    return render_template_to_response(
        'templates/list_feeds.pt',
        request=request,
        layout=layout,
        page_title="List Feeds",
        feeds=feeds)

def add_feed_view(context, request):

    layout = get_template('templates/layout.pt')

    is_submitted = request.POST.get('form.submitted', False)
    if is_submitted is not False:
        title = request.POST['title']
        url = request.POST['url']
        feed = Feed(title, url)

        name = title.replace(' ', '-').lower()
        context[name] = feed

        return HTTPFound(location=model_url(feed, request))

    # Form was not submitted, return the page

    return render_template_to_response(
        'templates/add_feed.pt',
        request=request,
        layout=layout,
        page_title='Add Feed',
        is_submitted=is_submitted)

The views/templates/add_feed.pt template is unchanged:

<div xmlns="http://www.w3.org/1999/xhtml" 
     xmlns:tal="http://xml.zope.org/namespaces/tal"
     xmlns:metal="http://xml.zope.org/namespaces/metal"
     metal:use-macro="layout.macros['master']">

  <div metal:fill-slot="content">

    <form method="post" action="${request.url}">
      <div>
	<label>Title</label> <input name="title"/>
      </div>
      <div>
	<label>URL</label> <input name="url"/>
      </div>
      <div>
	<input type="submit" name="form.submitted"/>
      </div>
    </form>

  </div>

</div>

Neither is the views/templates/layout.pt changed:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" 
	  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:tal="http://xml.zope.org/namespaces/tal"
      xmlns:metal="http://xml.zope.org/namespaces/metal" 
      metal:define-macro="master">
  <head>
    <title>${page_title} Feed</title>
    <link href="${request.application_url}/static/default.css" 
	  rel="stylesheet" type="text/css" />
  </head>
  <body>
    <!-- start header -->
    <div id="logo">
      <h2><code>FeedsTool</code>, a <code>repoze.bfg</code> application</h2>
    </div>
    <div id="header">
      <div id="menu">
      </div>
    </div>
    <!-- end header -->
    <div id="wrapper">
      <!-- start page -->
      <div id="page">

	<!-- start content -->
	<div class="post">
	  <h1 class="title">${page_title}</h1>
	  <div metal:define-slot="content" id="content"/>
	</div>
	<!-- end content -->

	<!-- start sidebar -->
	<div id="sidebar">
	  <ul>
	    <li>
	      <a href="/">Home</a>
	    </li>
	    <li>
	      <a href="/add_feed.html">Add Feed</a>
	    </li>
	  </ul>
	</div>
	<!-- end sidebar -->
	<div style="clear: both;">&nbsp;</div>
      </div>
    </div>
    <!-- end page -->
    <!-- start footer -->
    <div id="footer">
      <p id="legal">( c ) 2008. All Rights Reserved. Template design
	by <a href="http://www.freecsstemplates.org/">Free CSS
	  Templates</a>.</p>
    </div>
    <!-- end footer -->
  </body>
</html>

Nor is the views/templates/list_feeds.pt template:

<div xmlns="http://www.w3.org/1999/xhtml" 
     xmlns:tal="http://xml.zope.org/namespaces/tal"
     xmlns:metal="http://xml.zope.org/namespaces/metal"
     metal:use-macro="layout.macros['master']">

  <div metal:fill-slot="content">

    <p>Welcome to the FeedsTool.  Below is a list of current
    feeds.</p>

    <ul>
      <li tal:repeat="feed feeds">
	<a href="${feed['model_url']}">${feed['title']}</a>
      </li>
    </ul>

  </div>

</div>

Finally, the views/templates/show_feed.pt is also unchanged:

<div xmlns="http://www.w3.org/1999/xhtml" 
     xmlns:tal="http://xml.zope.org/namespaces/tal"
     xmlns:metal="http://xml.zope.org/namespaces/metal"
     metal:use-macro="layout.macros['master']">

  <div metal:fill-slot="content">

    <p>This is a feed
      at <a href="${feed_url}"><code>${feed_url}</code></a>.</p>

  </div>

</div>

We reorganized the view tests a bit, giving each unit test class a name that corresponds to the module. (In KARL3, each unit test corresponds to a view.) First, views/tests/test_feed.py:

import unittest

from zope.testing.cleanup import cleanUp
from repoze.bfg import testing

class FeedViewTests(unittest.TestCase):

    """ These tests are unit tests for the view.  They test the
    functionality of *only* the view.  They register and use dummy
    implementations of repoze.bfg functionality to allow you to avoid
    testing 'too much'"""

    def setUp(self):
        """ cleanUp() is required to clear out the application registry
        between tests (done in setUp for good measure too)
        """
        cleanUp()
        
    def tearDown(self):
        """ cleanUp() is required to clear out the application registry
        between tests
        """
        cleanUp()

    def test_show_feed_view(self):
        from feedstool.views.feed import show_feed_view
        context = testing.DummyModel()
        context.title = "This Feed"
        context.url = "This URL"
        request = testing.DummyRequest()
        renderer = testing.registerDummyRenderer('templates/show_feed.pt')
        response = show_feed_view(context, request)
        renderer.assert_(page_title='Show Feed This Feed')
        renderer.assert_(feed_url='This URL')



class ViewIntegrationTests(unittest.TestCase):
    """ These tests are integration tests for the view.  These test
    the functionality the view *and* its integration with the rest of
    the repoze.bfg framework.  They cause the entire environment to be
    set up and torn down as if your application was running 'for
    real'.  This is a heavy-hammer way of making sure that your tests
    have enough context to run properly, and it tests your view's
    integration with the rest of BFG.  You should not use this style
    of test to perform 'true' unit testing as tests will run faster
    and will be easier to write if you use the testing facilities
    provided by bfg and only the registrations you need, as in the
    above ViewTests.
    """
    def setUp(self):
        """ This sets up the application registry with the
        registrations your application declares in its configure.zcml
        (including dependent registrations for repoze.bfg itself).
        """
        cleanUp()
        import feedstool
        import zope.configuration.xmlconfig
        zope.configuration.xmlconfig.file('configure.zcml',
                                          package=feedstool)

    def tearDown(self):
        """ Clear out the application registry """
        cleanUp()

Next, views/tests/test_site.py:

import unittest

from zope.testing.cleanup import cleanUp
from repoze.bfg import testing

class SiteViewTests(unittest.TestCase):

    """ These tests are unit tests for the view.  They test the
    functionality of *only* the view.  They register and use dummy
    implementations of repoze.bfg functionality to allow you to avoid
    testing 'too much'"""

    def setUp(self):
        """ cleanUp() is required to clear out the application registry
        between tests (done in setUp for good measure too)
        """
        cleanUp()
        
    def tearDown(self):
        """ cleanUp() is required to clear out the application registry
        between tests
        """
        cleanUp()

    def test_list_feeds_view(self):
        from feedstool.views.site import list_feeds_view
        context = testing.DummyModel()
        request = testing.DummyRequest()
        renderer = testing.registerDummyRenderer('templates/list_feeds.pt')
        response = list_feeds_view(context, request)
        renderer.assert_(page_title='List Feeds')

    def test_add_feed_view(self):
        from feedstool.views.site import add_feed_view
        context = testing.DummyModel()
        request = testing.DummyRequest()
        renderer = testing.registerDummyRenderer('templates/add_feed.pt')
        response = add_feed_view(context, request)
        renderer.assert_(page_title='Add Feed')

    def test_add_feed_notsubmitted(self):
        from feedstool.views.site import add_feed_view
        context = testing.DummyModel()
        request = testing.DummyRequest()
        renderer = testing.registerDummyRenderer(
            'templates/add_feed.pt')
        response = add_feed_view(context, request)
        self.failIf(renderer.is_submitted)

    def test_add_feed_submitted_valid(self):
        from feedstool.views.site import add_feed_view
        context = testing.DummyModel()
        request = testing.DummyRequest(
            params={
                'form.submitted':True,
                'title':'Some Title',
                'url': 'someurl',
                }
            )
        renderer = testing.registerDummyRenderer(
            'templates/add_feed.pt')
        response = add_feed_view(context, request)
        self.assertEqual(response.location, 
                         'http://example.com/some-title/')


class SiteViewIntegrationTests(unittest.TestCase):
    """ These tests are integration tests for the view.  These test
    the functionality the view *and* its integration with the rest of
    the repoze.bfg framework.  They cause the entire environment to be
    set up and torn down as if your application was running 'for
    real'.  This is a heavy-hammer way of making sure that your tests
    have enough context to run properly, and it tests your view's
    integration with the rest of BFG.  You should not use this style
    of test to perform 'true' unit testing as tests will run faster
    and will be easier to write if you use the testing facilities
    provided by bfg and only the registrations you need, as in the
    above ViewTests.
    """
    def setUp(self):
        """ This sets up the application registry with the
        registrations your application declares in its configure.zcml
        (including dependent registrations for repoze.bfg itself).
        """
        cleanUp()
        import feedstool
        import zope.configuration.xmlconfig
        zope.configuration.xmlconfig.file('configure.zcml',
                                          package=feedstool)

    def tearDown(self):
        """ Clear out the application registry """
        cleanUp()

    def test_list_feeds_view(self):
        from feedstool.views.site import list_feeds_view
        context = testing.DummyModel()
        request = testing.DummyRequest()
        result = list_feeds_view(context, request)
        self.assertEqual(result.status, '200 OK')
        body = result.app_iter[0]
        self.failUnless('Welcome to' in body)
        self.assertEqual(len(result.headerlist), 2)
        self.assertEqual(result.headerlist[0],
                         ('content-type', 'text/html; charset=UTF-8'))
        self.assertEqual(result.headerlist[1], ('Content-Length',
                                                str(len(body))))

Restarting Your Server¶

As before, remember to delete your Data.* files before running paster server FeedsTool.ini --reload to start the server. Why? Because we changed the location of the class definition for the root object.

Better Error Reporting¶

In the case that you have caused a programming error so far, you probably saw in your browser a Internal Server Error page, forcing you to find the window where the server was running to see the traceback.

Wouldn’t it be great to have nicely-formatted tracebacks in the browser? Paste has such a thing. Edit FeedsTool/FeedsTool.ini and add the egg:Paste#cgitb shown below:

[DEFAULT]
debug = true

[app:zodb]
use = egg:FeedsTool#app
reload_templates = true
debug_authorization = false
debug_notfound = false
zodb_uri = file://%(here)s/Data.fs?connection_cache_size=20000

[pipeline:main]
pipeline =
    egg:Paste#cgitb
    egg:repoze.zodbconn#closer
    egg:repoze.tm#tm
    zodb

[server:main]
use = egg:Paste#http
host = 0.0.0.0
port = 6543

Next, go put an error in some view. For example, put a random x in views/site.py:

def list_feeds_view(context, request):
    layout = get_template('templates/layout.pt')
    x
    feeds = []
    for feed in context.values():
        feeds.append({
                'title': feed.title,
                'model_url': model_url(feed, request),
                })

    return render_template_to_response(
        'templates/list_feeds.pt',
        request=request,
        layout=layout,
        page_title="List Feeds",
        feeds=feeds)

Now when you go to http://localhost:6543/ you should see a nicer error:

Note

Perhaps you don’t want your customers to see such an error message. Fortunately there are ways to address that, either in middleware or the external server.

Perhaps, though, you want even more geeky message. For example, whouldn’t it be nice to have a clickable “debugger” that let you expand the stacks in the traceback? That, also, is possible, using different middleware than Paste#cgitb.

Side Note on ZCML¶

Other frameworks, particularly “mega-frameworks” that use magic to do a bunch of work it thinks you are asking for, eschew the idea of configuration. Instead, they tout the ability to use “convention”, namely statements right in the Python code the developer is writing, to wire up the application.

Stated differently, why can’t we eliminate the ugly XML and extra work of ZCML?

We can do that in repoze.bfg, which supports Grok-style Python decorators as a replacement for ZCML configuration.

Why aren’t we using that in this tutorial, then? Because KARL3 doesn’t use it.

Why doesn’t KARL3 use it? Because of a great feature of ZCML that is not only not possible in “convention”, but simply counter-intuitive. With KARL3, you can write new code that overrides existing code, from the outside, without touching the existing code. Want to make your own show_feeds.pt template, or customize the view handling of just the add_feed_view function? With KARL3, you can write a new Python package, get it registered in the ZCML, and modify the core software.

That’s a powerful feature, particular for the “pilots” model of KARL.

Feed Entries and New Module Layout¶

Tasks¶

Re-organized `feedstool/models` Directory¶

Using `nose` For Test Suites¶

Re-organized `feedstool/views` Directory¶

Restarting Your Server¶

Better Error Reporting¶

Side Note on ZCML¶

Table Of Contents

Previous topic

Next topic

This Page

Navigation

Feed Entries and New Module Layout¶

Tasks¶

Re-organized feedstool/models Directory¶

Using nose For Test Suites¶

Re-organized feedstool/views Directory¶

Restarting Your Server¶

Better Error Reporting¶

Side Note on ZCML¶

Table Of Contents

Previous topic

Next topic

This Page

Quick search

Navigation

Re-organized `feedstool/models` Directory¶

Using `nose` For Test Suites¶

Re-organized `feedstool/views` Directory¶