Navigation

Cataloging

Getting Setup

$ easy_install -i http://dist.repoze.org/lemonade/dev/simple repoze.catalog

Cataloging Support in Models

Add a catalog to the root object by modifying models/site.py:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
from repoze.folder import Folder
from zope.interface import implements

from feedstool.models.interfaces import IFeedsContainer

from repoze.catalog.catalog import Catalog
from repoze.catalog.indexes.text import CatalogTextIndex
from repoze.catalog.indexes.keyword import CatalogKeywordIndex
from repoze.catalog.document import DocumentMap
from zope.interface.declarations import Declaration
from zope.interface import providedBy


def get_interfaces(object, default):
    # we unwind all derived and immediate interfaces using spec.flattened()
    # (providedBy would just give us the immediate interfaces)
    provided_by = list(providedBy(object))
    spec = Declaration(provided_by)
    ifaces = list(spec.flattened())
    return ifaces

def get_textrepr(object, default):
    return getattr(object, 'title', default)


class Site(Folder):
    implements(IFeedsContainer)

    def __init__(self):
        super(Site, self).__init__()
        self.catalog = sc = Catalog()
        sc['interfaces'] = CatalogKeywordIndex(get_interfaces)
        sc['texts'] = CatalogTextIndex(get_textrepr)
        sc.document_map = DocumentMap()

def appmaker(zodb_root):
    if not 'app_root' in zodb_root:
        app_root = Site()
        zodb_root['app_root'] = app_root
        import transaction
        transaction.commit()
    return zodb_root['app_root']
  • More imports
  • Two functions to get representations of an object during indexing
  • An __init__ in Site that sets up the catalog. This means blow away Data.fs and friends!
  • Explain document_map

Update tests to make sure a catalog (and document map) exists by editing models/tests/test_site.py:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
import unittest

from zope.testing.cleanup import cleanUp

class SiteTests(unittest.TestCase):

    def setUp(self):
        cleanUp()

    def tearDown(self):
        cleanUp()

    def _getTargetClass(self):
        from feedstool.models.site import Site
        return Site

    def _makeOne(self):
        tc = self._getTargetClass()
        return tc()

    def test_class_conforms_to_IFeedsContainer(self):
        from zope.interface.verify import verifyClass
        from feedstool.models.interfaces import IFeedsContainer
        verifyClass(IFeedsContainer, self._getTargetClass())

    def test_instance_conforms_to_IFeedsContainer(self):
        from zope.interface.verify import verifyObject
        from feedstool.models.interfaces import IFeedsContainer
        verifyObject(IFeedsContainer, self._makeOne())

    def test_verify_catalog_present(self):
        from zope.interface.verify import verifyObject
        from repoze.catalog.interfaces import ICatalog
        site = self._makeOne()
        self.failUnless(hasattr(site, 'catalog'))
        catalog = site.catalog
        verifyObject(ICatalog, catalog)
        self.failUnless(hasattr(catalog, 'document_map'))
  • New test for verifying the presence of a catalog and document_map

Now run the tests and see if you have Ran 11 tests in 0.7 sec.

Since we have a catalog in place, we can update models/subscribers.py to actually index on add events:

1
2
3
4
5
6
7
8
from repoze.bfg.traversal import find_root
from repoze.bfg.traversal import model_path

def index_content(object, event):
    catalog = getattr(find_root(object), 'catalog', None)
    path = model_path(object)
    docid = catalog.document_map.add(path)
    catalog.index_doc(docid, object)

Adding a Search Form to Views

We have a search engine, let’s put it to use. First, let’s have a box appear on every page by editing views/templates/layout.pt:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" 
	  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:tal="http://xml.zope.org/namespaces/tal"
      xmlns:metal="http://xml.zope.org/namespaces/metal" 
      metal:define-macro="master">
  <head>
    <title>${page_title} Feed</title>
    <link href="${request.application_url}/static/default.css" 
	  rel="stylesheet" type="text/css" />
  </head>
  <body>
    <!-- start header -->
    <div id="logo">
      <h2><code>FeedsTool</code>, a <code>repoze.bfg</code> application</h2>
    </div>
    <div id="header">
      <div id="menu">
      </div>
    </div>
    <!-- end header -->
    <div id="wrapper">
      <!-- start page -->
      <div id="page">

	<!-- start content -->
	<div class="post">
	  <h1 class="title">${page_title}</h1>
	  <div metal:define-slot="content" id="content"/>
	</div>
	<!-- end content -->

	<!-- start sidebar -->
	<div id="sidebar">
	  <ul>
	    <li>
	      <a href="/">Home</a>
	    </li>
	    <li>
	      <a href="/add_feed.html">Add Feed</a>
	    </li>
	    <li>
		<form method="get" action="/search.html">
		  <div><input name="searchterm"/></div>
		  <div><input type="submit" value="Search"/></div>
		</form>
	    </li>
	  </ul>
	</div>
	<!-- end sidebar -->
	<div style="clear: both;">&nbsp;</div>
      </div>
    </div>
    <!-- end page -->
    <!-- start footer -->
    <div id="footer">
      <p id="legal">( c ) 2008. All Rights Reserved. Template design
	by <a href="http://www.freecsstemplates.org/">Free CSS
	  Templates</a>.</p>
    </div>
    <!-- end footer -->
  </body>
</html>
  • We’re adding the <form> in the sidebar

This submits to a URL /search.html to show the search results, so we need a view in views/site.py:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
from repoze.bfg.chameleon_zpt import render_template_to_response
from repoze.bfg.chameleon_zpt import get_template

from webob.exc import HTTPFound
from repoze.bfg.url import model_url
from repoze.bfg.view import static

from repoze.lemonade.content import create_content
from feedstool.models.interfaces import IFeed

import itertools
from repoze.bfg.traversal import find_model
from repoze.bfg.traversal import find_root

static_view = static('templates/static')

def list_feeds_view(context, request):
    layout = get_template('templates/layout.pt')

    feeds = []
    for feed in context.values():
        feeds.append({
                'title': feed.title,
                'model_url': model_url(feed, request),
                })

    return render_template_to_response(
        'templates/list_feeds.pt',
        request=request,
        layout=layout,
        page_title="List Feeds",
        feeds=feeds)


def catalog_search(context, searchterm):

    catalog = getattr(find_root(context), 'catalog', False)
    if catalog is False:
        # We are testing from the unit test, so just return an empty
        # list. Long term we'd make a DummyCatalog in the unit test to
        # mimic the semantics of an actual catalog.
        return []
    batch_start = 0
    limit = 100
    query = {'texts': searchterm}
    num, iter = catalog.search(**query)

    info = []
    for docid in itertools.islice(iter, batch_start, limit):
        path = catalog.document_map.address_for_docid(docid)
        instance = find_model(context, path)

        info.append(instance)

    return info


def search_view(context, request):
    layout = get_template('templates/layout.pt')

    # Get the search results, pick them apart into a list of dicts
    searchterm = request.params.get('searchterm', False)

    results = []
    for result in catalog_search(context, searchterm):
        results.append({
                'title': result.title,
                'model_url': model_url(result, request),
                })
    
    return render_template_to_response(
        'templates/search.pt',
        request=request,
        layout=layout,
        page_title="Search Results for " + searchterm,
        results=results)

def add_feed_view(context, request):

    layout = get_template('templates/layout.pt')

    is_submitted = request.POST.get('form.submitted', False)
    if is_submitted is not False:
        title = request.POST['title']
        url = request.POST['url']
        feed = create_content(IFeed, title, url)

        name = title.replace(' ', '-').lower()
        context[name] = feed

        return HTTPFound(location=model_url(feed, request))

    # Form was not submitted, return the page

    return render_template_to_response(
        'templates/add_feed.pt',
        request=request,
        layout=layout,
        page_title='Add Feed',
        is_submitted=is_submitted)
  • New imports
  • We added a search_view but also a helper function catalog_search
  • The latter builds the query, issues the search, and “flattens” the result into boring Python data. This makes it easier to deal with in the templates. Plus, lots of times we want to modify the data for view purposes (e.g. date formatting).
  • We try to use iterators

A view is needed at templates/search.pt to display the search results:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
<div xmlns="http://www.w3.org/1999/xhtml" 
     xmlns:tal="http://xml.zope.org/namespaces/tal"
     xmlns:metal="http://xml.zope.org/namespaces/metal"
     metal:use-macro="layout.macros['master']">

  <div metal:fill-slot="content">

    <ul>
      <li tal:repeat="result results">
	<a href="result['model_url']">${result['title']}</a>
      </li>
    </ul>

  </div>

</div>

Wire up the view in views/configure.zcml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
<configure xmlns="http://namespaces.repoze.org/bfg">

  <!-- this must be included for the view declarations to work -->
  <include package="repoze.bfg.includes" />

  <view
     for="feedstool.models.interfaces.IFeedsContainer"
     view=".site.list_feeds_view"
     />

  <view
     for="feedstool.models.interfaces.IFeedsContainer"
     view=".site.static_view"
     name="static"
     />

  <view
     for="feedstool.models.interfaces.IFeedsContainer"
     view=".site.add_feed_view"
     name="add_feed.html"
     />

  <view
     for="feedstool.models.interfaces.IFeed"
     view=".feed.show_feed_view"
     />

  <view
     for="feedstool.models.interfaces.IFeedsContainer"
     view=".site.search_view"
     name="search.html"
     />

</configure>
  • Last directive at the end

Add another view test at SiteViewTests.test_search_view in views/test_site.py:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
import unittest

from zope.testing.cleanup import cleanUp
from repoze.bfg import testing

class SiteViewTests(unittest.TestCase):

    """ These tests are unit tests for the view.  They test the
    functionality of *only* the view.  They register and use dummy
    implementations of repoze.bfg functionality to allow you to avoid
    testing 'too much'"""

    def setUp(self):
        """ cleanUp() is required to clear out the application registry
        between tests (done in setUp for good measure too)
        """
        cleanUp()
        import feedstool
        import zope.configuration.xmlconfig
        zope.configuration.xmlconfig.file('configure.zcml',
                                          package=feedstool)
        
    def tearDown(self):
        """ cleanUp() is required to clear out the application registry
        between tests
        """
        cleanUp()

    def test_list_feeds_view(self):
        from feedstool.views.site import list_feeds_view
        context = testing.DummyModel()
        request = testing.DummyRequest()
        renderer = testing.registerDummyRenderer('templates/list_feeds.pt')
        response = list_feeds_view(context, request)
        renderer.assert_(page_title='List Feeds')

    def test_add_feed_view(self):
        from feedstool.views.site import add_feed_view
        context = testing.DummyModel()
        request = testing.DummyRequest()
        renderer = testing.registerDummyRenderer('templates/add_feed.pt')
        response = add_feed_view(context, request)
        renderer.assert_(page_title='Add Feed')

    def test_add_feed_notsubmitted(self):
        from feedstool.views.site import add_feed_view
        context = testing.DummyModel()
        request = testing.DummyRequest()
        renderer = testing.registerDummyRenderer(
            'templates/add_feed.pt')
        response = add_feed_view(context, request)
        self.failIf(renderer.is_submitted)

    def test_add_feed_submitted_valid(self):
        from feedstool.views.site import add_feed_view
        context = testing.DummyModel()
        request = testing.DummyRequest(
            params={
                'form.submitted':True,
                'title':'Some Title',
                'url': 'someurl',
                }
            )
        renderer = testing.registerDummyRenderer(
            'templates/add_feed.pt')
        response = add_feed_view(context, request)
        self.assertEqual(response.location, 
                         'http://example.com/some-title/')

    def test_search_view(self):
        from feedstool.views.site import search_view
        context = testing.DummyModel()
        request = testing.DummyRequest(
            params={
                'searchterm': 'someword',
                }
            )
        renderer = testing.registerDummyRenderer('templates/search.pt')
        response = search_view(context, request)
        renderer.assert_(page_title='Search Results for someword')



class SiteViewIntegrationTests(unittest.TestCase):
    """ These tests are integration tests for the view.  These test
    the functionality the view *and* its integration with the rest of
    the repoze.bfg framework.  They cause the entire environment to be
    set up and torn down as if your application was running 'for
    real'.  This is a heavy-hammer way of making sure that your tests
    have enough context to run properly, and it tests your view's
    integration with the rest of BFG.  You should not use this style
    of test to perform 'true' unit testing as tests will run faster
    and will be easier to write if you use the testing facilities
    provided by bfg and only the registrations you need, as in the
    above ViewTests.
    """
    def setUp(self):
        """ This sets up the application registry with the
        registrations your application declares in its configure.zcml
        (including dependent registrations for repoze.bfg itself).
        """
        cleanUp()
        import feedstool
        import zope.configuration.xmlconfig
        zope.configuration.xmlconfig.file('configure.zcml',
                                          package=feedstool)

    def tearDown(self):
        """ Clear out the application registry """
        cleanUp()

    def test_list_feeds_view(self):
        from feedstool.views.site import list_feeds_view
        context = testing.DummyModel()
        request = testing.DummyRequest()
        result = list_feeds_view(context, request)
        self.assertEqual(result.status, '200 OK')
        body = result.app_iter[0]
        self.failUnless('Welcome to' in body)
        self.assertEqual(len(result.headerlist), 2)
        self.assertEqual(result.headerlist[0],
                         ('content-type', 'text/html; charset=UTF-8'))
        self.assertEqual(result.headerlist[1], ('Content-Length',
                                                str(len(body))))

All 12 tests should now pass. Delete your Data.fs and restart the server:

.. code-block:: bash
$ rm Data.* $ paster serve FeedsTool.ini –reload

Add a new Feed with a title of The first feed. Then do a search for first and a search for second.

Table Of Contents

Previous topic

Formal Content Types with repoze.lemonade

Next topic

Processing Feeds

This Page

Navigation

© Copyright 2008, Open Society Institute. Last updated on Nov 14, 2011. Created using Sphinx 1.1.2.