What's New In Pyramid 1.0
=========================

This article explains the new features in Pyramid version 1.0 as compared to
its predecessor, :mod:`repoze.bfg` 1.3.  It also documents backwards
incompatibilities between the two versions and deprecations added to Pyramid
1.0, as well as software dependency changes and notable documentation
additions.

Major Feature Additions
-----------------------

The major feature additions in Pyramid 1.0 are:

- New name and branding association with the Pylons Project.

- BFG conversion script

- Scaffold improvements

- Terminology changes

- Better platform compatibility and support

- Direct built-in support for the Mako templating language.

- Built-in support for sessions.

- Updated URL dispatch features

- Better imperative extensibility

- ZCML externalized

- Better support for global template variables during rendering

- View mappers

- Testing system improvements

- Authentication support improvements

- Documentation improvements

New Name and Branding
~~~~~~~~~~~~~~~~~~~~~

The name of ``repoze.bfg`` has been changed to Pyramid.  The project is also
now a subproject of a new entity, "The Pylons Project".  The Pylons Project
is the project name for a collection of web-framework-related technologies.
Pyramid was the first package in the Pylons Project. Other packages to the
collection have been added over time, such as support packages useful for
Pylons 1 users as well as ex-Zope users.  Pyramid is the successor to both
:mod:`repoze.bfg` and :term:`Pylons` version 1.

The Pyramid codebase is derived almost entirely from :mod:`repoze.bfg`
with some changes made for the sake of Pylons 1 compatibility.

Pyramid is technically backwards incompatible with :mod:`repoze.bfg`, as it
has a new package name, so older imports from the ``repoze.bfg`` module will
fail if you do nothing to your existing :mod:`repoze.bfg` application.
However, you won't have to do much to use your existing BFG applications on
Pyramid. There's automation which will change most of your import statements
and ZCML declarations. See
http://docs.pylonsproject.org/projects/pyramid/current/tutorials/bfg/index.html
for upgrade instructions.

Pylons 1 users will need to do more work to use Pyramid, as Pyramid shares no
"DNA" with Pylons.  It is hoped that over time documentation and upgrade code
will be developed to help Pylons 1 users transition to Pyramid more easily.

:mod:`repoze.bfg` version 1.3 will be its last major release. Minor updates
will be made for critical bug fixes.  Pylons version 1 will continue to see
maintenance releases, as well.

The Repoze project will continue to exist. Repoze will be able to regain its
original focus: bringing Zope technologies to WSGI. The popularity of
:mod:`repoze.bfg` as its own web framework hindered this goal.

We hope that people are attracted at first by the spirit of cooperation
demonstrated by the Pylons Project and the merging of development
communities. It takes humility to sacrifice a little sovereignty and work
together. The opposite, forking or splintering of projects, is much more
common in the open source world. We feel there is a limited amount of oxygen
in the space of "top-tier" Python web frameworks and we don’t do the Python
community a service by over-crowding.  By merging the :mod:`repoze.bfg` and
the philosophically-similar Pylons communities, both gain an expanded
audience and a stronger chance of future success.

BFG Conversion Script
~~~~~~~~~~~~~~~~~~~~~

The ``bfg2pyramid`` conversion script performs a mostly automated conversion
of an existing :mod:`repoze.bfg` application to Pyramid.  The process is
described in "Converting a BFG Application to Pyramid".

Scaffold Improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- The scaffolds now have much nicer CSS and graphics.

- The ``development.ini``, generated by all scaffolds, is now configured to
  use the WebError interactive exception debugger by default.

- All scaffolds have been normalized: each now uses the name ``main``
  to represent the function that returns a WSGI application, and each now has
  roughly the same shape of development.ini style.

- All preexisting scaffolds now use "imperative" configuration
  (``starter``, ``routesalchemy``, ``alchemy``, ``zodb``) instead of ZCML
  configuration.

- The ``pyramid_zodb``, ``routesalchemy`` and ``pyramid_alchemy``
  scaffolds now use a default "commit veto" hook when configuring the
  ``repoze.tm2`` transaction manager in ``development.ini``.  This prevents a
  transaction from being committed when the response status code is within
  the 400 or 500 ranges.
  
  .. seealso::
     
     See also http://docs.repoze.org/tm2/#using-a-commit-veto.

Terminology Changes
~~~~~~~~~~~~~~~~~~~

- The Pyramid concept previously known as "model" is now known as "resource".
  As a result, the following API renames have been made.  Backwards
  compatibility shims for the old names have been left in place in all cases::

      pyramid.url.model_url -> 
                        pyramid.url.resource_url

      pyramid.traversal.find_model -> 
                        pyramid.url.find_resource

      pyramid.traversal.model_path ->
                        pyramid.traversal.resource_path

      pyramid.traversal.model_path_tuple ->
                        pyramid.traversal.resource_path_tuple

      pyramid.traversal.ModelGraphTraverser -> 
                        pyramid.traversal.ResourceTreeTraverser

      pyramid.config.Configurator.testing_models ->
                        pyramid.config.Configurator.testing_resources

      pyramid.testing.registerModels ->
                        pyramid.testing.registerResources

      pyramid.testing.DummyModel ->
                        pyramid.testing.DummyResource

- All documentation which previously referred to "model" now refers to
  "resource".

- The ``starter`` scaffold now has a ``resources.py`` module instead
  of a ``models.py`` module.

- Positional argument names of various APIs have been changed from
  ``model`` to ``resource``.

- The Pyramid concept previously known as "resource" is now known as "asset".
  As a result, the following API changes were made.  Backwards compatibility
  shims have been left in place as necessary::

      pyramid.config.Configurator.absolute_resource_spec ->
                        pyramid.config.Configurator.absolute_asset_spec

      pyramid.config.Configurator.override_resource ->
                        pyramid.config.Configurator.override_asset


- The (non-API) module previously known as ``pyramid.resource`` is now
  known as ``pyramid.asset``.

- All docs that previously referred to "resource specification" now refer
  to "asset specification".

- The setting previously known as ``BFG_RELOAD_RESOURCES`` (envvar) or
  ``reload_resources`` (config file) is now known, respectively, as
  ``PYRAMID_RELOAD_ASSETS`` and ``reload_assets``.

Better Platform Compatibility and Support
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We've made Pyramid's test suite pass on both Jython and PyPy.  However,
Chameleon doesn't work on either, so you'll need to use Mako or Jinja2
templates on these platforms.

Sessions
~~~~~~~~

Pyramid now has built-in sessioning support, documented in
:ref:`sessions_chapter`.  The sessioning implementation is pluggable.  It
also provides flash messaging and cross-site-scripting prevention features.

Using ``request.session`` now returns a (dictionary-like) session object if
a :term:`session factory` has been configured.

A new argument to the Configurator constructor exists: ``session_factory``
and a new method on the configurator exists:
:meth:`pyramid.config.Configurator.set_session_factory`.

Mako
~~~~

In addition to Chameleon templating, Pyramid now also provides built-in
support for :term:`Mako` templating.  See
:ref:`available_template_system_bindings` for more information.

URL Dispatch
~~~~~~~~~~~~

- URL Dispatch now allows for replacement markers to be located anywhere
  in the pattern, instead of immediately following a ``/``.

- URL Dispatch now uses the form ``{marker}`` to denote a replace marker in
  the route pattern instead of ``:marker``. The old colon-style marker syntax
  is still accepted for backwards compatibility. The new format allows a
  regular expression for that marker location to be used instead of the
  default ``[^/]+``, for example ``{marker:\d+}`` is now valid to require the
  marker to be digits.

- Addded a new API :func:`pyramid.url.current_route_url`, which computes a
  URL based on the "current" route (if any) and its matchdict values.

- Added a ``paster proute`` command which displays a summary of the routing
  table.  See the narrative documentation section entitled
  :ref:`displaying_application_routes`.

- Added ``debug_routematch`` configuration setting (settable in your ``.ini``
  file) that logs matched routes including the matchdict and predicates.

- Add a :func:`pyramid.url.route_path` API, allowing folks to generate
  relative URLs.  Calling ``route_path`` is the same as calling
  :func:`pyramid.url.route_url` with the argument ``_app_url`` equal to the
  empty string.

- Add a :meth:`pyramid.request.Request.route_path` API.  This is a
  convenience method of the request which calls
  :func:`pyramid.url.route_url`.

- Added class vars ``matchdict`` and ``matched_route`` to
  :class:`pyramid.request.Request`.  Each is set to ``None`` when a route
  isn't matched during a request.

ZCML Externalized
~~~~~~~~~~~~~~~~~

- The ``load_zcml`` method of a Configurator has been removed from the
  Pyramid core.  Loading ZCML is now a feature of the :term:`pyramid_zcml`
  package, which can be downloaded from PyPI.  Documentation for the package
  should be available via
  http://docs.pylonsproject.org/projects/pyramid_zcml/en/latest/, which describes how to
  add a configuration statement to your ``main`` block to reobtain this
  method.  You will also need to add an ``install_requires`` dependency upon
  the ``pyramid_zcml`` distribution to your ``setup.py`` file.

- The "Declarative Configuration" narrative chapter has been removed (it was
  moved to the ``pyramid_zcml`` package).

- Most references to ZCML in narrative chapters have been removed or
  redirected to ``pyramid_zcml`` locations.

- The ``starter_zcml`` paster scaffold has been moved to the ``pyramid_zcml``
  package.

Imperative Two-Phase Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To support application extensibility, the :app:`Pyramid`
:term:`Configurator`, by default, now detects configuration conflicts and
allows you to include configuration imperatively from other packages or
modules.  It also, by default, performs configuration in two separate phases.
This allows you to ignore relative configuration statement ordering in some
circumstances.  See :ref:`advconfig_narr` for more information.

The :meth:`pyramid.config.Configurator.add_directive` allows framework
extenders to add methods to the configurator, which allows extenders to avoid
subclassing a Configurator just to add methods.  See :ref:`add_directive` for
more info.

Surrounding application configuration with ``config.begin()`` and
``config.end()`` is no longer necessary.  All scaffolds have been
changed to no longer call these functions.

Better Support for Global Template Variables During Rendering
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A new event type named :class:`pyramid.interfaces.IBeforeRender` is now sent
as an event before a renderer is invoked.  Applications may now subscribe to
the ``IBeforeRender`` event type in order to introspect the and modify the
set of renderer globals before they are passed to a renderer.  The event
object iself has a dictionary-like interface that can be used for this
purpose.  For example::

    from repoze.events import subscriber
    from pyramid.interfaces import IRendererGlobalsEvent

    @subscriber(IRendererGlobalsEvent)
    def add_global(event):
        event['mykey'] = 'foo'

View Mappers
~~~~~~~~~~~~

A "view mapper" subsystem has been extracted, which allows framework
extenders to control how view callables are constructed and called.  This
feature is not useful for "civilians", only for extension writers.  See
:ref:`using_a_view_mapper` for more information.

Testing Support Improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The :func:`pyramid.testing.setUp` and :func:`pyramid.testing.tearDown` APIs
have been undeprecated.  They are now the canonical setup and teardown APIs
for test configuration, replacing "direct" creation of a Configurator.  This
is a change designed to provide a facade that will protect against any future
Configurator deprecations.

Authentication Support Improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- The :class:`pyramid.interfaces.IAuthenticationPolicy` interface now
  specifies an ``unauthenticated_userid`` method.  This method supports an
  important optimization required by people who are using persistent storages
  which do not support object caching and whom want to create a "user object"
  as a request attribute.

- A new API has been added to the :mod:`pyramid.security` module named
  ``unauthenticated_userid``.  This API function calls the
  ``unauthenticated_userid`` method of the effective security policy.

- The class :class:`pyramid.authentication.AuthTktCookieHelper` is now an
  API.  This class can be used by third-party authentication policy
  developers to help in the mechanics of authentication cookie-setting.

- The :class:`pyramid.authentication.AuthTktAuthenticationPolicy` now accepts
  a ``tokens`` parameter via :func:`pyramid.security.remember`.  The value
  must be a sequence of strings.  Tokens are placed into the auth_tkt
  "tokens" field and returned in the auth_tkt cookie.

- Added a ``wild_domain`` argument to
  :class:`pyramid.authentication.AuthTktAuthenticationPolicy`, which defaults
  to ``True``.  If it is set to ``False``, the feature of the policy which
  sets a cookie with a wilcard domain will be turned off.

Documentation Improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~

- Casey Duncan, a good friend, and an excellent technical writer has given us
  the gift of professionally editing the entire Pyramid documentation set.
  Any faults in the documentation are the development team's, and all
  improvements are his.

- The "Resource Location and View Lookup" chapter has been replaced with a
  variant of Rob Miller's "Much Ado About Traversal" (originally published at
  http://blog.nonsequitarian.org/2010/much-ado-about-traversal/).

- Many users have contributed documentation fixes and improvements including
  Ben Bangert, Blaise Laflamme, Rob Miller, Mike Orr, Carlos de la Guardia,
  Paul Everitt, Tres Seaver, John Shipman, Marius Gedminas, Chris Rossi,
  Joachim Krebs, Xavier Spriet, Reed O'Brien, William Chambers, Charlie
  Choiniere, and Jamaludin Ahmad.

Minor Feature Additions
-----------------------

- The ``settings`` dictionary passed to the Configurator is now available as
  ``config.registry.settings`` in configuration code and
  ``request.registry.settings`` in view code).

- :meth:`pyramid.config.Configurator.add_view` now accepts a ``decorator``
  keyword argument, a callable which will decorate the view callable before
  it is added to the registry.

- Allow static renderer provided during view registration to be overridden at
  request time via a request attribute named ``override_renderer``, which
  should be the name of a previously registered renderer.  Useful to provide
  "omnipresent" RPC using existing rendered views.

- If a resource implements a ``__resource_url__`` method, it will be called
  as the result of invoking the :func:`pyramid.url.resource_url` function to
  generate a URL, overriding the default logic.  See
  :ref:`generating_the_url_of_a_resource` for more information.

- The name ``registry`` is now available in a ``pshell`` environment by
  default.  It is the application registry object.

- Added support for json on Google App Engine by catching
  :exc:`NotImplementedError` and importing ``simplejson`` from
  ``django.utils``.

- Added the :mod:`pyramid.httpexceptions` module, which is a facade for the
  ``webob.exc`` module.

- New class: :class:`pyramid.response.Response`.  This is a pure facade for
  ``webob.Response`` (old code need not change to use this facade, it's
  existence is mostly for vanity and documentation-generation purposes).

- The request now has a new attribute: ``tmpl_context`` for benefit of
  Pylons users.

- New API methods for :class:`pyramid.request.Request`: ``model_url``,
  ``route_url``, and ``static_url``.  These are simple passthroughs for their
  respective functions in :mod:`pyramid.url`.

Backwards Incompatibilities
---------------------------

- When a :class:`pyramid.exceptions.Forbidden` error is raised, its status
  code now ``403 Forbidden``.  It was previously ``401 Unauthorized``, for
  backwards compatibility purposes with :mod:`repoze.bfg`.  This change will
  cause problems for users of Pyramid with :mod:`repoze.who`, which
  intercepts ``401 Unauthorized`` by default, but allows ``403 Forbidden`` to
  pass through.  Those deployments will need to configure :mod:`repoze.who`
  to also react to ``403 Forbidden``.  To do so, use a repoze.who
  ``challenge_decider`` that looks like this::

     import zope.interface
     from repoze.who.interfaces import IChallengeDecider

     def challenge_decider(environ, status, headers):
         return status.startswith('403') or status.startswith('401')
     zope.interface.directlyProvides(challenge_decider, IChallengeDecider)

- The ``paster bfgshell`` command is now known as ``paster pshell``.

- There is no longer an ``IDebugLogger`` object registered as a named utility
  with the name ``repoze.bfg.debug``.

- These deprecated APIs have been removed:
  ``pyramid.testing.registerViewPermission``,
  ``pyramid.testing.registerRoutesMapper``, ``pyramid.request.get_request``,
  ``pyramid.security.Unauthorized``,
  ``pyramid.view.view_execution_permitted``, ``pyramid.view.NotFound``

- The Venusian "category" for all built-in Venusian decorators
  (e.g. ``subscriber`` and ``view_config``/``bfg_view``) is now
  ``pyramid`` instead of ``bfg``.

- The ``pyramid.renderers.rendered_response`` function removed; use
  :func:`pyramid.renderers.render_to_response` instead.

- Renderer factories now accept a *renderer info object* rather than an
  absolute resource specification or an absolute path.  The object has the
  following attributes: ``name`` (the ``renderer=`` value), ``package`` (the
  'current package' when the renderer configuration statement was found),
  ``type``: the renderer type, ``registry``: the current registry, and
  ``settings``: the deployment settings dictionary.  Third-party
  ``repoze.bfg`` renderer implementations that must be ported to Pyramid will
  need to account for this.  This change was made primarily to support more
  flexible Mako template rendering.

- The presence of the key ``repoze.bfg.message`` in the WSGI environment when
  an exception occurs is now deprecated.  Instead, code which relies on this
  environ value should use the ``exception`` attribute of the request
  (e.g. ``request.exception[0]``) to retrieve the message.

- The values ``bfg_localizer`` and ``bfg_locale_name`` kept on the request
  during internationalization for caching purposes were never APIs.  These
  however have changed to ``localizer`` and ``locale_name``, respectively.

- The default ``cookie_name`` value of the
  :class:`pyramid.authentication.AuthTktAuthenticationPolicy` now defaults to
  ``auth_tkt`` (it used to default to ``repoze.bfg.auth_tkt``).

- The :func:`pyramid.testing.zcml_configure` API has been removed.  It had
  been advertised as removed since :mod:`repoze.bfg` 1.2a1, but hadn't
  actually been.

- All environment variables which used to be prefixed with ``BFG_`` are now
  prefixed with ``PYRAMID_`` (e.g. ``BFG_DEBUG_NOTFOUND`` is now
  ``PYRAMID_DEBUG_NOTFOUND``)

- Since the :class:`pyramid.interfaces.IAuthenticationPolicy` interface now
  specifies that a policy implementation must implement an
  ``unauthenticated_userid`` method, all third-party custom authentication
  policies now must implement this method.  It, however, will only be called
  when the global function named
  :func:`pyramid.security.unauthenticated_userid` is invoked, so if you're
  not invoking that, you will not notice any issues.

- The ``configure_zcml`` setting within the deployment settings (within
  ``**settings`` passed to a Pyramid ``main`` function) has ceased to have any
  meaning.

- The ``make_app`` function has been removed from the :mod:`pyramid.router`
  module.  It continues life within the ``pyramid_zcml`` package.  This
  leaves the :mod:`pyramid.router` module without any API functions.

Deprecations and Behavior Differences
-------------------------------------

- :class:`pyramid.configuration.Configurator` is now deprecated.  Use
  :class:`pyramid.config.Configurator`, passing its constructor
  ``autocommit=True`` instead.  The
  :class:`pyramid.configuration.Configurator` alias will live for a long
  time, as every application uses it, but its import now issues a deprecation
  warning.  The :class:`pyramid.config.Configurator` class has the same API
  as the :class:`pyramid.configuration.Configurator` class, which it means to
  replace, except by default it is a *non-autocommitting* configurator. The
  now-deprecated ``pyramid.configuration.Configurator`` will autocommit every
  time a configuration method is called.  The :mod:`pyramid.configuration`
  module remains, but it is deprecated.  Use :mod:`pyramid.config` instead.

- The :func:`pyramid.settings.get_settings` API is now deprecated.  Use
  ``pyramid.threadlocals.get_current_registry().settings`` instead or use the
  ``settings`` attribute of the registry available from the request
  (``request.registry.settings``).

- The decorator previously known as ``pyramid.view.bfg_view`` is now known
  most formally as :class:`pyramid.view.view_config` in docs and scaffolds.

- Obtaining the ``settings`` object via
  ``registry.{get|query}Utility(ISettings)`` is now deprecated.  Instead,
  obtain the ``settings`` object via the ``registry.settings`` attribute.  A
  backwards compatibility shim was added to the registry object to register
  the settings object as an ISettings utility when ``setattr(registry,
  'settings', foo)`` is called, but it will be removed in a later release.

- Obtaining the ``settings`` object via :func:`pyramid.settings.get_settings`
  is now deprecated.  Obtain it instead as the ``settings`` attribute of the
  registry now (obtain the registry via
  :func:`pyramid.threadlocal.get_registry` or as ``request.registry``).

Dependency Changes
------------------

- Depend on Venusian >= 0.5 (for scanning conflict exception decoration).

Documentation Enhancements
--------------------------

- Added a :mod:`pyramid.httpexceptions` API documentation chapter.

- Added a :mod:`pyramid.session` API documentation chapter.

- Added an API chapter for the :mod:`pyramid.response` module.

- Added a :ref:`sessions_chapter` narrative documentation chapter.

- All documentation which previously referred to ``webob.Response`` now uses
  :class:`pyramid.response.Response` instead.

- The documentation has been overhauled to use imperative configuration,
  moving declarative configuration (ZCML) explanations to an external
  package, :term:`pyramid_zcml`.

- Removed ``zodbsessions`` tutorial chapter.  It's still useful, but we now
  have a SessionFactory abstraction which competes with it, and maintaining
  documentation on both ways to do it is a distraction.

- Added an example of ``WebTest`` functional testing to the testing narrative
  chapter at :ref:`functional_tests`.

- Extended the Resources chapter with examples of calls to resource-related
  APIs.

- Add "Pyramid Provides More Than One Way to Do It" to Design Defense
  documentation.

- The (weak) "Converting a CMF Application to Pyramid" tutorial has been
  removed from the tutorials section.  It was moved to the
  ``pyramid_tutorials`` Github repository at
  http://docs.pylonsproject.org/projects/pyramid_tutorials/dev/.

- Moved "Using ZODB With ZEO" and "Using repoze.catalog Within Pyramid"
  tutorials out of core documentation and into the Pyramid Tutorials site
  (http://docs.pylonsproject.org/projects/pyramid_tutorials/dev/).

- Removed API documentation for deprecated ``pyramid.testing`` APIs named
  ``registerDummySecurityPolicy``, ``registerResources``, ``registerModels``,
  ``registerEventListener``, ``registerTemplateRenderer``,
  ``registerDummyRenderer``, ``registerView``, ``registerUtility``,
  ``registerAdapter``, ``registerSubscriber``, ``registerRoute``, and
  ``registerSettings``.