Django: Testing

This post is part of my Django series. You can see an overview of the series along with instruction on how to get all the source code here.

This article assumes you are comfortable creating a Django project, can create apps and register them into the INSTALLED_APPS list of the settings file. If not please read my Django HelloWorld article.

Django uses the Python package unittest. This article assumes that you have a basic knowledge of creating unittests with this package. If this is not the case please see my blog post on unittest.

This article assumes you have a project called DjangoSandBox in an application called testintroduction.

This article tests the various parts we have built up over the series; please see here for any explanations of the code we are testing.

Testing Models

Models are pretty easy to test; our first example is using the HelloWorld class we have created previously. Nothing special, a name and date field, a __str()__ function and a Django get_absolute_url function.

#helloworld/models.py
from django.db import models
from django.core.urlresolvers import reverse

class HelloWorld(models.Model):
    first_name = models.CharField(max_length=20)
    now = models.DateTimeField()

    def __str__(self):
        return "{0} {1}".format(self.first_name, self.now)

    @staticmethod
    def get_absolute_url():
        return reverse('helloworld:model')

Testing these functions is nothing more than creating an instance of the class, calling the require function or property and asserting that our result is as required.

#test_models.py

from datetime import date
from django.test import TestCase
from helloworld.models import HelloWorld

class TestHelloWorld(TestCase):
    def setUp(self):
        self.first_name = "Luke"
        self.now = date.today()

        self.a_helloworld = HelloWorld(first_name=self.first_name, now=self.now)

    def test_str(self):
        self.assertEqual("{0} {1}".format(self.first_name, self.now), str(self.a_helloworld))

    def test_get_absolute_url(self):
        self.assertEqual("/helloworld/model/", self.a_helloworld.get_absolute_url())

Our model could contain validation. We can test validation by creating an instance of the model in an invalid state and calling full_clean. If the model is invalid an exception is raised with all the errors contained within a dictionary upon the message_dict property.

# test_models.py
from datetime import date
from django.core.exceptions import ValidationError
from django.test import TestCase
from modelsadvanced.models import ContactDetails, ProxyChild

class TestModelValidation(TestCase):

    def test_model_validation(self):

        try:
            a_contact = ContactDetails(name="Luke", age=101, contactDate=date(year=2015, month=7, day=2))
            a_contact.full_clean()
        except ValidationError as e:

            self.assertEqual({'name': ['Ensure this value has at least 10 characters (it has 4).', 'Luke is barred'],
                              'age': ['Ensure this value is less than or equal to 100.'],
                              'contactDate': ['2015-07-02 is not a future date.']}, e.message_dict)

If we want to test class meta data we can access this via the _meta property:

#test_models.py
class TestModelMeta(TestCase):
    def test_model_meta(self):
        self.assertEqual(True, ProxyChild()._meta.proxy)

We can use list and map to get a list of all field names contained within the fields property of the meta class.:

#test_models.py
class TestModelFields(TestCase):
    def test_model_meta(self):
        field_names = list(map((lambda x: x.name), ContactDetails._meta.fields))

        self.assertEqual(4, len(ContactDetails._meta.fields))
        self.assertEqual(['id', 'age', 'name', 'contactDate'], field_names)

Testing Views

When we test views we can use the test client; it acts as a dummy Web browser. We access it via the client property of our unit test class.

We call get upon the test client along with the URL.

response = self.client.get("/index")

We can use reverse to access the URL via our routing config. Don’t worry we will explicitly test the routing config later/

response = self.client.get(reverse("home:home"))

The response.status_code can be used to assure our response code is as expected.

The response.resolver_match can be use to ensure that the view function used was as expected.

The function self.assertTemplateUsed can be used to assure that the correct template was used.

We can assert that the html is as expected by calling self.assertContains along with sections of html or strings which should be contained within the rendered HTML.

#test_views.py

from django.core.urlresolvers import reverse
from django.test import TestCase
from home import views as home_views

class TestFunctionView(TestCase):
    def test_function_view(self):
        response = self.client.get(reverse("home:home"))

        self.assertEqual(200, response.status_code)
        self.assertEqual(home_views.home, response.resolver_match.func)
        self.assertTemplateUsed(response, template_name='home/home.html')

        self.assertContains(response, '<a href="/helloworld/index/">Hello World</a>')
        self.assertContains(response, '<a href="/templatesintroduction/index">Templates</a>')
        self.assertContains(response, '<a href="/formsintroduction/basic/">Forms</a>')
        self.assertContains(response, '<a href="/viewsintroduction/contact/list">Views</a>')

We can access the context data which was passed to the view via the context property of the response.

#test_views.py

from django.core.urlresolvers import reverse
from django.test import TestCase
from helloworld import views as helloworld_views

class TestContextData(TestCase):
    def test_context_data(self):
        response = self.client.get(reverse("helloworld:template"))

        self.assertEqual('Obelix', response.context["who"])
        self.assertEqual(False, response.context["islong"])

Testing Class Based Views

In the example two up we used the following code to ensure the correct view function was used:

self.assertEqual(home_views.home, response.resolver_match.func)

If we want to assure that the correct class view was used we need to compare the name of the class against that which was used:

self.assertEqual(PhoneAddressListView.as_view().__name__, response.resolver_match.func.__name__)

Where a base template is used we can simply call the function assertTemplateUsed with the base template name.

In the example below we test the list view with and without records to ensure our “no records here” message is displayed to the user.

When records are returned to our template we have a few ways we can test the results along with the assertContains function we have seen in the previous examples.

  • Check the record count of the object list in the contect view
  • Compare elements
  • Compare the query set. To do this we need to convert the expected query set to a list of strings from the repr function as this is what is contained in the response.
#test_views.py

class TestClassView(TestCase):
    def test_with_no_records(self):
        response = self.client.get(reverse("viewsintroduction:address_list"))

        self.assertEqual(200, response.status_code)
        self.assertEqual(PhoneAddressListView.as_view().__name__, response.resolver_match.func.__name__)

        self.assertTemplateUsed(response, template_name='viewsintroduction/base.html')
        self.assertTemplateUsed(response, template_name='viewsintroduction/phoneaddresslist.html')

        self.assertEqual(0, len(response.context["phoneaddress_list"]))
        self.assertContains(response, '<tr><td colspan="6">There are no addresses</td></tr>')

    def test_with_one_record(self):
        an_address = PhoneAddress(number=1, street_name="A", city="A City")
        an_address.save()

        response = self.client.get(reverse("viewsintroduction:address_list"))

        self.assertEqual(200, response.status_code)
        self.assertEqual(PhoneAddressListView.as_view().__name__, response.resolver_match.func.__name__)

        self.assertTemplateUsed(response, template_name='viewsintroduction/base.html')
        self.assertTemplateUsed(response, template_name='viewsintroduction/phoneaddresslist.html')

        context_addresses = response.context['phoneaddress_list']
        expected_addresses = [repr(r) for r in PhoneAddress.objects.all()]

        self.assertEqual(1, len(context_addresses))
        self.assertEqual(an_address, context_addresses.first())

        self.assertQuerysetEqual(context_addresses, expected_addresses, ordered=False)

        self.assertContains(response, an_address.city)
        self.assertContains(response, an_address.number)
        self.assertContains(response, an_address.street_name)

        self.assertNotContains(response, '<tr><td colspan="6">There are no addresses</td></tr>')

Testing Errors

We now move on to test some of our error views; by comparing the status code.

#test_views.py

class Test404Error(TestCase):
    def test_404_error_is_raised(self):
        response = self.client.get(reverse("helloworld:error_as_404"))
        self.assertEqual(404, response.status_code)

class TestNotAllowed(TestCase):
    def test_not_allowed(self):
        response = self.client.get(reverse("helloworld:not_allowed"))

        self.assertEqual(405, response.status_code)


class TestOnlyPost(TestCase):
    def test_raises_error_on_get(self):
        response = self.client.get(reverse("helloworld:error_if_not_post"))

        self.assertEqual(405, response.status_code)

    def test_all_ok_on_post(self):
        response = self.client.post(reverse("helloworld:error_if_not_post"), {'name': 'Jim'})

        self.assertEqual(200, response.status_code)
        self.assertContains(response, "{0}, this can only be called with a post".format("Jim"))

When we expect a redirect is made we can use the assertRedirects function to validate this. The function takes the status codes of our page load and also the page we direct to. 301 indicates that this is a page which has moved permanently.

#test_views.py

class TestRedirect(TestCase):
    def test_redirect(self):
        response = self.client.post(reverse("viewsintroduction:redirect_view"))

        self.assertTrue(301, response.status_code)
        self.assertRedirects(response, "http://djangoproject.com", 301, 200)  # Status codes Us/Them

Test URLS

When we test URLs we should test two things;

  • The reverse function returns the correct URL string
  • The resolve function returns the correct view function. This does not only include the function and URL name but the app name of the view, the URL namespace and any arguments which will be passed into the view function.
#test_urls.py

from django.core.urlresolvers import reverse, resolve
from django.test import TestCase

from home import views

class TestHomeUrls(TestCase):
    urls = 'home.urls'

    def test_reverse(self):
        self.assertEqual('/', reverse('home'))

    def test_resolve(self):
        resolver = resolve(reverse('home'))

        self.assertEqual(views.home, resolver.func)
        self.assertEqual('home', resolver.url_name)

        self.assertEqual((), resolver.args)
        self.assertEqual({}, resolver.kwargs)
        self.assertEqual(None, resolver.app_name)
        self.assertEqual('', resolver.namespace)

Note: Setting urls = ‘home.urls’ upon the test class above defines the URL routing config we are using. Here we test only an apps config which explains why we reference the URL by their name only without the namespace which we set in the project’s urls.py

When our URL config contains get query parameters we can pass them into the reverse function with the kwargs parameter which is a dictionary.

#test_urls.py

class TestViewsIntroductionUrls(TestCase):
    urls = 'viewsintroduction.urls'

    def test_reverse(self):
        self.assertEqual('/address/1/update/', reverse('address_update', kwargs={'pk': 1}))

Testing Forms

When testing forms we pass the input data as a dictionary into the constructor.

We can access the form fields via the fields property; below we ensure that we have the correct fields and that they contain the right configuration.

We can assert that the form validates as expected with form.is_valid(). Any errors raised will be found within the property errors on the form which is a dictionary of errors. Field errors will be found under a key named after the field name; it returns a list of errors.

#test_forms.py

from django.core.urlresolvers import reverse
from django.forms import DecimalField
from django.test import TestCase

from formsintroduction.forms.bastic_form_example import BasicFormExample
from formsintroduction.forms.class_form_example import ClassBasedForm
from viewsintroduction.models import PhoneAddress


class TestBasicForm(TestCase):
    def test_fields(self):
        form = BasicFormExample({})

        self.assertEqual(2, len(form.fields))
        self.assertTrue("height" in form.fields.keys())
        self.assertTrue("name" in form.fields.keys())
        self.assertIsInstance(form.fields['height'], DecimalField)
        self.assertEqual(2, form.fields['height'].max_value)

    def test_valid_data(self):
        form = BasicFormExample({"name": "Luke", "height": 1.11})

        self.assertTrue(form.is_valid())

    def test_invalid_data(self):
        form = BasicFormExample({"name": "Luke"})

        self.assertFalse(form.is_valid())
        self.assertTrue("name" not in form.errors.keys())
        self.assertTrue("height" in form.errors.keys())

        self.assertEqual(form.errors["height"], ['This field is required.'])

Testing Class Derived Forms

Class based views are tested in pretty much the same way. However when we call save we get an instance of the record created which we can test against.

In the example below we test the form meta data to ensure that the correct fields are set along with the right model class.

#test_forms.py

class TestClassBasedForm(TestCase):
    def test_valid_data(self):
        form = ClassBasedForm({"number": 123, "street_name": "The high street", "city": "Bristol"})

        self.assertTrue(form.is_valid())

        a_record = form.save()

        self.assertEqual(123, a_record.number)
        self.assertEqual("The high street", a_record.street_name)
        self.assertEqual("Bristol", a_record.city)

    def test_invalid_data(self):
        form = ClassBasedForm({})

        self.assertEqual(form.errors["number"],
                         ['We need a number of the house!', 'None of the fields have been set!!!'])

    def test_meta(self):
        self.assertEqual(PhoneAddress, ClassBasedForm.Meta.model)
        self.assertEqual(("city", "street_name", "number"), ClassBasedForm.Meta.fields)

Posting Forms

We use the test client to test posting a form to the server. We pass the form data as a dictionary along with the URL.

#test_forms.py

    def test_post_invalid(self):
        response = self.client.post(reverse("formsintroduction:form_class_create"),
                                    {"city": "Word_Word", "street_name": "street", "number": 1},
                                    follow=True)

        self.assertEqual(200, response.status_code)
        self.assertFormError(response, 'form', 'city', ["Name must be a capitalised single word"])

    def test_post_valid(self):
        response = self.client.post(reverse("formsintroduction:form_class_create"),
                                    {"city": "Foo", "street_name": "Foo", "number": 1},
                                    follow=True)

        self.assertEqual(200, response.status_code)
        self.assertRedirects(response, "http://testserver/viewsintroduction/address/1/", 302)

Testing An Authorised View

Unsuccessful Authentication

The Django authorisation system will automatically redirect a user to the registered login page when they try to access a view which requires authentication. Our first test case assures that we are redirected successfully.

from django.contrib.auth.models import User
from django.core.urlresolvers import reverse
from django.test import TestCase


class TestAuthedView(TestCase):
    def test_redirect_to_loging(self):
        response = self.client.post(reverse("auth:users"))

        self.assertTrue(302, response.status_code)
        self.assertRedirects(response, "auth/login/?next=/auth/users/", 302, 200)

In the above example the 302 and 200 parameters passed into assertRedirects are status codes of the HTTP for the initial page and then the redirect page. 302 is a redirect while 200 is a successful request.

Successful Authentication

The Django test web client comes with the ability to log a user in:

self.client.login(username='username', password='password')

Note: don’t forget that when we are running tests we start with a blank database so you will need to create a login first. The example below creates a new user and activates it in line; this should be moved to a test fixture set-up function.

from django.contrib.auth.models import User
from django.core.urlresolvers import reverse
from django.test import TestCase

class TestAuthedView(TestCase):

    def test_can_login(self):
        user = User.objects.create_user(username='FOOTESTFOO', password='pwd')
        user.is_active = True
        user.save()

        login = self.client.login(username='FOOTESTFOO', password='pwd')

        self.assertTrue(login)

        response = self.client.post(reverse("auth:users"))

        self.assertTrue(200, response.status_code)
        self.assertContains(response, "Registered Users")
        self.assertContains(response, "First Name")
        self.assertContains(response, "Last Name")
        self.assertContains(response, "FOOTESTFOO")

Testing Cookies

We can access cookies from the cookies property of the Django test web client once we have called post or get. It is a dictionary keyed upon the cookie name:

self.client.cookies
self.client.cookies["hits"]

The cookie is returned as a http.cookies.Morsel. We can access the cookie value with the value property. The class responds as a dictionary for other data, in the following example we ensure the expiry has been set correctly with the max-age key.

from django.core.urlresolvers import reverse
from django.test import TestCase

class TestSessions(TestCase):
    def test_cookie(self):
        response = self.client.post(reverse("sessionsandcookies:cookies"))

        self.assertTrue(200, response.status_code)

        self.assertTrue("hits" in self.client.cookies)
        self.assertEqual('1', self.client.cookies["hits"].value)
        self.assertEqual(30, self.client.cookies["hits"]["max-age"])

Testing Session Data

We can access session data upon the Django test web client after we have called post or get. It returns a SessionBase which responds as a dictionary to retrieve the session data from their key names. All functionality of the SessionBase class is available; below we call get_expiry_age to ensure our session expiry policy has worked as expected:

from django.core.urlresolvers import reverse
from django.test import TestCase


class TestSessions(TestCase): 

    def test_sessions(self):
        response = self.client.post(reverse("sessionsandcookies:sessions"))

        self.assertTrue(200, response.status_code)

        self.assertTrue("has_visited" in self.client.session)
        self.assertEqual('True', self.client.session["has_visited"])
        self.assertEqual(10, self.client.session.get_expiry_age())

Additional Assertions

As mentioned previously, Django provides some additional assert functions to the standard Python unittest. class We have covered many of them above though there are two others which I think will be quite useful.

The assertFieldOutput assertion allows us to validate against good and bad field input data. I tried to get this to work against the defined fields or a model or form however it seems to only work with the class type.

The assertNumQueries assertion will ensure that the database has been touched the correct number of times.

#test_additionl_assertions.py

from django.forms import EmailField, CharField
from django.test import TestCase

from viewsintroduction.models import PhoneAddress


class TestAdditionalAssertions(TestCase):
    def test_assertFieldOutput(self):
        self.assertFieldOutput(EmailField, {'a@a.com': 'a@a.com'}, {'aaa': ['Enter a valid email address.']})

        self.assertFieldOutput(CharField, {'aa': 'aa'},
                               {'word': ['Ensure this value has at most 3 characters (it has 4).'],
                                'a': ['Ensure this value has at least 2 characters (it has 1).']}, [],
                               {"max_length": 3, "min_length": 2})

    def test_assert_num_queries(self):
        with self.assertNumQueries(1):
            PhoneAddress.objects.create(city="Foo", street_name="Foo", number=1)

        self.assertNumQueries(1, lambda: PhoneAddress.objects.create(city="Foo", street_name="Foo", number=1))

References

Sessions & Cookies

This post is part of my Django series. You can see an overview of the series along with instruction on how to get all the source code here.

This article assumes you are comfortable creating a Django project, can create apps and register them into the INSTALLED_APPS list of the settings file. If not please read my Django HelloWorld article.

This article assumes you have a project called DjangoSandBox in an application called sessionsandcookies.

Sessions

By default Django comes with the sessions middleware and applications installed:

#settings.py

# Application definition
INSTALLED_APPS = (
    'django.contrib.sessions',
)

MIDDLEWARE_CLASSES = (
    'django.contrib.sessions.middleware.SessionMiddleware',
)

Note: If the settings were not present or you are starting a new project, ensure you have run migrate to create all the required schema and run all the required configuration scripts:

python manage.py migrate

Setting

Session data is stored as key value pairs within request.session which behaves as a dictionary. We set a value with [“key”] = value and get a value with get(“key”).

Session data is only valid for a certain amount of time. This is set to SESSION_COOKIE_AGE which defaults to two weeks.

We can use the get_expiry_date function to see the expiry date of the session data.

We can set the expiry with the set_expiry function which takes a time in seconds.

The following uses session data to set mark that a user has visited a page.

#views.py

from datetime import datetime

def sessions(request):
    context = {
        "has_visited": request.session.get("has_visited", False),
        "visited": request.session.get("visited", None),
        "expiry": request.session.get_expiry_date().strftime("%x %X")
    }

    if "has_visited" not in request.session:
        request.session['has_visited'] = str(True)
        request.session['visited'] = datetime.now().strftime("%x %X")
        request.session.set_expiry(10)  # Time in seconds. 0 expires when the browser is closed

    return render(request, 'sessionsandcookies/sessions.html', context)

A simple template to display the data we store and retrieve from the session data.

<!-- templates/sessionsandcookies/sessions.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Sessions</title>
</head>
<body>
<h1>Sessions</h1>

{% if has_visited %}
    <span>Visited: {{ visited }}</span>
    <span>Expires: {{ expiry }}</span>
{% else %}
    First Page Land
{% endif %}

<span>
    <a href="{% url 'sessionsandcookies:sessions_clear' %}">Clear</a>
</span>
</body>
</html>

Hooking in our view function into the URL routing config:

# urls.py
url(r'^sessions/$', views.sessions, name="sessions"),

Note: don’t forget to hook our app into the project URL routing config.

To test run the development server and navigate to:

http://127.0.0.1:8000/sessionsandcookies/sessions/

Clearing

We can use the clear function upon request.session to remove all session data. In this example we clear the session data and then redirect to the page in the previous example to prove we revert to the default value.

#view.py
def sessions_clear(request):
    if 'has_visited' in request.session:
        del request.session['has_visited']

    request.session.clear()

    return redirect(reverse("sessionsandcookies:sessions"))

Note: Above we call delete upon the session data element as well as clear; we can do both depending upon what we require.

Hooking in our view function into the URLrouting config.

#urls.py
url(r'^sessions/clear/$', views.sessions_clear, name="sessions_clear"),

To test run the development server and navigate to:

http://127.0.0.1:8000/sessionsandcookies/sessions/clear/

Cookies

Not all browsers accept cookies so our first example is a way to test if the browser accepts cookies. We call set_test_cookie and then test_cookie_worked. The latter returns true if we can create cookies.

Testing

#views.py
def cookies_test(request):
    can_create_cookie = False
    request.session.set_test_cookie()

    if request.session.test_cookie_worked():
        request.session.delete_test_cookie()
        can_create_cookie = True

    return render(request, "sessionsandcookies/cookies_test.html", {"can_create_cookie": can_create_cookie})

A simple view which displays our result.

<!-- templates/sessionsandcookies/cookies_test.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Test Cookie</title>
</head>
<body>
<h1>Test Cookie</h1>

{% if can_create_cookie %}
    <span>Test cookie worked</span>
{% else %}
    <span>Test cookie did not work</span>
{% endif %}

<span>
    <a href="{% url 'sessionsandcookies:cookies' %}">Back</a>
</span>
</body>
</html>

Hooking in our function view into the URL routing config.

#urls.py
url(r'^cookies/test$', views.cookies_test, name="cookies_test"),

To test run the development server and navigate to:

http://127.0.0.1:8000/sessionsandcookies/cookies/test

Setting

Setting cookie information is a little clunky in Django; we need to call set_cookie on the response object though we have to render our template to get hold of the response. We call render_to_response before calling set_cookies upon the returned response object.

#views.py
def cookies(request):
    hits = int(request.COOKIES.get('hits', '0'))
    first_hit = 'hits' not in request.COOKIES
    response = render_to_response('sessionsandcookies/cookies.html',
                                  {"first_hit": first_hit, "hits": hits}, RequestContext(request))

    response.set_cookie('hits', hits + 1, max_age=30)
    return response

A simple template view displays the number of hits which is passed into our template.

<!-- templates/sessionsandcookies/cookies.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Cookies</title>
</head>
<body>

<h1>Cookies</h1>

<span>
{% if first_hit %}
    Your first hit!
{% else %}
    Hits: {{ hits }}
{% endif %}
</span>

</body>
</html>

Hooking in our view function into our URL routing config.

#urls.py
url(r'^cookies/$', views.cookies, name="cookies"),

To test run the development server and navigate to:

http://127.0.0.1:8000/sessionsandcookies/cookies/

Expiring

We can expire our cookie by calling delete_cookie along with our cookie name.

#views.py
def cookies_expire(request):
    response =         render_to_response('sessionsandcookies/cookies_clear.html',
            {"first_hit": True, "hits": 0}, RequestContext(request))

    response.delete_cookie('hits')
    return response

A simple template which tells the user the cookies have been cleared along with a URL back to our page which shows the cookies data. We can then see the cookie hit count is reset to ‘first hit!’.

<! templates/sessionsandcookies/cookies_clear.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Cookie Clear</title>
</head>
<body>
<h1>Cookie Clear</h1>

<span>The cookie was cleared</span>

<span>
    <a href="{% url 'sessionsandcookies:cookies' %}">Back</a>
</span>
</body>
</html>

Hooking in our view function into our URLrouting config.

#urls.py
url(r'^cookies/expire', views.cookies_expire, name="cookies_expire"),

To test run the development server and navigate to:

http://127.0.0.1:8000/sessionsandcookies/cookies/expire/

References

Django: Errors and Logging

This post is part of my Django series. You can see an overview of the series along with instruction on how to get all the source code here.

This article assumes you are comfortable creating a Django project, can create apps and register them into the INSTALLED_APPS list of the settings file. If not please read my Django HelloWorld article.

This article assumes you have a project called DjangoSandBox in an application called errorsandlogging.

Errors

We can enforce HTTP GET/POST methods upon a view function with the @require_http_methods decorator. Access denied error methods are raised for other HTTP methods.

#views.py
from django.views.decorators.http import require_http_methods

@require_http_methods(["GET", "POST"])
def error_if_not_get_or_post(request):
    return HttpResponse("This can only be called with a get or a post")

If we only want HTTP POST calls to our view function we can use the @require_POST decorator; access denied errors are raised for other HTTP methods.

#views.py

from django.views.decorators.http import require_POST

@require_POST
def error_if_not_post(request):
    return HttpResponse("{0}, this can only be called with a post".format(request.POST.get("name", "")))

If we only want HTTP GET method calls to our view function we can use the @require_GET decorator; access denied errors are raised for other http methods.

#views.py

from django.views.decorators.http import require_GET

@require_GET
def error_if_not_get(request):
    return HttpResponse("This can only be called with a get")

We can return a HTTP 404 error (response not found) by raising a Http404 error. This will use the 404 html file placed within the site templates directory.

#views.py
from django.http import Http404

def error_as_404(request):
    raise Http404("There is nothing to see here")

We can also return a custom HTTP 404 error by returning an instance of HttpResponseNotFound. This is similar to raising a Http404 but allows us to use custom html rather than the stock site 404 page.

#views.py

from django.http import HttpResponseNotFound

def error_as_custom_404(request):
    return HttpResponseNotFound('<h1>There is nothing to see here</h1>')

We can return a HTTP 405 error (response not allowed) by returning an instance of HttpResponseNotAllowed.

#views.py
from django.http import HttpResponseNotAllowed

def not_allowed(request):
    return HttpResponseNotAllowed("<h1>You are not allowed to see this</h1>")

Logging

This assumes that you have a basic working knowledge of Python logging. If this is not the case then you can come up to speed with my post here.

The logging config is defined within the settings config of the Django project; any instances of the logger will then automatically pick these settings up.

Below we create two loggers as defined within the ‘handlers’ section.

  • A terminal logger which will log anything with a level of DEBUG or higher; it is called ‘console’.
  • A file logger which will log anything with a level of ERROR or higher; it is called ‘file’.

We create two formatters and assign them to a logger via the formatter variable.

  • A simple formatter which will prefix our log message with the time and the log level. It is called ‘simple’ and is assigned to the console logger.
  • A verbose formatter which will prefix our log message with the level, time, module, process id and thread id. It is called verbose and is assigned to the file logger.

The loggers or handlers are then assigned in the loggers section to our helloworld app.

#settings.py

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        },
        'simple': {
            'format': '%(asctime)s %(levelname)s %(message)s'
        }
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'simple'
        },
        'file': {
            'level': 'ERROR',
            'class': 'logging.FileHandler',
            'filename': 'debug.log',
            'formatter': 'verbose'
        },
    },
    'loggers': {
        'helloworld': {
            'handlers': ['console', 'file'],
            'level': 'WARNING',
            'propagate': True,
        },
    },
}

The following is a test view to check our custom Django log config is working. We throw a log message of each logging level at the logger as well as a trace stack and an exception.

#views.py

import logging
from django.http import HttpResponse

logger = logging.getLogger(__name__)

def with_logger(request):
    logger.debug('This is some debug')
    logger.info('This is information')
    logger.warning('This is a warning')
    logger.error('This is a error')
    logger.critical('This is a critical')

    logger.critical("Output with trace", exc_info=True)

    try:
        boom = 1 / 0
        logger.debug("Looks like we have broke the compiler as 1/0 = {0}".format(boom))
    except ZeroDivisionError as e:
        logger.exception(e)

    logger.critical('Final output to show we are logging still')

    return HttpResponse("Logged output")

The logging.getLogger function call passes in the name of the file; this ensures that we will only ever use one logger instance for any view function within this file; as long as they use the logger returned into the logger variable.

We can hook this view into our URL routing config.

#urls.py

from django.conf.urls import patterns, url

from . import views

urlpatterns = 
    patterns('',
             url(r'^logger/$', views.with_logger, name="logger")
             )

We hook into our errorsandlogging application into our project URL routing config:

# DjangoSandBox/urls.py
from django.conf.urls import patterns, include, url

from django.contrib import admin

from errorsandloggingimport urls as errorsandlogging_urls

urlpatterns = 
    patterns('',
url(r'^errorsandlogging/', include(errorsandlogging_urls, namespace="errorsandlogging")),
             )

To test run the development server and navigate to:

http://127.0.0.1:8000/errorsandlogging/logger/

References

Python: Parameter Scope

This is part of my Python & Django Series which can be found here including information on how to download all the source code.

Variables can be local or global.

Global Variables exist at the top level; i.e they are not inside a class or a method. They are implicitly global.

Local Variables are any variables inside a method. They will shadow a variable with the same name in an outer scope, i.e the calling method or class class etc. Changes to a local variable does not affect any variables which they shadow.

Global keyword is used to import a global variable. It allows a global variable to be used and set inside a method. It can only import global variables and not outer scope variables such as those within an outer method or class.

Nonlocal keyword is similar to global but will import any variables which are in an outer scope regardless if they are global or not global.

The following code and output can be used to demonstrate the usage of the above concepts.

Code:

def print_scope_msg(a_msg):
    print("Inside scope_test:", a_msg)

msg = "Initial"

def scope_test():

    def local_test():
        # msg is local. Changes are only existing inside the method
        msg = "local"
        print("Inside local_test():", msg)

    def nonlocal_test():
        # msg is non local. This allows the msg inside scope_test() to be imported and edited
        nonlocal msg
        msg = "nonlocal"
        print("Inside nonlocal_test:", msg)

    def global_test():
        # msg is global. This allows the msg inside global but not scope_test() to be imported and edited
        # Changes to msg affect the global msg but not the scope_test (it skips it)
        global msg
        msg = "global"
        print("Inside global_test: ", msg)

    msg = "scope"

    local_test()
    print_scope_msg(msg)

    nonlocal_test()
    print_scope_msg(msg)

    global_test()
    print_scope_msg(msg)

scope_test()
print("Inside global:", msg)

Output:

1: Inside local_test(): local
2: Inside scope_test: scope
3: Inside nonlocal_test: nonlocal
4: Inside scope_test: nonlocal
5: Inside global_test: global
6: Inside scope_test: nonlocal
7: Inside global: global

So what happened? Each numbered output above has an explanation as described below.

  1. Reports “local” as we have changed msg to local.
  2. Reports “scope” as the change inside local_test() was made to a shadowed version of msg which does not affect the value of msg inside scope_test().
  3. Reports “nonlocal” as we have changed msg to nonlocal.
  4. Reports “nonlocal” as inside nonlocal_test() we imported msg with the nonlocal keyword before changing the value.
  5. Reports “globaltest” as we have changed msg to globaltest.
  6. Reports “nonlocal” as the change to msg in global_test() used the global keyword which imported the global reference of msg and not that defined within scope_test() where we are now.
  7. Reports “globaltest” as the change to msg in global_test() used the global keyword.

Django Authorisation

Authorisation

This post is part of my Django series. You can see an overview of the series along with instruction on how to get all the source code here.

This article assumes you are comfortable creating a Django project, can create apps and register them into the INSTALLED_APPS list of the settings file. If not please read my Django HelloWorld article.

This article assumes you have a project called DjangoSandBox in an application called authmodel.

Prerequisites

Django provides authentication functionality through apps and middle ware.

Within the INSTALLED_APPS list of the settings.py add the following:

  • ‘django.contrib.auth’
  • ‘django.contrib.contenttypes’

Within the MIDDLEWARE_CLASSES list of the settings.py add the following:

  • SessionMiddleware
  • AuthenticationMiddleware
  • SessionAuthenticationMiddleware

If they were not added or you are starting a new project ensure to run migrate to create the required database schema and run any required configuration scripts:

python manage.py migrate

**Note: ** by default these apps and middle ware settings are already added into the settings.py

The User Model

By default the user class is django.contrib.auth.models.User and is found within the auth_user table. It contains many fields including the following; username, password, email, first_name, last_name

To create a user:

new_user = User.objects.create_user(username="user_login", email="foo@email.com", password="password", first_name="first_name", last_name="last_name")

By default the user is not created as active; it is presumed we would like some type of email confirmation from the user before activating the account. We can make the user active with the following code:

new_user.is_active = True
new_user.save()

We can reset their password with the set_password function. It is important that we use this method and not set the field directly to ensure their password is encrypted. Setting the field directly would mean their password would be stored in plain text which would not be secure.

new_user.set_password("new_pw")
new_user.save()

Logging in a user is a two step process. Initially we need to ensure that their username and password are valid, if this is the case then we log them in with the login function.

user = authenticate(username=username, password=password)

if user:
    login(request, user)

Django handles everything else; we simply need to tell Django which view functions require an authenticated user.

Django uses sessions to add the authentication into the request object.

The user template variable or the request.user property can be used to access the logged in user. If the user is not logged in then an instance of AnonymousUser is returned. The function user.is_authenticated() can be used to determine if the user is logged in:

if request.user.is_authenticated():
    # Authenticated 
else:
    # Anonymous 

Enforcing Authorisation

First lets make a view with some sensitive information. The following returns all user records:

#views.py

from django.contrib.auth.models import User
from django.shortcuts import render

def users(request):
    return render(request, 'authmodel/users.html', {"users": User.objects.all()})

To enforce a user to log in we wrap the view function with the login_required function in the URL routing config of our authmodel application:

# urls.py

from django.conf.urls import patterns, url
from django.contrib.auth.decorators import login_required

from . import views

url(r'^users/$', login_required(views.users), name="users"),

We need to hook in our authmodel application URL routing config into our project:

from django.conf.urls import patterns, include, url
from django.contrib import admin

from authmodel import urls as authmodel_urls

urlpatterns = 
    patterns('',
             url(r'^auth/', include(authmodel_urls, namespace="auth")),
             )

Now if a user tries to access http://127.0.0.1:8000/auth/users and they are not logged in they are redirected to the defined login URL which we set in the LOGIN_URL variable of our settings.py. We will fix this later on.

An alternative to using the login_require function in our URL routing config is to decorate our view function with the @login_required decorator:

#views.py

from django.contrib.auth.decorators import login_required
from django.contrib.auth.models import User
from django.shortcuts import render

from .forms import UserForm, UserLoginForm

@login_required
def users(request):
    return render(request, 'authmodel/users.html', {"users": User.objects.all()})

We can now remove the login_require function wrap within our url routing config:

#urls.py

from django.conf.urls import patterns, url

from . import views

urlpatterns = 
    patterns('',
             url(r'^users/$', views.users, name="users"),
             )

Here is our view which displays information about the users passed in to the context data as the template variable named users:

<!--templates/authmodel/users.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Registered Users</title>
</head>
<body>
<h2>Registered Users</h2>

<table>
    <tr>
        <th>First Name</th>
        <th>Last Name</th>
        <th>Email</th>
        <th>User Name</th>
    </tr>
    {% for user in users %}
        <tr>
            <td>{{ user.first_name }}</td>
            <td>{{ user.last_name }}</td>
            <td>{{ user.email }}</td>
            <td>{{ user.username }}</td>
        </tr>
    {% endfor %}
</table>

<span>
    <a href="{% url 'auth:index' %}">Back</a>
</span>
</body>
</html>

We now need to create pages for logging in and out as well as the ability to create a new login.

Create Login

We are going to create a login form which matches the data we want to collect from the user when they create a login.

We override the clean function of the form to validate that the password and the password_confirm are identical.

forms.py
from django.forms import Form, CharField, EmailField, PasswordInput, ValidationError

class UserForm(Form):
    first_name = CharField(max_length=30)
    last_name = CharField(max_length=30)
    login = CharField()
    email = EmailField()
    password = CharField(widget=PasswordInput)
    password_confirm = CharField(widget=PasswordInput)

    def clean(self):
        super(UserForm, self).clean()

        password = self.cleaned_data.get('password')
        password_confirm = self.cleaned_data.get('password_confirm')

        if password and password_confirm and password != password_confirm:
            raise ValidationError("Passwords do not match")

        return self.cleaned_data

We now hook this form into a function view; here we create a new user after the form has been posted and validates correctly.

Once we create the user we activate them. In a live system we might want to think about sending and receiving a confirmation email before we activate the user.

#views.py
def create_login(request):
    form = UserForm(request.POST or None)

    if request.POST and form.is_valid():
        email = form.cleaned_data['email']
        user_login = form.cleaned_data['login']
        password = form.cleaned_data['password']
        last_name = form.cleaned_data['last_name']
        first_name = form.cleaned_data['first_name']

        new_user = User.objects.create_user(username=user_login, email=email, password=password, first_name=first_name, last_name=last_name)

        new_user.is_active = True
        new_user.save()   

        return redirect(reverse('auth:index'))

    return render(request, 'authmodel/create_login.html', {'form': form})

A simple template which renders the required inputs via the form we have created above.

<!-- templates/authmodel/create_login.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Create Login</title>
</head>
<body>
<h1>Create Login</h1>

<form method="post" action="">
    {% csrf_token %}
    <table>{{ form.as_table }}</table>
    <p><input type="submit" value="Create"/></p>
</form>

<span>
    <a href="{% url 'auth:index' %}">Back</a>
</span>
</body>
</html>

Adding our new function into the url routing config:

#urls.py

from django.conf.urls import patterns, url

from . import views

urlpatterns = 
    patterns('',
             url(r'^login/create/$', views.create_login, name="login_create")
             )

To test run the development server and navigate to:

http://127.0.0.1:8000/auth/login/create/

Login

Our login page simply requires a text box for the username and password along with the ability to post this data back to the server.

First we create a form to represent the data required; username and password:

#forms.py
from django.forms import Form, CharField, EmailField, PasswordInput, ValidationError

class UserLoginForm(Form):
    login = CharField()
    password = CharField(widget=PasswordInput)

We then create a view function around the form we have just created.

If Django redirected the user to this page upon accessing a page which required authentication; the original URL will be available via the NEXT get query parameter. We will redirect the user back to this page if applicable.

#views.py

from django.contrib.auth import authenticate, login
from django.shortcuts import render, redirect

from .forms import UserLoginForm

def login_page(request):
    form = UserLoginForm(request.POST or None)

    if request.POST and form.is_valid():
        username = form.cleaned_data["login"]
        password = form.cleaned_data["password"]

        user = authenticate(username=username, password=password)

        if user:
            login(request, user)

            if request.GET.get('next') is not None:
                return redirect(request.GET['next'])
            else:
                return redirect(reverse('auth:index'))

    return render(request, 'authmodel/login.html', {'form': form})

Now we create the template which renders the user login form.

When a user is logged into any page we have access to their user details via the template variable called user. We use the user.is_authenticated property to determine if someone is logged in already.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Login</title>
    <style>
        span {
            display: block;
        }
    </style>
</head>
<body>

<h1>Login</h1>

{% if user.is_authenticated %}
    <span>{{ user.first_name }} {{ user.last_name }} you are already logged in!</span>
{% else %}
    <form method="post" action="">
        {% csrf_token %}
        <table>{{ form.as_table }}</table>
        <input type="submit" class="button" value="Login"/>
    </form>
{% endif %}

<span>
    <a href="{% url 'auth:index' %}">Back</a>
</span>
</body>
</html>

Adding in our new function into our URL routing config:

#urls.py

from django.conf.urls import patterns, url

from . import views

urlpatterns = 
    patterns('',
             url(r'^login/$', views.login_page, name="login"),
             )

Add our login URL into our settings file. Any attempt to access a page which requires authorisation will redirect here automatically if a user is not logged in.

#settings.py
LOGIN_URL = '/auth/login/'

To test run the development server and navigate to:

http://127.0.0.1:8000/auth/login/

Logout

We can log a user out by passing the request object to the logout function. We do this inside a view function:

#views.py

from django.contrib.auth import logout
from django.shortcuts import render


def logout_page(request):
    logout(request)
    return render(request, 'authmodel/logout.html')

A simple page which confirms to the user that they are now logged out:

<!--templates/authmodel/logout.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Logout</title>
    <style>
        span {
            display: block;
        }
    </style>

</head>
<body>
<h1>Logout</h1>

<span>You are now logged out</span>

<span>
    <a href="{% url 'auth:index' %}">Back</a>
</span>
</body>
</html>

Hooking in our logout page into our URL routing config:

#urls.py

from django.conf.urls import patterns, url

from . import views

urlpatterns = 
    patterns('',
             url(r'^logout/$', views.logout_page, name="logout"),
             )

To test run the development server and navigate to:

http://127.0.0.1:8000/auth/login/logout/

Extending User

If the default User profile is not suitable for your purposes there are a couple options.

Firstly you can create your own table with the additional fields and create a join to the existing user table, accessing it via ettings.AUTH_USER_MODEL:

# models.py
user = models.OneToOneField(settings.AUTH_USER_MODEL)

Alternatively you can subclass either AbstractUser or BaseUserManager. If you go down this route you will need to register the new table as replacing the default Djang Auth.User model table:

#settings.py
AUTH_USER_MODEL = "authapp.NewAuthUser"

References

https://docs.djangoproject.com/en/1.8/topics/auth/
https://docs.djangoproject.com/en/1.8/topics/auth/default/
https://docs.djangoproject.com/en/1.8/topics/auth/default/#permissions-and-authorization

Django Forms

This post is part of my Django series. You can see an overview of the series along with instruction on how to get all the source code here.

This article assumes you are comfortable creating a Django project, can create apps and register them into the INSTALLED_APPS list of the settings file. If not please read my Django HelloWorld article.

This article assumes you have a project called DjangoSandBox in an application called formsintroduction.

A Django Form instance automatically provides all the heavy work when working with HTTP forms; rendering UI, validation, cleaning and collection of data.

Basic Form

A basic Django Form is a class which inherits from forms.Form. It defines fields and looks similar to a Django model.

The following example defines a form which has a name and height field.

# forms.py
from django import forms

class BasicFormExample(forms.Form):
    name = forms.CharField(label="Name", min_length=3, max_length=30)

    height = forms.DecimalField(max_value=2, help_text="Height in meters", label="Height (M.CM)", decimal_places=2)  

We can now use an instance of BasicFormExample within a function view; we pass it to the context object dictionary with a key of ‘form’.

When posting back we create an instance of the form initiated with the POST data. We can then ask the form if it is valid; this returns false if the model or additional form validation (which we will add later) is invalid.

We can ask for cleaned_data of the fields from the form; this gets us validated data fields. In this example we simply collect the data and pass them into a HttpResponse instance.

If the form is not considered valid we pass the form instance back to the request object.

#views/py

from django.http import HttpResponse
from django.shortcuts import render

from .forms.bastic_form_example import BasicFormExample

def basic(request):
    if request.POST:
        form = BasicFormExample(request.POST)
        if form.is_valid():
            name = form.cleaned_data['name']
            height = form.cleaned_data['height']

            return HttpResponse("{0} is {1} tall".format(name, height))
        else:
            return render(request, "formsintroduction/basic_form_example.html", {'form': form})
    else:
        form = BasicFormExample()
        return render(request, 'formsintroduction/basic_form_example.html', {'form': form})

For our template we simply need to ask for the form to render itself. We use as_table function upon the form instance passed into the template context data.

Django provides the functions as_table, as_ul and as_p; this simply defines which html element to surrounds the rendered input element; TD, UI or P.

We also need to add a csrf_token to ensure we are protected from Cross-site request forgery attacks. When working with forms ‘{% csrf_token %}’ should always be included on the page; as long as we include this Django will take care of the rest.

We also add a button to submit the form.

<!-- templates/formsintroduction/basic_form_example.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title></title>
</head>
<body>
<form method="post" action="">
    {% csrf_token %}
    <table>
        {{ form.as_table }}
    </table>
    <p><input type="submit" value="Create"/></p>
</form>
</body>
</html>

We now hook in our view function into our URL routing config.

#formsintroduction/urls.py
from django.conf.urls import patterns, url

from . import views

urlpatterns = 
    patterns('',
url(r'^basic/$', views.basic, name="form_basic"),

We need to make sure that our app’s urls.py is configured within the project urls.py. I was working with an app called formsintroduction inside a project called DjangoSandBox.

#DjangoSandBox/urls.py
from django.conf.urls import patterns, include, url

from formsintroduction import urls as formsintroduction_urls

urlpatterns = 
    patterns('',
              url(r'^formsintroduction/', include(formsintroduction_urls, namespace="formsintroduction")),
             )

To test the page run the development server and navigate to:

http://127.0.0.1:8000/formsintroduction/basic/

Validation

This assumes that you have an understanding of adding validation onto Django models. If not you can read about this in my post here.

We automatically get server and client side validation for our model. For the example above this would ensure the name is of a minimum 3 characters and a maximum of 30. For the height it ensures a maximum value of 2 and a maximum number of two decimal places.

In short the constraints and validation that we place upon our model are automatically validated against in our form.

We can add additional validation onto our form.

Note: Where possible validation should be placed upon the model to support code reuse. Only bespoke validation for the page should be added directly onto the form.

Lets first write a custom validation function for our name field. We want to ensure that it starts with an upper case letter and then has at least 2 lower case letters. Upon failure we raise a django.core.exceptions.ValidationError.

# validators.py 
from re import match

from django.core.exceptions import ValidationError

def validate_name(a_string):
    a_match = match(r"[A-Z][a-z]{2,}$", a_string)
    if a_match:
        return True

    raise ValidationError("Name must be a capitalised single word")

We now change our form to look like the following:

# forms/basic_form_example.py
from django import forms
from ..validators import validate_name

error_name = {
    'required': 'You must enter a name!',
    'invalid': 'Invalid name format.'
}


class BasicFormExample(forms.Form):
    name = forms.CharField(label="Name", min_length=3, max_length=30, error_messages=error_name,
                           initial="Jim", validators=[validate_name])

    height = forms.DecimalField(max_value=2, help_text="Height in meters", label="Height (M.CM)", decimal_places=2)

    def clean(self):
        cleaned_data = super(BasicFormExample, self).clean()

        name = cleaned_data.get("name")
        height = cleaned_data.get("height")

        if name is None and height is None:
            msg = "Both name and height are required!!!"
            self.add_error('name', msg)
            self.add_error('height', msg)
            raise forms.ValidationError(msg)

        return self.cleaned_data

    def clean_name(self):
        name = self.cleaned_data.get("name")

        if " " in name:
            raise forms.ValidationError("No spaces in the name please")

        return name

In the example above:

  • Field.validators takes a list of custom validators or inbuilt Django validators. These have been covered in my post about Django Model Validation.
  • Field.error_messages takes a dictionary of error key to error message. We use it to provide custom error messages to the existing validation. This is the same funcitonality as per models. This has been covered in my post about Django Model Validation.
  • The clean_field provides a hook to place specific field validation. The above is a bad example as it could easily be added as a validator against the model or form field using regular expressions; however this is only an example.
    • Raising a ValidationError here automatically associates the error to the field
  • The clean function provides an additional hook to perform cross field validation or anything else which might not be applicable or possible anywhere else.
    • We can add validation messages against a field with the add_error function.
    • We can raise a general validation message against the form by raising a ValidationError.

Note: Validation error messages are associated to a field or their form. the association affects where they are displayed to the user. Field errors are displayed next to the field they arose from, form errors are displayed above all fields. These have been covered in my post about Django Model Validation.

To test the page run the development server and navigate to:

http://127.0.0.1:8000/formsintroduction/basic/

Basic Form With HTML

In the above example we use the form passed into the template context data to render itself. If we need more control over the placement or composition of the html there is nothing stopping us from writing the html ourselves.

This example renders the form using html and Django template operators.

<!-- templates/formsintroduction/basic_form_with_html_example.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title></title>
</head>
<body>
<form method="post" action="">
    {% csrf_token %}
    {{ form.non_field_errors }}
    <div class="fieldWrapper">
        {{ form.message.errors }}
        <label for="{{ form.name.id_for_label }}">Name:</label>
        {{ form.name }}
    </div>
    <div class="fieldWrapper">
        {{ form.height.errors }}
        {{ form.height.label_tag }}
        {{ form.height }}
    </div>
    <p><input type="submit" value="Create"/></p>
</form>
</body>
</html>

We have access to some properties on the form which come in handy when building a working form page:

  • The form.name and form.height renders the applicable input for the fields name and height respectively;. This includes any client side validation.
  • The form.height.errors and form.name.errors renders the fields errors when and if applicable.
  • The form.name.id_for_label reutrns the id to be used for the name field label.
  • The form.height.label_tag is used to render the label for the height column instead of constructing it ourselves with the id_for_label property.
  • The form.non_field_errors returns true or false depending upon if we have any validation errors which are not related directly to a field.
  • The form.message.errors renders all non field validation error messages.

We can now hook in our new template into what is a virtual copy of the view function called in the example above.

#views.py

from django.http import HttpResponse
from django.shortcuts import render

from .forms.class_form_example import ClassBasedForm
def basic_html(request):
    if request.POST:
        form = BasicFormExample(request.POST)
        if form.is_valid():
            name = form.cleaned_data['name']
            height = form.cleaned_data['height']

            return HttpResponse("{0} is {1} tall".format(name, height))
        else:
            return render(request, "formsintroduction/basic_form_with_html_example.html", {'form': form})
    else:
        form = BasicFormExample()
        return render(request, 'formsintroduction/basic_form_with_html_example.html', {'form': form})

Hook in our new view into our URL route config:

url(r'^basichtml/$', views.basic_html, name="form_basic_html"),

To test the page run the development server and navigate to:

http://127.0.0.1:8000/formsintroduction/basichtml/

Class Form

If we have a model class we can automatically generate a form from our model definition.

First our models; we are going to resuse the PhoneAddress and PhoneContact from the previous article when we looked at views. They sit in the viewsintroduction application.

# viewsintroduction/models.py
class PhoneAddress(Model):
    number = models.IntegerField()
    street_name = models.CharField(max_length=20)
    city = models.CharField(max_length=20)

    def __str__(self):
        return "{0} {1} {2}".format(self.number, self.street_name, self.city)

    def get_absolute_url(self):
        return reverse('viewsintroduction:address', args=[self.id])

We can create a form based upon our model by simply inheriting from ModleForm and setting the model property of the Meta internal class.

We also define the fields property which defines which fields of the model are going to be displayed; this is mandatory.

#forms/clased_based_form.py

.from django import forms
from django.forms import ModelForm

from viewsintroduction.models import PhoneAddress

class ClassBasedForm(ModelForm):        
    class Meta:
        model = PhoneAddress
        fields = ("city", "street_name", "number")
        labels = {'number': "House No."}
        help_texts = {'number': "This is the number of the house."}

With regards to validation:

  • We can implement the clean and clean_field functions
  • We can add custom validation messages via the error_messages property of the class meta. This takes a dictionary keyed upon each field name, which in itself takes a dictionary keyed upon each error type and the custom error message.
  • We can override the init function and set any settings as required; for example adding a custom validator onto a field.

A fuller example adding in validation and more meta information:

#forms/clased_based_form.py

from django import forms
from django.forms import ModelForm

from viewsintroduction.models import PhoneAddress
from ..validators import validate_name

class ClassBasedForm(ModelForm):
    def __init__(self, *args, **kwargs):
        super(ClassBasedForm, self).__init__(*args, **kwargs)
        self.fields["city"].validators.append(validate_name)

    def clean(self):
        cleaned_data = super(ClassBasedForm, self).clean()

        city = cleaned_data.get("city")
        street_name = cleaned_data.get("street_name")
        number = cleaned_data.get("number")

        if city is None and street_name is None and number is None:
            msg = "None of the fields have been set!!!"
            for a_field in ('city', 'street_name', 'number'):
                self.add_error(a_field, msg)
            raise forms.ValidationError(msg)

        return self.cleaned_data

    def clean_city(self):
        city = self.cleaned_data.get("city")

        if " " in city:
            raise forms.ValidationError("No spaces in the city name please")

        return city

    class Meta:
        model = PhoneAddress
        fields = ("city", "street_name", "number")
        labels = {'number': "House No."}
        help_texts = {'number': "This is the number of the house."}
        error_messages = {
            'number': {
                'required': "We need a number of the house!",
                'max_length': "We only accept houses up to 20!"}
        }

Our view function looks similar to previous examples with a few changes:

  • Our function view is going to be passed in a parameter called pk which defaults to none. We will strip this from the URL within our URL routing config shortly.
  • We use get_object_or_404 to return the record. It is passed the primary key value and the model class. It will return the record or raise a 404 HTTP response if no record exists.
  • We need to initiate the form with the post data if we are posting back otherwise the record instance. This is done with request.POST or None and instance=an_address being passed into the form constructor. If both are not set then we initiate the form from no data; i.e. the starting point.
  • Calling form.save will automatically create or update the address record.
  • Our address record knows its own URL via the get_absolute_url function. We can simply call redirect upon the record to redirect to the detail view we made in the previous article.
# views.py
from django.http import HttpResponse
from django.shortcuts import render, redirect, get_object_or_404

from .forms.class_form_example import ClassBasedForm
from viewsintroduction.models import PhoneAddress

def class_form(request, pk=None):
    if pk:
        an_address = get_object_or_404(PhoneAddress, pk=pk)
        form = ClassBasedForm(request.POST or None, instance=an_address)
    else:
        form = ClassBasedForm(request.POST or None, initial={'city': 'Plymouth'})

    if request.POST and form.is_valid():
        an_address = form.save(commit=True)
        return redirect(an_address)
    else:
        return render(request, 'formsintroduction/class_based_form_example.html', {'form': form})

In our template we will simply get our form to render itself:

<!-- templates/formsintroduction/class_based_form_example.html>
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title></title>
</head>
<body>
<form method="post" action="">
    {% csrf_token %}
    <table>
        {{ form.as_table }}
    </table>
    <p><input type="submit" value="Create"/></p>
</form>
</body>
</html>

We now hook in our view function with two a create and edit URL:

#urls.py

url(r'^class/create/$', views.class_form, name="form_class_create"),
url(r'^class/(?P<pk>[0-9]+)/edit/$', views.class_form, name="form_class_edit"),

With regards to the edit URL matching. We match the URL class/edit/xxx/ where xxx is the primary key. The primary key is made up of one or more numerical digits as noted by [0-9]+. We assign the integral into a variable called pk. The string ?P reads make a parameter for reference later on.

To test the page run the development server and navigate to:

http://127.0.0.1:8000/formsintroduction/class/create
http://127.0.0.1:8000/formsintroduction/class/1/edit

Note: replace 1 with the id of your record

Model Form Factory

If we don’t want to customise the class based form we can use the modelform_factory class.

#views.py

from django.forms import modelform_factory
from django.shortcuts import render, redirect, get_object_or_404

from viewsintroduction.models import PhoneAddress
#views.py

def model_form_factory_form(request, pk=None):
    phone_address_form = modelform_factory(PhoneAddress, fields=("city", "street_name", "number"))

    if pk:
        an_address = get_object_or_404(PhoneAddress, pk=pk)
        form = phone_address_form(request.POST or None, instance=an_address)
    else:
        form = phone_address_form(request.POST or None)

    if request.POST and form.is_valid():
        an_address = form.save(commit=True)
        return redirect(an_address)
    else:
        return render(request, 'formsintroduction/form_factory_example.html', {'form': form})

A simple template which asks the form to render itself.

<!-- templates/formsintroduction/form_factory_example.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Form Factory Example</title>
</head>
<body>
<form method="post" action="">
    {% csrf_token %}
    <table>
        {{ form.as_table }}
    </table>
    <p><input type="submit" value="Create"/></p>
</form>
</body>
</html>

Again we set us two URLs; one for creating and one for editing.

#urls.py
url(r'^factory/create/$', views.model_form_factory_form, name="form_factory_create"),
url(r'^factory/(?P<pk>[0-9]+)/edit/$', views.model_form_factory_form, name="form_factory_edit"),

To test the page run the development server and navigate to:

http://127.0.0.1:8000/formsintroduction/factory/create
http://127.0.0.1:8000/formsintroduction/factory/1/edit

Note: replace 1 with the id of your record

Widgets

Widgets provide extra customisation onto our fields; for example we can add a PasswordInput widget onto a CharField to give us a password input box which displays * instead of the password.

PasswordInput = forms.CharField(max_length=10, widget=forms.PasswordInput)

We can can add a Textarea onto a CharField to allow multi-line text input.

TextField = forms.CharField(widget=forms.Textarea)

We can render an input as hidden:

HiddenInput = forms.CharField(max_length=10, widget=forms.HiddenInput, initial='a')

The normal choice field be be changed to radios or a multi select list box:

TITLE_CHOICES = (
    ('MR', 'Mr.'),
    ('MRS', 'Mrs.'),
    ('MS', 'Ms.'),
)

# Drop Down
ChoiceField = forms.CharField(max_length=3, widget=forms.Select(choices=TITLE_CHOICES))

# Radio
RadioSelect = forms.CharField(max_length=10, widget=forms.RadioSelect(choices=TITLE_CHOICES))

# Multiple display and select
CheckboxSelectMultiple = forms.CharField(max_length=10, widget=forms.CheckboxSelectMultiple(choices=TITLE_CHOICES))

We can create a date selection widget limited to certain years or months:

YEARS_ = years = (2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009)

SelectDateWidget = forms.CharField(max_length=10, widget=SelectDateWidget(years=YEARS_))

A lot of the widgets can take parameters for extra customisation. Check the Django documentation for available widgets and their options.

We provide a working example for all of these widgets in the “A Full Example” section below.

Joins

Joins fields for 1:1 and 1:n are implemented by a ModelChoiceField form field defining the possible records to be joined to with the queryset parameter.

ForeignKey = forms.ModelChoiceField(queryset=ChildOfMany.objects.all())

OneToOneField = forms.ModelChoiceField(queryset=ChildOfOne.objects.all())

Many to many join fields are implemented in the same way but with a ModelMultipleChoiceField form field.

ManyToManyField = forms.ModelMultipleChoiceField(queryset=ChildManyToMany.objects.all())

We provide a working example for all of these in the “A Full Example” section below.

A Full Example

For every model field there is a recommended form field and as such editor and potentially a widget. The following is an example with “one of virtually everything”.

Lets take a model which has a field of virtually every type including joins of all types; 1:1, 1:n and 1:m.

#models.py
from django.db import models
from django.db.models import Model

TITLE_CHOICES = (
    ('MR', 'Mr.'),
    ('MRS', 'Mrs.'),
    ('MS', 'Ms.'),
)


class ChildOfMany(Model):
    CharField = models.CharField(max_length=10)

    def __str__(self):
        return self.CharField


class ChildOfOne(Model):
    CharField = models.CharField(max_length=10)

    def __str__(self):
        return self.CharField


class ChildManyToMany(Model):
    CharField = models.CharField(max_length=10)

    def __str__(self):
        return self.CharField

    class Meta:
        verbose_name_plural = "children"


class ModelFieldsToFormFields(Model):
    ChoiceField = models.CharField(max_length=3, choices=TITLE_CHOICES)
    CharField = models.CharField(max_length=10)
    CommaSeparatedIntegerField = models.CharField(max_length=50)
    EmailField = models.EmailField()
    TextField = models.TextField()
    URLField = models.URLField()

    DateField = models.DateField()
    DateTimeField = models.DateTimeField()
    TimeField = models.TimeField()

    # FileField = models.FileField()
    # ImageField = models.ImageField()
    # FilePathField = models.FilePathField()

    BigIntegerField = models.BigIntegerField()
    BooleanField = models.BooleanField()
    NullBooleanField = models.NullBooleanField()

    PositiveIntegerField = models.PositiveIntegerField()
    PositiveSmallIntegerField = models.PositiveSmallIntegerField()
    SlugField = models.SlugField()
    SmallIntegerField = models.SmallIntegerField()
    DecimalField = models.DecimalField(decimal_places=2, max_digits=5)
    FloatField = models.FloatField()
    IntegerField = models.IntegerField()
    GenericIPAddressField = models.GenericIPAddressField()

    # Joins
    ForeignKey = models.ForeignKey(ChildOfMany)
    ManyToManyField = models.ManyToManyField(ChildManyToMany)
    OneToOneField = models.OneToOneField(ChildOfOne)

Our form, using Django advised mappings of form field and widgets to model fields will then look like this:

# forms/complex_form_with_widgets.py

from django import forms
from django.forms.extras.widgets import SelectDateWidget

from ..models import TITLE_CHOICES, ChildOfOne, ChildManyToMany, ChildOfMany, ModelFieldsToFormFields

YEARS_ = years = (2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009)


class ComplexFormWithWidgets(forms.ModelForm):
    ChoiceField = forms.CharField(max_length=3, widget=forms.Select(choices=TITLE_CHOICES))
    CharField = forms.CharField(max_length=10)
    CommaSeparatedIntegerField = forms.CharField()
    EmailField = forms.EmailField()
    TextField = forms.CharField(widget=forms.Textarea)
    URLField = forms.URLField()

    DateField = forms.DateField()
    DateTimeField = forms.DateTimeField()
    TimeField = forms.TimeField()

    # FileField = models.FileField()
    # ImageField = models.ImageField()
    # FilePathField = forms.FilePathField((match="*.py", recursive=True)

    BigIntegerField = forms.IntegerField(min_value=-9223372036854775808, max_value=9223372036854775807.)
    BooleanField = forms.BooleanField()
    NullBooleanField = forms.NullBooleanField()

    PositiveIntegerField = forms.IntegerField(min_value=0, max_value=2147483647)
    PositiveSmallIntegerField = forms.IntegerField(min_value=0, max_value=32767)
    SlugField = forms.SlugField()
    SmallIntegerField = forms.IntegerField(min_value=-32768, max_value=32767)
    DecimalField = forms.DecimalField()
    FloatField = forms.FloatField()
    IntegerField = forms.IntegerField()
    GenericIPAddressField = forms.GenericIPAddressField()

    # Joins
    ForeignKey = forms.ModelChoiceField(queryset=ChildOfMany.objects.all())
    ManyToManyField = forms.ModelMultipleChoiceField(queryset=ChildManyToMany.objects.all())
    OneToOneField = forms.ModelChoiceField(queryset=ChildOfOne.objects.all())

    # More Widgets
    PasswordInput = forms.CharField(max_length=10, widget=forms.PasswordInput)
    HiddenInput = forms.CharField(max_length=10, widget=forms.HiddenInput, initial='a')
    RadioSelect = forms.CharField(max_length=10, widget=forms.RadioSelect(choices=TITLE_CHOICES))
    CheckboxSelectMultiple = forms.CharField(max_length=10, widget=forms.CheckboxSelectMultiple(choices=TITLE_CHOICES))
    SelectDateWidget = forms.CharField(max_length=10, widget=SelectDateWidget(years=YEARS_))

    class Meta:
        model = ModelFieldsToFormFields
        exclude = ()  # Better to set fields explicitly

There is no real change in our view function from the model form example previously.

#views.py

from django.http import HttpResponse
from django.shortcuts import render, redirect, get_object_or_404

from .forms.complex_form_with_widgets import ComplexFormWithWidgets
from .models import ModelFieldsToFormFields
def complete_model_example(request, pk=None):
    if pk:
        an_instance = get_object_or_404(ModelFieldsToFormFields, pk=pk)
        form = ComplexFormWithWidgets(request.POST or None, instance=an_instance)
    else:
        form = ComplexFormWithWidgets(request.POST or None)

    if request.POST and form.is_valid():
        if form.is_valid():
            an_instance = form.save(commit=True)
            return HttpResponse("Created with id={0}".format(an_instance.id))
    else:
        return render(request, 'formsintroduction/complete_model_form_example.html', {'form': form})

A simple html template asking the form to render itself:

<!-- templates/formsintroduction/complete_model_form_example.html-->

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title></title>
</head>
<body>
<form method="post" action="">
    {% csrf_token %}
    <table>
        {{ form.as_table }}
    </table>
    <p><input type="submit" value="Create"/></p>
</form>
</body>
</html>

Hook in our view to a create and edit URL within our URL routing config:

# urls.py
url(r'^complete_model/create/$', views.complete_model_example, name="form_complete_create"),
url(r'^complete_model/(?P<pk>[0-9]+)/edit/$', views.complete_model_example, name="form_complete_edit")

To test the page run the development server and navigate to:

http://127.0.0.1:8000/formsintroduction/complete_model/create
http://127.0.0.1:8000/formsintroduction/complete_model/1/edit

Note: replace 1 with the id of your record

Additional Notes

We can pass data to the HTML via the attrs parameter which takes a dictionary of property to name. The following makes our char field take on a CSS class name.

CharField = forms.CharField(max_length=10, attrs={'class': 'special'})

References

Django Views

This post is part of my Django series. You can see an overview of the series along with instruction on how to get all the source code here.

This article assumes you are comfortable creating a Django project, can create apps and register them into the INSTALLED_APPS list of the settings file. If not please read my Django HelloWorld article.

This article assumes you have a project called DjangoSandBox and an application called viewsintroduction.

Django Views provide out of the box functionality for displaying and CRUD ( Create, Update, Delete) of a model.

The Models

First we need a couple models to work upon. We have a simple model of a Contact and Address which are joined with a one to one relationship.

#formsintroduction/models.py
from django.db import models
from django.db.models import Model
from django.core.urlresolvers import reverse

OPTION_SEX_MALE = 'M'
OPTION_SEX_FEMALE = 'F'
OPTION_SEX_MALE_DESCRIPTION = 'MALE'
OPTION_SEX_FEMALE_DESCRIPTION = 'FEMALE'

OPTIONS_SEX = (
    (OPTION_SEX_MALE, OPTION_SEX_MALE_DESCRIPTION),
    (OPTION_SEX_FEMALE, OPTION_SEX_FEMALE_DESCRIPTION)
)


class PhoneAddress(Model):
    number = models.IntegerField()
    street_name = models.CharField(max_length=20)
    city = models.CharField(max_length=20)

    def __str__(self):
        return "{0} {1} {2}".format(self.number, self.street_name, self.city)

    def get_absolute_url(self):
        return reverse('viewsintroduction:address', args=[self.id])


class PhoneContact(Model):
    name = models.CharField(max_length=20, unique=True)
    surname = models.CharField(max_length=20, unique=True)

    height = models.FloatField()
    date_of_birth = models.DateField()
    is_family = models.BooleanField(default=False)
    sex = models.CharField(max_length=1, choices=OPTIONS_SEX)

    address = models.OneToOneField(PhoneAddress)

    def __str__(self):
        return "{0} {1}".format(self.name, self.surname)

    def get_absolute_url(self):
        return reverse('viewsintroduction:contact', args=[self.id])

Create View

The CreateView will provide us with the UI for collecting all the information for a model, the validation as defined within the model and also the ability to create a model from the post data.

#views.py
from django.contrib import messages
from django.core.urlresolvers import reverse
from django.views.generic.edit import CreateView

from .models import PhoneAddress

class PhoneAddressCreateView(CreateView):
    model = PhoneAddress
    fields = ["city", "street_name", "number"]
    template_name = "viewsintroduction/phoneaddresscreate.html"

    def form_valid(self, form):
        messages.success(self.request, "Address created successfully")
        return super(PhoneAddressCreateView, self).form_valid(form)

    def form_invalid(self, form):
        messages.error(self.request, "Address has not been created")
        return super(PhoneAddressCreateView, self).form_invalid(form)
  • The model property registers the model class to the view
  • The fields property is a list of model field names which will be displayed, collected and populated.
  • The template_name property is the template to be used
  • The form_valid and form_invalid are not mandatory but provide us with a hook to place code upon valid and invalid validation. We use these to add a success or failure message to the Django message system. We will display these in a common base template later on.

Next we need a template which renders our form.

<!-- viewsintroduction/templates/phoneaddresscreate/base.html -->
{% extends "viewsintroduction/base.html" %}

{% block title %}
    <h2>Create Address</h2>
{% endblock title %}

{% block content %}
    <form method="post" action="{% url "viewsintroduction:address_create" %}">
        {% csrf_token %}
        <table>
            {{ form.as_table }}
        </table>
        <p><input type="submit" value="Create"/></p>
    </form>
{% endblock content %}
  • The csrf_token is a cross site forgery token; we should always include this.
  • The form.as_table renders our auto generated Django form object as html.

Finally we will create our base view which will display our messages:

<!-- viewsintroduction/templates/viewsintroduction/base.html -->
<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
</head>
<body>
<h1>Views Introduction</h1>
{% block title %}
{% endblock %}

{% if messages %}
    <ul class="messages">
        {% for message in messages %}
            <li id="message_{{ forloop.counter }}"
                    {% if message.tags %} class="{{ message.tags }}"{% endif %}>
                {{ message }}
            </li>
        {% endfor %}
    </ul>
{% endif %}

{% block content %}
{% endblock %}
</body>
</html>
  • The messages template variable is automatically available as we used the Django messaging middle ware. The app ‘django.contrib.messages’ is registered automatically by default in the INSTALLED_APPS of the settings file; it should work out of the box.

Hook in our view into out URL routing config:

#viewsintroduction/urls.py
url(r'^address/create/$', PhoneAddressCreateView.as_view(), name="address_create"),

Our project’s urls.py should register the URL prefix of viewsintroduction to our app’s url.py:

#DjangoSandBox/url.py
from django.conf.urls import patterns, include, url

from viewsintroduction import urls as viewsintroduction_urls

urlpatterns = 
    patterns('',
             url(r'^viewsintroduction/', include(viewsintroduction_urls, namespace="viewsintroduction")),
           )

And I kid you not, we are done! The ability to create an Address record including all validation!

To test run the development server and navigate to:

http://127.0.0.1:8000/viewsintroduction/address/create/

Update View

The update view follows the same pattern though we inherit the UpdateView.

#views.url
from django.views.generic.edit import UpdateView

from .models import PhoneAddress

class PhoneAddressUpdateView(UpdateView):
    model = PhoneAddress
    fields = ["city", "street_name", "number"]
    template_name = "viewsintroduction/phoneaddressupdate.html"

    def form_valid(self, form):
        messages.success(self.request, "Address updated successfully")
        return super(PhoneAddressUpdateView, self).form_valid(form)

    def form_invalid(self, form):
        messages.error(self.request, "Address has not been updated")
        return super(PhoneAddressUpdateView, self).form_invalid(form)

A simple template using the as_table functionality of the Django Form to render itself:

<!--templates/viewsintroduction/phoneaddressupdate.html -->
{% extends "viewsintroduction/base.html" %}

{% block title %}
    <h2>Address Update</h2>
{% endblock title %}

{% block content %}
    <form method="post" action="">
        {% csrf_token %}
        <table>     {{ form.as_table }}   </table>
        <p><input type="submit" value="Update"/></p>
    </form>
{% endblock content %}

The only real difference from our previous CreateView example is that we need to have an identifier for the record we are updating. Here we parse the URL to extract the primary key value which should be passed into the URL:

/address/1/update
#urls.py

url(r'^address/(?P<pk>[0-9]+)/update/$', PhoneAddressUpdateView.as_view(), name="address_update"),
  • (?Pregex) means we are matching a section which we can reference later on by the word name. In our case the name is pk and the regex match is [0-9]+
  • [0-9]+ matches any number of times the digits 0 to 9.

To test this page run the development server and navigate to:

http://127.0.0.1:8000/viewsintroduction/address/1/update/

Note: Replace 1 with your required id.

Note; there is no choice in the name of the pk parameter here as the Django code we are using requires it to be explicitly called pk.

Delete View

The delete view implementation is very similar to the update, the main difference is that we inherit form the DeleteView.

#views.url
from django.views.generic.edit import DeleteView

from .models import PhoneAddress

class PhoneAddressDeleteView(DeleteView):
    model = PhoneAddress
    template_name = "viewsintroduction/phoneaddressdelete.html"

    def get_success_url(self):
        return reverse("viewsintroduction:address_list")

    def delete(self, request, *args, **kwargs):
        messages.success(self.request, "Address deleted successfully")
        return super().delete(request, *args, **kwargs)

Our template view:

<!-- templates/viewsintroduction/phoneaddressdelete.html -->
{% extends "viewsintroduction/base.html" %}

{% block title %}
    <h2>Delete Address</h2>
{% endblock title %}

{% block content %}
    <form method="post" action="">
        {% csrf_token %}
        <table>     {{ form.as_table }}   </table>
        <p><input type="submit" value="Delete"/></p>
    </form>
{% endblock content %}

Our URL routing config:

#urls.py
   url(r'^address/(?P<pk>[0-9]+)/delete/$', PhoneAddressDeleteView.as_view(), name="address_delete"),

To test this page run the development server and navigate to:

http://127.0.0.1:8000/viewsintroduction/address/2/delete/

Note: Replace 1 with your required id.

Detail View

The DetailView allows us to display a record from if the URL contains the primary key.

#views.py
from django.views.generic import DetailView

from .models import PhoneAddress

class PhoneAddressDetailView(DetailView):
    model = PhoneAddress
    fields = ["city", "street_name", "number"]
    template_name = "viewsintroduction/phoneaddress.html"
  • The model property contains our model class
  • The fields property contains a list of model field names as string
  • The template_name property contains the template identifier

The main difference here is that there is no auto functionality to render the record within a read-only table. We have to manually render the template details ourself.

The record is added into the template context data variable named object.

<!--template/viewsintroduction/phoneaddress.html-->
{% extends "viewsintroduction/base.html" %}

{% block title %}
    <h2>Address Details</h2>
{% endblock title %}

{% block content %}
    <table>
        <tr>
            <td>No.</td>
            <td>{{ object.number }}</td>
        <tr>
            <td>Street</td>
            <td>{{ object.street_name }}</td>
        <tr>
            <td>City</td>
            <td>{{ object.city }}</td>
        </tr>
    </table>

    <table>
        <tr>
            <td><a href="{% url "viewsintroduction:address_list" %}">Back</a></td>
            <td><a href="{% url "viewsintroduction:address_update" object.id %}">Update</a></td>
            <td><a href="{% url "viewsintroduction:address_delete" object.id %}">Delete</a></td>
        </tr>
    </table>
{% endblock content %}

We have added links to update and delete the record via the URL template operator and the URL names with their namespace such as:

{% url "viewsintroduction:address_update",  object.id %}

Hook in our URL routing config:

#urls.py

url(r'^address/(?P<pk>[0-9]+)/$', PhoneAddressDetailView.as_view(), name="address"),

To test this page run the development server and navigate to:

http://127.0.0.1:8000/viewsintroduction/address/1/

Note: Replace 1 with your required id.

List View

The ListView class allows us to retrieve and display a list of objects:

#views.py

from django.views.generic.list import ListView

from .models import PhoneAddress

class PhoneAddressListView(ListView):
    model = PhoneAddress
    template_name = 'viewsintroduction/phoneaddresslist.html'
    paginate_by = 5

    def get_queryset(self):
        return PhoneAddress.objects.all().order_by("city", "street_name", "number")
  • The model is the name of the model class we will be displaying
  • The template_name is the name of the template we will be using to render the data
  • The paginate_by is the number of records in each page; this is optional
  • The get_queryset is a function used to return the data. Here we are returning all objects ordered by city then street_name and then number.

By default the records are contained in a list under the template variable. named object_list .

Paing is set up using the following tempalte variables;

  • is_paginated set to true or false if we have paging defined on the view
  • page_obj is a class with various properties for working with paging
  • page_obj.has_previous returns true or false if we have a previous page; i.e we are not on the first page
  • page_obj.previous_page_number returns the number of the previous page
  • page_obj.number returns the current page number
  • page_obj.paginator.num_pages returns the total number of pages
  • page_obj.has_next returns true or false depending upon if we have following pages; i.e. we are not on the last page
  • page_obj.next_page_number returns the number of the next page

We then construct the URL with the page number as a get query string variable; i.e. it ends with ?page=x where x is the number of the page we want to view.

<!-- templates/viewsintroduction/base.html -->
{% extends "viewsintroduction/base.html" %}

{% block title %}
    <h2>Address List</h2>
{% endblock title %}

{% block content %}
    <table>
        <tr>
            <th>No.</th>
            <th>Street Name</th>
            <th>City</th>
            <th></th>
            <th></th>
            <th></th>
        </tr>
        {% if not object_list %}
        <tr><td colspan="6">There are no addresses</td></tr>
        {% endif %}
        {% for address in object_list %}
            <tr>
                <td>{{ address.number }}</td>
                <td>{{ address.street_name }}</td>
                <td>{{ address.city }}</td>
                <td><a href="{% url "viewsintroduction:address" address.id %}">View</a></td>
                <td><a href="{% url "viewsintroduction:address_update" address.id %}">Update</a></td>
                <td><a href="{% url "viewsintroduction:address_delete" address.id %}">Delete</a></td>
            </tr>
        {% endfor %}
    </table>
    {% if is_paginated %}
        <div class="pagination">
            <span>
                {% if page_obj.has_previous %}
                    <a href="{% url "viewsintroduction:address_list" %}?page={{ page_obj.previous_page_number }}">Previous</a>
                {% endif %}
                <span>
                    Page {{ page_obj.number }} of {{ page_obj.paginator.num_pages }}.
                </span>
                {% if page_obj.has_next %}
                    <a href="{% url "viewsintroduction:address_list" %}?page={{ page_obj.next_page_number }}">Next</a>
                {% endif %}
            </span>
        </div>
    {% endif %}

    <span><a href="{% url "viewsintroduction:address_create" %}">Create</a></span>
    <span><a href="{% url "viewsintroduction:contact_list" %}">Contacts</a></span>
{% endblock %}

Hook in the view into the URL routing config:

#urls.py
url(r'^address/list$', PhoneAddressListView.as_view(), name="address_list"),

To view this page run the development server and navigate to:

http://127.0.0.1:8000/viewsintroduction/address/list/

Contact Views

Please see the associated source code for how to hook in the parent Contact model views. They are basically the same as the address model. The only cool point to note is that we get a UI drop down of addresses pre-populated which allow us to set the associated joined address record.

Redirect View

The redirect view allows us to perform a server redirect via the URL routing config:

#urls.py

from django.views.generic.base import RedirectView

url(r'^redirect/$', RedirectView.as_view(url='http://djangoproject.com'), name='redirect_view'),

To view this page run the development server and navigate to:

http://127.0.0.1:8000/viewsintroduction/redirect/

Template View

The template view allows us to display static data:

#urls.py

from django.views.generic.base import TemplateView

url(r'^about/$', TemplateView.as_view(template_name="viewsintroduction/about.html"), name="template_view")

A template would then look like this:

<!-- templats/viewsintroduction/about.html-->
{% extends "viewsintroduction/base.html" %}

{% block title %}
    <h2>Template View</h2>
{% endblock title %}

{% block content %}
    Some static information would be here!
{% endblock content %}

To view this page run the development server and navigate to:

http://127.0.0.1:8000/viewsintroduction/about/

Django Templates

This post is part of my Django series. You can see an overview of the series along with instruction on how to get all the source code here.

This article assumes you are comfortable creating a Django project, can create apps and register them into the INSTALLED_APPS list of the settings file. If not please read my Django HelloWorld article.

This article assumes you have a project called DjangoSandBox in an application called templatesintroduction.

Collections

for endfor

We can loop through a collection with the for and endfor tags

{%  for colour in colours %}
    {{  colour }}
{% endfor %}

Where each element in a collection is a tuple we can unpack the tuple elements into loop variables.

If out context data is:

'lettersandordinals': [(1, 'a'), (2, 'b'), (3, 'c')],

We could loop like this:

{% for number, letter in lettersandordinals %}
    {{ number }} ({{ letter  }})
{% endfor %}

When we are looping through a dictionary we can access the key and value.

If our context data is:

'worldcities': {'UK': ['Plymouth', 'Bristol'], 'France': ['Grenoble']},

We could loop like this:

{% for key, value in worldcities.items %}
    {{ key }} : {{ value }}
{% endfor %}

for empty

We can use the empty tag as a conditional check when looping through an empty collection.

{% for empty in emptycollection %}
    The collection is not empty
    {%  empty %}
    This collection is empty!!!
{% endfor %}

Loop Variables

When within a loop we can access various loop variables from the forloop template variable;

Variable Description
forloop.counter The current iteration count where 1 is the first count
forloop.counter0 The current iteration count where 0 is the first count
forloop.revcounter The number of iterations from the end of the loop; where 1 is the end
forloop.revcounter0 The number of iterations from the end of the loop; where 0 is the end
forloop.first Returns true if this is the first iteration
forloop.last Returns true if this is the last iteration
forloop.parentloop Allows access to a parent loop variable if we are a nested loop
<table>
    <thead>
        <tr>
            <th>Number</th>
            <th>forloop.counter</th>
            <th>forloop.counter0</th>
            <th>forloop.revcounter</th>
            <th>forloop.revcounter0</th>
            <th>forloop.first</th>
            <th>forloop.last</th>
        </tr>
    </thead>
    {% for number in numbers  %}
    <tr>
        <td>
            {{ number }}
        </td>
        <td>
            {{ forloop.counter }}
        </td>
        <td>
            {{ forloop.counter0 }}
        </td>
        <td>
            {{ forloop.revcounter }}
        </td>
        <td>
            {{ forloop.revcounter0 }}
        </td>
        <td>
            {{ forloop.first }}
        </td>
        <td>
            {{ forloop.last }}
        </td>
    </tr>
    {% endfor %}
</table>

reversed

We can iterate through a collection backwards with the reversed tag

{% for number in numbers reversed %}
    {{ number }}
{% endfor %}

cycle

The cycle flag allow us to identify iterations. We can define 2 or more strings, numbers or variables along with the cycle tag. Each iteration will iterate through all the elements passed in. Here we assign the output to the oddeven variable which we can access later on.

{% for colour in colours %}
    {{  colour }}
    {% cycle 'odd' 'even' as oddeven%}
    {{ oddeven }}
{% endfor %}

ifchanged

We can determine when a variable has changed between iterations with the ifchanged tag.

If out context data was:

 'duplicatednumbers': [1, 1, 1, 1, 2, 2, 2, 3, 3, 3, 3],
<ul>
    {% for number in duplicatednumbers %}
        <li>
            {{ number }}
            {% ifchanged number %} {#  can take multiple arguments #}
                (changed)
            {% else %}
                (not changed)
            {% endifchanged %}
        </li>
    {% endfor %}
</ul>

Would output:

1 (changed)
1 (not changed)
1 (not changed)
1 (not changed)
2 (changed)
2 (not changed)
2 (not changed)
3 (changed)
3 (not changed)
3 (not changed)
3 (not changed)

unordered_list

We can generate html for a list using UL elements with the unordered_list tag

{{ numbers|unordered_list }}

This would generate something like this:

<ul>
    <li>1</li>
    <li>2</li>
</ul>

If the collection is a dictionary of lists we generate nested lists.

<li>A
<ul>
        <li>B
        <ul>
                <li>1</li>
                <li>2</li>
        </ul>
        </li>
        <li>C</li>
</ul>
</li>

dictsort

Where we have a list of dictionaries; we can sort by a common child column.

If our context data was:

boysandgirls': [
                   {'name': 'Lukelad', 'age': 26, 'sex': 'male'},
                   {'name': 'Lukette', 'age': 66, 'sex': 'female'},
                   {'name': 'Luke', 'age': 36, 'sex': 'male'},
                   {'name': 'Lukecy', 'age': 36, 'sex': 'female'}]}

We can sort by the age column ascending by :

{{ boysandgirls|dictsort:"age" }}

Would output:

[{'name': 'Lukelad', 'age': 26, 'sex': 'male'}, {'name': 'Luke', 'age': 36, 'sex': 'male'}, {'name': 'Lukecy', 'age': 36, 'sex': 'female'}, {'name': 'Lukette', 'age': 66, 'sex': 'female'}]

dictsortreversed

We can also sort in a reversed order with dictsortreversed pipe:

{{ boysandgirls|dictsortreversed:"age" }}

Would output:

[{'name': 'Lukette', 'age': 66, 'sex': 'female'}, {'name': 'Luke', 'age': 36, 'sex': 'male'}, {'name': 'Lukecy', 'age': 36, 'sex': 'female'}, {'name': 'Lukelad', 'age': 26, 'sex': 'male'}]

regroup

Continuing with our list of people as a dictionary we can group upon a common field to produce a collection for each grouped set. Here we group by the sex column to produce a collection of boys and girls.

We have access to the group value via the grouper variable upon each group.

If our context data was:

boysandgirls': [
                   {'name': 'Lukelad', 'age': 26, 'sex': 'male'},
                   {'name': 'Lukette', 'age': 66, 'sex': 'female'},
                   {'name': 'Luke', 'age': 36, 'sex': 'male'},
                   {'name': 'Lukecy', 'age': 36, 'sex': 'female'}]}

We can group by sex and loop through the groups and then elements in each group:

{% regroup boysandgirls|dictsort:"sex" by sex as boys_and_girls_grouped  %}
{#The grouping does not sort the data. | through dictsort if required#}
<ul>
{% for sex_group in boys_and_girls_grouped %}
    <li>{{ sex_group.grouper | capfirst }}
    <ul>
        {% for item in sex_group.list %}
          <li>{{ item.name }}: {{ item.age }}</li>
        {% endfor %}
    </ul>
    </li>
{% endfor %}
</ul>

The following would be outputted:

Female
    Lukette: 66
    Lukecy: 36
Male
    Lukelad: 26
    Luke: 36

join

We can join all elements in collection as a string with the join filter along with the separator character.

{{ numbers|join:":" }}

If our context data was:

numbers is: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],

Would output:

1:2:3:4:5:6:7:8:9:10

first and last

The first and last pipes can be used to extract the first and last element of a collection.

First of {{ numbers }} is {{ numbers|first }}
Last of {{ numbers }} is {{ numbers|last }}

length and length_is

The length filter returns the number of elements in a collection while the length_is tag returns true or false depending upon if our collection size is equal to our comparison argument.

{{ numbers }} has {{ numbers|length }} elements
Is {{ numbers }} 10 in length? {{ numbers|length_is:"10" }}

If out context data was:

  'numbers': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],

Would output:

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10] has 10 elements
/ Is [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] 10 in length? True

make_list

The make_list can be used to create a list from a literal. 123456789 would become a list of the numbers 0 – 9.

123456789 becomes {{ 123456789|make_list  }}
Hello becomes {{ "Hello"|make_list  }}

Would output:

123456789 becomes ['1', '2', '3', '4', '5', '6', '7', '8', '9']
Hello becomes ['H', 'e', 'l', 'l', 'o']

random

We can extract a random element from a collection with the random pipe.

A random of {{ numbers }} is {{ numbers|random }}

Collection Slicing ### {#Collection Slicing}

We can use the python collection slicing functionality to slice elements from a collection.

<br />The first two of {{ numbers }} is {{ numbers|slice:":2"}}
The last two of {{ numbers }} is {{ numbers|slice:"-2:"}}
Elements 5 to 8 {{ numbers }} is {{ numbers|slice:"4:8"}}

Would output

The first two of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] is [1, 2]
The last two of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] is [9, 10]
Elements 5 to 8 [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] is [5, 6, 7, 8]

Conditions

if, elif, else and endif

Django templates supports if, elif, else and endif tags for if conditions.

<span>
    {% if age < 50 %}
        Less than 50
    {% elif age == 50 %}
        Equal to 50
    {% else  %}
        The else clause
    {% endif %}
</span>

or & and

We can compound conditions with the ‘or’ and ‘and’ operators.

<span>
    {% if age < 50 and age > 10 %}
        Less than 50 but more than 10
    {% elif age == 50 or age == 51 %}
        Equal to 50
    {% else  %}
        The else clause
    {% endif %}
</span>

not

Boolean negation can be made with the not operator.

<span>
    {%  if not false %}
        Not False
    {% endif %}
</span>

in

We can check for the inclusion of an element within a collection with the in operator.

<span>
    {%  if 1 in numbers%}
        1 is in {{  numbers }}
    {% endif %}
</span>

not in

We can negate the in operator with the not operator to check for the non inclusion of an element within a collection.

<span>
    {%  if 11 not in numbers%}
        11 is not in {{  numbers }}
    {% endif %}
</span>

ifequal

The ifequal, else endifequal tags are a shortcut for if x = y, else and endif.

<span>
    {% ifequal valueone valueone%}
        {{ valueone }} and {{ valueone }} are equal
    {% else %}
        {{ valueone }} and {{ valueone }} are not equal
    {% endifequal %}
</span>

ifequal, endifequal, ifnotequal and endifnotequal

Negation of ifequal/endifequal is made with the ifnotequal and endifnotequal operators.

<span>
    {% ifnotequal valueone valuetwo%}
        {{ valueone }} and {{ valuetwo }} are not equal
    {% else %}
        {{ valueone }} and {{ valuetwo }} are equal
    {% endifnotequal %}
</span>

firstof

The firstof tag outputs the first true (non zero) element; this can be used with boolean and integer values.

<span>
    {% firstof 0 0 0 1 "All were false"  %}
    {% firstof 0 0 0 0 "All were false"  %}
</span>

|

Filters allow piping results through multiple tags; here we filter the html string to be all lower case and then capitalised upon the first word:

<span>
    {% filter lower|capfirst %}This text will be HTML-escaped.{% endfilter %}
</span>

Filters can be used with other tags; here we check the length of a collection and pipe it to the >= comparison operator.

<span>
    {% if numbers|length >= 5 %}
        There are more than five numbers!
    {% endif %}
</span>

with

Variables can be declare with the with operator.

{% with count=10 %}
    There are {{ count }} item{{ 2|pluralize }}
{% endwith %}

Operator Precedence

Parenthesis is not supported within templates so you must rely on operator precedence

<span>
    The precedence is the same as python: or, and, not, in, ==, !=, <, >, <=, >=
</span>

Dates & Times

now

The now keyword can be used to get the current date and time.

It is: {% now "c"  %}

Formatting Dates

The “c” above is a date formatter specifying the ISO 8601 format; we can use any date time formatters as defined by Django date template formatters.

It is {% now "f A jS of F Y " %}

Outputs something similar to this:

It is 10:52 AM 12th of July 2015
  • f: Time, in 12-hour hours and minutes, with minutes left off if they’re zero. Proprietary extension.
  • A: ‘AM’ or ‘PM’.
  • j: Day of the month without leading zeros.
  • S: English ordinal suffix for day of the month, 2 characters.
  • o: SO-8601 week-numbering year, corresponding to the ISO-8601 week number (W)
  • f: Time, in 12-hour hours and minutes, with minutes left off if they’re zero. Proprietary extension.
  • F: Month, textual, long
  • Y: Year, 4 digits.

We also have access to various predefined formats which respect the local settings.

{% now "DATE_FORMAT" %}
{% now "DATETIME_FORMAT" %}
{% now "SHORT_DATE_FORMAT" %}
{% now "SHORT_DATETIME_FORMAT" %}

Which outputs the following UK formatted dates.

July 12, 2015
July 12, 2015, 10:52 a.m.
07/12/2015
07/12/2015 10:52 a.m.

date and time

We can use the date and time filter with a date time object along with formatting criteria. We can use all the date time formats as mentioned above.

{{ when|date:"D d M Y" }} 
{{ when|time:"H:i" }}

If our context data was:

when is timezone.now()

Would output:

Sun 12 Jul 2015 
10:52

timesince

We can use the the timesince filter to output the difference between two date times.

{{ yesterday }} since {{ when }} = {{ yesterday|timesince:when}}
{{ when }} until {{ tomorrow }} = {{ when|timesince:tomorrow}}

If our context was:

'when': timezone.now()
'yesterday': timezone.now() - timedelta(days=1)
'tomorrow': timezone.now() + timedelta(days=1)

Would output:

July 11, 2015, 10:52 a.m. since July 12, 2015, 10:52 a.m. = 23 hours, 59 minutes
July 12, 2015, 10:52 a.m. until July 13, 2015, 10:52 a.m. = 1 day

Debugging

Comment

Comments are made with the comment tag. All code between the comment and endcomment tags are commented out. This includes template code.

{% comment "Optional note" %}
<p>This is a comment. Even template code is commented out {{ who }}</p>
{% endcomment %}

PPRINT

A wrapper around Python’s pretty print library; pprint.pprint(). Use as a filter against any template parameter.

{{ request|pprint }}

Debug

The debug tag can be used to output a whole bunch of debug information about Django; I would favour using the Django Debug Toolbar

{%  debug %}}

Block & Extends

The block and extends tags are used to reuse sections of templates and is similar to ASP.NET master pages.

Imagine we had static data and format between pages which we would like to reuse. We can extract this into a template page. For example the menus, placement of the title and the static files such as css files.

The parent page would declare the required sections with the block tag; for example the page title along with the actual physical content of the page.

Physical child pages would then extend the page using the extends tag and provide content for the defined block tag sections of the parent template.

Our parent template would be:

<!-- templates/templatesintroduction/base_template.html -->
<h1>Welcome to the site</h1>
{%  block title %}
{%  endblock %}

{% block content %}
{% endblock %}

Our child template would be:

{%  extends 'templatesintroduction/base_template.html' %}

{%  block title %}
    <h2>This is the title</h2>
{%  endblock %}

{% block content %}
    <h3>This is the content</h3>
{% endblock %}

Would output:

<h1>Welcome to the site</h1>
<h2>This is the title</h2>
<h3>This is the content</h3>

Include

The include tag allows us to include a template into the current template; we have access to the context data of the parent page within the include page.

  • All context data from the parent page is passed into the included template
  • We can prevent passing in the context data with the ‘only’ option
  • We can explicitly pass additional context data into the template with the ‘with’ command and providing key value paired data
  • We can combine the only and with tags; only the additional context data is passed in

Imagine the following template which displays the msg template parameter along with a default message.

<!-- templatesintroduction/include_one.html -->
{{ msg|default:"There was no message!!!" }}

We can then include this template into another as follows:

{% include "templatesintroduction/include_one.html" %}
{% include "templatesintroduction/include_one.html" with msg="This was overridden!!!" %}
{% include "templatesintroduction/include_one.html" with wrongmsg="This was overridden!!!" only %}

If our context to the outer template had the following:

'msg': "This is a message of hello!!!"

Would output:

This is a message of hello!!!
This was overridden!!!
There was no message!!!
  • The first include passes all context data into the include template; the parent context data is rendered
  • The second include passes all context data into the include templates but also passes in additional context data which overrides the value in the parent page context data
  • The third include uses the only tag. The parent context data is not passed in as as such the default message was used

Note: it seems that using the only keyword requires at least one context data element passed in with the ‘with’ option.

SSI

SSI includes the contents of a file without using the template rendering system; i.e. the contents of the file are displayed as is.

You need to add a permission to the file as static data otherwise you will get a permission denied error.

{% ssi "foo.css" %}

Numbers

widthratio

The widthratio can work out the ratio of a width based upon a value and its maximum value; {% widthratio 50 100 100 %}. This would read what is the width of 50 if it’s maximum value is 100 and the maximum width we want is 100.

The width is: {% widthratio 50 100 100 %}

Would output:

The width is: 50

We can assign the value to a variable with the ‘as’ command to use later on.

{% widthratio 50 100 100 as width %}
The width is: {{ width }}

divisibleby ###{#divisibleby}

The divisibleby tag can be used to determine if a modulus division would return 0.

Is 21 dividable by 3? = {{ 21|divisibleby:"3" }}
Is 21 dividable by 5? = {{ 21|divisibleby:"5" }}

Would output:

Is 21 dividable by 3? = True
Is 21 dividable by 5? = False

filesizeformat

The filesizeformat tag provides an easy way of outputting a number as a file size format with the units.

{{ 1|filesizeformat }}
{{ 1024|filesizeformat }}
{{ 1048576|filesizeformat }}
{{ 1073741824|filesizeformat }}

Would output the following:

1 byte
1.0 KB
1.0 MB
1.0 GB

floatformat

The floatformat tag can be used to round a floating point number to a defined number of decimal places.

{{ 11.1111|floatformat:"3" }}
{{ 11.1111|floatformat:"0" }}

Would output:

11.111
11

If the number being rounded has less decimal places than it is being rounded to 0’s are placed upon the entity to make up the remaining decimal places. We can use a negative number to prevent these surplus digits.

{{ 11|floatformat:"3" }}
{{ 11.1111|floatformat:"-3" }}
{{ 11|floatformat:"-3" }}

Would output:

11.111
11.111
11

get_digit

The get_digit tag takes an integer as an ordinal position and returns the associated digit at this position from another number. The ordinal position starts at element 1 on the right hand side of the number; the following would return 8.

{{ 123456789|get_digit:"2" }}

phone2numeric

The phone2numeric tag can be used to convert telephone numbers containing letters to their numerical counter part.

{{ "+44 1752 HELLO"|phone2numeric }}

Would output:

+44 1752 43556

add

We can use the add tag to perform addition. This not only adds two numbers together but it can concatenate two strings as well as appending one collection onto the end of another. Where a string can be cast to a number a conversion is made.

2 + 1 = {{ 2|add:1 }}
'2' + '1' = {{ '2'|add:'1' }}
'a' + 'b' = {{ 'a'|add:'b' }}
{{ onetwo }} + {{ onetwo }} = {{ onetwo|add:onetwo }}

If the context data was:

'onetwo': [1, 2]

Would output:

2 + 1 = 3
'2' + '1' = 3
'a' + 'b' = ab
[1, 2] + [1, 2] = [1, 2, 1, 2]

Strings

{{ }}

The {{ }} tag is used to write a value to the response stream; i.e to render the value of a variable to the template. This can be use with literals in the page or context data passed in as template variables.

{{ who }}

lower, upper, capfirst, title

  • The lower filter will make all letters lower case
  • The upper filter will make all letters upper case
  • The capfirst will make all letters lower case except the very first letter which will be upper case
  • The title filter will make all letters lower case except the first letter of every word which will be upper case
{{ who | lower }}
{{ who | upper }}
{{ who | capfirst }}
{{ who | title }}

addslashes

The addslashes tag can be used to escape any quotes in a string with a backslash.

{{ "Simon's slashes"|addslashes }}

This would output:

Simon's slashes

ljust, center, rjust

We can space pad strings to be be a minimum of x characters. Where the string does not contain this number of characters spaces are used to build up the string to this size.

  • ljust: the string is aligned left by placing spaces on the right had side
  • center: the string is aligned in the centre by placing spaces evenly on each side
  • rjust: the string is aligned right by placing spaces on the left hand side
{{ "Hello"|ljust:20 }}
{{ "Hello"|center:20 }}
{{ "Hello"|rjust:20 }}

cut

The cut command can be used to remove all instances of a string from another string. Here we remove the string el from the word Hello to leave Hlo.

{{ "Hello"|cut:"el" }}

default, default_if_none

The default filter allows us to provide a default value.

{{ NotProvided|default:"Nothing was provided here!!!" }}
{{ False|default:"The value was false!" }}
{{ None|default_if_none:"Only displayed if this is None" }}

If our context data was:

'False': False,
'None': None,

Would output:

Nothing was provided here!!!
The value was false!
Only displayed if this is None
  • The first call simply did not provided the template variable; the default message was outputted
  • The second uses the False template variable which equates to a False boolean. Any expression equating to false triggers the default value
  • The third call has a template variable but it is set to None. In this case we can trigger the default message by using the default_if_none filer.

pluralize

We can use the pluralize filter to change a word to its pluralised variation. For example the word ‘example’ would become ‘examples’. The default appends the character s onto the word.

The filter works with a collection or an integer. Where a collection is used if it contains two or more elements the plural will be used. Where an integer is used if it is 2 or more the plural will be used.

Where a pluralized word differs from simply adding an ‘s’ character we can provide the single and plural endings. Here we append our static prefix of ‘pe’ with ‘rson’ or ‘ople’ to make person or people depending if we have 0/1 or 2 or more entries.

The plural of colour is colour{{ colours|pluralize }}.
The plural of colour is colour{{ 2 | pluralize }}.
The plural of person is pe{{ 2|pluralize:"rson,ople"}}.

If our context data was:

'colours': {'Red', 'Blue', 'Green'},

Would output:

The plural of colour is colours.
The plural of colour is colours.
The plural of person is people.

truncatechars, truncatechars_html, truncatewords and truncatewords_html

We can truncate strings with the truncatechars, truncatechars_html, truncatewords and truncatewords_html filters.

  • truncatechars truncates a string to be the first x characters
  • truncatechars_html truncates a string to be the first x characters. Where HTML tags are removed it ensures that any open tags are completed.
  • truncatewords truncates a string to be the first x characters. It will allow any words to be completed even if it means the final string is longer than the required x characters
  • truncatewords_html truncates a string to be the first x characters. It will allow words and HTML tags to be completed.
{{ longmessage|truncatechars:20 }}
{{ longboldmessage|truncatechars_html:20 }}
{{ longmessage|truncatewords:3 }}
{{ longboldmessage|truncatewords_html:3 }}

spaceless

The spaceless tag can be used to “compress” html such that it contains no white space. White space which is displayed to the user is not affected.

{% spaceless %}
    <span>
    <b>
        <i>
            This is some text
        </i>
    </b>
    </span>
{% endspaceless %}

Would output:

<span><b><i>This is some text</i></b></span>

templatetag

The template tag allows us to output template revered symbols such as {{ and {%.

{% templatetag openblock %} {% templatetag closeblock %}
{% templatetag openvariable %} {% templatetag closevariable %}
{% templatetag openbrace%} {% templatetag closebrace %}
{% templatetag opencomment %} {% templatetag closecomment %}

Would output:

{% %}
{{ }}
{ }
{# #}

verbatim

The verbatim tag takes any characters between the verbatim and endverbatim and renders them into the HTML directly, regardless of what they contain.

{% verbatim %}
    {{this will output}}{%  as it nothing has happened %}
{% endverbatim %}

Would output:

{{this will output}}{%  as it nothing has happened %}

linebreaks and linebreaksbr

We can render newline characters in strings with the
html tag with the linebreaks and linebreaksbr filters. The difference is that linebreaks will append

tags around each line.

{{ messagewithnewlines|linebreaks }}
{{ messagewithnewlines|linebreaksbr }}

wordwrap

We can apply word wrap to x characters with the wordwrap tag

{{ longmessage|wordwrap:5 }}

linenumbers

We can output line number counts against each line with the linenumbers filter.

{{ messagewithnewlines|linenumbers|linebreaks }}

If the context data was:

'messagewithnewlines': "Thisncontainsnnewlines"

Would output:

1. This
2. contains
3. newlines

wordcount

We can use the wordcount filter to provide a count of the number of words in a string.

There are {{ longmessage|wordcount }} words in "{{ longmessage }}"

stringformat

We have access to all the normal Python string formatting options:

{{ 1234.567|stringformat:"E" }}
{{ 1234.567|stringformat:"F" }}
{{ 1234.567|stringformat:"d" }}

Would output:

1.234567E+03
1234.567000
1234

yesno

We can convert boolean values to custom strings with the yesno filter. An optional value for None can be provided. If no None value is provided the False value is used.

{{ False|yesno:"yeah,no,maybe" }}
{{ True|yesno:"yeah,no,maybe" }}
{{ None|yesno:"yeah,no,maybe" }}
{{ None|yesno:"yeah,no" }}

Would output:

no
yeah
maybe
no

URLs

We can render a URL by referencing it via it’s name and namespace as defined within the URL routing config.

<a href="{% url 'templatesintroduction:urls'%}">URLs</a>

We can pass in any URL arguments in the order they are found in the view function:

<a href="{% url 'templatesintroduction:urls' 1 2%}">URLs</a>

We can name the arguments to match the view function arguments to by pass the order.

<a href="{% url 'templatesintroduction:urls' two=2 one=1%}">URLs</a>

Static Files

Any static files such as css or js files need to be registered within the settings file of the project:

#settings.py
STATIC_URL = '/static/'

On each template requiring a static file we need to load the static files:

{% load staticfiles %}

We can then include static files on our page:

<link rel="stylesheet" type="text/css" href="{% static 'css/styles.css' %}"/>
<img src="{% static 'imgs/tintin.jpg' %}" alt="My image"/>

Escaping

autoescape

Auto escaping allows automatically escaping html special characters to their asci equivalent; < becomes &lt. This can help protecting against running vulnerable code on a client web browser. Important if we allow collecting data from the user to display later on.

By default autoescaping is on.

We can turn it on and off along when opening a new autoescape block.

{% autoescape on %}
    {{  toescape }}
    {% autoescape off %}
        {{  toescape }}
    {% endautoescape %}
{% endautoescape %}

If our context data was:

'toescape': "1 is < 2"

We would output:

1 is < 2
    1 is < 2

safe

Where escaping is currently on we can output a variable without escaping by marking it as safe

{{ value|safe}}

safeseq

Where escaping is currently on we can output all elements of a collection without escaping:

{{ list_data|safeseq}}

Django Models Advance

Django Models Advance

This post is part of my Django series. You can see an overview of the series along with instruction on how to get all the source code here.

This article assumes you are comfortable creating a Django project, can create apps and register them into the INSTALLED_APPS list of the settings file. If not please read my Django HelloWorld article.

This article assumes you have a project called DjangoSandBox in an application called modelsadvanced.

JOINS / Relationships

Django provides three relationship types between models; these translate to joins and foreign keys at the SQL and schema level.

  • One to One (1:1)
  • One to Many (1:n)
  • Many to Many (n:m)

They are all applied as fields to the models; Django then provides an API for walking, querying, setting and deleting the related model instances.

One To One

A 1:1 relationship is where only one model instance is allowed to be associated on either end of the join field.

In this example we create a Man and a Dog model and associate them with the field owner which is a one to one join field set upon the Dog model. The Man is considered the parent and the Dog is considered the child model.

The parent model will have a field automatically added onto it named after the child model; in this case dog.

class Man(Model):
    name = models.CharField(max_length=20)

class Dog(Model):
    name = models.CharField(max_length=20)
    owner = models.OneToOneField(Man, on_delete=models.CASCADE, blank=True, null=True)

The on_delete option defines what happens when the parent record is deleted; here we state CASCADE which means the dog record will also be deleted at the same time as its associated Man record.

The join is optional (not mandatory) in the database as we set null=True. If we were to use Django forms or class based views the relationship would also be considered optional as we set blank=True.

We can create the Man record first then the Dog record passing in the man record to the owner field.

Note: here we use the get_or_create function; to prevent multiple records being generated we need to pass in the man instance into the defaults parameter.

a_man = Man.objects.get_or_create(name="John")[0]
a_dog = Dog.objects.get_or_create(name="Fido", defaults={'owner': a_man})[0]

We could have use the owner field to associate the man onto the dog.

a_man = Man(name="John")
a_man.save()

a_dog = Dog(name="Fido")
a_dog.owner = a_man

We could also have used the auto generated dog field on man to associate the dog onto the man.

a_dog = Dog(name="Fido")
a_dog.save()

a_man = Man(name="John")
a_man.dog = a_man
a_man.save()

If we have a mandatory join then we would have to create the parent record first, and associate the parent (Man) onto the child (Dog) before saving. Trying to save a dog without an owner, when we have a mandatory join, will cause an error to be raised.

We can read data via the dog and owner fields, this traverses the join and causes a database read.

for aMan in Man.objects.all():
    print("{0} --> {1}".format(aMan, aMan.dog))

for a_dog in Dog.objects.all():
    print("{0} --> {1}".format(a_dog, a_dog.owner))

Note: Calling a joined model field will cause a trip to the database upon the first call. Subsequent calls will used a cached version of the data. We can prevent the initial database trip by using the select_related function when we initially get the data.

We can query using the join field using an instance of the associated record and the join field name. Below we find the Dog record who has the owner set to our instance of the Man model found within the variable a_man.

a_dog = Dog.objects.get(name="Fido")
a_man = Man.objects.get(dog=a_dog)

a_man = Man.objects.get(name="Luke")
Dog.objects.get(owner=a_man)

Alternatively we can query using the joined table field by a double underscore between the join field name, the joined table field name and the operator required.

Dog.objects.get(owner__name="Luke")
Dog.objects.get(owner__name__icontains="L")

If no joined record exists accessing the join field will raise a ObjectDoesNotExist error.

One To Many

A one to many (1:n) relationship is where one parent record is associated to multiple children record.

In this example we create a one to many relationship between a model called Parent and a model called Child.

class Parent(Model):
    name = models.CharField(max_length=10, unique=True, null=True)

class Child(Model):
    name = models.CharField(max_length=10, unique=True)
    parent = models.ForeignKey(Parent, null=True)

The parent record can be accessed from the child record via the field called parent.

The children can be accessed from the parent record via a field called child_set. A QuerySet containing all children is returned; all QuerySet functions including filter and order_by can be called.

for aParent in Parent.objects.all():
    print("{0} --> {1}".format(aParent, aParent.child_set.all()))

A QuerySet is always returned, all be it empty, regardless if any children records have been saved. However calling parent upon a child record will raise a ObjectDoesNotExist error if no association has been made.

Django automatically names the field ChildModelName_set though this can be overridden with the related_name option. The following would rename the automatically generated field child_set to children upon the model Parent.

parent = models.ForeignKey(Parent, null=True, related_name=children)

We can create the parent record and then each child passing the parent into the parent field.

a_parent = Parent.objects.get_or_create(name="Bob")[0]
a_child = Child.objects.get_or_create(name="John", defaults={'parent': a_parent})[0]

We can also call add upon the child_set field to associate a child onto a parent. You don’t need to call save afterwards as add saves the data to the database.

a_parent = Parent(name="Bob")
a_parent.save()

a_child = Child(name="John")
a_child.save()
a_parent.child_set.add(a_child)

The add function of the child field can take any number of children.

a_parent.child_set.add(a_child, a_child_two)

Alternatively we can simply assign a list of children onto the parent field. Any existing associated children would be removed. This does not save to the database; a call to save is required afterwards.

a_parent.child_set = [a_child, a_child_two]
a_parent.save()

We can also save a parent onto a child; here we would need to call save.

a_parent = Parent(name="Bob")
a_parent.save()

a_child = Child(name="John")
a_child.parent = a_parent
a_child.save()

We can navigate to the parent record from the child with the join field:

parent_name = Child.objects.get(name="Sophie").parent.name

We can search by the actual joined records. The first line finds the parent who has the child called Sophie associated to it. The later finds the child record who has the parent named bob associated to it.

Parent.objects.get(child=Child.objects.get(name="Sophie"))
Child.objects.get(parent=Parent.objects.get(name="Bob"))

We can search against the parent fields in the format ParentTableName__ParentFieldName__operator. Here we find the child of Bob by using the parent’s name field.

child_of_bob = Child.objects.get(parent__name="Bob")
parent_of_sophie = Parent.objects.get(child__name="Bob")

We can use all the normal filter operators:

Parent.objects.filter(child__name="Sport")
Parent.objects.filter(child__name__contains="S")
Parent.objects.filter(child__name__contains="S").filter(child__name__contains="p")

Where a the join is optional and an association has not been made to a record Django fills the field with null. We can use the isnull operator to query records if they have an association or not.

parent_with_children = Parent.objects.filter(child__name__isnull=True)
parent_with_no_children - Parent.objects.filter(child__name__isnull=False)

The child_set returns a QuerySet which we can predicate, order and iterate through etc.

Parent.objects.get(name="Dave").child_set.filter(name__contains="S").count()

Many operations upon child records will cause subsequent database hits as we traverse the relational model. The select_related function pre-populates the join objects so we don’t have to hit the database again

Parent.objects.select_related().get(name="Dave").child_set.count()
Child.objects.select_related().get(name="Sophie").parent.name

We can annotate the models with aggregate statistics of the associated child records.

parents_with_name = Parent.objects.annotate(children_count=Count("child"))
parents_with_name.get(name="Luke").children_count

Many To Many

A many to many (m:n) join allows any parent record to be associated to any number of children as well as any child record to be associated to any number of parents.

The following example creates the tables Author and Books and then creates a many to many relationship between them.

class Author(Model):
    name = models.CharField(max_length=10, unique=True)

class Book(Model):
    authors = models.ManyToManyField(Author)
    name = models.CharField(max_length=10, unique=True)

Books can access their authors via the authors field, an author can access their books with the auto generated book_set field unless the related_name option is used to define the field name.

for anAuthor in Author.objects.all():
    for a_book in anAuthor.book_set.all():
        print("{0} --> {1}".format(anAuthor, a_book))

for a_book in Book.objects.all():
    for anAuthor in a_book.author.all():
        print("{0} --> {1}".format(a_book, anAuthor))

We can assign records with the add function upon either end of the join field; we only need to associate the records once. Calling add saves the change to the database; there is no need to call save afterwards.

an_author = Author.objects.get_or_create(name="Author 1)[0]
a_book = Book.objects.get_or_create(name=name)[0]
a_book.authors.add(author)

As long as the join is non mandatory we can associate the records from either end via the authors or book_set field just the same way as associating children to a parent in the example of 1:n joins above.

a_parent.book_set.add(a_child)
a_parent.book_set.add(a_child, a_child_two)
a_parent.book_set = [a_child, a_child_two]

Calling either side’s join field returns a QuerySet which supports the the same functionality as with calling the child field on a 1:n relationship; i.e calling parent.child_set as shown above.

Book.objects.get(name="1984").authors.filter(name__contains="o")

Author.objects.get(name__contains="George").book_set
Author.objects.get(name__contains="George").book_set.filter(name__contains="b")

Additional Join Notes

We can place restrictions upon the the the valid join records via the limit_choices_\to option.

fieldname = models.ForeignKey(MyModel, limit_choices_to={'field_name': 'value'})

By default the foreign key and joins for relationships are made with the primary key field; we can use to_field to define another field.

fieldname = models.ForeignKey(MyModel, to_field='field_name')

As mentioned previous the on_delete=models.CASCADE flag means that when we delete a record all assigned entities are also deleted. Other options include:

  • PROTECT: Raises an error if any joined records are found
  • SET_NULL: Joined records are assigned null instead of the record which is being deleted
  • SET_DEFAULT: Joined records are assigned the default value for the field; you must set default option on the field.

Model Inheritance

As per inheritance in Object Orientated programming we can inherit models to reuse model definitions.

Duplicating Fields

The simplest option is to simply copy and paste the fields; though this should be kept to a minimum to prevent abusing the DRY principal. For simply a duplicating a field or two between a couple models this might be the simplest option.

Abstract Parent

An abstract parent allows inheriting fields form a parent model. When we apply our migrations a table named after the child model is found with all the fields from the child and parent. No table is made for the parent model.

class AbstractBaseParent(Model):
    name = models.CharField(max_length=10, unique=True)

    def __str__(self):
        return self.name

    class Meta:
        abstract = True

class AbstractBaseChild(AbstractBaseParent):
    age = models.IntegerField()

    def __str__(self):
        return "Name: {0}, Age: {1}".format(self.name, self.age)

Note: Marking the parent table as abstract within the class meta ensures that no table is generated for the parent table.

We save a child record with the fields of both the parent and child.

a_child = AbstractBaseChild(age=1, name"foo")
a_child.save()

We have access to read the parent models fields from the child model instance.

for child in AbstractBaseChild.objects.all():
    print(child.name, child.age)

A child model can inherit from any number of parent models.

class AChild(ParentOne, ParentTwo, ParentThree):
    pass

Multi-Parent base Child

A multi parent base child creates table definitions for all parent models as well as the child model. Django creates foreign keys within the schema and automatically generates model fields and the required SQL for traversing the joins

Data access is always made via the child which has access to all parent fields as if they are local. Django magically saves and reads to and from the parent tables.

class MultiTableBaseParent(Model):
    name = models.CharField(max_length=10, unique=True)
class MultiTableBaseChild(MultiTableBaseParent):
    age = models.IntegerField()

    def __str__(self):
        return "Name: {0}, Age: {1}".format(self.name, self.age)

Note: This more normalised approached will have the overhead of extra data reads and writes. Use with care.

We access child and parent fields through an instance of the child model. This includes when we create an instance of a child record. We pass in all parent fields at the same time; Django handles saving the parent record.

a_child = MultiTableBaseChild(age=age, name=name)
a_child.save()

We have access to the parent fields as if they are local to the child.

for child in MultiTableBaseChild.objects.all():
    print(child.name, child.age)

Proxy

A Proxy child allows us to wrap another model with extra class meta information. Data is read and written to the true model table.

In the following example the real table is ProxyParent and the proxy model is called ProxyChild.

The proxy child inherits from the ProxyParent model and sets proxy=True in the class meta.

class ProxyParent(Model):
    name = models.CharField(max_length=10, unique=True)

class ProxyChild(ProxyParent):
    class Meta:
        proxy = True

We save an instance of ProxyChild as per normal however the record is saved into the ProxyParent table.

ProxyChild.objects.get_or_create(name="Luke")

We can even access the data via the ProxyChild.

for child in ProxyChild.objects.all():
    print(child)

So why bother? The main point here is that we can have a model class with different meta information from it’s parent. One possible use is to set a different Data Manager. Here we can place additional predicates upon what is considered all of the data. For example we would return only records which are validated or considered live data rather than having to continuously adding in the predicate when required.

Validation

When using Django forms you call is_valid() to ensure all validation is called. When you are working at the model level you can call full_clean().

The functions is_valid() and full_clean() calls the following functions upon the model:

  1. Model.clean_fields() which validates the model’s fields
  2. Model.clean() which validates the model as a whole entity
  3. Model.validate_unique() which validate the field uniqueness against the database and any field constraints.

Simply calling save() upon a model does not raise validation; only database schema constraints will raise errors.

All the functions above allow include/exclude fields to control which fields are validated against.

Django comes with the following validators:

  • MinValueValidator and MaxValueValidator for validating against min and max numerical values.
  • MinLengthValidator and MaxLengthValidator for validating against min and max string fields length.
  • RegexValidator for more complex string validation.

Validators are initiated and assigned to the validators property when defining the model field; it takes a list.

We can also define our own custom validator; simply write a function which takes a value and raises a ValidationError if required.

In the following example we add the following validation onto our ContactDetails model.

  • clean() validates to ensure that the model does not have the name Luke. This is a poor example; clean should be used to validate between fields or where applicable to touch the database.
    • The code sets the error type identifier; this will default to invalid
    • The error raised takes a dictionary taking the field name and then the error.
  • age integer field has a min and max value of 10 and 100
  • name has a min and length of 3 and 10. We also provide a regular expression to ensure it starts with an upper case letter (A-Z) and then contains at least one lower letter.
  • contactDate takes our custom validator to ensure that the date is in the future.
def is_future_date_validator(value):
    if value <= date.today():
        raise ValidationError("{0} is not a future date.".format(value))


class ContactDetails(Model):
    def clean(self):
        if self.name == 'Luke':
            raise ValidationError({'name': "Luke is barred"}, code="Invalid")

    age = models.IntegerField(
        validators=[
            MinValueValidator(10),
            MaxValueValidator(100)])

    name = models.CharField(max_length=20,
                            validators=[
                                MinLengthValidator(3),
                                MaxLengthValidator(10),
                                RegexValidator("^[A-Z][a-z]{1,}$")])

    contactDate = models.DateField(
        validators=[
            is_future_date_validator])

Calling full_clean raises a ValidationError which contains a dictionary of validation error messages within the message_dict property. We can see the error messages raised with the following test.

from unittest import TestCase
from datetime import date

from django.core.exceptions import ValidationError

from ..models import ContactDetails

class TestCreateAuthorBook(TestCase):

    def test_create_author_book(self):

        try:
            a_contact = ContactDetails(name="Luke", age=101, contactDate=date(year=2015, month=7, day=2))
            a_contact.full_clean()
        except ValidationError as e:

            self.assertEqual({'name': ['Ensure this value has at least 10 characters (it has 4).', 'Luke is barred'],
                              'age': ['Ensure this value is less than or equal to 100.'],
                              'contactDate': ['2015-07-02 is not a future date.']}, e.message_dict)

Overriding Error Messages

Django provides a number of default error messages for various types of invalid data; for example when we try to save a model without setting a mandatory field or if we invalidate a maximum value of the field. We can override the default messages by providing a dictionary of error types and validation messages to the error_messages argument when defining the model field.

 name = models.CharField(max_length=20, 
     error_messages={'blank': 'Please provide a value'
                     'invalid': "Names must start with an upper case letter and contain only letters"})

Valid keys include null, blank, invalid, invalid_choice, unique, and unique_for_date. We can create our own when manually raising ValidationError; see the example above in the clean function.

Error Scope

Error messages are associated to where they have been validated against. Where this is a field it is associated to the field via it’s name. The error messages are found within a dictionary upon the exception raised via the message_dict property. When we raise a ValidationError manually we can specify the origin of the error; in the example above we associate it to the name field.

 def clean(self):
        if self.name == 'Luke':
            raise ValidationError({'name': "Luke is barred"}, code="Invalid")

When we hook in the UI the association is important as it affects where the error message is displayed. Associating it to a field displays it next to the field, otherwise it is displayed at the top above all fields.

Transactions

Django’s default behaviour is autocommit mode; each touch of the database is immediately committed.

We can change this to a transaction to ensure all database touches either pass or fail together; use the with statement along with the atomic class to achieve this.

Leaving the scope of the with statement by passing naturally outside will cause the database to be committed. If an error is thrown inside the with scope which is handled anywhere outside of the with statement the database is rolled back losing all changes made since the start of the with statement.

Note: It is important to ensure that the error handling code is outside of the with scope otherwise the database will be committed if we don’t re-throw the exception.

from django.db import transaction, IntegrityError

def another_function():
    try:
        with transaction.atomic():
            # Touch db
    except IntegrityError:
        # Handle error. 

Alternatively we can decorate a function with the atomic decorator; any touching of the database is wrapped in a transaction. The database is committed when we leave the function without error.

We can manually roll back the database by simply raising an exception which is not caught or handled inside the function decorated by the @transaction.atomic decorator.

from django.db import transaction

@transaction.atomic
def a_function():
    pass

Transactions can be nested; failure only rolls back database touches which are within scope. Take the following example.

with transaction.atomic():       
    MyModel.object.all().first().delete()
    try:
        with transaction.atomic():
            MyModel.object.all().first().delete()
            raise Exception()
    except IntegrityError:
    print("Inner")

We have a two with satement; one nested inside the other.

The outer with statement deletes a record, we then enter the inner with statement where we delete another record.

Immediately after the second delete, still in scope of both with statements, we raise an exception which is caught outside of the scope of the inner with statement. The deletion of the second record is undone.

The exception handler catches the exception and allows code execution to continue naturally outside of the outer with statement causing the database to be committed and saving the initial delete.

The result is only the initial delete is persisted to the database.

Additional Information

  • Model Meta Options can be used for various configs from ordering, table names etc.
    Reference:
  • Managers can be used provident different perspectives of the data.

References

Relationships
Inheritance
Many to Many
Many to One
One to One
Transactions

Django Models Introduction

Models Introduction

This post is part of my Django series. You can see an overview of the series along with instruction on how to get all the source code here.

This article assumes you are comfortable creating a Django project, can create apps and register them into the INSTALLED_APPS list of the settings file. If not please read my Django HelloWorld article.

It also assumes that you have a database created and configured for connection. By default Django comes preconfigured for SQLite3. The database and default schema for the required Django default applications can be created with the following command.

$ python manage.py migrate

This article assumes you have a project called DjangoSandBox and an app called modelsintrodcution.

ORM

Django comes with an inbuilt ORM with drivers for many database vendors. By default it is configured to use SQLite3. The ORM removes, for the most part, interaction directly with the database. Django will create and manage schema along with generating any SQL which is required for selection, inserts, updates and deletes of data.

Any class which is contained within models.py and inherits from django.db.models.Model will have a table created within the configured database. We also get a rich framework for querying, creating, updating and deleting records.

A simple model would look like:

# models.py
from django.db import models

class Sample(models.Model):
    name = models.CharField(max_length=20)

To apply the model where modelsintroduction is the application you have placed the model in.

$ python manage.py makemigrations modelsintroduction
$ python manage.py migrate

We now have a table called modelsintroduction_sample within the database. It has one field called name which is a string field and can contain up to 20 characters.

By default Django will create an id field for any table where the model does not explicity mark any field as the primary key. The auto generated primary key field will be of an integer type and have an auto incrementing identity index.

If we want to change the schema of the table we simply change the model definition and rerun the migration; Django takes care of the rest.

$ python manage.py makemigrations modelsintroduction
$ python manage.py migrate

Note: You need to have consideration when modifying models; under the hood Django makes schema changes via SQL; check the SQL to ensure that the changes are as expected and are possible. Don’t expect to be able to add a mandatory column with no default to a table which is already populated.

Migration scripts are added into the migrations directory of the application directory. You can view the file in an editor or use the sqlmigrate command to view them on the terminal formatted. The following will view the migration named 0001 in the myapp application.

$ python manage.py sqlmigrate myapp 0001

Django also provides the check function to validate that the current outstanding model changes can be applied.

$ python manage.py check myapp

Model Creation

Django has many field types all specialising for a certain situation, each field is also then configurable.

Lets create our model:

#modelsintroduction/models.py
from django.db import models

OPTION_SEX_MALE = 'M'
OPTION_SEX_FEMALE = 'F'
OPTION_SEX_MALE_DESCRIPTION = 'MALE'
OPTION_SEX_FEMALE_DESCRIPTION = 'FEMALE'

OPTIONS_SEX = (
    (OPTION_SEX_MALE, OPTION_SEX_MALE_DESCRIPTION),
    (OPTION_SEX_FEMALE, OPTION_SEX_FEMALE_DESCRIPTION)
)


class Person(models.Model):
    name = models.CharField(max_length=20, unique=True)
    height = models.FloatField()
    date_of_birth = models.DateField()
    sex = models.CharField(max_length=1, choices=OPTIONS_SEX)  
    # get_sex_display() to get the display of MALE/FEMALE
    validated = models.BooleanField(default=False)

    def __str__(self):
        return "{0} - {1} - {2}- {3}- {4}".format(self.name, self.sex, self.date_of_birth, self.height, self.validated)

    class Meta:
        get_latest_by = 'date_of_birth'
  • The name field is a string field of up to 20 characters with a unique index placed on the schema.
  • The height field is a float field.
  • The sex field is a string field with one character only. We provide a choice list of M/F with mapping to Male and Female strings. When we ask our person class it’s sex we return M/F though we can translate this to Male/Femail with get_sex_display(); this function is automatically created by Django upon our model class.
  • The validated field is a boolean field with a default option of False.

Models can contain an inner class called Meta. Here we can set various meta options which are used throughout the Django framework.

Field Types ## {#Field Types}

Django provides a host of field types which you can read about more here. Below is a selection of the most common entities and their settings.

String Fields

  • max_length is the maximum number of characters permitted.
models.CharField(max_length=10)      # Displays as a text box
models.TextField(max_length=100)     # Displays as a text input area; multi lines
models.EmailField(max_length=100)    # Provides regex validation for a email address
models.CommaSeparatedIntegerField(max_length=100)   # List of csv integers

Boolean Fields

models.BooleanField()           # true/false
models.NullBooleanField()       #null/true/false

Integer Fields

models.BigIntegerField() # 64 bit: -9223372036854775808 to 9223372036854775807
models.IntegerField() # 32 bit: -2147483648 to 2147483647
models.PositiveIntegerField() # 0 to 2147483647
models.SmallIntegerField() # -32768 to 32767
models.PositiveSmallIntegerField # 0 to 3276

Real Number Fields

  • max_digits is the maximum number of digits allowed
  • decimal_places is the maximum number of digits allowed to the right hand side of the decimal point
models.DecimalField(max_digits = 8, decimal_places = 2)     # Python Decimal type.
models.FloatField()       # Python Float type. Does not take max_digits or decimal_places

Date Fields

  • auto_now set as true will update the field value to now upon every save
  • auto_now_add set as true will update the field value to now upon record creation
models.DateField()       # Date
models.TimeField()       # Time
models.DateTimeField()   # Date and time 
models.DurationField()   # modelled by Python time delta

Common Field Type Attributes

A list of the common field attributes which we pass into the field when we define it.

Attributes Possible Value Description
null True/False If true sets the nullable constrains on the field in the db
blank True/False Whether Django validation allows no data for the model field. This allows extra control over the null option. We can enforce collection upon certain forms and not on others
choices A list of possible strings A list of possible values for string fields. Acts like an enumerator
db_column String Override the field name created in the table. By default it is set as the name of the field in the model
db_index True/False If true an index is added onto the table field
default Any value of the field type Sets a default value. If no field value provided a record takes this value for this field
editable True/False When Django automatically creates the UI for creating and updating this table, determines if this field will be displayed or not.
error_messages Dictionary of strings Custom error messages
help_text String Defines a help message to be displayed next tot he field upon forms
primary_key True/False Makes the field a primary key
unique True/False Adds uniqueness constraints to the db for the field value
unique_for_date True/False Adds a uniqueness constraints to the db for the date value
unique_for_month True/False Adds uniqueness constraints to the db but only for the month value of a date
unique_for_year True/False Adds uniqueness constraints to the db but only for the year value of a date
verbose_name adds String Provides a custom external name for the table. Used when displaying to the user on the UI. This value defaults to the field name with underscores replaced with spaces and the words capitalised

CRUD

Create, Update and Delete; how we create and maintain our data.

Creation

Djago provides us with a number of ways of creating records.

The first option is via the constructor function, we pass in the fieldnanes and values as **kwargs. Arguments marked with ** collect all named arguments and places them within a dictionary.

In the following example the record is not created until we call save upon the model instance.

from mypass.models import Person

a_person = Person(name="Luke, sex="M", height=1.82, date_of_birth=date(1975, 9, 17))
a_person.save()

We can access all records within the modelsintroduction_person table via the Person model class and the all function. Here we use print to output a string returned from __str__ function of the model to the terminal.

for a_model in Person.objects.all():
    print(a_model)

We can access individual fields via their name.

for a_model in Person.objects.all():
    print(a_model.name)

We can use an empty constructor call and update record via the fields we defined upon the model.

a_person = Person()
a_person.name = "Luke"
a_person.sex = "M"
a_person.height = 1.82,
a_person.date_of_birth = date(1975, 9, 17)
a_person.save()

We can use the get_or_create function to only create our record once, even after subsequent calls. All fields passed in as named/valued arguments are used to predicate the table. If the record already exists it is simply returned.

The function returns a tuple of the record and a boolean indicating if the record was created.

a_person = Person.objects.get_or_create(name="Luke", sex="M", height=1.82, date_of_birth=date(1875, 9, 17))[0]

If we only wanted a subset of the fields to be used to determine if the record exists we can pass the other fields into the parameter named defaults which is a dictionary. The following would only use the name field to determine if the record exists. The second call would simply return the record created in the first call despite having all fields different except name .

a_person = Person.objects.get_or_create(name="Luke", defaults={'sex': "M", 'height': 1.82, 'date_of_birth' : date(1875, 9, 17))

a_person = Person.objects.get_or_create(name="Luke", defaults={'sex': "F", 'height': 2.82, 'date_of_birth' : date(1885, 9, 11))

The update_or_create funcion works similar to the get_or_create though subsequent fields will update the record. Here only one record is created though the state of the record resembles the second function call.

a_person = Person.objects.update_or_create(name="Luke", defaults={'sex': "M", 'height': 1.82, 'date_of_birth' : date(1875, 9, 17))

a_person = Person.objects.update_or_create(name="Luke", defaults={'sex': "F", 'height': 2.82, 'date_of_birth' : date(1885, 9, 11))

The create function is similar to the constructor though no call to save is required.

Person.objects.create(name="Luke", sex="M", height=1.82, date_of_birth=date(1875, 9, 17))

The bulk_create can create a number of records within the same transaction. By default Django runs in a mode called autocommit; all data is committed upon every touch of the database. This means that nothing runs in a transaction unless especially asked for.

people = []
people.append(Person(name="Luke", sex="M", height=1.82, date_of_birth=date(1875, 9, 17)))
people.append(Person(name="Lukey, sex="M", height=1.82, date_of_birth=date(1875, 9, 17)))

Person.objects.bulk_create(people)

Updating

Updating is made by modifying a model instance and then calling save.

tallest_man.validated = True
tallest_man.save()

The save function() can be passed a collection of fields to save; omitted fields are not saved even if they are modified.

a_person.save(update_fields=["name", "sex"])

We can update multiple records with the update command. Here we update all records to be validated.

Person.objects.all().update(validated=True)

Deleting

Deleting is made by calling delete upon a model instance.

a_person.delete()

Cloning Records

There is no specific clone function though we can reset the id to 0 and call save upon the record. Django will generate a SQL insert statement if the id/pk is None, “” or 0, otherwise it will generate a SQL update statement.

a_person.id = 0
a_person.save()

Predicates

Predicates allow the ability to conditionally select data from the database.

All

Whenever we deal with a collection of records or a handle upon the table we deal with a QuerySet. This provides use with the ability to query, order, update and many other things.

The easiest way to get a handle on a QuerySet is to call object.all() upon a model. We can then iterate through the QuerySet.

for a_model in Person.objects.all():
    print(a_model)

Filter

The filter function allows predicating or searching upon the data; a SQL where clause.

Each field on the model can be searched upon by passing key value pairs into the filter function in the format; [field-name]__[operator]=value.

Note: between the field and the operator there are two underscores.

Querying a field directly for equality is made by the [field-name]__[exact] keyword.

Person.objects.filter(id__exact=1)

There is a shortcut to the exact operator; we can simply use the field name.

Person.objects.filter(id=1)

For the primary key field we can also use the ‘pk’ alias which in our case would equate to id.

Person.objects.filter(pk=1)

IsNull

The isnull operator can be used to see if a column has a null value.

Person.objects.filter(name__isnull=False)

Strings

Django provides a number of operators for string fields.

By default exact, when used upon string fields, uses case sensitive searching though ultimately this would depend upon the database and the configuration used.

We can explicitly ask for case insensitive searching with the iexact option.

Person.objects.filter(name__iexact="sophie")

The contains argument will search for a string or characters anywhere within the stored field value. The icontains is a case insensitive version.

Person.objects.filter(name__contains="Sop")
Person.objects.filter(name__icontains="sop")

The startswith and endswith operators allows searching for a string of characters which are found at the start or end of the stored field value respectively. Once more there are case insensitive versions which are prefixed with i.

Person.objects.filter(name__startswith="Sop")
Person.objects.filter(name__endswith="hie")
Person.objects.filter(name__istartswith="sop")
Person.objects.filter(name__iendswith="hie")

If the database supports full-text searching then we can improve performance of string searching by using the search operator.

# SQLite3 does not support full-text search
Person.objects.filter(name__search="sop")

Note: SQLite3 does not allow this functionality.

We can search for complex string pattern with regular expressions and the regex and iregex operators.

Person.objects.filter(name__regex="^[SJ]")
Person.objects.filter(name__iregex="^[sj]")

Dates

Date fields can have their individual components queried with the year, month, day, hour, min, second and week_day operators.

Person.objects.filter(date_of_birth__year=1977)
Person.objects.filter(date_of_birth__month=8)
Person.objects.filter(date_of_birth__day=25)

Choices

For fields which are enumerated with the choice setting we can search upon their field value directly with the exact operator. We can also use the in operator where we are interested in several of the choice values.

Person.objects.filter(sex=OPTION_SEX_FEMALE)
Person.objects.filter(name__in=["Sophie", "Claire", "Jim"])

Comparisons

Comparison operators consist of the greater than (gt), greater than or equal to (gte), less than (lt) and less than or equal to (lte). We can also search for all entries between a min and max value with the range operator. This translates to a SQL BETWEEN clause.

Comparisons operators can be used with all field types; strings, numericals and dates.

Person.objects.filter(height__gt=1.75)
Person.objects.filter(height__gte=1.75)
Person.objects.filter(height__lte=1.75)
Person.objects.filter(height__lt=1.75)
Person.objects.filter(height__range=(1.75, 2.00))

AND & OR

The filter function can take multiple operators separated by commas, by default these use an AND notation. The following would search for people named Sophie and are greater than 1.75 in height.

Person.objects.filter(name="Sophie", height__gt=1.75)

We can use the Q function along with either I or & operators to use a OR or AND notation between predicate entries.

The following searches for People who are called Sophie or Emma. The latter searches for people called Sophie who are 1.7 hight.

from django.db.models import Q

Person.objects.filter(Q(name="Sophie") | Q(name="Emma"))
Person.objects.filter(Q(name="Sophie") & Q(height=1.7))

We can even match the comma (AND) along with Q and I or &. The following searches for women ( SEX=”F”) and who are called Sophie or Emma.

Person.objects.filter(Q(sex="F"), Q(name="Sophie") | Q(name="Emma"))

F

The F function allows us to use the value of one field to search upon another. Here we find anyone who has the letter of their sex (F/M) in their name.

from django.db.models import F

Person.objects.filter(name__contains=F("sex"))

Exists

The exists function returns true/false depending upon if the current QuerySet will yield any records. Calling this upon all() will return true/false depending upon if the table is empty or not.

Person.objects.filter(name="Jimmy").exists()
Person.objects.all().exists()

Chaining

We can chain many QuerySet functions together and even multiple filter calls; this can aid code readability. Below we search for women called Sophie; i.e. AND notation is used.

Person.objects.filter(sex=OPTION_SEX_FEMALE).filter(name="Sophie")

The interesting thing about chaining is that we don’t actually hit the database until we use the data; i.e. looping through, calling first or last upon the QuerySet.

Each QuerySet also has a cache against the query; so even if we were to loop through a QuerySet twice it would only hit the database once. This does not hold true for the all() function upon the model.objects as this returns a new QuerySet each time.

Exclude

The exclude function can be used to provide a predicate which all returned records will not match. It can take any of the operators as defined with the filter function. We can also chain this to the filter function. Here we find people who are female but are not called Sophie.

Person.objects.filter(sex=OPTION_SEX_FEMALE).exclude(name="Sophie")

Ordering

We can order the data with the order_by function along with the field name. This will order the data ascending though we can call reverse to order the data descending.

people_by_height = Person.objects.all().order_by("height")
people_by_height_desc = Person.objects.all().order_by("height").reverse()

We can order upon multiple columns by providing multiple field names.

people_by_height = Person.objects.all().order_by("height", "name")

Calling order_by subsequent times will overwrite the ordering; the following will order the data by the name column, the order upon the height column will be discarded.

people_by_height = Person.objects.all().order_by("height").order_by("name")

First, Last, Latest, Earliest

We can ask for the first or last record within the database or QuerySet. If there is no order criteria upon the QuerySet then the data will be returned ordered by the primary key. If the QuerySet is empty then None will be returned.

Person.objects.all().order_by("name").first()
Person.objects.all().order_by("name").last()

In fact as QuerySet is a collection and as such we can access the elements by their ordinal position; a call to first is equivalent to [0].

Person.objects.all().order_by("name")[0]

For date fields we can use the latest and earliest functions along with a field name or names. This will return the first or last entry once the data has been ordered. If no fields are defined then the field set upon the model’s class meta entry of get_latest_by is used.

Person.objects.latest("date_of_birth")
Person.objects.earliest("date_of_birth")

If the QuerySet is empty then calls to these functions will raise a DoesNotExist error.

Get

Get works exactly the same as filter though it assumes one record will be returned. If no records are returned then DoesNotExist is raised. If multiple records are returned then MultipleObjectsReturned is raised.

Get can take any operators as defined with the filter function.

a_person = Person.objects.get(name="Emma")

Limiting Returned Records

QuerySets respond to collection slicing.

two_elements = Person.objects.all().order_by("name")[1:3]
first = Person.objects.all().order_by("name")[0]

Aggregates

Django provides a few aggregate functions.

The count function can be used to return the number of elements in the QuerySet; it is equivalent to the len function.

all_people = Person.objects.all().count()

Django provides aggregates functions for count, average, minimum, maximum and sum. They are passed into the aggregate function along with the field name.

The aggregate function returns a dictionary of in the format:

{ [field]__[aggregate], value }

For example asking for the average aggregate upon the height column would yield something similar to this:

{ "height__avg", 1.11 }

Below are examples of how to use aggregates:

all_people = Person.objects.all()

sex_count = all_people.aggregate(Count('sex'))["sex__count"]
avg_height = all_people.aggregate(Avg('height'))["height__avg"]
max_height = all_people.aggregate(Max('height'))["height__max"]
min_height = all_people.aggregate(Min('height'))["height__min"]
total_height = all_people.aggregate(Sum('height'))["height__sum"]

The count function has a switch which can be used to return a count of the distinct values.

distinct_sex_count = all_people.aggregate(Count('sex', True))["sex__count"]

We can also call the distinct function which will return a collection of distinct field values. This will generate SQL containing SELECT DISTINCT.

Note: SQLite3 does not support distinct

# Won't work with SQLite3
distinct_sex_count = Person.objects.all().distinct("sex")

Adding Database Indexes

We can add indexes onto the table by either defining the field with db_index=True or via the index_together class Meta option if we require an index with multiple fields.

Here we create an index on field called ‘one’ and another on fields called ‘two’ and ‘three’.

class DbIndexedModelExample(models.Model):
    one = models.CharField(db_index=True)
    two = models.CharField()
    three = models.CharField()

    class Meta:
        index_together = ["two", "three"]

Refresh From DB

A model can have its data refreshed or re-synced from the database with the refresh_from_db function.

a_person.refresh_from_db()

We can limit the fields which are to be reloaded from the database by using the fields parameter.

a_person.refresh_from_db(fields={"name", "height"})

Equality

Model equality in Django uses the python equality or (__eq__ ) function.

By default equality is based upon the types of the model being the same and the equality of the primary key values.

If the model class is a proxy (see the next post) then the mode class is defined as the first non proxy class in the ancestry of the model.

References