Thursday, 18 May 2017

Implementing sitemaps in django application

Sitemap:

  • Sitemap is a collection of web pages of website that are placed in single page that is available for both search engines as well as users.
  • Xml is used to generate sitemap for a website.
  • Sitemap tells a search engine that how frequently your contents in a page changes and follow the priority while indexing the web pages.

Rules of a sitemap:

  • A sitemap can have at most 50,000 URL's.
  • A sitemap file size should not greater than 10 MB.
  • If you have more than  50,000 URL's then you can categorize the URLS and and create sub sitemaps for it.
  • Use sitemap index to hold all the sub sitemaps of a website.
  • Submit the sitemap index to the search engine bots. It will automatically crawls the all web pages  from a given website.

Sitemaps framework in django

  • It automates the creation of sitemap xml
  • We can classify the urls of a website as static and dynamic

Configurations of sitemaps framework

  • We have to add 'django.contrib.sitemaps' in INSTALLED_APPS setting.
  • You must add 'APP_DIRS': True in TEMPLATES setting.
Sitemaps framework provides two classes Sitemap and GenericSitemap.

Sitemap Class:

  • It has the following attributes or methods that will help in creating the sitemap.
  • items:

    • It is a method and It returns list of objects.
    • Every object that is returned from this method will be passed to location(), lastmod(), changefreq() and priority() methods.
  • location:

    • It is either an attribute or a method
    • If it is a method, It takes an item/object as a argument.
    • It's responsible for returning the absolute path.
    • Absolute path should not contains protocol(http, https), domain name.
    • Example: "/contact-us/", "/country/india/"
    • If you do not override it will call "obj.get_absolute_url()" by default.
    • If the object doesn't have get_absolute_url it will raises an error.
  • lastmod: 

    • It is either an attribute or a method.
    • It is responsible for returning a date or datetime object.
    • It returns none by default.
    • It is used to tell a search engine that when the page is last modified.
     
  • changefreq:

    • It is either an attribute or a method.
    • If it is a method, It takes an item/object as a argument.
    • It tells search engines that how frequently a web page changes.
    • Possible values are "always", "hourly", "daily", "weekly", "monthly", "yearly", "never"
  • priority:

    • The priority of this URL relative to other URLs on your site.
    • It's valid values range from 0.0 to 1.0.
    • It only lets the search engines know which pages you deem most important for the crawlers.
    • The default priority of a page is 0.5.
       
  • protocol:

    • It is an attribute. It specifies which version of protocol to use while generating full url.
    • Possible values are "http", "https"
    • Default values is "http"
  • i18n:

    • It is a boolean attribute.
    • If it is true then it will add a language prefix for all urls.
    • By  default it is set to "False"

GenericSitemap:

  • It simply inherits the Sitemap class. So, It gets all the above mentioned properties.
  • It takes arguments "info_dict", "priority", "changefreq", "protocol".
  • info_dict contains two keys 1. "queryset", 2. "date_field"
  •  We don't need to write extra classes for Sitemaps instead we can use it
Now, It's coding time.
If we use sitemap view for urls below 50,000 and sitemap index for above 5,000.

Friday, 12 May 2017

django base template structure for template inheritance

django template inheritance
django template inheritance
  • Generally we develop websites to show our products or services to customers all over the web.
  • Our customers only can find our services or products through search engines like Google, Yahoo, Bing, etc.
  • We/Owner of website register his/her website at search engines to crawl and index our products in the search engine's database.
  • The indexed products appears in the customers search when they search in search engines.
  • Based on the meta data, web page content and speed of the web page(load time) search engine will give rank to our web page.
  • In order to get good rankings in search engines we need to develop our django templates by considering above factors.
  • To reduce the code and to increase the maintainability we use template inheritance in django templates.
Every web page must contain two types HTML tags head and body.

head Tag:

  • Inside head tag we provide information like web page title, search key words, web page description.
  • Above information is useful for search engines. When search engines crawl our web page they look for information inside head tag.
  • It will help us to index our products or services in search engines.
  • It works fine but It would be better if we have a good speed in page loading. This would certainly help us to get better rankings in search engines.
  • web page loads and applies the styles from style sheet resources.
  • In order to get style sheets browser will send a request to the resource server. If server takes more time if the resource size is more. 
  • To reduce the size of style sheets/resources we use compressor tools like "django compressor".
  • It  will reduce stylesheet(css) size so that server can serve it quickly.
  • It's a best approach not to load the javascript resources inside the head tag.

Body Tag: 

  • Inside the body tag we display our information like our services and products to the customer.
  • It's a best approach to load the javascript resources just before the end of body tag.
  •  This will improve the page displaying time and It's better to compress the javascript also by using the "django compressor" tool.

Let's design the base template structure:

{% load compress %}
<!DOCTYPE html>
<html>
  <head>
    <title> {% block title %} Learn Html {% endblock %}</title>
    {% block metadata %}
        <meta charset="utf-8">
        <meta name="description" content="easy way to learn HTML">
        <!-- other html tags here -->
    {% endblock %}
    
    {% block cdn_css %}
        <link rel="stylesheet" 
        href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
    {% endblock %}
    
    {% compress css %}
      {% block local_css %}
        <link rel="stylesheet" href="{% static 'css/main.css' %}">
      {% endblock %}
      
      {% block internal_css %}
        <style>
          body{
           color: black;
          }
        </style>
      {% endblock %}
    {% endcompress %}
  </head>
  <body>
    {% block content %}
     Main content
    {% endblock %}
    
    {% block cdn_js %}
       <script 
        src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
        <!-- other cdn javascript links -->
    {% endblock %}

    {% compress js %}
      {% block local_js %}
      <script src="{% static 'js/main.js' %}"></script>
        <!-- other cdn javascript links --> 
      {% endblock %}
      {% block internal_js %}
        <script type="text/javascript">
          console.log(" I'm working...")
          <!-- extra script -->
        </script>   
      {% endblock %}
    {% endcompress %} 
  </body>
</html> 

title Block:

Title will change from page to page. After extending or inheriting the base.html we can override the title block to define our page title.

metadata Block:

Metadata also changes page to page we can also override/overload it by defining the block in inherited page.

cdn_css Block:

This block is used to link the cdn css files from could that will be used to style the page. Do not include the all css files in base.html file.

local_css Block:

This block is used to link our local external css files. This should be defined below the cdn_css block because we may override/overload styles of cdn css files.

internal_css Block:

Some times we also need internal styles for every page. In this kind of scenarios we can override/overload the internal_css block.

content Block:

This block is used to render our main HTML content.

cdn_js Block:

Some times we use external javascript plugins in some of the pages. If it it the case then override/overload cdn_js block.

local_js Block:

We use this block to link our local javascript libraries to the page. This will improves the page structure.

internal_js Block:

In many cases we write internal javascript. For this type of cases we can override/overload "internal_js" block.

what is the use of writing css blocks inside "compress" tag ?

If we write javascript block inside the compress tag then django compressor will gather all the css data and minimizes it.
Note: do not include cloud css links inside "compress" tag block.

what is the use of writing javascript blocks inside "compress" tag ?

If we write javascript block inside the compress tag then django compressor will gather all the javascript data and minimizes it.

Notes:
  • Do not include cloud javascript or css links inside "compress" tag block.
     
  • Use "{{ block.super }}" to inherit the content from the block in base.html.

Thursday, 4 May 2017

django debug toolbar - improve application performance

django debug toolbar
django debug toolbar

debugging:

  • It is the process of locating and correcting the errors in a computer program or code.
  • debugging is an integral part of software testing.
  • debugging improves  code quality and eases the maintainability of a software.

 debugging tool:

  • A debugging tool is comprised of one or more computer programs that are  used to locate the errors in other programs.

django debug toolbar: 

  • It is one of the best debugging tools for the django applications.
  • debug toolbar makes developer life simple. Developer need not to worry about common lookups like settings, request data, etc.
  • By using this tool we can quickly find out the error location. It helps developers to keep headache away.

tools provided by debug toolbar: 

  • versions:

    • It provides the information of packages that are used in our django application.
    • It lists out package, package name and package version.
    • In some cases we need to have a specific version of a package but we may use advanced version of the package. We may get errors  by using toolbar we can quickly have a look on it. 
    • example: wee need djcelery==2.9.1 but we used djcelery==3.2.1
     
  • time:

    • It helps to use our system resources(CPU) effectively.
    • It lists out the time used by each resource(user CPU time, System CPU time, etc) in milli seconds.
    • It also provides the timing browser when our web page is loaded.
    • It provides the time information like domain lookup, connect, request, response, DOM loading, DOM ContentLoadedEvent, load event.
     
  • Settings:

    • It quickly lists out  our django project settings in a tabular format.
    • We need not go to the ide/editor to see project settings.
  • Headers: 

    • It summaries the information of  HTTP request.
    • It provides information of headers sent in the HTTP request as well as  headers received in the HTTP response.
    • In some cases we need to send authentication information like tokens in HTTP request headers.
    • It helps us to check whether we sent authentication token in headers or not.
  • Request:

    • It contains the information of view, session data, cookies, GET data, POST data
    •  By using this we can quickly navigate to the respective view
    • And also can be able to see data handled by the request
  • SQL:

    • It shows the number of queries executed on a page and time taken each query in milliseconds.
    • It also shows the django equivalent of raw SQL.
    • Reduce the number of queries to improve the application performance.
  • Static files:

    • It lists the static files and paths of static files for quick navigation. 
  • Templates: 

    • It shows template directory paths
    • It  also lists out the templates used in creating the response.
    • It helps us for quick navigation to a template.
  • Cache: 

    • It summaries the  cache of our django application
  • Signals: 

    • It lists out the signals and receivers used in the page rendering.
    • It helps if we write custom signals or receivers.
       
  • Logging: 

    • It shows  logging messages information

How to use django debug toolbar ?

add the below code at the end of the settings file.
settings.py
if DEBUG:
    MIDDLEWARE_CLASSES = MIDDLEWARE_CLASSES + [
        'debug_toolbar.middleware.DebugToolbarMiddleware',
    ]
    INSTALLED_APPS = INSTALLED_APPS + [
        'debug_toolbar',
    ]
    INTERNAL_IPS = ('127.0.0.1', )
    DEBUG_TOOLBAR_CONFIG = {
        'INTERCEPT_REDIRECTS': False,
    }
add the below code at the end of the project urls file. 
urls.py 
if settings.DEBUG:
    import debug_toolbar
    urlpatterns += [
        url(r'^__debug__/', include(debug_toolbar.urls)),
    ]

Sunday, 30 April 2017

django send one time use account activation email

django one time activation link email
  • After success registration of the user, we will send activation link email to the users email account.
  • After clicking the activation link we have to activate users account & make user login. 
  • This process will avoid the fake accounts in our application.
app/urls.py
from django.conf.urls import url
from app import views

urlpatterns = [
    # ......
    url(r'^activate/(?P[0-9A-Za-z_\-]+)/(?P[0-9A-Za-z]{1,13}-[0-9A-Za-z]{1,20})/account/$',
        views.activate_user_account, name='activate_user_account'),
    # ......
]
app/views.py
from django.core.urlresolver import reverse
from django.contrib.auth import login
from django.conf import settings
from django.utils.encoding import force_bytes
from django.utils.encoding import force_text
from django.utils.http import urlsafe_base64_encode
from django.core.mail import EmailMultiAlternatives
from django.contrib.auth.tokens import default_token_generator
from django.core.urlresolvers import reverse
from django.template.loader import render_to_string
from django.http import HttpResponse

from app.models import User


def send_account_activation_email(request, user):
    text_content = 'Account Activation Email'
    subject = 'Email Activation'
    template_name = "emails/account/activation.html"
    from_email = settings.DEFAULT_FROM_EMAIL
    recipients = [user.email]
    kwargs = {
        "uidb64": urlsafe_base64_encode(force_bytes(user.pk)).decode(),
        "token": default_token_generator.make_token(user)
    }
    activation_url = reverse("app:activate_user_account", kwargs=kwargs)

    activate_url = "{0}://{1}{2}".format(request.scheme, request.get_host(), activation_url)

    context = {
        'user': user,
        'activate_url': activate_url
    }
    html_content = render_to_string(template_name, context)
    email = EmailMultiAlternatives(subject, text_content, from_email, recipients)
    email.attach_alternative(html_content, "text/html")
    email.send()


def activate_user_account(request, uidb64=None, token=None):
    try:
        uid = force_text(urlsafe_base64_decode(uidb64))
        user = User.objects.get(pk=uid)
    except User.DoesNotExist:
        user = None
    if user and default_token_generator.check_token(user, token):
        user.is_email_verified = True
        user.is_active = True
        user.save()
        login(request, user)
        return redirect('hr:user_profile')
    else:
        return HttpResponse("Activation link has expired")
emails/account/activation.html
<html>
<title> Email Activation </title>
<body>
    <p> Dear {{ user }}</p>
    <p>Your account has successfully created. Please click below link to activate your account.</p>
    <p><a href="{{ activate_url }}"> Activate your account </a></p>
</body>
</html>
In the last blog post "User Registration" we have seen how to register a user.

Let's start our discussion,
  • "urlsafe_base64_encode" takes user id and generates the base64 code(uidb64)
  • "default_token_generator.make_token" takes the user object and generates the onetime usable token for the user(token)
  • we create activation email using uidb64 and token and send it to user's email.
  • after clicking the activation url it will dispatched to the "activate_user_account" view
  • Here we receive "uidb64", "token". By using the "urlsafe_base64_decode" we decode the base64 encoded "uidb64" user id. We query the database with user id to get user.
  • We check the token validation using "default_token_generator.check_token" and "user object"
Reference: https://github.com/django/django/blob/master/django/contrib/auth/tokens.py#L80
Please comment if you have any questions.
 

Saturday, 29 April 2017

django form usage and validations

django form usage and validations
Let's start with an example, If you are applying for a college admission you will fill up the college admission form with your details and submit it to the college.
It's general process for college admission. No web interactivity.

If we are applying for a college in web. We go to college website and we fill the details in a web form(HTML Form) and submit it.

In backend web application will store the data submitted by the student/user.
To avoid the data redundancy or invalid data we have to validated the form and show the errors to the user to take relevant information from the user.

Developer part:
To perform the above process developer has to write the html forms with hands. It consumes the time of developer. we can avoid writing of html forms with django forms.

Advantages of using the django forms


1. Automated HTML creation
2. Basic validation of data
3. Custom validations for fields
4. Automatic saving of data for model forms

Let's start using Django Forms

Use Case:  User Registration for web application

app/forms.py
from django import forms
from app.models import User

class RegistrationForm(forms.Form):
    first_name = forms.CharField()
    last_name = forms.CharField()
    email = forms.EmailField()
    password = forms.CharField(widget=forms.HiddenInput())
    profile_pic = forms.ImageField()
    address = forms.CharField(widget=forms.Textarea())

    def clean_email(self):
        email = self.cleaned_data.get('email')
        # check and raise error if other user already exists with given email
        is_exists = User.objects.filter(email=email).exists()
        if is_exists:
            raise forms.ValidationError("User already exists with this email")
        return email
    
    def clean(self):
        data = self.cleaned_data
        # use your logic for non field errors
        return data

Django forms consists of three components.


1. Fields
2. Widgets
3. Custom Validators

1.Fields:
  • It is responsible for basic validation of data
  • Every field has default validation logic and  a default widget.
  • It generates the HTML form elements with the help of widgets.
  • HTML form elements "input", "textarea", "select", "checkbox", "radio", etc.
  • Django Form Fields:
2.Widgets:
  • It is responsible for generation of HTML form elements.
  • we can write and use custom widgets
3. Custom Validators
  •  The Custom Validator allows us to create a validation control with customized validation logic.

Levels of validations in django forms


1. Basic Field Validation
2. Custom Field Validation
3. Non Fields Validation / Common Fields Validation  

1. Basic Field Validation
  • It is the first method to validate the data ("to_python")
  • Every field has a basic validation logic based on the type 
  • Example: "EmailField" has the logic to validate whether the given data is valid email or not.
2. Custom Field Validation
  • It is called after basic validation method "to_python".
  • It has method signature as "clean_<field name>"
  • Validation logic goes inside the method
3. Non Fields Validation / Common Fields Validation
  • It is called after the custom field validation method.
  • method signature is "clean"
  • we override it only if we need to validate multiple fields at the same time.

 Handling django forms

form = RegistrationForm(data={}, files={}, initial={})
django form class takes the three initial arguments "data", "files", "initial" if we provide it otherwise it takes it as empty.
data: form data
files: form files
initial: initial data to be rendered with widgets to show to the user.

Form usage in views

def register(request):
    """
        create user & send one time activation email to user
    """
    print(request.POST, request.FILES)
    # make sure to keep enctype="multipart/form-data" in html form
    if request.POST:
        form = RegistrationForm(request.POST, request.FILES)
        if form.is_valid():
            print(form.cleaned_data)
            first_name = form.cleaned_data.get("first_name")
            last_name = form.cleaned_data.get("last_name")
            email = form.cleaned_data.get("email")
            profile_pic = form.cleaned_data.get("profile_pic")
            password = form.cleaned_data.get("password")
            user = User.objects.create(
                first_name=first_name,
                last_name=last_name,
                email=email,
                profile_pic=profile_pic,
                is_active=False
            )
            user.set_password(password)
            user.save()
            print(user)
            # send one time activation email
            # send_account_activation_email(request, user)
            return redirect('books_list')
    else:
        form = RegistrationForm()
    return render(request, 'registration.html', {'form': form})

form.cleaned_data => returns a dictionary with validated data
form.is_valid() => returns a boolean True if form data is valid otherwise False

Form usage in template

{% extends 'base.html' %}

{% block content %}
<div class="wrapper" style="color: blue; padding: 100px;">
        <form action="" enctype="multipart/form-data" method="POST">
            <legend> Registration </legend>
     {% csrf_token %}
     {{ form.as_p }}
     <input type="submit" value="submit" />
 </form>
</div>
{% endblock %}

In the next blog post we will see sending one time use activation link email.
References:
https://docs.djangoproject.com/en/1.11/topics/forms/
https://github.com/django/django/blob/master/django/forms/forms.py

Saturday, 1 April 2017

usage of css, javascript, images in django

In the last post we have seen how to use html template in django view. In this tutorial we will learn about usage of static files in django.

Before we are going to use CSS and JavaScript, we have to know why we use them. Let's start with HTML

HTML - Hyper Text Markup Language

HTML is the basic language of the web. It describes the how a web page should look. When a browser loads the web page it reads the HTML instructions line by line and follows it to present the web page.
Once the HTML instructions in the are completed then the browser stops working, because the HTML program is over.

CSS - Cascading Style Sheets

You may arise a question that we have HTML why we need CSS.

In above picture we are comparing the HTML and HTML + CSS.
HTML just provides us basic structure of a web page, If we apply styles(CSS) to it we can make awesome web pages. 

JavaScript

By using HTML + CSS can only provide then best presentation, but user can't able to interact with the web page. Here comes our superman JavaScript.

JavaScript allows a user to interact with the web page.
For example,
mouse over me
If you mouse over the above box, it will say thanks.
By using JavaScript we can make web page more lively.
We can make animations, games, etc. by using HTML + CSS + JavaScript
Best Websites using JavaScript
http://www.filippobello.com/portfolio
http://www.legworkstudio.com/

Now, It's time to use the static files in our django application.
If you don't setup static settings yet, setup static settings in settings.py

static/img/books_back.jpg
It is the background image used in the template.
static/css/main.css
html, body {
  height: 100%;
  margin: 0;
}
.wrapper {
  min-height: 85%;

  /* Equal to height of footer */
  /* But also accounting for potential margin-bottom of last child */
  margin-bottom: -50px;
  background: #ece4e4;
  background-image: url("/static/img/books_back.jpg");
  background-repeat: no-repeat;
}
.footer,
.push {
  height: 50px;
}
.header{
    height: 50px;
    background: #d4d0d0;
    line-height: 2.5;
    text-align: center;
    font-size: 17pt;
    color: green;
    font-weight: 600;
}
.footer{
    line-height: 3.5;
    background: #d4d0d0;
    text-align: center;
    vertical-align: text-bottom;
}

static/js/main.js
alert("Javascript Test");
console.log("Javascript is working");

templates/home.html

{% load  static %}
<!DOCTYPE html>
<html>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
<link rel="stylesheet" type="text/css" href="{% static 'css/main.css' %}">
<head>
 <title> Library Management </title>
</head>
<body>
 <div class="header">Library Management</div>
    <div class="wrapper">
     <div class="container"> 
       <h2><a href="#" class="btn btn-primary">Login</a></h2>
       <h2><a href="#" class="btn btn-primary">Register</a></h2>
      </div>
     </div>
    </div>
    <div class="push"></div>
  </div>
  <footer class="footer">copyrights &copy; 2017</footer>
</body>
<script type="text/javascript" src="{% static 'js/main.js' %}"></script>
</html>


  • {% load  static %}  will load the built-in template tags from the static library.
  • static is a built-in template tag that will return the static path.
  • {% static 'css/main.css' %} is equivalent to static/css/main.css
  • If it works fine you will get an alert message.

Friday, 31 March 2017

django usage of template and view

django usage of template and view
django templates & views

In the last blog post we have seen how to write models for project. In this blog post we are going learn about usage of templates in django views.

Before start writing the view read the django request life cycle for better understanding.

Url dispatcher passes the request to the view. View is responsible for the processing of the request. As we know django follows the MVT architecture.

M -  model
V  -  view
T  -  template

  1. Model's allow us to connect with the databases for information.
  2. View takes the advantage of django models to get the information from the database[using queries] and processes it. We write business logic in views.
    We process the data in views and passes it to the template for showing processed data to the user.
  3. Template takes the processed data from the view and represent the data in a required formats like html, xml, json, etc.

We have discussed the theory part. Let's start the coding.
library_management/urls.py
from django.conf.urls import url, include

urlpatterns = [
    ....
    url(r'^', include('library.urls')),
    ....
]
This is the root urls file of our application. 'include' will add the app(library) urls. Based on the regular expression pattern users request will be served.

library/urls.py
from django.conf.urls import url
from library import views

urlpatterns = [
    url(r'^$', views.home, name="home"),
]
library/views.py
from django.shortcuts import render

def home(request):
    return render(request, 'home.html', {'say_hello': 'Hello World'})
templates/home.html
<!DOCTYPE html>
    <html> <head>
        <title> Library Management </title> </head>
    <body>
        <h1> {{ say_hello }}</h1>
    </body>
</html>
We have imported 'views' module from the library app and mapped it to url. Whenever we request the url http://localhost:8000 in the browser it will the passed to the function home.
If we observe library/views.py we have imported the render from the django shortcuts module. render takes request, template name(path from template directories) and context data(the dictionary) and renders the template with context data and returns the http response. Browser receives the response and displays it to the user.
{{ say_hello }}  replaces with Hello World. Context is a dictionary with keys and values. say_hello is a key in the context so it will replaced by it's value.
It's just like string formatting in python.