Django Complex Query

    Recently I got a chance to work on database query handling in a project. There were lot of objects filtering using field names, some advanced filtering etc. There I met a simple piece of code that was doing something amazing, our Django's Q object. Normally when we use a get or filter on object we specify some parameters inside the filter or get or ... So what this Q object does is it gives you the power to add logical operation to our filtering query. We can do our logical AND or OR operation in Q object. I'll list the syntax of Q object operation one by one.

First of all you need to import the Q object. (obviously...)

from django.db.models import Q 

Consider our models.py as

class Author(models.Model):
    name = models.CharField(max_length=200)
    
    def __unicode__(self):
        return self.name


class Store(models.Model):
    name = models.CharField(max_length=200)
    
    def __unicode__(self):
        return self.name


class Books(models.Model):
    author_name = models.ForeignKey(Author)
    name = models.CharField(max_length=200)
    store_name = models.ForeignKey(Store)

    def __unicode__(self):
        return self.name

Now our logical operations:

AND Operation:

    Search for book written by specific author and in specific store.

Books.objects.filter(Q(author_name__contains='John Doe') & Q(store_name__contains='amazon'))

OR Operation:

Search for book written by specific author or books in specific store.

Books.objects.filter(Q(author_name__contains='John Doe') | Q(store_name__contains='amazon'))

NOT Operation:

Search for book not in specific store.

Books.objects.filter(~Q(store_name__contains='amazon'))

Now suppose you have a list of store names and authors you need to query and its not good to use above approach since length will be high and makes things more complex. For these kinds of situation you can use the following approach:

To check books present in all store. (AND Operation)

my_stores = [list of store names]
q_object = Q()
for store in my_stores:
    q_object.add(Q(store_name__contains=store), Q.AND)
Books.objects.filter(q_object)

To check books present in any one of the store. (OR Operation)

my_stores = [list of store names]
q_object = Q()
for store in my_stores:
    q_object.add(Q(store_name__contains=store), Q.OR)
Books.objects.filter(q_object)

Note: By default Q object operation will be AND

Python Profilers

Code optimization is one of the obstacles that every developer now a days come across. There are lot of tools present now a days to optimize code. You can find python builtin or 3rd party modules for code optimization. Below I have mentioned a few codes optimization tools that I have used.

Builtin Modules

• Profile

  
  
  How To :  First of all import profile into you python code. Then at the main method use the profile.run() method. On executing the py file we will provide the profiler output. An example code is shown below:
  
  

   
  import profile
  
  
  def add(first, second):
      return first + second
  
  
  profile.run(' print add(1,2); print ')
  
  
  

        Profile module also provides runtx( ) for custom profiler generation.
  

• cProfile

  
  
  How To :This module can be used in two ways. One is to import the module and call it as cProfile.run( <your method> )Second is to call the cProfile module at the time of file execution.
  
  

    
     python -m cProfile pythonfile.py
  
  

• pstat

  
  
  How To: Standard report produced by the profile module is not very flexible. So depending upon the need the user can save the output of the run() or runtx() and process it using the Stats class from pstat.

Third Party Modules

• Line Profiler

  
         How To :  First of all install line profile using the following command.
  

   
     pip install line_profiler
  
  

  Then add the @profile above your method definition. Finally execute the code using the following command.

  
    $ kernprof.py -l -v fib.py
  
  

  
  

Converting rml file to pdf

PDF generation using reportlab's rml file

What is RML?
RML is the Report Markup Language - a member of the XML family of languages, and the XML dialect used by rml2pdf to produce documents in Adobe's Portable Document Format (PDF).RML documents can be written automatically by a program or manually using any word processor that can output text files (e.g. using a "Save as Text" option from the save menu). Since RML documents are basic text files, they can be created on the fly by scripts in Python, Perl, or almost any other language. RML makes creating documents in PDF as simple as creating a basic web page - RML is as easy to write as HTML, and uses "tags" just like HTML. It is much easier than trying to write PDF programmatically.
User guide for rml can be found here.


Installation:
 You will have to download and install reportlab plus. You can find the installation steps here.


(Assuming that you have already created a django app)

Now consider the case that I have an xml file which I need to convert to pdf. For that first you will have to create an rml file depending upon your need and save with .rml extension.

RML sample files can be found here.

In views.py import the required modules.

from rlextra.rml2pdf import rml2pdf
import xml.etree.ElementTree as ElementTree
from xml.dom import minidom

Next we write a function to obtain the rml file. The function at first extracts data from the xml file (as dict) the pass it on to the rml through context. Finally the function returns an rml in the specified encoded format  .

def getRML():
    xmldict = {}
    tree = ElementTree.parse('path to your xml file')
    root = tree.getroot()
    xmldict = XmlDictConfig(root)
    t = Template(open('path your rml file').read())
    c = Context({"xmldata": xmldict}) 
    rml = t.render(c)
    #django templates are unicode, and so need to be encoded to utf-8
    return rml.encode('utf16')

Last step is the pdf generation process.

class ConvertView(View):
   

    def get(self, request, *args, **kwargs):
        rml = getRML()
        buf = cStringIO.StringIO()
        
        #create the pdf
        rml2pdf.go(rml, outputFileName='output.pdf')
        buf.reset()
        pdfData = buf.read()
        
        #send the response
        response = HttpResponse(mimetype='application/pdf')
        response.write(pdfData)
        response['Content-Disposition'] = 'attachment; filename=output.pdf'
        return response

Now in urls.py :

from app.views import ConvertView

urlpatterns = patterns('',
    url(r'^admin/', include(admin.site.urls)),
    url(r'^convert/$', ConvertView.as_view(), name="convert2pdf"),
      
)

Finally call the 'convert' url to obtain the pdf document.

PDF generation using easy pdf

 

PDF generation using easy pdf

Easy pdf is a pdf rendering django package. You can install it with the following command:

$ pip install django-easy-pdf
$ pip install "xhtml2pdf>=0.0.6" "reportlab>=2.7,<3"
 

Easy pdf documentation can be found here

Now in your views.py file:

import easy_pdf
from easy_pdf.views import PDFTemplateView 



class MyView(PDFTemplateView):
     template_name = "converter.html"
     pdf_filename = "Output.pdf"


    def post(self, request, args, kwargs):
          objects = MyReport.objects.all()           
          context = { 'objects'  : objects}
          return  easy_pdf.rendering.render_to_pdf_response(request,  self.template_name, context, filename='Output.pdf', encoding=u'utf-8', **kwargs)

Now in urls.py :

from app.views import MyView

urlpatterns = patterns('',
    url(r'^admin/', include(admin.site.urls)),
    url(r'^convert/$', MyView.as_view(), name="convert2pdf"),
      
)

Finally call the 'convert' url to obtain the pdf document.