Home » python

Category: python

Cherokee, uWSGi, Django, Python server set up.

Preamble:

As promised in my last blog post this post is going to be all about how to set up a server to run Cherokee with uWSGI for deploying Django or other Python applications.

Server setup:

I am going to make some assumptions about your server environment:

  1. You are running Ubuntu 10.04 or similar. Older versions may have some changes that need to be made.
  2. This is a brand new server or VPS running at either Slicehost or The Rackspace Cloud. Other companies may work but I’ve only tested here.
  3. You have set the machine up, at least doing the steps here and here.
  4. You have a Django project you can deploy.
  5. You are running an operating system that is intelligent enough to give you a shell and open ssh tunnels.

Now that your server is set up you can run this shell script to install all of the software we will be using:

Yes I am installing Python modules server wide and no I don’t care about the problems that can come with that, since, I am only going to be running one application from this server. If you think you are going to have a problem with that install the python modules in the appropriate directories or sub-directories how ever you want to do that.

Now that everything is installed, let’s get it set up. Here are the steps:

  1. Set up our postgres username, password, and database. Here is a link to get setup.
  2. Copy your Django project to: /home/username/
  3. Sync your database
  4. Start the cherokee admin by running: cherokee-admin
  5. Open a second shell and run: ssh -l username -L 9090:localhost:9090 IPADDRESSFORYOURSERVER (Now you can go to localhost:9090 on your computer’s browser and get to the Cherokee admin interface.) You’ll need to use the username and password that come up from Step 4 above.

You can now start the Cherokee server by clicking the start server button (cool huh?) Once it is started follow the directions here:
http://www.cherokee-project.com/doc/cookbook_uwsgi.html
Edit your django_wsgi.py and uwsgi.xml files to match your application’s settings. Once that is done you should be able to start your site as long as things are correct. Oh, and don’t put your Django files in /var/www.

Configuration Options:

Here are some more configuration options:

1. Have the Cherokee server match multiple domains to the same virtual host:

Dock

  1. a. Serve your admin media: 

Dock

b. Create a new rule as type Directory with the web directory being your admin media path (in this case mine is /media):

Dock

c. Change to the Handler tab choose Static Content and put the full path to your admin media in the Document Root including the trailing slash:

Dock

You can play with the other settings for it like gzip, deflate, and traffic shaping.
3. Change the user that the uWSGI process runs as, click Sources and select your uWSGI app:

Dock

I had to change this to the user where my code was deployed: /home/username/ for things to work correctly.

  1. Traffic monitoring. You can enable it by going to Status, then clicking on the virtual server, and checking enable at the bottom. You may need to choose RRDTool Graphs in General -> Network -> Information Collector:

Dock

Dock

  1. Looking at your log files if something isn’t working:
    You just need to cat or tail /var/log/cherokee/cherokee.error. If you want it to log somewhere else you can configure it by clicking vServers -> your vServer -> Logging and changing the path there:

Dock

Tips and Tricks:

If you get something like:
uWSGI Error, wsgi application not found
Make sure that you have set the user that the uWSGI process runs as (step 4 in Configuration Options), save and hard restart the server.

New Django/Python deployment stack: Django, Cherokee, and uWSGI.

What I had:

In the past couple of months I have been looking for alternatives to my current web stack. My current web stack is:

  • Django (my favorite web framework),
  • nginx (frontend web server),
  • apache + mod_wsgi (to serve the Django application), and
  • PostgreSQL the database, obviously.

Django isn’t getting replaced because I like it and I can’t get my coding partner to switch to anything else no matter how magical or revolutionary.

I fucking hate nginx configuration files so that was going out the window as soon as I could find a suitable replacement. Apache + mod_wsgi I’m not entirely unhappy with but the extra memory usage from Apache bloatware and, again, the ugly configuration files are annoying, especially, with non-standard PYTHONPATHs. Yes, I know I can do sys.path magic to work around it but I shouldn’t have to, this is 2010 not 1990, fuck that.

PostgreSQL is a great database server and I’m happy that the Django ORM just works with it so I don’t have to build my own ORM, big plus. I would love to give CouchDB a chance but I don’t have the time to rewrite _everything_ to work with it, I’ll use it with something else.

What I Found:

Looking for other web servers I came across Cherokee (http://www.cherokee-project.com/). The documentation on the Cherokee site is phenomenal, when have you EVER seen screen casts for how to configure your web server? Oh, and what is that? A screencast about deploying Django using uWSGI or Flup? Why, yes, that what you are seeing, isn’t that great? That isn’t the only good thing, Cherokee stacks up, speed wise, with nginx, has plenty of configuration “wizards”, and has a web front end for configuring the web server. I was mostly sold just from the documentation, compare this with this. The Apache docs may be informational but they are completely worthless if I want to get something done, today.

Configuring Cherokee is like using the WebFaction hosting interface, easy to use and as powerful as you want it to be. In fact, in the next blog post about how to set everything up you can see the similarities. (Disclaimer: I work for WebFaction so I am a little biased, however, in the 10 years that I have been dealing with web hosts I have never seen another hosting company build something so great at what it does: allow you to do as much or as little as you need, to get things done.)

After hours of looking through the different ways to serve WSGI applications I finally settled on uWSGI, mostly because it is written completely in C and has a really easy install process (read: one command, make). uWSGI is also able to used with Cherokee and Django really easily, see the screencast here: http://www.cherokee-project.com/screencasts.html#django_uwsgi. Now that is what I was looking for, easy to configure, easy to use, powerful, and fast. Also, setting the PYTHONPATH is as simple as 1 or more XML tags, like so:

<pythonpath>/path/you/are/adding/</pythonpath>

Fantastic! Have another path you want used? Add another line, that’s it.

Processes are automatically managed and restarted when they are killed. uWSGI has amusing errors, such as:

  • …brutally killing workers…
  • DAMN ! process 14955 died 🙁 trying respawn …

Log files are much more fun to read when they include some humor even if shit is wildly broken.

Next post will be about how to set all of this up, from nothing to a working deployment of a working Django app using Cherokee and uWSGI.

 

Patches or suggestions aren’t always welcome.

I believe that Zed Shaw adequately summed up the state of Python learning material, my time trying to learn enough Python from online using the material he is talking about was completely pointless because of it being outdated.

The couple of responses he got was less than “patches welcome.”

So Zed put up in an epic fucking way. Thanks Zed, now new programmers don’t have to struggle with the shit that was there.

Thoughts about Java and Python.

In my Computer Science course I was asked to write a program, in Java, that would compute the powers of 2 from 2^1 to 2^20 using only a while statement. At first I tried, with just the basics that we had learned so far, to create the program. Things did not go well because I couldn’t get the powers to work correctly using pre or post decrements. After I looked through the extensive Java Math library here is the solution I came up with:

Coming from a completely Python world, I wondered how I would have implemented the same program in Python. This solution is probably not the most elegant solution but Java was very heavy on the brain while I wrote it.

A couple of things I really missed from Java when coming back to writing Python were:

  1. Post and pre increment, I understand why Python doesn’t have them but damn are they useful
  2. Brackets for the while statement, I know brackets are horrible and messy but it really makes the start/stop of a code block easy to follow…fuck Ruby and its “end” command that is just ugly.

I really like the way Python’s “print” values work. It is much nicer to use %s inside the code and then add % (count, multiplier**count) at the end of the statement rather than using the “+” and variable names.

In the Python world Java gets a lot of shit from a lot of people. Granted, Java does have its share of problems and oddities but it is an extremely powerful, well backed language. Oracle’s support of Java after their acquisition of Sun Microsystems has dramatically increased since a great number of their products are built on Java. Probably the strongest piece of the Java programming language is the Java Virtual Machine. The JVM is so versatile, that, practically, every language can be run on it. There are implementations for Ruby (JRuby), Python (Jython)[1], and even, my favorite web framework, Django (Django-Jython)[2].

With Django and Python gaining traction in the Enterprise world Django-Jython and Jython are perfect building blocks for organizations that *have* to run Java for some of their applications or middleware but want the ease of use that comes with Django and Python for their internal or even external web teams. I like where things are going and I am on the bandwagon with both languages.

[1]: http://www.jython.org/

[2]: http://code.google.com/p/django-jython/

Meta and random musings.

Shorter (read: less content) blog post this week because life has been overly strenuous this week. School has become, almost overwhelmingly, full of homework and reading assignments as well as programming assignments. Besides that, work has been extremely tiring because of the workload and being short handed (for completely valid reasons –  I don’t blame any one for being where they are, under the same circumstances I would do the exact same things). Life at home has also been stressful because of all of the other factors.

Perhaps the most amusing thing I have seen today is http://www.michaelv.org/ which is a Javascript rendition of Windows 3.1 (the third operating system I ever used the first being DOS the second Apple something). This is where the meta part of this post comes in. Inside of the Javascript Windows 3.1 you are allowed to open a browser and because the browser runs Javascript you can once again open the same web page (Windows 3.1) over and over again, currently I am at 10 windows open but the screen shot only shows 2:

This can go on, I am assuming, indefinitely as long as you correctly format your window sizes. This  project gets 5/5 pyl0ns from me.

More amusing software that I found today, because of Jeff Croft (http://twitter.com/jcroft/status/81415197100) is exactly what I was looking for to make my transition from work, where I use Firefox as the default browser, to my regular browser (Chrome|Safari) much easier. I had looked for AppleScripts and tutorials to see if I could find some way of programmatically setting which browser was my default but my Google-Fu failed me. Now I can simply create 2 “programs” that can be launched from LaunchBar that will set my default browser without all of the extraneous clicking, plus it runs on Jython which is just cool since at this point in my life Java and Python are the only things I am focusing on. I highly suggest checking Project Sikuli out, there has to be something in your life that you can automate.

TextMate to VIM and back, why I made the switch.

About 6 months ago I made the switch from TextMate to MacVim because of a couple things:

  • TextMate is kinda, not really being developed anymore
  • Many other options are out there
  • TONs of Python developers post about their editors all the time
  • VIM is hardcore and installed by default on many Linux systems (which I work with all day)
  • I wanted to take a side in the holy war (Emacs vs. VIM)

After doing a lot of research on the editors that were available I tried out Aquamacs and played around with it for a while. Emacs is powerful, as long as you install the right plugins…and know LISP(ugh!). Emacs is just too much pressing of the control button and shift and other modifier keys, I have carpal tunnel  and pressing Control + (modifier key) is too much if I have to do it repeatedly.

I switched to MacVim, fewer modifier keys, much simpler to use and no knowledge of LISP is required. VIM syntax is much better, but still coding to be able to code isn’t really ideal for me. Luckily I found the right plugins and the right tutorials and got everything working, I even switched to MacVim as the default editor for everything. I spent the next 5 months learning how to use MacVim and since, I got a new job, doing support for the best hosting company in the world, using VIM was very helpful.

Recently on Twitter I saw this: http://justinlilly.com/blog/editor-stats/ After looking through the screenshots of all of the different editors that are out there I began to miss using TextMate with its Cocoa-ness and modern feel, VIM is very archaic and keyboard dependent, while being ideal for some people is not idea for me). I installed all of the bundles that I wanted, and, there was a bundle for everything that I wanted. IMO, using “svn co …” is much easier than downloading, unzipping, and copying the files to the right directories and editing vimrc to make sure that the new modules are being used when they are supposed to be.

So now I am back to using TextMate as my default editor and I am enjoying it tremendously. There is no coding involved to change the background color, no command to invoke to have the sidebar open, no navigating with the keyboard only when using NERDTree, no more having to type NERDTree, and most of all bundles are so much better, faster to install and modify than with VIM.

I am still, eagerly, awaiting the release of TextMate 2 but for now I am happy to be able to change things without code so that I can spend my coding time doing something productive instead of configuring my editor.

Installing mod_wsgi on OpenSolaris.

*Not for the faint of heart. (See below)*

This is the process I followed to install mod_wsgi on OpenSolaris 2008.11.
First I would like to thank Graham Dumpleton and Thurner Rupert from the mod_wsgi mailing list and @yksingisyyteni from #opensolaris.
The way to do it:
When trying to install mod_wsgi with a fresh install of OpenSolaris and the amp-dev, gcc-dev, Sun Studio Express, and all other relevant packages; I kept ending up with an error saying, “/opt/SUNWspro/bin/cc not found”. WTF is SUNWspro? Let me tell you: SUNWspro is a piece of the Sun Studio 12 program used to compile C for the Solaris platform. It is the same thing that compiled the version of Apache you installed through PKG. Which means that that GNU cc program that you chose to download is not going to work to compile it. What you need to do is get Sun Studio 12. Once you have it you can follow the instructions on the Sun website for how to install it (which I could never get to work) or you could do this:
  • bunzip2 -vv SunStudio…tar.bz2 (this will take some time; let it run)
  • tar xvf SunStudio…tar (this will also take some time)
  • ./installer & (now wait)
  • Follow the GUI tools to finish the install.

Once you have Sun Studio installed you can compile like the Installation Instructions say. Just a quick make, make install, edit your apache conf file and you are done.

*Not the way to do it:
This is what I had to do to get to the final, finished product you see above.
  • Realize I don’t know what SUNWspro is.
  • Think, “Fuck, I will just reinstall.”
  • mailing list, irc
  • Reinstall, damn it still doesn’t work.
  • I know, I will try to install Sun Studio.
  • I am running bzcat SunStudio…tar.bz2 | tar xvf – and it is not working.
  • I waited…
  • and waited…
  • OK, scratch that. I will compile my own versions of Apache, MySQL, and any other thing that I would need to run a server.
  • That was fun. Apache is a blast to compile.
  • MySQL…pkg, WTF? I can’t just double click?
  • I will install the tar MySQL.
  • Fuck, this takes a long time and is kind of hard.
  • Pkgadd -d, wait that’s how I can add the MySQL.pkg file?
  • rm -rf mysql
  • pkgadd -d mysql.pkg
  • mysqld
  • ERRORS!!!1!111!!
  • Much “googling”
  • Damn. I got mod_wsgi to work I don’t want to reinstall again.
  • mysql ERRORS!!1!!1!!
  • Reinstall
  • Now I know the drill
  • Install amp-dev and all other relevant packages
  • Install SunStudio
  • Compile mod_wsgi
  • HOORAY!!!