Home » django

Tag: django

Cherokee, uWSGi, Django, Python server set up.

Preamble:

As promised in my last blog post this post is going to be all about how to set up a server to run Cherokee with uWSGI for deploying Django or other Python applications.

Server setup:

I am going to make some assumptions about your server environment:

  1. You are running Ubuntu 10.04 or similar. Older versions may have some changes that need to be made.
  2. This is a brand new server or VPS running at either Slicehost or The Rackspace Cloud. Other companies may work but I’ve only tested here.
  3. You have set the machine up, at least doing the steps here and here.
  4. You have a Django project you can deploy.
  5. You are running an operating system that is intelligent enough to give you a shell and open ssh tunnels.

Now that your server is set up you can run this shell script to install all of the software we will be using:

Yes I am installing Python modules server wide and no I don’t care about the problems that can come with that, since, I am only going to be running one application from this server. If you think you are going to have a problem with that install the python modules in the appropriate directories or sub-directories how ever you want to do that.

Now that everything is installed, let’s get it set up. Here are the steps:

  1. Set up our postgres username, password, and database. Here is a link to get setup.
  2. Copy your Django project to: /home/username/
  3. Sync your database
  4. Start the cherokee admin by running: cherokee-admin
  5. Open a second shell and run: ssh -l username -L 9090:localhost:9090 IPADDRESSFORYOURSERVER (Now you can go to localhost:9090 on your computer’s browser and get to the Cherokee admin interface.) You’ll need to use the username and password that come up from Step 4 above.

You can now start the Cherokee server by clicking the start server button (cool huh?) Once it is started follow the directions here:
http://www.cherokee-project.com/doc/cookbook_uwsgi.html
Edit your django_wsgi.py and uwsgi.xml files to match your application’s settings. Once that is done you should be able to start your site as long as things are correct. Oh, and don’t put your Django files in /var/www.

Configuration Options:

Here are some more configuration options:

1. Have the Cherokee server match multiple domains to the same virtual host:

Dock

  1. a. Serve your admin media: 

Dock

b. Create a new rule as type Directory with the web directory being your admin media path (in this case mine is /media):

Dock

c. Change to the Handler tab choose Static Content and put the full path to your admin media in the Document Root including the trailing slash:

Dock

You can play with the other settings for it like gzip, deflate, and traffic shaping.
3. Change the user that the uWSGI process runs as, click Sources and select your uWSGI app:

Dock

I had to change this to the user where my code was deployed: /home/username/ for things to work correctly.

  1. Traffic monitoring. You can enable it by going to Status, then clicking on the virtual server, and checking enable at the bottom. You may need to choose RRDTool Graphs in General -> Network -> Information Collector:

Dock

Dock

  1. Looking at your log files if something isn’t working:
    You just need to cat or tail /var/log/cherokee/cherokee.error. If you want it to log somewhere else you can configure it by clicking vServers -> your vServer -> Logging and changing the path there:

Dock

Tips and Tricks:

If you get something like:
uWSGI Error, wsgi application not found
Make sure that you have set the user that the uWSGI process runs as (step 4 in Configuration Options), save and hard restart the server.

New Django/Python deployment stack: Django, Cherokee, and uWSGI.

What I had:

In the past couple of months I have been looking for alternatives to my current web stack. My current web stack is:

  • Django (my favorite web framework),
  • nginx (frontend web server),
  • apache + mod_wsgi (to serve the Django application), and
  • PostgreSQL the database, obviously.

Django isn’t getting replaced because I like it and I can’t get my coding partner to switch to anything else no matter how magical or revolutionary.

I fucking hate nginx configuration files so that was going out the window as soon as I could find a suitable replacement. Apache + mod_wsgi I’m not entirely unhappy with but the extra memory usage from Apache bloatware and, again, the ugly configuration files are annoying, especially, with non-standard PYTHONPATHs. Yes, I know I can do sys.path magic to work around it but I shouldn’t have to, this is 2010 not 1990, fuck that.

PostgreSQL is a great database server and I’m happy that the Django ORM just works with it so I don’t have to build my own ORM, big plus. I would love to give CouchDB a chance but I don’t have the time to rewrite _everything_ to work with it, I’ll use it with something else.

What I Found:

Looking for other web servers I came across Cherokee (http://www.cherokee-project.com/). The documentation on the Cherokee site is phenomenal, when have you EVER seen screen casts for how to configure your web server? Oh, and what is that? A screencast about deploying Django using uWSGI or Flup? Why, yes, that what you are seeing, isn’t that great? That isn’t the only good thing, Cherokee stacks up, speed wise, with nginx, has plenty of configuration “wizards”, and has a web front end for configuring the web server. I was mostly sold just from the documentation, compare this with this. The Apache docs may be informational but they are completely worthless if I want to get something done, today.

Configuring Cherokee is like using the WebFaction hosting interface, easy to use and as powerful as you want it to be. In fact, in the next blog post about how to set everything up you can see the similarities. (Disclaimer: I work for WebFaction so I am a little biased, however, in the 10 years that I have been dealing with web hosts I have never seen another hosting company build something so great at what it does: allow you to do as much or as little as you need, to get things done.)

After hours of looking through the different ways to serve WSGI applications I finally settled on uWSGI, mostly because it is written completely in C and has a really easy install process (read: one command, make). uWSGI is also able to used with Cherokee and Django really easily, see the screencast here: http://www.cherokee-project.com/screencasts.html#django_uwsgi. Now that is what I was looking for, easy to configure, easy to use, powerful, and fast. Also, setting the PYTHONPATH is as simple as 1 or more XML tags, like so:

<pythonpath>/path/you/are/adding/</pythonpath>

Fantastic! Have another path you want used? Add another line, that’s it.

Processes are automatically managed and restarted when they are killed. uWSGI has amusing errors, such as:

  • …brutally killing workers…
  • DAMN ! process 14955 died 🙁 trying respawn …

Log files are much more fun to read when they include some humor even if shit is wildly broken.

Next post will be about how to set all of this up, from nothing to a working deployment of a working Django app using Cherokee and uWSGI.

 

Building a community, 1 year invested.

Building a community is hard work. There are a lot of steps that have to be taken to attract visitors, draw them in, and retain them. Fortunately, this project came with a community that was already there, just dispersed across 3 or 4 different sites on forums; 1 point for us. The downside is that the hunting/fishing crowd are not really very technical, are used to sites that are visual trash like this one, and are habitual people so they don’t leave the community they are already in if there is not some incentive to move.

One year ago today my company launched our first site, trackmytrophies.com. Of course, the company didn’t actually exist yet and wouldn’t for over a month, we were brand new to the technology stack we were building the site with, neither of us were extremely familiar with design or design tools, and we both had full time jobs working for someone other than ourselves. The site was built on Pinax 0.5.1. There was an early beta of Pinax 0.7 at that point but, for people new to the Django/Pinax world it was completely impossible to deploy, I spent ~17 hours, over 2 days, trying without success to deploy the site. At that point I decided to revert to the stable Pinax 0.5.1  which still took me 8 hours to deploy, but deploy I did.

Over this last year a lot of things have changed: we have become a lot more familiar with the platform we are using, our design skills and tools have increased, and we are working for ourselves…technically (we actually work for WebFaction). We have experimented with different layouts and designs, worked with serving our own ads (OpenX community is shit, don’t use it…), we’ve broken shit, read upgraded Django to an incompatible version, and we’ve even helped patch a minor bug upstream.

Enough of the boring stuff, here are some community stats:

  • 297 registered users
  • 612 uploaded photos
  • 10,000 total visits

…and we are not done yet. We have  a lot of ideas that we are still working on; there is a completely new design coming along with *many*, other backend upgrades that will be transparent for end users.

Here’s to the next year ending stronger than the first!

Thoughts about Java and Python.

In my Computer Science course I was asked to write a program, in Java, that would compute the powers of 2 from 2^1 to 2^20 using only a while statement. At first I tried, with just the basics that we had learned so far, to create the program. Things did not go well because I couldn’t get the powers to work correctly using pre or post decrements. After I looked through the extensive Java Math library here is the solution I came up with:

Coming from a completely Python world, I wondered how I would have implemented the same program in Python. This solution is probably not the most elegant solution but Java was very heavy on the brain while I wrote it.

A couple of things I really missed from Java when coming back to writing Python were:

  1. Post and pre increment, I understand why Python doesn’t have them but damn are they useful
  2. Brackets for the while statement, I know brackets are horrible and messy but it really makes the start/stop of a code block easy to follow…fuck Ruby and its “end” command that is just ugly.

I really like the way Python’s “print” values work. It is much nicer to use %s inside the code and then add % (count, multiplier**count) at the end of the statement rather than using the “+” and variable names.

In the Python world Java gets a lot of shit from a lot of people. Granted, Java does have its share of problems and oddities but it is an extremely powerful, well backed language. Oracle’s support of Java after their acquisition of Sun Microsystems has dramatically increased since a great number of their products are built on Java. Probably the strongest piece of the Java programming language is the Java Virtual Machine. The JVM is so versatile, that, practically, every language can be run on it. There are implementations for Ruby (JRuby), Python (Jython)[1], and even, my favorite web framework, Django (Django-Jython)[2].

With Django and Python gaining traction in the Enterprise world Django-Jython and Jython are perfect building blocks for organizations that *have* to run Java for some of their applications or middleware but want the ease of use that comes with Django and Python for their internal or even external web teams. I like where things are going and I am on the bandwagon with both languages.

[1]: http://www.jython.org/

[2]: http://code.google.com/p/django-jython/

WTF is with "developers" using Windows to write code? *updated*

Seriously, WTF? Demo? Check out this moron.

I read the #django-users’ list and it seems that, almost daily, there is someone that has a problem installing django on Vista or XP. Who would host a server running Vista or XP? I hope, NO ONE. What kind of moron would trust Microsoft with his/her data? Also, seriously, who runs server farm hosting businesses on Windows? I know people do, but, self flagellation would be a little less painful, really.
I read the mod_wsgi users’ list and it seems that, almost daily, someone has a problem installing mod_wsgi on Windows. (I have had my fair share of mod_wsgi problems (see below)) but a local Apache server on Windows, c’mon people.
Most of the screencasts I have seen about anything django or Python related has been done on Windows as well. Save us FSM (flying spaghetti monster).
I guess I have to applaud their efforts, these lost intellects, I got off the Windows juice many years ago. I have been consistently Windows free for the better part of 6 years. What should they use? Unix, almost any flavor. I was never much for the distros that had BSD in the name, my hardware was too advanced to install it (god I miss those days). Nowadays, I run OS X on my MBP and OpenSolaris on my in home staging server. Both Unix, both very powerful and all that. Both > Windows.
Seriously “developers” get an operating system that will allow you to actually have a key to the hood so you can fix your own itches.
(This kind of ended up as an opensource parade but it is still true; friends don’t let friends use Windows.)

Installing mod_wsgi on OpenSolaris.

*Not for the faint of heart. (See below)*

This is the process I followed to install mod_wsgi on OpenSolaris 2008.11.
First I would like to thank Graham Dumpleton and Thurner Rupert from the mod_wsgi mailing list and @yksingisyyteni from #opensolaris.
The way to do it:
When trying to install mod_wsgi with a fresh install of OpenSolaris and the amp-dev, gcc-dev, Sun Studio Express, and all other relevant packages; I kept ending up with an error saying, “/opt/SUNWspro/bin/cc not found”. WTF is SUNWspro? Let me tell you: SUNWspro is a piece of the Sun Studio 12 program used to compile C for the Solaris platform. It is the same thing that compiled the version of Apache you installed through PKG. Which means that that GNU cc program that you chose to download is not going to work to compile it. What you need to do is get Sun Studio 12. Once you have it you can follow the instructions on the Sun website for how to install it (which I could never get to work) or you could do this:
  • bunzip2 -vv SunStudio…tar.bz2 (this will take some time; let it run)
  • tar xvf SunStudio…tar (this will also take some time)
  • ./installer & (now wait)
  • Follow the GUI tools to finish the install.

Once you have Sun Studio installed you can compile like the Installation Instructions say. Just a quick make, make install, edit your apache conf file and you are done.

*Not the way to do it:
This is what I had to do to get to the final, finished product you see above.
  • Realize I don’t know what SUNWspro is.
  • Think, “Fuck, I will just reinstall.”
  • mailing list, irc
  • Reinstall, damn it still doesn’t work.
  • I know, I will try to install Sun Studio.
  • I am running bzcat SunStudio…tar.bz2 | tar xvf – and it is not working.
  • I waited…
  • and waited…
  • OK, scratch that. I will compile my own versions of Apache, MySQL, and any other thing that I would need to run a server.
  • That was fun. Apache is a blast to compile.
  • MySQL…pkg, WTF? I can’t just double click?
  • I will install the tar MySQL.
  • Fuck, this takes a long time and is kind of hard.
  • Pkgadd -d, wait that’s how I can add the MySQL.pkg file?
  • rm -rf mysql
  • pkgadd -d mysql.pkg
  • mysqld
  • ERRORS!!!1!111!!
  • Much “googling”
  • Damn. I got mod_wsgi to work I don’t want to reinstall again.
  • mysql ERRORS!!1!!1!!
  • Reinstall
  • Now I know the drill
  • Install amp-dev and all other relevant packages
  • Install SunStudio
  • Compile mod_wsgi
  • HOORAY!!!