Category Archives: Django

Fixing a Django/Vagrant error in ifup/ifup-eth

I normally test out a django project using a local vagrant instance. Vagrant creates a virtual machine running the django project which instantly recognizes changes to code. However, I was running a test recently and the VM suddenly started outputting error messages as follows. Note: I’ve redacted some parts of these terminal outputs.

(Scroll to the bottom of this post to skip to the solution)

No translation files? I’ve never heard of them. Thinking it could be a random error, I tried to run the test again and got:

I tried logging out of the instance and got an I/O error:

Attempting to ssh into the VM again is refused:

Sometimes the “turning it off and on again” solution can work, so let’s try vagrant reload:

A timeout error this round. The error seems related to authentication, but it’s not the whole story. The VM is running according to global-status, despite not being correctly set up, which is a bit strange.

I also run a GUI called VirtualBox which is sometimes handy for visualizing all VMs on your laptop. Checking there, it also seems to be present – so global-status wasn’t lying to us. Let’s try halting it:

A forced shutdown this time. So far debugging this error isn’t going well. It is totally strange because the VM was working perfectly beforehand. Vagrant up after this produces the same timeouts:

Let’s try destroying the vagrant instance completely….

And starting from scratch:

That’s a very weird error. Perhaps something failed randomly?

Apparently it still created the VM. SSHing into it gives this:

Bizarrely there are no files! Something still didn’t startup properly.

At this point I was getting a bit frustrated and started to Google the error. The most relevant blog post I could find was Mike Berggren’s solution here: http://mikeberggren.com/post/100289806126/if-up. He fixed this issue and reports: “I’ll spare you the rest of the gory details but suffice it to say, we eventually circumvented that check and ran that same command from the host. It came back with another MAC address claiming ownership“. Does this mean he literally commented out the lines of code that make that check? He may have had a very different problem and I’m no dev-ops expert but perhaps there’s a better solution – maybe that check is there for a reason.

Let’s look back at that network error mentioned before:

Cat the vagrant file and it reveals the same IP address. This means that this particular vagrant is always assigned that IP address. If this is the only instance, how is it that something else on this virtual network is using it? Something inside my mac is conflicting with it.

We can check to see what’s really running (aside from what global-status and VirtualBox say) by checking running processes:

Aha! There are actually 2 of them. Let’s destroy the non-functioning blank VM that we created before and see if anything has changed:

Yup, it has gone:

Use kill -9 [pid] to remove it.

Now let’s try recreating the VM:

It works!

Conclusions:

  • I’m still not sure why this happened in the first place. The errors at the start came totally out of the blue – I’d been running that VM for several weeks without issues. Perhaps it was the fact that I’d been running it for so long?
  • Vagrant global-status and VirtualBox seem to not entirely accurately report running VMs so definitely check all running vbox processes using ps and remove any extras that didn’t shut down properly. This reminds me of the –prune option in global-status (https://www.vagrantup.com/docs/cli/global-status.html) which can fix persistent old entries.

Big thanks to Tony (http://blog.tonns.org/) for helping work this issue out!