Shell script to download files from anywhere

As the title suggests, this is just a simple script I figured out for downloading files. The list of urls for downloading are located in an xml document that I'm parsing, storing briefly, looping through and downloading what I wanted to a predetermined location (though that could be modified for user input).

Script:

read_dom () {
    local IFS=\>
    read -d \< ENTITY CONTENT
}

while read_dom; do
    if [[ $ENTITY = "xmlTag" ]]; then
echo $CONTENT
if [[ $CONTENT != "null" ]]; then
wget -Nrkpl 0 $CONTENT -nd -Pdownloads
fi      
    fi
done < listOfXML.xml > temporaryXMLPage.txt

echo "File cleanup"
rm temporaryXMLPage.txt

source:
http://stackoverflow.com/questions/893585/how-to-parse-xml-in-bash
and
http://stackoverflow.com/questions/15407884/javascript-download-images-from-url

Using Drush to deploy themes and modules to multi-sites on CentOS

Previously I discussed setting up multi-site on CentOS, now, using Drush, we can also download, enable, and set themes and modules for individual sites.  The default Drupal7 site gets all the glory of files downloaded to it, but the individual sites should be the ones showing the updates.

Steps to use Drush:

1) cd /var/www/vhosts/drupal7/sites/your.alternative.site
get into the correct folder that you want to have the theme enabled
2) drush dl theme
This is the last part of a url available from drupal, i.e. drupal.org/project/omega would use omega

3) drush en theme
This enables the them for all sites. From what I understand, the files themselves get downloaded to the main drupal7/sites/all/themes/ directory. Somehow Drupal knows to find the themes for each individual site, which in the next step we set as the default. Setting it this way makes it available to all sites to enable themselves, but not as a default for all sites. If you'd like it to be exclusive just for your one site, and not maintained for the rest of them, you might have to just move it around or not use drush for that...not sure? ask on the forums.

4) drush vset theme_default theme
This sets the default theme for the site that you're in. This is awesome because it does it only for the specified site. Even if I set the Drupal7 default site default theme, it would have no impact on the other sites because they all get their own choices.

5) drush pm-list --type=theme
this last one is pretty neat because drush has some pretty cool project management tools. this one just lists out all of the themes available for your main installation, including the package it comes with, the name, status, and version.

Enjoy!

Drupal 7 multi-site install on CentOS6

Tricks to getting a multi-site Drupal7 installed on CentOS w/ Apache2, mySQL.

Follow these two posts, more or less. The biggest point of advice is that for each url that isn't a subdomain should point to the same url (i.e. your.alternative.site [192.168.10.1] v. original.alternative.site [192.168.10.1] v. original.alternative.site/different [192.168.10.1])

These are suggested reading sites, but none of them are exact, always referencing ubuntu or other linux distros...mine was a little different.

https://drupal.org/node/290768

https://drupal.org/node/138889#comment-2510252

https://drupal.org/node/1114158

http://www.seascapewebdesign.com/blog/how-setup-multi-sites-drupal-6-and-drupal-7

The most important thing is to have drush installed (mentioned in my other post). That makes installing everything else move quite quickly.

This doesn't have to be the same for everyone, but here are my steps:
1) cd /var/www/vhosts/drupal7 (if vhosts or drupal7 doesn't exist, create them)
2) drush dl drupal
before step 3, make sure to create the databases you need
pre-3a) mysql -u userName -p (this user is your sql admin user name, same or different from drupal db's)
pre-3b) create database YourMySQLDbName;
pre-3c) grant all on YourMySQLDbName.* to YourMySQLUserName@localhost identified by 'PasswordYouSet';
3) drush site-install standard --account-name=admin --account-pass=admin --db-url=mysql://YourMySQLUserName:PasswordYouSet@localhost/YourMySQLDbName
4) vi /etc/http/conf.d/drupal7.conf (make this if it doesn't exist yet)
5) paste the following in, save, then quit:

Alias drupal7 127.0.0.1

<Directory /var/www/vhosts/drupal7>
Options +FollowSymLinks
AllowOverride All
order allow,deny
allow from all
</Directory>

6) vi /etc/http/conf.d/00_drupal.conf (make this if it doesn't exist yet)
one thing I've learned about apache is that the apache configuration file works alphabetically, and this file is the configuration for all virtualhosts (which allows us to have the multi-sites installed correctly)
7) paste the following in, save, then quit:
NameVirtualHost *:80

<Directory /var/www/vhosts>
  AllowOverride All
</Directory>

<VirtualHost *:80>
   ServerPath /drupal7
   DocumentRoot /var/www/vhosts/drupal7
   ServerName drupal7
</VirtualHost>

8) apachectl restart

you should still be in the /var/www/vhosts/drupal7 directory
9) cp -a default/ sites/your.alternative.site/ (create it if it doesn't exist)
this should move over the files/ folder, settings.php and default.settings.php

10) chmod a+w sites/your.alternative.site/settings.php
&
chmod a+w sites/your.alternative.site/files

11) vi sites/sites.php
at the bottom of the file, you should add a line for each site you're installing. This hash is useful for finding the website and folders based on the apache setup (we'll get to that shortly).  Add the following:

$sites['nickname'] = 'your.alternative.site' (this is the folder path in drupal7/sites/your.alternative.site)

save and exit

12) now for your newly added site, we need to append info to the conf file so apache knows to redirect correctly to the sites.php file which will redirect to the correct folder which will show the correct site
vi /etc/http/conf.d/00_drupal.conf

at the bottom, add:

<VirtualHost *:80>
   DocumentRoot /var/www/vhosts/drupal7/sites/your.alternative.site
   ServerName nickname
</VirtualHost>

repeat for each new site

13) Finally change the settings for the newest site added:
vi sites/your.alternative.site/settings.php

change the database information for your new site

14) now setup your site: your.alternative.site/install.php?

15) after setup, change the folder and file permissions back

16) pat yourself on the back, it's all done :) - maybe a drush script for multi-site would be useful?

Drupal 7 on OSX 10.8, with Apache 2.2, MySQL, and PHP 5

This post is more about my experience of using a local instance of Drupal on my OSX box. The following are links about resources I found extremely helpful. Basically, I installed 2 versions (Drupal 7.2 and , and you have to jump through some hoops because although OSX ships with Apache and PHP, you need to now enable some settings (via cmd line or other tools) in order to have web sharing activated again (the silly prompt from the Preferences Pane is now gone *insert_sad_face*)

SITES:

1) This one helped me get the AMP stack going on my box (it also had some good step by step instructions about using phpMyAdmin)
http://www.coolestguyplanettech.com/downtown/install-and-configure-apache-mysql-php-and-phpmyadmin-osx-108-mountain-lion#apache

2) Getting a few steps closer, this site gave me some more info about getting the info setup correctly for Drupal as well as AMP stack
http://drewish.com/2012/07/26/drupal-on-mountain-lion-os-x-10-8/

3) A few stack overflow links
http://stackoverflow.com/questions/1676688/php-mysql-connection-not-working-2002-no-such-file-or-directory
and
http://stackoverflow.com/questions/1819592/error-when-connecting-to-mysql-using-php-pdo

4) And I cannot forget the infallible directions from Drupal.org
https://drupal.org/documentation/install

5) and for good measure, once I installed phpMyAdmin, the help docs helped :)
phpMyAdmin/doc/html/setup.html#setup_script
and
phpMyAdmin/doc/html/faq.html

So almost all of this got me to the right place, though it took a few hours to get screens working, and the DB didn't want to initially work. Why? Well, because the pdo_mysql.default_socket was not commented out...huh? Yeah, it didn't make sense to me for a bit. I searched lord google for things about the following error messages:

Warning: PDO::__construct(): [2002] No such file or directory (trying to connect via unix:///tmp/mysqld.sock)

Yeah, that kind of message didn't help at all, until I started changing my php.ini file around, and realized that that specific line, though correct (in a couple different ways, might I add), was wrong. My uneducated guess is that pdo_mysql is for Unix only, and as I am on OSX, it was wrong. Just commenting it out made it all better, install scripts started running on both installs, cats and dogs started sleeping together, and donuts began raining from the heavens...Ok, the last two didn't happen, but still, I hope this helps someone else besides me.

Setting up Jenkins on CentOS

It's been a while since I needed to install Jenkins, but I'm getting to the point where I'd like my code in GitHub (eventually pushed down to my own Git server on my CentOS box, once I figure out SSH), to be checked against tests whenever commits are made. Jenkins is the most kick-ass CI tool I know about. Setting it up is relatively easy, and getting it to play with Github is a cinch.

I followed the following steps to get it mostly up and running:
http://www.andrewzammit.com/blog/installing-jenkins-ci-on-centos-6-x-tomcat-with-an-ajp-proxy/

The only thing I'm not completely sure about is AJP. This is something outside the realm of my experience, and if it breaks, or if I have to learn more, I will, but for now, I want my box to pull against my repo, and if there's a change, pull, run tests, notify developers (rinse, repeat).

The only issue I really ran into was that Jenkins didn't run on the default java I had installed, so a quick update to my CentOS java via:

yum install java-1.7.0-openjdk

And now I have Jenkins up and running on CentOS!

Next is to figure out some good ole VPN fun, and then SSH (or the other way around).

Writing tests: RSpec's reason for Lambda

Most of my tests have been using the standard test format:

it "should add the thing to the db" do
  post :message, :content => "yadda yadda", :id => @user.user_id
  response.should change(Message, count).by(1)
end

However I'm now learning about Lambda's, which there have been some great posts recently about (i.e. Blocks v Procs v Lambdas). Lambda's themselves are a mathematical convention for improving readability of a formula. In the case of RSpec, Lambda's allow me to test the function, and not the response, for a change in my database.

My previous example actually says that the response would be the reason that the count changed. But if I have an empty db, or want to display data in my db temporarily to demonstrate that the function is correct, I can use a Lambda to my benefit. It essentially wraps my code in the before/after, so I can just do the following:

it "should add the thing to the db" do
  lambda do
    post :message, :content => "yadda yadda", :id => @user.user_id
  end.should change(Message, count).by(1)
end

Overall, the 2nd version is checking that I created a Message based on the action inside the Lambda, versus the 1st version is looking at the response to change the action.

Rails default JS files in application.html.erb to avoid defaults.js 404 error

Although Rails cannot let you post a DELETE action, it can pretend to do so with javascript. JQuery is the default js lib of choice for what I've studied so far, and in this post, a simple change in the application.html.erb allows my js file to be called correctly.

My Gemfile does have:
gem 'jquery-rails'

I tried updating my application.rb with a config.action_view.javascript_expansions[:defaults] = %w(jquery rails), but that caused Zeus to puke. No good :(

Instead, I made a slight change to my application.html.erb and was able to now magically have JS destroy for me (WIN!).

<%= javascript_include_tag "application %>

did the trick!

Rails 4.0.0, local CI with RSpec, Zeus, and Guard, using PostgreSQL, deployed to Heroku

This post is an initial rambling about what I've been doing on some side projects. Aside from learning about Sinatra, and remembering my SQL-fu commands, I'm getting well acclimated to Heroku, re-enjoying Git, and having a blast working around concepts in RoR.

This is just a mini post about my experience with Hartl's exercises, but using Ruby 1.9.3-p194, rvm, Rails 4.0.0 to build the micro-post blogging engine.

Really, in summation, there were a few tricks that made this experience more painful than necessary.

I should also mention that due to other client work, I'm now on Xcode5 DP3. This is only relevant for the time being. Once iOS7 is out of beta and Xcode5 is RC, then maybe the command line tools will work better. For now, the nokogiri gem refused to install, and that's because the libxml2 and libxlts gems refused to install because something in the gcc from Xcode5 DP3 did not like playing well. That said, I could not run capybara or webrat to utilize have_selector("attribute", "value") in any of my rspec's. It was rather unfortunate, but kept me on my toes.

Basically, my gemfile (after the jump), has a couple of gems commented out b/c of the aforementioned Xcode differences. I did not use webrat (or capybara) or annotate-models. I did use PostgreSQL, which I found easier to manipulate, and though it required some config changes, allowed me to deploy to Heroku with no problems. Because of Heroku, and resetting the db, I had to put the faker gem in the normal list, and not development, b/c Heroku tried to use Faker for resetting the db :(

The one gem I did not anticipate ever having to add was the protected_attributes. Rails 4 removed the mass assignment of attributes for security, and there is plenty of good reason for it (just google 'rails protected_attributes, or 'rails 4 attr_accessible'). To follow Hartl's Rails 3 example in Rails 4, I used the protected_attributes and devise gems.

source 'https://rubygems.org'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '4.0.0'

# Faker for dev and heroku
gem 'faker'

# Strong protection
gem 'protected_attributes'
gem 'devise', '3.0.0.rc'

# gravatars
gem 'gravatar_image_tag'

# Pagination
gem 'will_paginate'

# Use postgres as the database for Active Record
gem 'pg'

# Use SCSS for stylesheets
gem 'sass-rails', '~> 4.0.0'

# Use Uglifier as compressor for JavaScript assets
gem 'uglifier', '>= 1.3.0'

# Use CoffeeScript for .js.coffee assets and views
gem 'coffee-rails', '~> 4.0.0'

# See https://github.com/sstephenson/execjs#readme for more supported runtimes
# gem 'therubyracer', platforms: :ruby

# Use jquery as the JavaScript library
gem 'jquery-rails'

# Turbolinks makes following links in your web application faster. Read more: https://github.com/rails/turbolinks
gem 'turbolinks'

# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
gem 'jbuilder', '~> 1.2'

group :doc do
  # bundle exec rake doc:rails generates the API under doc/api.
  gem 'sdoc', require: false
end

# Use ActiveModel has_secure_password
# gem 'bcrypt-ruby', '~> 3.0.0'

# Use unicorn as the app server
# gem 'unicorn'

# Use Capistrano for deployment

group :development do
  gem 'rspec-rails'
  gem 'guard'
  gem 'guard-rspec'
  gem 'zeus'
  gem 'factory_girl_rails'
  #gem 'annotate-models'
  #gem 'webrat'
end

group :test do
  gem 'rspec'
  gem 'zeus'
  gem 'factory_girl_rails'
  #gem 'webrat'
end
# gem 'capistrano', group: :development

# Use debugger
# gem 'debugger', group: [:development, :test]

Rails w/ IntelliJ IDEA and BASH

After spending a lot of time w/ IDEs, I realize why I don't like VIM (nor eMacs, Less, More, Pico, Nano, etc). They're great, if you can memorize all of the shortcuts, keyboard tasks, and integration modules, as well as editing techniques, etc. But frankly, I have a very strong knowledge of IntelliJ, and people like John Lindquist show how to kick ass even better in an IDE than other systems.

My workflow with BASH lets me run my CI with Guard (using Zeus server), psql development (thanks Heroku!), and irb/rails c for any quick tests. However for autocompletion (because the Ruby and Rails frameworks are sooooo freakishly huge, that I'd never remember any of it, but I can also dive into any definition simply with ctrl + click, I'm learning much faster (even without the mustachio).

A few years ago I had a client switch me from Eclipse to IJ, and I haven't looked back. Rails is super easy to setup, but so is Python, and Scala, and Javascript. All of which rock to be able to switch back and forth on, even have multiple windows for different projects open, and multiple terminals running different servers, doing lots of different tests.

</rant>

Importing a Python Django project into IntelliJ IDEA with the Plugin using virtualenv

I recently was importing a project that was created by another developer.  Before importing the project, I wanted to install a virtualenv for my python and Django version (it is kind of annoying that if you create a virtualenv for each project, you have to re-install libs like Django every time).  I came across this great post that got me where I needed to be on OSX.8: http://hackercodex.com/guide/python-virtualenv-on-mac-osx-mountain-lion-10.8/

That got me most of the way there, and during this time I became intimately closer w/ virtualenv.  However, the problem was the IntelliJ was not enjoying the Django, no matter what set for my settings and manager scripts...finally, with the right google-fu I found this question that got me mostly there: http://stackoverflow.com/questions/13994846/intellij-python-plugin-run-classpath

In OSX, in the project settings, select the module, and then select the parent folder of the Django module, and there you can see if you have the virtualenv listed as part of your dependencies.  Once I got that in place, suddenly I could see into all of my Django libs and build my project.  Win!