RobL

Mysql2::Error: Incorrect string value

Waking up to an unexpected encoding error. Turns out the client has posted bullet points in their form data. It appears this isn’t working because the database is actually encoded in Latin1. Whereas it should be UTF-8.

An ActiveRecord::StatementInvalid occurred in jobs#create:

  Mysql2::Error: Incorrect string value: '\xE2\x80\xA8\x0D\x0A\x0D...' for column 'description' at row 1: INSERT INTO `stuffs`

Converted the character set to UTF-8 easily. However, through checking is probably prudent given that mixed encodings and converting them can have unexpected results

mysql> ALTER TABLE stuffs CONVERT TO CHARACTER SET utf8;
Query OK, 2350 rows affected (1.25 sec)
Records: 2350  Duplicates: 0  Warnings: 0

#HitIt - The Ultimate Guide to Programming Drums

Different kind of Programming this time Drum Programming Guide $15. Bargain.

Internal Server Error on new Rails 4 deploy

First deploy of a new app shows an Apache Internal Server Error and the following errors appear in the VirtualHost’s log. Not particularly helpful.

/var/log/apache/app-error.log

[Wed Oct 29 07:02:17 2014] [error] [client 81.111.42.20] Premature end of script headers: 
[Wed Oct 29 07:36:06 2014] [error] [client 81.111.42.20] Premature end of script headers: 
[Wed Oct 29 07:37:56 2014] [error] [client 81.111.42.20] Premature end of script headers: 

The default Apache error log shows more information. Seems I haven’t set up the Production secret key.

/var/log/apache/error.log

App 12630 stdout: 
App 12424 stderr: [ 2014-10-29 07:37:56.3925 12630/0x0000000157c988(Worker 1) utils.rb:68 ]: *** Exception RuntimeError in Rack application object (Missing `secret_key_base` for 'production' environment, set this value in `config/secrets.yml`) (process 12630, thread 0x0000000157c988(Worker 1)):
App 12424 stderr: 	from /var/www/app/shared/bundle/ruby/2.1.0/gems/railties-4.2.0.beta1/lib/rails/application.rb:510:in `validate_secret_key_config!'
App 12424 stderr: 	from /var/www/app/shared/bundle/ruby/2.1.0/gems/railties-4.2.0.beta1/lib/rails/application.rb:243:in `env_config'
App 12424 stderr: 	from /var/www/app/shared/bundle/ruby/2.1.0/gems/railties-4.2.0.beta1/lib/rails/engine.rb:510:in `call'
App 12424 stderr: 	from /var/www/app/shared/bundle/ruby/2.1.0/gems/railties-4.2.0.beta1/lib/rails/application.rb:161:in `call'
App 12424 stderr: 	from /home/deployer/.rvm/gems/ruby-1.9.3-p448/gems/passenger-4.0.41/lib/phusion_passenger/rack/thread_handler_extension.rb:74:in `process_request'
App 12424 stderr: 	from /home/deployer/.rvm/gems/ruby-1.9.3-p448/gems/passenger-4.0.41/lib/phusion_passenger/request_handler/thread_handler.rb:141:in `accept_and_process_next_request'
App 12424 stderr: 	from /home/deployer/.rvm/gems/ruby-1.9.3-p448/gems/passenger-4.0.41/lib/phusion_passenger/request_handler/thread_handler.rb:109:in `main_loop'
App 12424 stderr: 	from /home/deployer/.rvm/gems/ruby-1.9.3-p448/gems/passenger-4.0.41/lib/phusion_passenger/request_handler.rb:448:in `block (3 levels) in start_threads'

Which is true.

Robs-iMac:matotu rl$ cat config/secrets.yml 
# Be sure to restart your server when you modify this file.

# Your secret key is used for verifying the integrity of signed cookies.
# If you change this key, all old signed cookies will become invalid!

# Make sure the secret is at least 30 characters and all random,
# no regular words or you'll be exposed to dictionary attacks.
# You can use `rake secret` to generate a secure secret key.

# Make sure the secrets in this file are kept private
# if you're sharing your code publicly.

development:
  secret_key_base: somekey
test:
  secret_key_base: somekey

# Do not keep production secrets in the repository,
# instead read values from the environment.
production:
  secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>

I need to generate a new key, not the one I just posted though because that would be silly. I could place this in the repo replacing the ERB segment above but that’s not very secure.

Robs-iMac:app rl$ bundle exec rake secret
daf28cd197710437091a7bb8fef64b8fe0ccafc164f1c1a74f7b237e6a4b35cb9f6929259b41527035100c7a7a020b7bb1c2d6f6e4ce07df8a14a176ef50e40b

Instead I shall place it in /etc/apache/envvars

bash-4.3# cat /etc/apache2/envvars 
# envvars - default environment variables for apache2ctl

# Since there is no sane way to get the parsed apache2 config in scripts, some
# settings are defined via environment variables and then used in apache2ctl,
# /etc/init.d/apache2, /etc/logrotate.d/apache2, etc.
export APACHE_RUN_USER=www-data
export APACHE_RUN_GROUP=www-data
export APACHE_PID_FILE=/var/run/apache2.pid

## The locale used by some modules like mod_dav
export LANG=C
## Uncomment the following line to use the system default locale instead:
#. /etc/default/locale

export LANG

## The command to get the status for 'apache2ctl status'.
## Some packages providing 'www-browser' need '--dump' instead of '-dump'.
#export APACHE_LYNX='www-browser -dump'
export SECRET_KEY_BASE=somesecretkey

You’ll need to restart Apache, rather than just reload to get this to work. Obviously this is a environment variable for the entire Apache process, so if you wanted to host multiple applications on one server you could have to make the names of each secret unique.

And working…

Can I have my FFMPEG back, please?

It appears that FFmpeg has been removed from Ubuntu 14.04 and replaced with libav which is not actually very helpful if we use Semaphore for continuous integration tesing. However, there is a solution https://launchpad.net/~jon-severinsson/+archive/ubuntu/ffmpeg now we can just add the PPA and install. Phew.

sudo add-apt-repository -y ppa:jon-severinsson/ffmpeg
sudo apt-get update
sudo apt-get install -y ffmpeg libavcodec-extra-54

Rake tasks via Crontab with RVM

What seemed like a simple task has amounted to plenty of messing about. But here we go, the following didn’t work at all, it filled /var/mail/rails with junk and failure messages about missing a missing bundle command.

rails@fashion1-002:/var/www/app/current$ crontab -l
# m h  dom mon dow   command
0 5 * * * cd /var/www/app/current && bundle exec rake app:activity:rebuild
# 0 9 * * 1 cd /var/www/app/current && bundle exec rake app:jobs:digest
# */5 * * * * cd /var/www/app/current && bundle exec rails runner "SiteMailer.simple_message('contact@mydomain.me', 'Test', 'Test Cron Message').deliver"

You have to use Environment Variables in this case to specify RAILS_ENV although it is already defined in my .bashrc, also if you’re using RVM in order to use the correct version of Ruby you must use RVM’s own wrapper scripts. Lastly to stop the output from cron ending up in a dead mailbox or spamming you direct the output of the command to /dev/null

rails@fashion1-002:/var/www/app/current$ crontab -l
# m h  dom mon dow   command
0 5 * * * cd /var/www/app/current && RAILS_ENV=production /home/rails/.rvm/wrappers/ruby-2.1.2@app/bundle exec rake app:activity:rebuild > /dev/null
# 0 9 * * 1 cd /var/www/app/current && RAILS_ENV=production /home/rails/.rvm/wrappers/ruby-2.1.2@app/bundle exec rake app:jobs:digest > /dev/null
# */5 * * * * cd /var/www/app/current && RAILS_ENV=production /home/rails/.rvm/wrappers/ruby-2.1.2@app/bundle exec rails runner "SiteMailer.simple_message('contact@mydomain.me', 'Test', 'Test Cron Message').deliver" > /dev/null

Fixed.

How can you get a flash movie to accept parameters from the query string?

I have been using JWPlayer for a while to dynamically load videos into Facebook. Later versions of JWPlayer completely disable this ability because it can leave the player open to abuse from sites using your hosted player. How rude of them. Consequently I’ve been stuck on a quite an old version of JWPlayer just to retain this ability. However, this appears to have stopped working in Facebook recently and I need to re-implement this in a more recent version of JWPlayer.

I don’t know any ActionScript at all, but hunting through it appears to me that the config for setting up the player is here. In this case loading the config via Flash Vars passed to the movie via Javascript. So in theory I should optionally allow the config to be loaded from params around here.

https://github.com/braindeaf/jwplayer/blob/master/src/flash/com/longtailvideo/jwplayer/utils/Configger.as

private function loadExternal():void {
  if (ExternalInterface.available) {
    try {
      var flashvars:Object = ExternalInterface.call("jwplayer.embed.flash.getVars", ExternalInterface.objectID);
      if (flashvars !== null) {
        for (var param:String in flashvars) {
          setConfigParam(param, flashvars[param]);
        }
        dispatchEvent(new Event(Event.COMPLETE));
        return;
      }
    } catch (e:Error) {}
  }
  if (Security.sandboxType == Security.LOCAL_WITH_FILE) {
    dispatchEvent(new ErrorEvent(ErrorEvent.ERROR, false, false, "Error loading player: Offline playback not supported"));
  } else {
    dispatchEvent(new ErrorEvent(ErrorEvent.ERROR, false, false, "Error loading player: Could not load player configuration"));
  }
}

Ok, So it would appear there are copies of the JWPlayer Subversion repository floating around which show how configs are loaded in via RootReference. So in theory its just a question of putting this back in and rebuilding the player.swf. Might be nice to pull remote JSON instead of XML configs which would make things a bit more secure.

https://bitbucket.org/pombredanne/http-developer.longtailvideo.com-svn-tags/src/a84e56c637af0d725f9625c769bc715bb7dd88f3/mediaplayer-5.9.rc1/src/com/longtailvideo/jwplayer/utils/Configger.as?at=master

/**
 * @return
 * @throws Error if something bad happens.
 */
public function loadConfig():void {
  loadCookies();
  if (this.xmlConfig) {
    loadXML(this.xmlConfig);
  } else {
    loadFlashvars(RootReference.root.loaderInfo.parameters);
  }
}

/** Whether the "config" flashvar is set **/
public function get xmlConfig():String {
  return RootReference.root.loaderInfo.parameters['config'];
}

/**
 * Loads a config block from an XML file
 * @param url The location of the config file.  Can be absolute URL or path relative to the player SWF.
 */
public function loadXML(url:String):void {
  var xmlLoader:URLLoader = new URLLoader();
  xmlLoader.addEventListener(IOErrorEvent.IO_ERROR, xmlFail);
  xmlLoader.addEventListener(SecurityErrorEvent.SECURITY_ERROR, xmlFail);
  xmlLoader.addEventListener(Event.COMPLETE, loadComplete);
  xmlLoader.load(new URLRequest(url));
}

/**
 * Loads configuration flashvars
 * @param params Hash map containing key/value pairs
 */
public function loadFlashvars(params:Object):void {
  try {
    for (var param:String in params) {
      setConfigParam(param, params[param]);
    }
    dispatchEvent(new Event(Event.COMPLETE));
  } catch (e:Error) {
    dispatchEvent(new ErrorEvent(ErrorEvent.ERROR, false, false, e.message));
  }
}

Cross Domain Fonts with Nginx?

Having seen a few issues with cross domain web fonts with Firefox and a report of the same in Chrome. This small addition to an Nginx server config allows access from any origin, in this case for a fonts in a Rails asset pipeline. This config solves the problem of another site pinching our fonts. However, more importantly this allows cloudfront to sit in front of our origin server and for our site to serve fonts over cloudfront.

location ~ "^/assets/(.*/)*.*-[0-9a-f]{32}\.(woff|ttf|eot)$" {
  gzip_static on;
  expires     1h;
  add_header  Cache-Control public;
  add_header  Access-Control-Allow-Origin *;
  # debug to check this is being caught.
  add_header  X-Braindeaf-Pipeline true;
}

With the right syntax, we should probably limit the origin to *.cloudfront.net domains, and to our specific distrubitions in a more complete solution.

Static Google Maps

Since we’re heading off to Germany in the morning, here’s a quick stab at the Google Maps Image APIs Static Maps . Looks fairly straight forward if you have the Lat/Long of the places you want to create markers for.

https://maps.googleapis.com/maps/api/staticmap?zoom=6&size=900×500&maptype=roadmap&markers=color:blue|label:Brighton|50.84274,-0.106829&markers=color:green|label:Essen|51.450001,7.016700

Redirect / to another path in Nginx

I needed to tweak my Nginx config to redirect / (slash) only on a Wordpress site to a completely different location but leave all other routing intact. There we have it.

server {
  listen 80;
  server_name here.robl.me;
  root /var/www/wordpress;
  index index.html index.htm index.php;

  location = / {
    rewrite ^ http://there.robl.me permanent;
  }

  location / {
    try_files $uri $uri/ /index.php?$args;
    # auth_basic "Restricted";
    # auth_basic_user_file /etc/nginx/.htpasswd;
  }

  location ~ \.php$ {
    fastcgi_pass unix:/var/run/php5-fpm.sock;
    fastcgi_index index.php;
    include fastcgi_params;
  }
}

Wrote some PHP for the first time in about 8 years.

I think I liked it. Maybe. Associative Arrays and the like

<?php
$array = array(
  'units' => array(
    '1117411' => array(
      'is_shipped_at' => 'true',
      'shipped_at'    => '2014-10-14 T14:56:55+00:00'
    )
  )
);
var_dump($array);
?>
Robs-iMac:snarf rl$ php -f data.php
array(1) {
  ["units"]=>
  array(1) {
    [1117411]=>
    array(2) {
      ["is_shipped_at"]=>
      string(4) "true"
      ["shipped_at"]=>
      string(26) "2014-10-14 T14:56:55+00:00"
    }
  }
}