Well, after my last rant/blog post I decided that its time to release some of our software as open source. When we build a site, I tend to prefer using a lean html/css front end and use Ajax to interact with the server. This can make for some fresh, dynamic sites and keeps page load times to a minimum. On the backed we try to keep things light, as this is obviously where you’re going to hit scalability issues so the more work that can be done on the client side the better.
On the backed you only really need a handful of classes, a basic controller to handle incoming Ajax request which then process these request, which generally just require database reads or writes. Each controller is generally different for each project, but follows the same principles and requires the same, simple, supporting machinery; database manager, session manager, logging and security/validation.
These classes give you a minimal layer of abstraction, making it easy to change infrastructure without making any code changes.
You could say this constitutes a php framework, albeit a modest one. But, its not really, as these are just helper classes that merely extend the base php functionality.
In the coming weeks, I’ll release all of these classes under GPL for anyone to play with, and welcome feedback.
So, I’ve just spend the last week getting to grips with the php framework symfony. Now, I understand the perceived benefit of using a framework to help speed up your development and save you from re-inventing the wheel.
I’ve been a developer for over a decade, and I’ve also been a manager too and now I run my own company. I always look for ways to save money by reducing the development overhead and leveraging pre-existing tech. But, I have never worked a project where a php framework has saved time or money. The possible exception is the wordpress framework, but that is because sites that are built from wordpress need a blog and a cms. That is a lot of overhead to build it out yourself, so it makes sense. But, it still takes longer to build out basic pages in wordpress then it would to hand code them in html/css with some ajax goodness. But, in the case of a simple site with NO cms and NO blog, then why the hell use a php framework? And wordpress is slooow. It takes some love to get a wordpress install running nice and fast.
I also build out server farms for my clients. So, I’m pretty clued up on how to build out fast sites on small budgets. Having 100% of a site rendered in php with lots of MySQL reads is definitely a bad start, I know there are plenty of ways to cache php and MySQL. Put a basic html/css site on a nginx server and watch it scream.
Anyway, back to symfony. Its just way too complex for what you need, its hard to maintain and not simple to upgrade. Ever been handed a symfony project from another developer? Yeah, its not all that nice. And multiple config files in multiple locations for a single website is also not a good selling point for a framework.
The number one rule in engineering is KEEP IT SIMPLE. Symfony violates that rule in so many ways, it adds complexity where complexity was not needed, and results in slower page loads. I like to keep it lean and mean.
I’ve spent so much of my career working with engineers who violate that number one rule on a daily basis, the often heard excuse is “I’m building re-usable code” or “this will make it faster to build next time“. Yeah, right. I met a highly successful VP at ITT, and he had a rule, “Write once, throw away“. This sounds crazy, but I believe him that its probably cheaper in the long run. Watch this video of a presentation by Ken Schwaber (the inventor of Agile project management), its interesting what he has say about the growth of infrastructure code and its effect on productivity.
So, what frameworks do I like for web development? Well, I love jQuery for one. This massively simplifies your code, and with the wealth of plugins life is made easy. And, the syntax is easy to read for the next developer who works on your code. It follows the KIS principle. Symfony does not.
I’m sure lots of people disagree with me on this, this is obviously just my opinion based on my own personal experience. But, figured it was worth sharing.
We’ve been working on a huge, soon to announced project, that involves Wordpress MU. I’ve been working for months on this, and I’ve really got stuck into the deepest and darkest parts of Wordpress MU. I’ve got to say, I’m a fan of Wordpress. The code is nice to work with, its well documented (for the most part) and it just works. As opposed to some of the other open source CMS and blog engines out there.
But, I just kept hitting performance bottle necks with it. Which, is to be expected when you’re serving heavy php pages with lots of MySQL reads. I went down the well traveled road of installing various caching plugins (WP Super Cache was my favorite), and they made a huge difference – but still, performance not quite where I felt it should be. This project is for a very high traffic site, so its really worth squeezing every last percentage of performance out of your code.
I was about to roll out some kind of reverse caching proxy, such as Squid or a server cluster when I started to debate changing out Apache for something lighter and faster. I’d heard good things about lighthttp and Nginx. I was a little nervous, as this is a live production site, but the performance gains could be worth it. When I learned that wordpress.com uses Nginx, that sealed it for me. Nginx it is then!
So, after much trepidation, I pulled the trigger and started to rebuild the app on a new server running Nginx (a LEMP stack, as opposed to LAMP). It took me a long and arduous 2 days to get everything back up and running (on a dev server first, customers didn’t experience any downtime), but man, first impressions blew my mind. It is way faster, especially for serving images. I intend to write a detailed post about the setup, but for now here is a graph that shows a quick comparison. The site is monitored by Site24×7, and the following graph shows the ping times for a given page. This page had very simple content, so I wasn’t expecting much. But, see for yourself. Its a big difference, note that the site was cached before and after the change it was not cached. Once I roll out caching on the new nginx server then it will be fair comparison, i.e. this graph makes it looks like nginx is slower then it really could be.
Some quick lessons learned, the hard way.
PHP sessions did not work, I spent hours trying to get them to work under nginx but then just decided it was time to roll out a MySQL based session system, its more secure and portable and only took about 2 hours to get up and running on the live site. I’ll post a how-to article soon.
If images aren’t being served correctly it could be because FastCGI php cache’s directory needs to be owned by the same user that Nginx is running as. Obvious I know, but still, not so obvious at the time. Well, lots of things aren’t obvious at 2 in the morning!
We’ve worked on a few projects over the years where we’ve adopted agile project planning practices. As a result, we’ve used a few of the tools that are out there, such as Rally, and just haven’t been that impressed (not to mention the cost!). It seems to me that the beauty of Agile is that its simple, and I feel like the tools that go along with it should be too. I’m a big fan of 7 Signal’s Basecamp, and use it a lot. Its good for what it is, but its not a project planing tool (which is fair, as its not supposed to be).
When we run agile, most of the actual day-to-day planning activity focuses around a white board, with basically a bunch of post-it notes that we physically move around. So, why is it that none of these project planning tools for Agile seem to be as simple? In my experience, they seemed to get bogged down by trying to give us too many features.
So, at Ad Astra we developed our own tool, trying to keep it as simple as possible. And, finally, we’ve decided to release it into the wild as a free service with the hope that other people will find it useful. We’re going to open source the code, if we have any interest from the community to do so. So, if you’re interested, send us an email and we’ll get you hooked up with the code repo. Its Beta software, and a work in progress – when we have free time!
Its been out for a few months now, but we just wanted to announce the official release of Electronic Arts’ latest game in the Spore series, Spore Islands for Facebook.
We had the privilege of assisting Area/Code in its development. We were the lead php and AI developer for this game, it was a lot of fun and a technically challenging project. We have a lot of experience building websites that support high traffic, but supporting a game as advanced as this with as much back-end logic as this required, was always going to be a challenge. But, we rose to the challenge and we’re extremely proud to have been a part of this great game series!
Logging in php isn’t fun, lets face it. There are quite a few php loggers out there, but we’ve built our own over the years and we felt why not share this with the community? So, why did we build our own? Well, firstly I personally like log files that are actually readable and secondly I really need a stack trace, I need to know not only what part of the code fell over, but what called that piece of code? And, then what called that piece of code. Sometimes I like to use the system log file, and just have a terminal open (yeah, we’re a Mac shop here!) and just watch the log using tail -f php_errors.log. But, sometimes I like to view the log in html for small test scripts. And, another thing is that sometimes I want to setup the logger to email me any fatal errors, this is very handy as sometimes these can go under the radar especially on really busy sites.
Our Logger class is static, which is great for speed and means there’s only ever one instantiation floating around. It also initializes itself, so you don’t need to worry.
require_once("Logger.class.php");
// This tells the logger to echo html
Logger::echoLog();
// Tells the Logger to catch and log system errors
Logger::catchSysErrors();
// This tells the logger to email you fatal errors. The email will contain the error
// message as well as a dump of the POST, GET and SESSION variables
Logger::setupEmail($email, $projectName, $logLevel = self::$FATAL)
// You can set the logging level (the default is debug)
// Level can be (in order);
// Logger::$DEBUG, Logger::$INFO, Logger::$ERROR, Logger::FATAL
// e.g., if you set the level to ERROR, only ERROR and FATAL messages would be outputted
// e.g., if you set the level to INFO, only INFO, ERROR & FATAL messages would be outputted
Logger::setLevel(Logger::$DEBUG);
// To use, call any of these
Logger::fatal("Some error message");
Logger::error("Some error message");
Logger::debug("Some error message");
Logger::info("Some error message");
// You can also dump a variable, array or object's contents with
Logger::dump($variable);
Here is the Logger class (apologies that the indenting is a little screwy);
/**
* Logging class, use for all trace commands
*
*/
class Logger {
public static $DEBUG = 0;
public static $INFO = 1;
public static $WARNING = 2;
public static $ERROR = 3;
public static $FATAL = 4;
private static $debugLevel = 0;
private static $echoLog = false;
private static $email = "dev@adastrasystems.com";
private static $project = "Adastra Project";
private static $sendEmail = true;
private static $emailLevel = 4;
// //////////////////////////////////////////////////////////////////////////////////////
/**
* Class constructor
*/
public function __construct(){
self::init();
}
// //////////////////////////////////////////////////////////////////////////////////////
public static function echoLog(){
self::$echoLog = true;
}
// //////////////////////////////////////////////////////////////////////////////////////
/**
* Let the Logger catch any system errors
*/
public static function catchSysErrors(){
set_error_handler("Logger::sysErrorHandler");
}
// //////////////////////////////////////////////////////////////////////////////////////
/**
*
*/
private static function trace($level, $msg){
if ($level < self::$debugLevel){
return;
}
$bt = debug_backtrace();
$class = "";
$function = "";
// get class, function called by caller of caller of caller
if (isset($bt[2]['class'])){
$class = $bt[2]['class'];
$function = "." . $bt[2]['function'];
}
// get file, line where call to caller of caller was made
$file = $bt[1]['file'];
$line = $bt[1]['line'];
$file_name = basename($file);
if (self::$echoLog){
switch($level){
case self::$DEBUG: $levMsg = "debug"; break;
case self::$INFO: $levMsg = "info"; break;
case self::$WARNING:$levMsg = "warning"; break;
case self::$ERROR: $levMsg = "error"; break;
case self::$FATAL: $levMsg = "fatal"; break;
}
$msg = "[$levMsg $class$function] $msg";
$msg .= " on line $line of $file_name";
if (isset($bt[2]['file'])){
$fname = basename($bt[2]['file']);
$msg .= ", called from line ".$bt[2]['line']." of $fname";
}
$msg .= "\n";
echo $msg;
flush();
}
else {
// Get IP address....
// Translate level into text...
$levMsg = "????";
switch($level){
case self::$DEBUG: $levMsg = "DEBUG"; break;
case self::$INFO: $levMsg = "INFO"; break;
case self::$WARNING:$levMsg = "WARNING"; break;
case self::$ERROR: $levMsg = "ERROR"; break;
case self::$FATAL: $levMsg = "FATAL"; break;
}
$msg = "[$levMsg] $class$function $msg";
$msg .= " {on line $line of $file_name";
if (isset($bt[2]['file'])){
$fname = basename($bt[2]['file']);
$msg .= ", called from line ".$bt[2]['line']." of $fname";
}
if (isset($bt[3]['file'])){
$fname = basename($bt[3]['file']);
$msg .= ", called from line ".$bt[3]['line']." of $fname";
}
$msg .= "}";
error_log($msg);
}
if (self::$sendEmail){
self::sendEmail($msg, $level);
}
}
// //////////////////////////////////////////////////////////////////////////////////////
private static function sendEmail($msg, $level){
if ($level < self::$emailLevel){
return;
}
$headers = 'From: ' . self::$email;
$message = $msg . "\n\nPOST VARIABLES \n\n" . print_r($_POST, true) . "\nGET VARIABLES\n\n" . print_r($_GET, true) . "\nSESSION VARIABLES\n\n" . print_r($_SESSION, true);
mail(self::$email, self::$project, $message, $headers);
}
// //////////////////////////////////////////////////////////////////////////////////////
/**
* Use to replace system error handler, if desired. Use Logger::catchSysErrors() to
* activate
*/
public static function sysErrorHandler($errno, $errstr, $errfile, $errline){
switch ($errno) {
case E_USER_ERROR:
$levMsg = "SYS_ERROR";
break;
case E_USER_WARNING:
$levMsg = "SYS_WARNING";
break;
case E_USER_NOTICE:
$levMsg = "SYS_NOTICE";
break;
default:
$levMsg = "SYS_UNKNOWN";
break;
}
if (self::$echoLog){
switch($errno){
case E_USER_NOTICE: $levMsg = "info"; break;
case E_USER_WARNING:$levMsg = "warning"; break;
case E_USER_ERROR: $levMsg = "error"; break;
default: $levMsg = "debug"; break;
}
$msg = "[$levMsg #$errno] $errstr";
$msg .= " on line $errline of ".basename($errfile)."";
$msg .= "\n";
echo $msg;
flush();
}
else {
$msg = "[$levMsg] No: $errno Msg: $errstr";
$msg .= " {on line $errline of ".basename($errfile)."}";
error_log($msg);
}
/* Don't execute PHP internal error handler */
return true;
}
// //////////////////////////////////////////////////////////////////////////////////////
public static function setupEmail($email, $projectName, $logLevel = self::$FATAL) {
self::$sendEmail = true;
self::$emailLevel = $logLevel;
self::$project = $projectName;
}
// //////////////////////////////////////////////////////////////////////////////////////
public static function setLevel($newLevel) { $debugLevel = $newLevel; }
public static function setLevelDebug() { self::$debugLevel = self::$DEBUG; }
public static function setLevelInfo() { self::$debugLevel = self::$INFO; }
public static function setLevelWarning() { self::$debugLevel = self::$WARNING; }
public static function setLevelError() { self::$debugLevel = self::$ERROR; }
public static function setLevelFatal() { self::$debugLevel = self::$FATAL; }
// //////////////////////////////////////////////////////////////////////////////////////
public static function dump($var) { self::debug(print_r($var, true)); }
public static function debug($msg){ self::trace(self::$DEBUG, $msg); }
public static function warning($msg){ self::trace(self::$WARNING, $msg); }
public static function warn($msg){ self::trace(self::$WARNING, $msg); }
public static function error($msg){ self::trace(self::$ERROR, $msg); }
public static function info($msg){ self::trace(self::$INFO, $msg); }
public static function fatal($msg){ self::trace(self::$FATAL, $msg); die(); }
// //////////////////////////////////////////////////////////////////////////////////////
}
Ok, so this is more of a rant than anything – but I think its so wrong that the vast majority of scientific papers are not freely available online. Lets face it, the vast majority of research going on at Universities world-wide is paid for by governments. If that government is a democracy, then that work was paid for by tax payers. So, why is it that when I try to find papers online I have to pay for them. Take for example www.sciencedirect.com, they charge $32 for each paper! And, you can’t preview the paper so you have no idea if its useful or relevant (the abstract really doesn’t cut it). Typically, if you want to survey a field of study you might need to reviews tens if not hundreds of papers. This is simply not tenable for a small company such as ours. The internet was perhaps the greatest technological achievement in history, its about time that scientific information flowed as freely as mp3’s, porn and youtube videos!
So, to the sciencedirect.com’s of the world – either open your electronic vaults to the world, or at the very least lower your prices to a sensible amount for the good of us all. Imagine the world where even the hobbyist and bedroom inventor can access the world’s combined scientific knowledge, and contribute to it. Perhaps, that is what they are afraid of!?
We’ve been working lately on a BlackBerry application, currently targeted for the 8900 Curve. Its been over a year since I last did any work on the Blackberry, so I dusted of the JDE from RIM and started to setup a fresh install of Netbeans (6.7.1) to start cranking away, how hard can it be to get things setup? Right? So, several frustrating hours later and digging up some old projects, I finally figured it out. I thought I’d post how to do it here to save someone else the pain!
Ok, so I’m going to assume you already have installed Netbeans along with the Mobility pack.
The first step is to download the JDE that you need from here, the version you need will depend on your phone. For the 8900 its version 4.6.1. So, I hear you ask, how do I know which one? Grab your Blackberry and hit Options->About and you’ll see the version number, this corresponds to the JDE version you’ll need. My 8900 gives a a version number of 4.6.1.250, hence I used the version 4.6.1 of the JDE. Note that on the about page, it also gives a Platform version number, for me this is 4.2.0.113 – ignore this.
Then, once you’ve installed the JDE check out this article over at netbeans. Although this is old, this basically works. So, follow these steps but ignore the last step where it asks you to cut and paste some stuff into your build.xml file. This part doesn’t work, at least not for me. Specifically, it would build but it would not generate the alx and cod file that you’ll need to actually run the application!
So, this was the tricky part – well, kinda. Cut and paste the following into the bottom of you build.xml file, just above the tag;
Post clean
Change the first line to point to the location of the JDE that you installed. The next step is to add a alx file, this is needed to generate the cod file. We can create one manually and place in the same directory as your build file. Its name will need to be the the name of your midlet. You can find this out, or set this, by looking at your project properties. Under Application Descriptor you should see a MIDLet-Name, this will be the name of you alx file (but give it an alx extension). Don’t worry, if you get the name or file location wrong, you’ll get an error message when you try to build the app, this will tell you the expected location and name that Netbeans was expecting, which I found useful as I got it wrong a couple of times myself
You’ll also need to change the target phone, see where it says rim.blackberry.emulator, for me the target was the 8900 phone, hence the 8900 for its value.
Once you create this file, paste the following into it (but replace MyApp and MyCompany with your app name and company name, its only really important to get the name of the .cod file correct) ;
Ad Astra has just completed a major 7-month project developing a game for the facebook platform. Ad Astra was the lead developer for the server-side code. The game, well, more of a simulation-game, was written in php and MySQL and was pretty computationally intensive. We were selected because of our expertise in both Artificial Intelligence and also our expertise in developing large scale websites. The game itself hasn’t officially launched yet, so we’re not able to publicize any details of the game yet – but when it is we’ll post an update here. But, what I can say is that the publisher is one of the largest game publishers in the world, so it was very exciting to work on a project for them. And, as always, working on games is just fun – though there are some long hours to get it just right. And, it was also fun to work on a team of talented professionals, artists, producers and developers. Awesome, can’t wait to get another game contract!
Ad Astra just wrapped up development of another mobile phone game working for Massively Mobile, this time the publisher was Cartoon Network, and as a big fan of theirs, this was a lot of fun to work on. The game was written in J2ME, targeting a few different handsets. Can’t say much more, once the game is published I’ll follow up with more details!
