Top Ten Behat Tip

Using Behat for BDD with Symfony is always a good idea. Here are my top ten hints for getting the most out of this useful tool;

  1. The feature files should be understandable by a non-techical audience – so don’t use IDs of form elements and such like. Mink can find input boxes by label, name or placeholder. Every input on the web page must have something that identifies its function to the user so use that. What is easier to understand; When I enter “Fred” for “Username” – or – When I enter “Fred” for “#uname”
  2. Use the Behatch library to get step definitions that might be useful (https://github.com/Behatch/contexts)
  3. Tag the scenarios to group them into sensible sub-sets. Such as “@security”, “@routing”, “@profile” – then you can just run a subset of the tests if you need to
  4. Ensure the database is in a known state before the tests start to run. This is probably the most important tip – you and your tests need to know exactly what is in the database before the tests start. There is a useful Doctrine class called the Purger which can empty database tables between tests. https://github.com/doctrine/data-fixtures/blob/master/lib/Doctrine/Common/DataFixtures/Purger/ORMPurger.php
  5. The Behat tests should only test outcomes that a regular user could verify, so try to avoid testing to see if an entity is updated in the database for example. If a user updates an entity for example, there should be some way in the user interface for them to verify that the change has been made – not by poking around in the  database.
  6. The cheat sheet is useful – if you’re old school, print it out and keep it next to you. Otherwise, open it in your second screen http://blog.lepine.pro/images/2012-03-behat-cheat-sheet-en.pdf
  7. If you must insist on checking in the database for a property of an entity, bear in mind that Doctrine might have cached the entity. So, your Feature Context might have loaded the entity from the database and, in running the test, Symfony might have modified the entity and saved the change to the database. If the test then tries to verify that an attribute has changed, its default position is to check the cached version of the entity – which, of course, will be exactly the same as it was when it loaded before the test ran. You’ll have to refresh the entity before doing the assertion – this forces Doctrine to reload the entity and to ignore any cached version.
  8. Make sure each Scenario has a unique and descriptive description – so when it fails in Jenkins you know where to start looking
  9. Don’t assume that the features will be tested in a certain order – so if a test changes something in the database, another test further down the line might fail as it could be assuming that the database is in another state (see 4)
  10. Don’t waste time writing steps that already exist – don’t write steps like “I should see the text” – use “I should see” – a built-in step from Mink. do “behat -dl” to see all the steps that are already written and available to use.

Mercury Rev gigs

Went to see Mercury Rev on Monday, so tried to recall all my previous gigs. Here goes …

  • 26/11/1991 – Duchess of York, Leeds
  • Sometime in March 1992 – Leeds University, supporing Ride
  • 19/11/1992 – Wulfrun Hall, Wolverhampton
  • 16/07/1993 – Warehouse, Derby
  • October/November 1993? – Glasgow University, with Spiritualized and Dr Phibes and the House of Wax Equations
  • sometime in 1998? – Sheffield University, supporting Bob Mould
  • 1999 – The Junction, Cambridge
  • 5/11/2001 – Leadmill, Sheffield
  • 6/11/2008 – Academy, Leeds
  • 24/11/2015 – Royal Northern College of Music, Manchester

Crazy Programming Practice part #39291

Sometimes other developers just amaze with their brilliance, or their craziness. For the latter, here is an example.

We had a system whereby the ID of each entity in a database was being encrypted, before being included in a URL – which could then be used to access some service. For example, a user might have ID=1234, which when encrypted was ABCDEFG987. They could then access personalised content via link that ended ABCEDF987.

The backend code, therefore, had to take an encrypted ID, and then deduce which user it belonged to. The simple way would be to decrypt the encrypted string (ABCEDF987 in the example above) to get back to the user’s ID (1234 in our example), then issue a simple query to get the User with that ID.

What someone did, however, was loop through every single User in the database, encrypting their ID and then comparing this to the encrypted string from the URL. With six million-odd users in the database, and an encryption algorithm that is deliberately quite resource intensive, this was a recipe for disaster.

The importance of two-stage sign ups

We’ve probably all used sites where all you have to do to sign up is provide an email address and a password – and, hey presto, you’re a registered member and you can post messages, upload photos, fill in surveys, etc, etc.

A moment’s thought reveals that using such a system on your own sites is a daft idea – there are simply no checks to ensure that the submitted email address belongs to the person signing up, or even if it is a real email address. Of course, you can validate the email address (use the world’s most complicated regular expression, or PHP’s filter_var function, or just check the string contains an @ sign – I’ve seen all three methods used, and often in the same codebase), but this does not guarantee that an email sent that address will ever reach the person it is intended for.

But, from a business perspective, whenever you need to contact your registered users, you can fall foul of systems put in place to protect real inboxes. Suppose you have a few hundred registered users with non-existent email addresses – let’s say that most of them are @hotmail.com addresses that our users have helpfully just made up. Then, suppose we decide to email all our users – chances are, a database query will get the email addresses to send to (possibly in alphabetical order) and some code then sends off the emails. Most of them will bounce of course, and we’ll probably ignore the bounces.

But, from the point of view of hotmail – it just looks like some server is sending emails to random addresses – and the sender will probably be checking to see which ones don’t bounce – a classic sign of namespace-mining  – “bombard a domain with loads of made-up email addresses and see which ones don’t bounce”.

So, the moral of this tale is – if you need users to sign up to your site, and you need an email address, insist on a two-stage authentication where your user can confirm that they can receive emails at the address they give to you. Obvious really.

Get on the Cycle Path!

These two links popped into my consciousness recently;

  1. http://youtu.be/fp-G2bdt1_4 – bus driver road rage incident captured on camera;
  2. http://www.yorkpress.co.uk/features/readersletters/10628039.Why_don___t_cyclists_use_proper_paths_/?ref=m Typically bat-shit letter in local rag

Both touch on that age-old question of “why don’t cyclists use cycle paths?”.

Probably most worrying is the claim from that guy (who looks like he should know better, being associated with one of the biggest bus companies in the country) in the YouTube video that if there is cycle path next to road, cyclists are banned from using the road. This is also indicated in some of the more vitriolic responses to the letter in the York Press. Any cyclist, and anyone reasonably familiar with the Highway Code, or the Law, would know that cyclists are permitted to use any road unless a specific prohibition exists – motorways carry this prohibition by default, but other roads can have them – fairly few do, however. The presence or absence of a cycle path makes no difference. The HIghway Code makes it clear that use of such tracks by cyclists is optional.

But, the question leads to another question – “if a cycle path is there, why don’t cyclists use it through choice?”. The questioner is usually hinting at some nagging feeling that these cyclists are not using the cycle path just to be awkward, just to get in the way, just to be bloody minded. But, this misses the point. Cyclists, like every other transport user, are allowed to select whatever legal route they wish, taking into account their own personal preferences for speed, safety, energy-expenditure, etc.

As a thought experiment, imagine a brand new motorway was built and opened, bypassing a congested town centre, and no-one chose to drive on. Would anyone question the motives of the drivers? No, they would obviously think that whoever had planned and built the motorway had got something fundamentally wrong. If a motorway is being built, it is because it should offer a more attractive route that the one going through the town centre – and all those drivers stuck in the congestion of the town centre would see that the motorway offers a better route and would start using it. If no-one chooses to use it, the people who designed and built it got it wrong. No-one could possibly argue otherwise.

But, if a cycle path is built and cyclists choose not to use it, it is somehow felt to be the fault of the cyclist, not the people who designed and built it. One could argue that cycle paths are built to encourage those who currently drive to try cycling – so every car that drives on a road with an adjacent cycle path is almost a witness to the cycle path’s failings.

Anyway, here are my top 10 reasons why I don’t use cycle paths next to roads;

  1. They are shared with pedestrians, who deserve their own space;
  2. They inevitably make the cyclist give way at every side road, and as I don’t have eyes in the back of my head, it is impossible to check if any traffic is coming without stopping
  3. they are seldom gritted in winter
  4. they seldom have over-hanging branches cleared
  5. they seldom are designed to recognised design standards
  6. they often are obstructed by street furniture
  7. they are often obstructed by parked cars
  8. they often have blind corners
  9. they often have bus stops on them
  10. they are never wide enough to pass pedestrians or other cyclists

Many cyclists tell of drivers shouting “get on the cycle path” at them. This has only happened to me twice. Once, it was a Wakefield Council gritting lorry that passed me far too closely, then sprayed me with its grit. It soon got stuck in traffic, I overtook and the driver shouted “get on the cycle path [expletive deleted]!”. Being the driver of a gritting lorry might, one could suggest, have led him to think why I might have been avoiding the cycle-path at that time – it was covered in ice.

Another time, a fat and red-faced man shouted it at me – we got into an altercation;

Him: You should be on the cycle path, you [expletive deleted]!

Me: Why?

Him: Because you should – it is safer and better.

Me: Who says?

Him: I do!

Me: If it’s so good, why aren’t you cycling on it?

Him: I don’t cycle anywhere.

Me: Oh, you seem to know a lot about it.

Him: I don’t.

Me: Exactly.

Cross Browser Functionality – how our Government does it

Anyone who works in the field of web-development will be well aware of the problems of the inconsistencies between the ways different browsers work. All too often, we find ourselves saying such things as “Well, it worked fine in Firefox/Chrome/Opera, why won’t it work in Internet Explorer 6/7/8/9/10”.

The latest issue of Private Eye magazine led me to this site; http://www.dwp.gov.uk/eservice/need.asp – an online portal for claiming Disability Living Allowance. Obviously, the current Government is committed to making it as hard as possible for anyone to claim any sort of benefit, but this site takes it a little too far;

Operating systems and browsers

The service does not work properly with Macs or other Unix-based systems even though you may be able to input information.

You are likely to have problems if you use Internet Explorer 7, 8, 9 and 10, Windows Vista or a smartphone. Clearing temporary internet files may help but you may wish to claim in another way.

There is also a high risk that if you use browsers not listed below, including Chrome, Safari or Firefox, the service will not display all the questions you need to answer. This is likely to prevent you from successfully completing or submitting the form. You may wish to claim in another way.

In short – you can only claim for this benefit on-line if you haven’t updated your computer or browser for ten years.

Using multiple entity managers in Symfony2

When using a single Symfony2 install to manage multiple web-apps, each with its own database, you can set up multiple entity managers and tell each bundle which manager to use.

One thing that stumped me, and has seemed to stump others judging by a quick Google, is how to inject a non-default entity manager into a service.

Turns out, the way to do it use [name]_entity_manager, instead of entity_manager in the arguments line of services.yml. So, if you have an entity manager called ‘other’, use this:

services:
     my_bundle.selector.project_selector:
         class: Me\MyBundle\Form\Selector\ProjectSelectorType
         arguments: ["@doctrine.orm.other_entity_manager"]

 

Using PHPass with CodeIgniter

Storing passwords in a database is a necessary evil for most web-applications. No-one in their right mind would save plain-text passwords, and MD5 hashes aren’t much better. A useful library called PHPass exists to hash passwords in a more secure manner. Using this in CodeIgniter should be straightfoward, but it is a bit fiddly.

Using third-party libraries in CodeIgniter is simple – just make sure the class you wish to use is the appliction/libraries directory and then use

$this->load->library('some_library.php');

But, when CodeIgniter loads the library, it also instantiates an object using the class definition in the library file. If the class constructor requires some parameters, you can pass them by adding an array to the $this->load->library() function;

$this->load->library('some_library.php',$params);

In this case, $params must be an array.

The PHPass class definition, however, requires two parameters. So, the easy way round this is to adapt the PHPass class constructor to accept an array rather than two single parameters;

function PasswordHash($params)
{
        $iteration_count_log2 = $params[0];
        $portable_hashes = $params[1];
        $this->itoa64 = './0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';

        if ($iteration_count_log2 < 4 || $iteration_count_log2 > 31)
            $iteration_count_log2 = 8;
        $this->iteration_count_log2 = $iteration_count_log2;

        $this->portable_hashes = $portable_hashes;

        $this->random_state = microtime();
        if (function_exists('getmypid'))
            $this->random_state .= getmypid();
    }

Football Results Database – part 1

In order to store the results of the NFL games, we’re going to need some tables. My initial design is as follows;

  • a table to store details of the teams in the league. For simplicity, we’ll assume that we only want to store results for a single season (for now), so the teams won’t change divisions. The ‘teams’ table just needs to store a ‘short-name’ for each team (which might as well act as the Primary Key for the table), the full name of the team, which conference it plays in, and which division. That should do for now.
  • a table to store details of the two Conferences. We might as well use AFC and NFC as the Primary Key for this table
  • a table to store details of each game; the week number, the date, maybe the location (maybe we’ll add this later) and whether the game went into overtime
  • a table to sore the results. a result will take up two rows of the table, one row for each team. I’ve added a ‘home’ field, which will be 1 if that row contains the score of the home team.

So, here is my basic table schema for MySQL;

CREATE TABLE `conferences` (
  `short` char(3) NOT NULL,
  `name` varchar(50) NOT NULL,
  PRIMARY KEY (`short`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
insert  into conferences values 
('AFC', 'American Football Conference'), 
('NFC', 'National Football Conference');

CREATE TABLE `games` (
  `game_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `week` int(11) NOT NULL,
  `year` int(11) NOT NULL,
  `date` date NOT NULL,
  `overtime` tinyint(4) unsigned NOT NULL DEFAULT '0',
  PRIMARY KEY (`game_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `results` (
  `game_id` int(11) unsigned NOT NULL,
  `team` char(2) NOT NULL,
  `points` int(11) NOT NULL,
  `home` tinyint(4) NOT NULL DEFAULT '0',
  KEY `game_id` (`game_id`),
  KEY `team_fk` (`team`),
  CONSTRAINT `game_fk` FOREIGN KEY (`game_id`) REFERENCES `games` (`game_id`) ON DELETE CASCADE ON UPDATE CASCADE,
  CONSTRAINT `team_fk` FOREIGN KEY (`team`) REFERENCES `teams` (`short`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `teams` (
  `short` char(2) NOT NULL,
  `name` varchar(100) NOT NULL,
  `conference` char(3) NOT NULL,
  `division` varchar(5) NOT NULL,
  PRIMARY KEY (`short`),
  KEY `conf` (`conference`) USING BTREE,
  CONSTRAINT `conf` FOREIGN KEY (`conference`) REFERENCES `conferences` (`short`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO teams VALUES 
('AC', 'Arizona Cardinals', 'NFC', 'West'), 
('AF', 'Atlanta Falcons', 'NFC', 'South'), 
('BB', 'Buffalo Bills', 'AFC', 'East'), 
('BR', 'Baltimore Ravens', 'AFC', 'North'), 
('CB', 'Chicago Bears', 'NFC', 'North'), 
('CL', 'Cleveland Browns', 'AFC', 'North'), 
('CN', 'Cincinnati Bengals', 'AFC', 'North'), 
('CP', 'Carolina Panthers', 'NFC', 'South'), 
('DB', 'Denver Broncos', 'AFC', 'West'), 
('DC', 'Dallas Cowboys', 'NFC', 'East'), 
('DL', 'Detroit Lions', 'NFC', 'North'), 
('GB', 'Green Bay Packers', 'NFC', 'North'), 
('HT', 'Houston Texans', 'AFC', 'South'), 
('IC', 'Indianapolis Colts', 'AFC', 'South'), 
('JJ', 'Jacksonville Jaguars', 'AFC', 'South'), 
('KC', 'Kansas City Chiefs', 'AFC', 'West'), 
('MD', 'Miami Dolphins', 'AFC', 'East'), 
('MV', 'Minnesota Vikings', 'NFC', 'North'), 
('NE', 'New England Patriots', 'AFC', 'East'), 
('NG', 'New York Giants', 'NFC', 'East'), 
('NJ', 'New York Jets', 'AFC', 'East'), 
('NO', 'New Orleans Saints', 'NFC', 'South'), 
('OR', 'Oakland Raiders', 'AFC', 'West'), 
('PE', 'Philadelphia Eagles', 'NFC', 'East'), 
('PS', 'Pittsburg Steelers', 'AFC', 'North'), 
('SD', 'San Diego Chargers', 'AFC', 'West'), 
('SF', 'San Francisco 49ers', 'NFC', 'West'), 
('SL', 'St. Louise Rams', 'NFC', 'West'), 
('SS', 'Seattle Seahawks', 'NFC', 'West'), 
('TB', 'Tampa Bay Buccaneers', 'NFC', 'South'), 
('TT', 'Tennessee Titans', 'AFC', 'South'), 
('WR', 'Washington Redskins', 'NFC', 'East');