Eat my monkeydust

Background

Recently I was working on a form for updating a couple of very simple values for a single table. When creating such forms, where we only need a subset of the available columns to be editable, we always have the option of either unsetting the fields we don’t need, or overriding the widgetSchema. In this case, I opted for the latter, since I only needed to edit 2 columns out of a possible 10, I didn’t think adding 8 fields to the unset() function was the cleanest way. The following examples contain obfuscated data.

Overriding the widget schema

public function configure()
{
  $this->setWidgets(array(
    'amount'               => new sfWidgetFormInput(),
    'reduced_amount' => new sfWidgetFormInput(),
    ));
 
   //Labels and decorator stuff here
}

My plan was to inherit the validators that already exist in the base class, since they do the job for what I need.

The error

The form worked fine for an insert, but when I came to update an existing record, the error was quite strange:

SQLSTATE[23505]: Unique violation: 7 ERROR: 
  duplicate key value violates unique constraint 
  "body_fee_version_pkey"

The problem it seemed was that my versionable behaviour was not incrementing the version value before attempting to insert a new version record. After a long period of debugging the versionable behaviour, along with some of my other custom behaviours, I was no closer to a solution.

I started to dig into the form classes, working backwords through all the object update methods, save, dosave, etc. Until I finally stumbled across this line:

$this->values = $this->validatorSchema->clean(
   self::deepArrayUnion($this->taintedValues, 
   self::convertFileInformation($this->taintedFiles))
 );

Before this call, everything seemed ok, but after this call, my values array, which at this stage only contained the two fields that had been posted, now suddenly included a value for all the fields in the table. Why? It then occurred to me that the entire validator schema was being processed, not just the fields that are actually posted! This means that all the validators that are required=false will silently return a “clean” value, which is most likely the database default.

So what did this mean? Well, it meant that the validator was “cleaning” all the columns that had not been submitted with the form, including the version column, which was being set to null. When the versionable behaviour kicked in, it read this null value and incremented by 1 for the next version, which then became 1 – a version which of course already existed, causing the error.

The solution

The solution is blindingly simple, we don’t just declare the widget schema, we must also declare the validator schema. Whilst this seems like it makes sense, I feel that it is a shame that I have to essentially copy and paste the necessary validators from the base class. The alternative of course would have been to unset the offending fields, but then we are back to option 1 above, unsetting 8 of 10 fields when it seems cleaner just to declare the 2 fields I actially need.

  $this->setWidgets(array(
    'amount'               => new sfWidgetFormInput(),
    'reduced_amount' => new sfWidgetFormInput(),
    ));
 
// Messages declared here as array since they are the same
 
$this->setValidators(array(
  "amount"  => new sfNumberValidator(
                       array('required' => true), $messages),
  "reduced_amount" => new sfNumberValidator(
                       array(), $messages),
 ));
 
   //Labels and decorator stuff here

Conclusion

I have been working with Symfony for over a year and a half, and with Symfony forms since they were born – and yet I was still caught out by something that seems quite simple, because I assumed it would be ok. There are so many things that could have alerted me to this problem and saved me a lot of time, for example if any of the extra fields had been required=true then at least I would have had some form errors to give me a clue!

I may suggest that the default behaviour should be to only process the validators that match the widgets, or maybe at least for the form to err if extra validators are found that are not used, as I feel that this is a mistake that others could make, and as I mentioned above, it’s a shame that we have to re-declare validators that are already present in the base class.

Intro

Last Friday, reading Jonathan Wage’s Doctrine blog, I spotted a post in the top right hand corner which Read “Doctrine at Symfony Camp 2008″, and I thought “Symfony Camp”? Wassat? So I took a look and realised that this would be a great event to attend, as I’m currently between projects and about to move from Symfony 1.0 to Symfony 1.1/1.2.

Working for an open source company (Redpill-Linpro) has it’s benefits – it took around an hour on Monday afternoon to gather interested parties (Hannes was the only one that could join me unfortunately), check with the boss(es), book flights, hire car and register. We were booked and ready to go.

Day 0 – Polar Bears and Pizza

We made it Gardermoen airport in plenty of time, noticing that the helpful monitor on the express train was informing us that our flight was delayed by an hour, giving us chance to play a couple of games of Hedgewars while we waited

Once we arrived in Amsterdam, I had the amusing pleasure of watching Hannes search in desperation for a place to smoke in a non-smoking airport, we grabbed our bags and then with a newly-chilled Icelander in tow proceeded to wind up the Alamo guy and get a nicely upgraded Volvo S40 hire car.

After a slight tour of the airport, TomTom took us to Amersfoort, where we drew out some €uros and looked in vain for  a place to eat. After giving up on the search (it was around 23:30 by now), we headed off to find the camp. TomTom took us to the correct road, but from that point on we were on our own – after driving down a road with an almost cartoon-like amount of road signs informing us of bends, obstructions, rabbits, etc. we found our headlights pointing down a cycle path. The sign showed a “dead end” symbol (“T” with a red top) and a couple of pictures of bikes an mopeds. On the floor was a dotted white line, separating the “road” into two sections barely big enough for a couple of bikes each. After consulting Hannes, we decided that this was not something we should be driving down, and headed back for a search of the area.

About 15 minutes later we were back driving down the “cycle path”. Yes, to get to Dutch Open Projects you have to drive down a cycle path – which “technically” is also a road, until just after the gate to their offices when a new sign informs you that only bikes may pass.

Hungry from our trip, we were pleased when offered some sandwiches by our new hosts which we munched on the way to setting up our camp beds in the military style tents. Everybody (except Fabien) was still up, so we joined “the Italians” and the beer started flowing.

During the rest of the “evening”, random shouts of “Polar Bear!” (aimed at Hannes), “Tantra!” (aimed at the Polish beer “Tatra” that was scattered about on the table and “Pizza!” (aimed at the Italians) kept us amused until around 4am, when I gave up and went to bed with a fag-smoke induced headache the size of a beach-ball.

While I tried to sleep, I was reminded of my current location by the constant shouts (see above), phone calls to Iceland and many general beer-fuelled activities, which fortunately involved avoiding the swimming pool. Early days yet…

On Thursday night there were relatively few people – on Friday there will be 80-90, more beer and a casino (With Symfony money no less). Stay tuned.

Day 1 – Community, Casinos and Karaoke

I woke up at about 5 past 10 to a sea of empty Italian beds, which was in plenty of time for the introductory fun and games at 10… hmm… well, anyway, by about 10:15 I had joined the conference and owing to the relaxed atmosphere here it was not embarrassing at all

Our Dutch host recreated a form of what he described as the Large Hadron Collider, although some may argue that it was just a line of red warning tape. We then proceeded to order ourselves in various categories, like how far we’d travelled (we were down at the end behind the Americans), how many Symfony apps we’d been part of, etc. It was clear that there was a broad mix of experience levels – Fabien at one end (of course) and Symfony virgins at the other.

Hannes was restful during most of the talks, but he made a noticeable shriek when Fabien mentioned that he considered PHP 5.3 to be PHP 6 because it wasn’t backwards compatible with 5.2… He quickly dozed off again though.

I won’t detail the talks here, since they will be well covered elsewhere, but we had an interesting day – often struggling to hear over the pounding rain but generally absorbing all the necessary info. We had a few chats about how the community can get more involved in the Symfony project, and some positive vibes were evident.

I have to mention that the barbecue was top notch, I had no less than 3 different types of veggie burger! They all tasted great along with the ample side salads and dressings – a welcome treat for me.

I also had a chat with Jonathan Wage about one of our future projects and whether Doctrine could “handle” it. I appreciate his time going through it with me and showing me some code examples, and helping me to formulate some options that I could present to the project – top bloke.

The rest of the evening consisted of some nifty casino action with possibly the worlds best croupier. He was letting us “cheat” to an extent which made the game of Blackjack a whole lot more fun – and when the game was over half of the table had watches, phones, wallets, car keys and credit cards down for the final hand. Great stuff. Following that was karaoke, and yes I did sing 2 or 3 numbers after warming up with a song I can’t remember (edit: it was “Pretty Fly for a White Guy”) right now because REM is playing in the background as I write this.

By 2am the barbecues had been lit for “munchie time”… not that I’m implying anything.

Day 2 – Write a ticket, submit a patch, write the documentation

I woke up pretty late today, but I feel refreshed at least! Maybe enough to try the pool later if I can convince anyone to join me… The sun has come out but it’s still a bit chilly all the same.

The weekend has been good, not only for learning, but also for networking – now we’ll be able to put names to faces the next time we are talking to someone in #Symfony or checking the commit logs. It has also increased our understanding of the project and the people involved, which is very important when it comes to trusting it for future developments.

The night ended with “the Italians” making pasta for us, which was a real treat (including my vegetarian option – thanks guys) and lots of wine drinking until the early hours with a couple of the French guys. It was a pretty cold night, so we even sneakily moved our beds inside for a warmer night’s sleep! It’s late in the day on Sunday when I write this, so we’d better get a move on and get ourselves into the City to meet “the Italians” for lunch.

I’ll do some more updates once we’ve left, and also add a few photos at some point.

Useful links

• Symfony camp
• Symfony Blog
• Rest support in Symfony 1.2 – Fabien Potencier
  
• Dependency injection in Symfony 2.0 – Fabien Potencier
• Lessons Learned at Yahoo! – Dustin Whittle
• Symfony Performance – Fabian Lange
• Doctrine – Jonathan Wage

• Plugins and Making your own – Lambert Beekhuis
• Developing for developers – Usability Applied to Programming – Francois Zaninotto
• Admin generator – Ian Christian
• Yahoo user interface libraries – Dustin Whittle
• Symfony debugging tools – Stefan Koopmanschap

I’m not sure if this will be covered in 1.1 (maybe someone can shed some light on it?) but currently when you logout using sfGuardAuth standard functionality, the session is not cleared/destroyed.

This only came to light recently, when I was scratching my head over why a parameter I had set was still available in the $sf_params array even after logging out, and logging back in again as a different user. This threw up an interesting security issue, because I started to wonder if I’d set any admin specific parameters elsewhere which could be reused by another user on the same machine.

The fix is fairly straightforward, but can only be run when not in test mode, because sfBrowser does not like the session to be destroyed! Maybe this is why it has never been written into the core functionality?

In apps/yourapp/modules/sfGuardAuth/actions/actions.class.php

 public function executeSignout()
   {
     if (sfConfig::get('sf_environment') != 'test')
     {
        session_destroy();
        session_write_close();
        session_regenerate_id();
     }
     parent::executeSignout();
   }

Adding the parent::executeSignout() line means that you can let the sfGuard plugin do the remainder of the work for you, so rather than overriding the function, you are just adding a bit to the start of it.

I was just working with a user list using an admin generator, and received a suggestion that username searches should automatically include wildcards in the searches (front and back), so it would not be necessary to add the asterisk. The problem was that the list was based on the sfGuardUser plugin, and we all know that modifying the contents of a plugin can be bad news.

The way to do this without modifying the plugin, is to override the action that is being called, and then call the parent method. In this case, the action that was processing the form submission was sfGuardUserActions->executeList(). So in apps/yourapp/modules/sfGuardUser/actions/actions.class.php add the following:

  public function executeList()
  {
    //Add fields to auto-wildcard to this list
    $wildcardFields = array("username");
 
    $filters = $this->getRequestParameter("filters")
    $originalFilters = $filters;
 
    if ($filters)
    {
      foreach ($wildcardFields as $fieldName)
      {
        // Only add the wildcards if the user is not already using them
        if (strpos($filters[$fieldName], "*") === false)
        {
          $filters[$fieldName]  = "*".$filters[$fieldName]."*";
        }
      }
      $this->getRequest()->getParameterHolder()->set("filters", $filters);
    }
    parent::executeList();
 
    // Set the filters back to their original values before the template is rendered
    $this->filters = $originalFilters;
  }

Now the request parameters for the selected fields are automatically prepended and appended with an asterisk before calling the parent function of the same name to continue with the request. Once the parent has finished, we then set the filters back to their original value, otherwise the asterisks we added would now appear in the fields when the page is presented.

An enhancement to this would be to specify the auto-wildcards field in the .yml file, rather than specifying it here in the action.

It seems that the sfGuard plugin only checks the “remember me” cookie if the user attempts to access a secure module. This is fine if your site requires a login for any access, and thus is always is_secure: on, however if you have any “public” pages which logged in and non-logged in users can access, users that have selected the “remember me” box will not be logged in automatically.

Solution:

Of course this could be fixed in the plugin, but if you do not want to (and shouldn’t) modify the plugin code directly, you can add another filter which will check the cookie. Assuming you have already installed the sfGuardPlugin, and have a “remember me” checkbox implemented, this is all you will need to do to get the automatic logins on non-secure pages:

in apps/yourapp/config/filters.yml add the remember me filter

rendering: ~
web_debug: ~
security:
  class: sfGuardBasicSecurityFilter
 
# generally, you will want to insert your own filters here

remember:
  class: rememberMeFilter
 
cache:     ~
common:    ~
flash:     ~
execution: ~

Now create rememberMeFilter.class.php in apps/yourapp/lib
This example uses Propel

class rememberMeFilter extends sfFilter
{
  public function execute ($filterChain)
  {
    // execute this filter only once, and if the user is not already logged in, and has a cookie set
    if ($this->isFirstCall() && !$this->getContext()->getUser()->isAuthenticated()
        && $this->getContext()->getRequest()->getCookie(sfConfig::get('app_sf_guard_plugin_remember_cookie_name', 'sfRemember')))
    {
      // See if a user exists with this cookie in the remember database
      $c = new Criteria();
      $c->add(sfGuardRememberKeyPeer::REMEMBER_KEY, $this->getContext()->getRequest()->getCookie(sfConfig::get('app_sf_guard_plugin_remember_cookie_name', 'sfRemember')));
      $c->add(sfGuardRememberKeyPeer::IP_ADDRESS, $this->getContext()->getRequest()->getHttpHeader ('addr','remote'));
 
      if ($resultArray = sfGuardRememberKeyPeer::doSelectJoinsfGuardUser($c))
      {
        $resultRow = current($resultArray);
        $this->getContext()->getUser()->signIn($resultRow->getSfGuardUser());
      }
    }
    // execute next filter
    $filterChain->execute();
  }
}

That should do it, now your users will be logged in on any page if they have the cookie set. The sfGuardPlugin will still take care of setting the cookie and clearing it on logout.

Remember when using these cookies, it’s good practice to ask the user to re-enter their password when doing anything sensitive, like submitting an order or changing any personal details like their password.

A follow up by Shiny explains that you can also get this working by securing modules with empty permission sets like this:

all:
  is_secure: on
 
index:
  credentials: []

Be careful with setting that globally though if you’re trying to set a “secure it first, grant permissions later” style system, because with the above – everything is accessible until secured.