Introduction

As a Symfony user and contributor for several years, it gives me great pleasure to announce that the company I work for Redpill Linpro along with sister company Varnish Software have agreed to be gold sponsors of the Symfony Live conference in Paris (March 2011).

As a company we have benefited from the Symfony framework in many ways including:

• Rapid development
• Consistency (easy to bring team members in and out)
• Community support
• Great documentation

And given back by providing time for our developers to:

• File bug reports
• Submit patches
• Develop plugins
• Help other users (particularly in IRC and the Symfony Forum)
• Update the documentation

By agreeing to gold sponsorship we are now also making a small financial contribution to the project, whilst at the same time giving ourselves a chance to become more known in the Symfony world, especially with Symfony2 just around the corner and its great support for Varnish!

Brief history of the companies

Until recently, Redpill and Linpro were two separate companies, and the work done by Varnish Software was a part of Linpro. In 2008, Linpro and Redpill merged to form Redpill Linpro, and in 2010 Varnish Software broke away to form a “daughter” company and concentrate on the Varnish cache. Confused?

The most important thing is that all of the companies involved are dedicated to open source, you can read more about Redpill Linpro and Varnish Software on their respective web sites.

Employment Opportunities for Symfony developers in “the North”

If you are looking for a company to work for with a strong PHP team, and either already live in or are considering a move to “the Nordic region¹” please get in touch. We also have opportunities for Java developers and various other posts related to Open Source technologies, and if we don’t have a specific post for you but you are a skilled open source enthusiast, we are also interested in your CV – who knows, we might be able to create one for you.

¹We have offices in Norway, Sweden, Denmark and Finland, the bulk of Symfony operations are presently in Oslo.

It’s been a while since Jonathan announced the availability of Doctrine 2 for Symfony 1.x (wouldn’t recommend trying it with anything less than 1.3) and a a few things have changed since then, so here’s a refresher.

We are at a crossroads now with Symfony 2 looming on the horizon, and many developers may wish to wait until that is more stable (later this year) to make the move to Symfony 2 and Doctrine 2 simultaneously. This is not such a bad idea, however Doctrine 2 is already at a level where you may wish to consider using it in your projects, you won’t be disappointed if you do!

It will take me a while to explain why Doctrine 2 is better than Doctrine 1, and more importantly why you should start using – but you can just take my word for it and take a look at a couple of Jonathan’s presentations to back up my claim. (I Recommend Doctrine 2 – not the same old PHP ORM)

Getting started

Check out the plugin and set up the database as described in Jon’s blog, but don’t configure the schema just yet

We’re going to go all out with the “Doctrine 2″ way of doing things, so we’ll be using annotations, not yaml or xml – although you can look up that syntax if you prefer. (A lot of the stuff below won’t work though – you can’t have multiple yaml files for example yet).

In your project configuration class, you will have access to some methods:

public function configureDoctrineConnection(\Doctrine\ORM\Configuration $config) {}
 
public function configureEntityManager(\Doctrine\ORM\EntityManager $em) {}

In the first of these, you need to add all the things to configure Doctrine before the entity manager can be created. Once Doctrine has created the entity manager, you can use the second method if you need to, to further configure the entity manager. (Registering a Doctrine listener for example).

So, in your configureDoctrineConnection() method, you’ll want to do something like this:

// Decide where you want the proxy classes to be stored, and which namespace they should use
$config->setProxyDir(sfConfig::get("sf_cache_dir")."/Proxies");
$config->setProxyNamespace("Proxies");
 
// You may want to make this environment specific for performance reasons
$config->setAutoGenerateProxyClasses(false);
$config->setSqlLogger(null);
 
// This will get things working, but later you will want to use APC
// or another "real" cache for production
$cache = new \Doctrine\Common\Cache\ArrayCache;
$reader = new \Doctrine\Common\Annotations\AnnotationReader($cache);
$config->setMetadataCacheImpl($cache);
$config->setQueryCacheImpl($cache);
 
// Tell Doctrine where to find your entities (you may have more than one location)
// This is mostly required for cli tasks that iterate over all of your entities
$paths = array(sfConfig::get("sf_lib_dir") . "/Entities"); // Populate this with all the locations of your entities
$reader->setDefaultAnnotationNamespace('Doctrine\ORM\Mapping\\');
$annotation = new \Doctrine\ORM\Mapping\Driver\AnnotationDriver($reader, $paths);
$config->setMetadataDriverImpl($annotation);
 
// Register all the classes that Doctrine needs to autoload
$classLoader = new \Doctrine\Common\ClassLoader('Entities\doctrine');
$classLoader->setIncludePath(sfConfig::get('sf_lib_dir'));
$classLoader->register();
 
$classLoader = new \Doctrine\Common\ClassLoader("Proxies");
$classLoader->setIncludePath($cachedir);
$classLoader->register();
 
$classLoader = new \Doctrine\Common\ClassLoader("Another\Namespace");
$classLoader->setIncludePath(sfConfig::get('sf_lib_dir'));
$classLoader->register();

Ok, that was a lot of stuff – but Doctrine 2 is more verbose – meaning less magic, but more explicit code. Some of the above calls may not be necessary – I need to do further testing, and also I’d like to get a lot of these things “standardised” in the plugin, so you can skip a lot of this if you follow a “default” path. The problem with the original release of the plugin was that it was not possible to have multiple class dirs, which makes it impossible to have plugins (for example) that also contain Entities. With the approach above, you explicitly add as many classes as you like to the annotation driver and the autoloader, and your plugin configuration classes can do the same.

Active Entity

The plugin comes bundled with Active Entity – this means that by extending ActiveEntity from our model classes (Entities) we can use our classes in much the same way as in Doctrine 1. Things like the following become available:

$user->toArray();
$user->save();
\Entities\User::find(1);
$user["name"];

ActiveEntity was written and included with good intentions, because for Doctrine 2 to continue to be compatible with symony in the same way as Doctrine 1, a bit of magic needed to be re-introduced. BUT – for anyone that has been excited by Doctrine 2, one of the major breakthroughs is the non-intrusive model, and ActiveEntity kills that (along with a kitten):

//With ActiveEntity:
class \Entities\User extends  \DoctrineExtensions\ActiveEntity {}
 
//Without ActiveEntity:
class \Entities\User extends-or-does-not-extend  \Whatever\The\Hell\You\Like {}

So I strongly recommend that you Don’t extend ActiveEntity – and gradually the plugin will be “fixed” so that is never necessary. Currently basic object forms and some widgets are working fine without it, but I haven’t tested it with admin generators or anything like that yet so feedback is appreciated.

If you follow my advice, you have to do everything the “Doctrine 2 way”. That means taking control of your own code! Write your own getters and setters, extend your own classes (if you want to) – be verbose, write OOP code, etc!

Ongoing development

Doctrine 2 is still in Alpha at the moment and is constantly changing. Since we are using it actively in a project at the moment, we normally spot the changes pretty quickly and update the plugin to keep up, however we are not using all the aspects of the plugin (like generators, all the form widgets, etc) so from time to time something might completely fail when you svn up.

We are also trying to slowly, and safely (for BC) remove the need for ActiveEntity – but we’re being careful with this because we don’t want to break existing projects that are using it.

Report bugs in the usual way using the symfony bug tracker (if it’s plugin related) or on Doctrine Jira (if you know it is a Doctrine issue). When reporting plugin bugs, please register the bug under the sfDoctrine component and add the keyword sfDoctrine2Plugin as I use a filtered query to keep an eye on Doctrine 2 related issues.

Coming in part 2…

Will either be based on feedback/suggestions (if any) or I’ll move on to some real world examples and best practices when it comes to integrating symfony and Doctrine 2.

Cascading / Transitive persistence

In Doctrine there are two ways to define a cascade, one will tell Doctrine what to do, the other defines how that condition is set up on your database when Doctrine builds the sql commands. The reason this was a gotcha for me, is that we only ever used one of the types, and were completely unaware of the other one! The Doctrine manual is quite long, and unless you’ve read it several times cover to cover it’s easy to miss these things – although next time I will at least be reading the appropriate section of the manual, regardless of what I think I already know.

So, you want a delete cascade? You might do something like this in your yaml file:

PhoneNumber:
  columns:
    user_id:
      type: integer
  relations:
    User:
      class:       User
      local:       user_id
      foreign:     id
      onDelete:    CASCADE

What the above definition says is that this phone number is connected to a user, and if the user record is deleted – in order to maintain database integrity with foreign keys, the corresponding phone number records should also be deleted. This is perfect behaviour – but Doctrine does not do anything with it, it simply sets up the database to handle it, so if you are using a dbms that supports delete cascades then everything will be handled at the database level.

Adding softDelete to the equation

So here comes the gotcha, all of our tables use the softDelete behaviour – which means that instead of the row being deleted, a flag is set (deleted=true) which then magically makes the row appear deleted in all your queries (Providing you have dql callbacks turned on). This means that the row is never actually deleted at the databse level, and thus the cascade is never applied. Now I always knew this, I mean I wouldn’t install a behaviour before knowing what it does right? What I wrongly assumed was that the call to onDelete: CASCADE would take care of “soft deleting” my relations too, but it doesn’t, in fact that line is effectively useless when combined with softDelete. For that you need to set the doctrine cascade parameter:

PhoneNumber:
  actAs: [softDelete]
  columns:
    user_id:
      type: integer
  relations:
    User:
      class:       User
      local:       user_id
      foreign:     id
      foreignType: one
      cascade:     [delete]

Now the behaviour will be as expected. If you are using softDelete and your application has a lot of delete operations, you should take some time to understand what is happening, because Doctrine must fetch and load all of the related objects and then check their relations and cascade settings and so on, so in a large system deleting a single user could result in quite a lot of database queries and cpu time. It is for this reason that database level cascades are preferable, so maybe being more picky about which models you apply the softDelete behaviour would be prudent. Learn more about Doctrine Transitive Persistance.

Defining types of foreign key relations

This one is also one of those “read the manual” situations, but similarly to the above example what caught us out here is that we again thought that there was only one option for defining what we wanted to do, when in fact there were too parameters we should have been considering. This gotcha also only applies if you are defining your relations in a yaml file, as the problem lies in its interpretation when Doctrine builds the base classes.

Look at the following example:

User:
  columns:
    id:
      type: integer
      notnull: true
  relations:
    Payment:
      class:       Payment
      local:       id
      foreign:     user_id
      foreignType: many

The intention was that a user could have multiple payments, and it seemed to make sense to define it using the foreignType parameter above, however when the model was built – this always resulted in a $this->hasOne(…) statement being created, because what was actually happening is that we were not defining the relationship at the other end, we were actually defining it at the local end! So foreignType: many actually means “Many users can have one payment” – hence the interpretation of $this->hasOne(..)

In actual fact using the foreignType key is mostly useless, as Doctrine “guesses” this side of the relationship. It’s only necessary if Doctrine guesses incorrectly and you need to override it, our example above should actually have been using the “type” parameter, which defines the relationship from the other perspective:

User:
  columns:
    id:
      type: integer
      notnull: true
  relations:
    Payments:
      class:    Payment
      local:    id
      foreign:  user_id
      type:     many

Adding [ foreignType: one ] to this schema declaration would have no offect, as Doctrine would guess it to be the case (in this example). The above declaration will now correctly form a $this->hasMany(…) statement in your base class. To make it easy to remember, just forget about foreignType all together unless you come across a special case where you nee it – and prepend the word “has” to the type parameter, that should give you a clue as to what you are defining.

So, with one-to-one relationships, the type parameter is unnecessary, with one-to-many you specify the type: many (Doctrine will guess the one part) and with many-to-one you also do not need to specify the type as it will be guessed based on how you’ve got the relation set up from the other model’s perspective.

Read more about relations in the Doctrine manual.

Conclusion

it’s hard to believe that after 6 months of coding a large application that currently has over 30,000 lines of code and growing we could have a schema file with several useless calls to onDelete: CASCADE and even more useless calls to foreignType. It’s even harder to believe that in the early days of the project, we simply overrode the setup methods to add the relations that we needed, when the Schema.yml file failed to come up with the goods. Next time I think I’ll spend a little more time with the documentation, although we are of course experts now – so maybe next time we won’t need to?

Recently I was working on a form for updating a couple of very simple values for a single table. When creating such forms, where we only need a subset of the available columns to be editable, we always have the option of either unsetting the fields we don’t need, or overriding the widgetSchema. In this case, I opted for the latter, since I only needed to edit 2 columns out of a possible 10, I didn’t think adding 8 fields to the unset() function was the cleanest way. The following examples contain obfuscated data.

Overriding the widget schema

public function configure()
{
  $this->setWidgets(array(
    'amount'               => new sfWidgetFormInput(),
    'reduced_amount' => new sfWidgetFormInput(),
    ));
 
   //Labels and decorator stuff here
}

My plan was to inherit the validators that already exist in the base class, since they do the job for what I need.

The error

The form worked fine for an insert, but when I came to update an existing record, the error was quite strange:

SQLSTATE[23505]: Unique violation: 7 ERROR: 
  duplicate key value violates unique constraint 
  "body_fee_version_pkey"

The problem it seemed was that my versionable behaviour was not incrementing the version value before attempting to insert a new version record. After a long period of debugging the versionable behaviour, along with some of my other custom behaviours, I was no closer to a solution.

I started to dig into the form classes, working backwords through all the object update methods, save, dosave, etc. Until I finally stumbled across this line:

$this->values = $this->validatorSchema->clean(
   self::deepArrayUnion($this->taintedValues, 
   self::convertFileInformation($this->taintedFiles))
 );

Before this call, everything seemed ok, but after this call, my values array, which at this stage only contained the two fields that had been posted, now suddenly included a value for all the fields in the table. Why? It then occurred to me that the entire validator schema was being processed, not just the fields that are actually posted! This means that all the validators that are required=false will silently return a “clean” value, which is most likely the database default.

So what did this mean? Well, it meant that the validator was “cleaning” all the columns that had not been submitted with the form, including the version column, which was being set to null. When the versionable behaviour kicked in, it read this null value and incremented by 1 for the next version, which then became 1 – a version which of course already existed, causing the error.

The solution

The solution is blindingly simple, we don’t just declare the widget schema, we must also declare the validator schema. Whilst this seems like it makes sense, I feel that it is a shame that I have to essentially copy and paste the necessary validators from the base class. The alternative of course would have been to unset the offending fields, but then we are back to option 1 above, unsetting 8 of 10 fields when it seems cleaner just to declare the 2 fields I actially need.

  $this->setWidgets(array(
    'amount'               => new sfWidgetFormInput(),
    'reduced_amount' => new sfWidgetFormInput(),
    ));
 
// Messages declared here as array since they are the same
 
$this->setValidators(array(
  "amount"  => new sfNumberValidator(
                       array('required' => true), $messages),
  "reduced_amount" => new sfNumberValidator(
                       array(), $messages),
 ));
 
   //Labels and decorator stuff here

Conclusion

I have been working with Symfony for over a year and a half, and with Symfony forms since they were born – and yet I was still caught out by something that seems quite simple, because I assumed it would be ok. There are so many things that could have alerted me to this problem and saved me a lot of time, for example if any of the extra fields had been required=true then at least I would have had some form errors to give me a clue!

I may suggest that the default behaviour should be to only process the validators that match the widgets, or maybe at least for the form to err if extra validators are found that are not used, as I feel that this is a mistake that others could make, and as I mentioned above, it’s a shame that we have to re-declare validators that are already present in the base class.