Eat my monkeydust

These are mistakes anyone can make, as proved by our team of experienced developers who only came across these late into a project – after developing with Doctrine for over 6 months.

Cascading / Transitive persistence

In Doctrine there are two ways to define a cascade, one will tell Doctrine what to do, the other defines how that condition is set up on your database when Doctrine builds the sql commands. The reason this was a gotcha for me, is that we only ever used one of the types, and were completely unaware of the other one! The Doctrine manual is quite long, and unless you’ve read it several times cover to cover it’s easy to miss these things – although next time I will at least be reading the appropriate section of the manual, regardless of what I think I already know.

So, you want a delete cascade? You might do something like this in your yaml file:

PhoneNumber:
  columns:
    user_id:
      type: integer
  relations:
    User:
      class:       User
      local:       user_id
      foreign:     id
      onDelete:    CASCADE

What the above definition says is that this phone number is connected to a user, and if the user record is deleted – in order to maintain database integrity with foreign keys, the corresponding phone number records should also be deleted. This is perfect behaviour – but Doctrine does not do anything with it, it simply sets up the database to handle it, so if you are using a dbms that supports delete cascades then everything will be handled at the database level.

Adding softDelete to the equation

So here comes the gotcha, all of our tables use the softDelete behaviour – which means that instead of the row being deleted, a flag is set (deleted=true) which then magically makes the row appear deleted in all your queries (Providing you have dql callbacks turned on). This means that the row is never actually deleted at the databse level, and thus the cascade is never applied. Now I always knew this, I mean I wouldn’t install a behaviour before knowing what it does right? What I wrongly assumed was that the call to onDelete: CASCADE would take care of “soft deleting” my relations too, but it doesn’t, in fact that line is effectively useless when combined with softDelete. For that you need to set the doctrine cascade parameter:

PhoneNumber:
  actAs: [softDelete]
  columns:
    user_id:
      type: integer
  relations:
    User:
      class:       User
      local:       user_id
      foreign:     id
      foreignType: one
      cascade:     [delete]

Now the behaviour will be as expected. If you are using softDelete and your application has a lot of delete operations, you should take some time to understand what is happening, because Doctrine must fetch and load all of the related objects and then check their relations and cascade settings and so on, so in a large system deleting a single user could result in quite a lot of database queries and cpu time. It is for this reason that database level cascades are preferable, so maybe being more picky about which models you apply the softDelete behaviour would be prudent. Learn more about Doctrine Transitive Persistance.

Defining types of foreign key relations

This one is also one of those “read the manual” situations, but similarly to the above example what caught us out here is that we again thought that there was only one option for defining what we wanted to do, when in fact there were too parameters we should have been considering. This gotcha also only applies if you are defining your relations in a yaml file, as the problem lies in its interpretation when Doctrine builds the base classes.

Look at the following example:

User:
  columns:
    id:
      type: integer
      notnull: true
  relations:
    Payment:
      class:       Payment
      local:       id
      foreign:     user_id
      foreignType: many

The intention was that a user could have multiple payments, and it seemed to make sense to define it using the foreignType parameter above, however when the model was built – this always resulted in a $this->hasOne(…) statement being created, because what was actually happening is that we were not defining the relationship at the other end, we were actually defining it at the local end! So foreignType: many actually means “Many users can have one payment” – hence the interpretation of $this->hasOne(..)

In actual fact using the foreignType key is mostly useless, as Doctrine “guesses” this side of the relationship. It’s only necessary if Doctrine guesses incorrectly and you need to override it, our example above should actually have been using the “type” parameter, which defines the relationship from the other perspective:

User:
  columns:
    id:
      type: integer
      notnull: true
  relations:
    Payments:
      class:    Payment
      local:    id
      foreign:  user_id
      type:     many

Adding [ foreignType: one ] to this schema declaration would have no offect, as Doctrine would guess it to be the case (in this example). The above declaration will now correctly form a $this->hasMany(…) statement in your base class. To make it easy to remember, just forget about foreignType all together unless you come across a special case where you nee it – and prepend the word “has” to the type parameter, that should give you a clue as to what you are defining.

So, with one-to-one relationships, the type parameter is unnecessary, with one-to-many you specify the type: many (Doctrine will guess the one part) and with many-to-one you also do not need to specify the type as it will be guessed based on how you’ve got the relation set up from the other model’s perspective.

Read more about relations in the Doctrine manual.

Conclusion

it’s hard to believe that after 6 months of coding a large application that currently has over 30,000 lines of code and growing we could have a schema file with several useless calls to onDelete: CASCADE and even more useless calls to foreignType. It’s even harder to believe that in the early days of the project, we simply overrode the setup methods to add the relations that we needed, when the Schema.yml file failed to come up with the goods. Next time I think I’ll spend a little more time with the documentation, although we are of course experts now – so maybe next time we won’t need to?

Background

Recently I was working on a form for updating a couple of very simple values for a single table. When creating such forms, where we only need a subset of the available columns to be editable, we always have the option of either unsetting the fields we don’t need, or overriding the widgetSchema. In this case, I opted for the latter, since I only needed to edit 2 columns out of a possible 10, I didn’t think adding 8 fields to the unset() function was the cleanest way. The following examples contain obfuscated data.

Overriding the widget schema

public function configure()
{
  $this->setWidgets(array(
    'amount'               => new sfWidgetFormInput(),
    'reduced_amount' => new sfWidgetFormInput(),
    ));
 
   //Labels and decorator stuff here
}

My plan was to inherit the validators that already exist in the base class, since they do the job for what I need.

The error

The form worked fine for an insert, but when I came to update an existing record, the error was quite strange:

SQLSTATE[23505]: Unique violation: 7 ERROR: 
  duplicate key value violates unique constraint 
  "body_fee_version_pkey"

The problem it seemed was that my versionable behaviour was not incrementing the version value before attempting to insert a new version record. After a long period of debugging the versionable behaviour, along with some of my other custom behaviours, I was no closer to a solution.

I started to dig into the form classes, working backwords through all the object update methods, save, dosave, etc. Until I finally stumbled across this line:

$this->values = $this->validatorSchema->clean(
   self::deepArrayUnion($this->taintedValues, 
   self::convertFileInformation($this->taintedFiles))
 );

Before this call, everything seemed ok, but after this call, my values array, which at this stage only contained the two fields that had been posted, now suddenly included a value for all the fields in the table. Why? It then occurred to me that the entire validator schema was being processed, not just the fields that are actually posted! This means that all the validators that are required=false will silently return a “clean” value, which is most likely the database default.

So what did this mean? Well, it meant that the validator was “cleaning” all the columns that had not been submitted with the form, including the version column, which was being set to null. When the versionable behaviour kicked in, it read this null value and incremented by 1 for the next version, which then became 1 – a version which of course already existed, causing the error.

The solution

The solution is blindingly simple, we don’t just declare the widget schema, we must also declare the validator schema. Whilst this seems like it makes sense, I feel that it is a shame that I have to essentially copy and paste the necessary validators from the base class. The alternative of course would have been to unset the offending fields, but then we are back to option 1 above, unsetting 8 of 10 fields when it seems cleaner just to declare the 2 fields I actially need.

  $this->setWidgets(array(
    'amount'               => new sfWidgetFormInput(),
    'reduced_amount' => new sfWidgetFormInput(),
    ));
 
// Messages declared here as array since they are the same
 
$this->setValidators(array(
  "amount"  => new sfNumberValidator(
                       array('required' => true), $messages),
  "reduced_amount" => new sfNumberValidator(
                       array(), $messages),
 ));
 
   //Labels and decorator stuff here

Conclusion

I have been working with Symfony for over a year and a half, and with Symfony forms since they were born – and yet I was still caught out by something that seems quite simple, because I assumed it would be ok. There are so many things that could have alerted me to this problem and saved me a lot of time, for example if any of the extra fields had been required=true then at least I would have had some form errors to give me a clue!

I may suggest that the default behaviour should be to only process the validators that match the widgets, or maybe at least for the form to err if extra validators are found that are not used, as I feel that this is a mistake that others could make, and as I mentioned above, it’s a shame that we have to re-declare validators that are already present in the base class.